Category: Programming

Before we learn about the money, let’s get this question out of the way:

What is Tableau?

Let’s have a look at the definition inspired by the official Tableau website:

Tableau is a visual data analytics platform focused on the business analytics use case that helps you use data to solve problems. It is great to visualize data (e.g., using dashboards) and perform complex data analytics tasks with relatively simple-to-use operations.

I recommend you watch the following 10-minute video at 1.5x speed to get a first intuition of the platform quickly:

Here’s a screenshot of the video that shows the data analysis process relevant for Tableau:

And here’s a screenshot of all the subproducts of the Tableau platform—all of them may be a potential specialization for a Tableau Developer:

So, Tableau helps you accomplish all five steps of business process and business intelligence:

1. Storing transactions
2. Analyse data
   • Data preparation
   • Data analysis
3. Sharing insights
4. Making decisions
5. Monitoring outcomes and results

What is a Tableau Developer?

A Tableau Developer analyzes data, develops software, and creates data visualizations using Tableau to make businesses more efficient and effective.

Tableau developers routinely generate Tableau dashboards, business intelligence (BI) reports, and data visualizations to improve decision making in a data-driven organization.

Now that you know about what it is, let’s have a look at what it earns next!

Annual Income of Tableau Developer (US)

Question: How much does a Tableau Developer in the US make per year?

The expected annual income of a Tableau Developer in the United States is between $71,807 and $114,559, with an average annual income of $93,564 and a median income of $93,460 per year.

This data is based on our meta-study of 8 salary aggregators sources such as Glassdoor, ZipRecruiter, and PayScale.

Source	Average Income
Glassdoor.com	$97,578
ZipRecruiter.com	$114,559
Talent.com	$110,000
Indeed.com	$105,000
PayScale.com	$75,401
Comparably.com	$71,807
Zippia.com	$89,343
Salary.com	$84,829

Note: This is the most comprehensive salary meta-study of tableau developer income in the world, to the best of my knowledge!

To learn more about the income potential and opportunity in the broad data space, have a look at our full guide on the Finxter blog:

• Data Engineer — Income and Opportunity
• Data Scientist — Income and Opportunity
• Machine Learning Engineer — Income and Opportunity

Let’s have a look at the hourly rate of tableau freelancers next!

Hourly Rate of Tableau Freelancers

Tableau Developers are well-paid on freelancing platforms such as Upwork or Fiverr.

• Related Article: What’s the best freelancing platform?

If you decide to go the route as a freelance Tableau Developer, you can expect to make between $40 and $80 per hour on Upwork (source). Assuming an annual workload of 2000 hours, you can expect to make between $80,000 and $160,000 per year.

Note: Do you want to create your own thriving coding business online? Feel free to check out our freelance developer course — the world’s #1 best-selling freelance developer course that specifically shows you how to succeed on Upwork and Fiverr!

Industry Demand

But is there enough demand? Let’s have a look at Google trends to find out how interest evolves over time (source):

Tableau is now a Salesforce company. So, here’s the general interest in Salesforce for your reference:

Both trends look optimistic—up into the right!

Learning Path, Skills, and Education Requirements

Do you want to become a Tableau Developer? Here’s a step-by-step learning path I’d propose to get started with Tableau :

• Step 1: Introduction to Computer Science (~40 hours)
• Step 2: Introduction to Machine Learning (~40 hours)
• Step 3: Introduction to Data Science (~40 hours)
• Step 4: Introduction to Tableau (~20 hours)
• Step 5: Introduction to Tableau (Free Video Courses) (~10 hours)
• Step 6: Official Tableau Training Program (~40 hours)

You can find many additional computer science courses on the Finxter Computer Science Academy (flatrate model).

But don’t wait too long to acquire practical experience!

Even if you have little skills, it’s best to get started as a freelance developer and learn as you work on real projects for clients — earning income as you learn and gaining motivation through real-world feedback.

Tip: An excellent start to turbo-charge your freelancing career (earning more in less time) is our Finxter Freelancer Course. The goal of the course is to pay for itself!

Related Video

You can find more job descriptions for coders, programmers, and computer scientists in our detailed overview guide:

• Get Your Job in IT [Overview]

Related Income of Professional Developers

The following statistic shows the self-reported income from 9,649 US-based professional developers (source).

The average annual income of professional developers in the US is between $70,000 and $177,500 for various programming languages.

Question: What is your current total compensation (salary, bonuses, and perks, before taxes and deductions)? Please enter a whole number in the box below, without any punctuation. If you are paid hourly, please estimate an equivalent weekly, monthly, or yearly salary. (source)

The following statistic compares the self-reported income from 46,693 professional programmers as conducted by StackOverflow.

The average annual income of professional developers worldwide (US and non-US) is between $33,000 and $95,000 for various programming languages.

Here’s a screenshot of a more detailed overview of each programming language considered in the report:

Here’s what different database professionals earn:

Here’s an overview of different cloud solutions experts:

Here’s what professionals in web frameworks earn:

There are many other interesting frameworks—that pay well!

Look at those tools:

Okay, but what do you need to do to get there? What are the skill requirements and qualifications to make you become a professional developer in the area you desire?

Let’s find out next!

General Qualifications of Professionals

StackOverflow performs an annual survey asking professionals, coders, developers, researchers, and engineers various questions about their background and job satisfaction on their website.

Interestingly, when aggregating the data of the developers’ educational background, a good three quarters have an academic background.

Here’s the question asked by StackOverflow (source):

Which of the following best describes the highest level of formal education that you’ve completed?

However, if you don’t have a formal degree, don’t fear! Many of the respondents with degrees don’t have a degree in their field—so it may not be of much value for their coding careers anyways.

Also, about one out of four don’t have a formal degree and still succeeds in their field! You certainly don’t need a degree if you’re committed to your own success!

Freelancing vs Employment Status

The percentage of freelance developers increases steadily. The fraction of freelance developers has already reached 11.21%!

This indicates that more and more work will be done in a more flexible work environment—and fewer and fewer companies and clients want to hire inflexible talent.

Here are the stats from the StackOverflow developer survey (source):

Do you want to become a professional freelance developer and earn some money on the side or as your primary source of income?

Resource: Check out our freelance developer course—it’s the best freelance developer course in the world with the highest student success rate in the industry!

Other Programming Languages Used by Professional Developers

The StackOverflow developer survey collected 58000 responses about the following question (source):

Which programming, scripting, and markup languages have you done extensive development work in over the past year, and which do you want to work in over the next year?

These are the languages you want to focus on when starting out as a coder:

And don’t worry—if you feel stuck or struggle with a nasty bug. We all go through it. Here’s what SO survey respondents and professional developers do when they’re stuck:

What do you do when you get stuck on a problem? Select all that apply. (source)

Related Tutorials

To get started with some of the fundamentals and industry concepts, feel free to check out these articles:

• Freelance Developer – How to Code From Home and Earn Six Figures [Industry Report]
• How to Become a Python Freelancer—and Earn $1,000 on the Side? [A Step-by-Step Tutorial]
• How Adam Earns $5000 per Month as a Python Freelancer on Upwork [Month 4]
• Tableau Wikipedia
• Learn Tableau Google

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

References

[1] The figure was generated using the following code snippet:

import matplotlib.pyplot as plt
import numpy as np
import math data = [97578, 114559, 110000, 105000, 75401, 71807, 89343, 84829] labels = ['Glassdoor.com', 'ZipRecruiter.com', 'Talent.com', 'Indeed.com', 'PayScale.com', 'Comparably.com', 'Zippia.com', 'Salary.com'] median = np.median(data)
average = np.average(data)
print(median, average)
n = len(data) plt.plot(range(n), [median] * n, color='black', label='Median: $' + str(int(median)))
plt.plot(range(n), [average] * n, '--', color='red', label='Average: $' + str(int(average)))
plt.bar(range(len(data)), data)
plt.xticks(range(len(data)), labels, rotation='vertical', position = (0,0.45), color='white', weight='bold')
plt.ylabel('Average Income ($)')
plt.title('Tableau Developer Annual Income - by Finxter')
plt.legend()
plt.show()

Problem Formulation

Say, you want to find a regex pattern in a given string. You know the pattern exists in the string. You use the re.match(pattern, string) function to find the match object where the pattern matches in the string.

Problem: The Python regular expression pattern is not found in the string. The pattern doesn’t match anything, and, thus, the match object is None. How to fix this?

Here’s an example in which you’re searching for the pattern 'h[a-z]+' which should match the substring 'hello'.

But it doesn’t match!

import re my_string = 'hello world'
pattern = re.compile('h[a-z]+') match = re.match(pattern, my_string) if match: print('found!')
else: print('not found!')

Output:

not found!

Where is the bug? And how to fix it, so that the pattern matches the substring 'hello'?

Learn More: Improve your regex superpower by studying character classes used in the example pattern 'h[a-z]+' by visiting this tutorial on the Finxter blog.

Solution: Use re.search() instead of re.match()

A common reason why your Python regular expression pattern is not matching in a given string is that you mistakenly used re.match(pattern, string) instead of re.search(pattern, string) or re.findall(pattern, string). The former attempts to match the pattern at the beginning of the string, whereas the latter two functions attempt to match anywhere in the string.

Here’s a quick recap of the three regex functions:

• re.match(pattern, string) returns a match object if the pattern matches at the beginning of the string. The match object contains useful information such as the matching groups and the matching positions.
• re.search(pattern, string) matches the first occurrence of the pattern in the string and returns a match object.
• re.findall(pattern, string) scans string from left to right, searching for all non-overlapping matches of the pattern. It returns a list of strings in the matching order when scanning the string from left to right.

Thus, the following code uses re.search() to fix our problem:

import re my_string = 'hello world'
pattern = re.compile('h[a-z]+') match = re.search(pattern, my_string) if match: print('found!')
else: print('not found!')

Output:

found!

Finally, the pattern 'h[a-z]+' does match the string 'hello world'.

Note that you can also use the re.findall() function if you’re interested in just the string matches of your pattern (without match object). We’ll explain all of this — re.match(), re.search(), re.findall(), and match objects — in a moment but first, let’s have a look at the same example with re.findall():

import re my_string = 'hello world'
pattern = re.compile('h[a-z]+') match = re.findall(pattern, my_string) print(match)
# ['hello'] if match: print('found!')
else: print('not found!')

Output:

['hello']
found!

Understanding re.match()

The re.match(pattern, string) method returns a match object if the pattern matches at the beginning of the string. The match object contains useful information such as the matching groups and the matching positions. An optional argument flags allows you to customize the regex engine, for example to ignore capitalization.

Specification:

re.match(pattern, string, flags=0)

The re.match() method has up to three arguments.

• pattern: the regular expression pattern that you want to match.
• string: the string which you want to search for the pattern.
• flags (optional argument): a more advanced modifier that allows you to customize the behavior of the function. Want to know how to use those flags? Check out this detailed article on the Finxter blog.

We’ll explore them in more detail later.

Return Value:

The re.match() method returns a match object. You may ask (and rightly so):

Learn More: Understanding re.match() on the Finxter blog.

What’s a Match Object?

If a regular expression matches a part of your string, there’s a lot of useful information that comes with it: what’s the exact position of the match? Which regex groups were matched—and where?

The match object is a simple wrapper for this information. Some regex methods of the re package in Python—such as search()—automatically create a match object upon the first pattern match.

At this point, you don’t need to explore the match object in detail. Just know that we can access the start and end positions of the match in the string by calling the methods m.start() and m.end() on the match object m:

>>> m = re.search('h...o', 'hello world')
>>> m.start()
0
>>> m.end()
5
>>> 'hello world'[m.start():m.end()] 'hello'

In the first line, you create a match object m by using the re.search() method. The pattern 'h...o' matches in the string 'hello world' at start position 0.

You use the start and end position to access the substring that matches the pattern (using the popular Python technique of slicing).

---

Now that you understood the purpose of the match object, let’s have a look at the alternative to the re.match() function next!

Understanding re.search()

The re.search(pattern, string) method matches the first occurrence of the pattern in the string and returns a match object.

Specification:

re.search(pattern, string, flags=0)

The re.search() method has up to three arguments.

• pattern: the regular expression pattern that you want to match.
• string: the string which you want to search for the pattern.
• flags (optional argument): a more advanced modifier that allows you to customize the behavior of the function. Want to know how to use those flags? Check out this detailed article on the Finxter blog.

We’ll explore them in more detail later.

Return Value:

The re.search() method returns a match object. You may ask (and rightly so):

Learn More: Understanding re.search() on the Finxter blog.

Understanding re.findall()

The re.findall(pattern, string) method scans string from left to right, searching for all non-overlapping matches of the pattern. It returns a list of strings in the matching order when scanning the string from left to right.

Specification:

re.findall(pattern, string, flags=0)

The re.findall() method has up to three arguments.

• pattern: the regular expression pattern that you want to match.
• string: the string which you want to search for the pattern.
• flags (optional argument): a more advanced modifier that allows you to customize the behavior of the function. Want to know how to use those flags? Check out this detailed article on the Finxter blog.

We will have a look at each of them in more detail.

Return Value:

The re.findall() method returns a list of strings. Each string element is a matching substring of the string argument.

Learn More: Understanding re.findall() on the Finxter blog.

---

Now, this was a lot of theory! Let’s get some practice.

In my Python freelancer bootcamp, I’ll train you on how to create yourself a new success skill as a Python freelancer with the potential of earning six figures online.

The next recession is coming for sure, and you want to be able to create your own economy so that you can take care of your loved ones.

Check out my free “Python Freelancer” webinar now!

Join 20,000+ ambitious coders for free!

Problem Formulation and Solution Overview

In this article, you’ll learn how to print the contents of a List without surrounding brackets in Python.

To make it more fun, we have the following running scenario:

You are a student and need to memorize the first 10 elements in the Periodic Table. This data is currently saved in a List format. However, you would prefer it to display without brackets to remove any distractions.

Question: How would we write Python code to print a List without brackets?

We can accomplish this task by one of the following options:

• Method 1: Use print() and join()
• Method 2: Use print(*, sep)
• Method 3: Use slicing
• Method 4: Use join() and map()
• Bonus: Strip quote characters from Method 3.

---

Method 1: Use print() and join()

This method uses join() to access each element of the List passed. Then print() lets join() know which separator to concatenate (append) to each element. The result is a String.

periodic_els = ['H', 'He', 'Li', 'Be', 'B', 'C', 'N', 'O', 'F', 'Ne']
print(' / '.join(periodic_els))

This code declares a List of the first 10 element names of the Periodic Table and saves them to periodic_els.

Next, join() passes periodic_els as an argument and accesses each element, adding the appropriate separator character(s) as indicated in the print statement (' / ').

Finally, the output is sent to the terminal as a String data type.

Output

H / He / Li / Be / B / C / N / O / F / Ne

If we modified the print statement to have a comma (',') as a separator, the output would be as follows:

elements = ['H', 'He', 'Li', 'Be', 'B', 'C', 'N', 'O', 'F', 'Ne']
print(','.join(elements))

Output

H,He,Li,Be,B,C,N,O,F,Ne

 Note: We recommend trying different separating characters, including a space (' ') and an empty space ('') to see the output.

---

Method 2: Use print(*, sep)

This method uses print() passing *periodic_els, and a separator character (' ') as arguments to the same.

periodic_els = ['H', 'He', 'Li', 'Be', 'B', 'C', 'N', 'O', 'F', 'Ne']
print(*periodic_els, sep=' ')

 Note: Placing an asterisk (*) before the variable lets Python know to unpack/extract the variable, in this case, a List.

This code declares a List of the first 10 elements in the Periodic Table and saves them to periodic_els.

Inside the print() statement *periodic_els is passed as the first argument. This tells Python to unpack the List as indicated above.

Next, the sep argument is passed to print(). This will concatenate (append) character(s) between each element, in this case, a space (' ').

Finally, the output is sent to the terminal as a String data type.

Output

H He Li Be B C N O F Ne

---

Method 3: Use Slicing

This method uses slicing to remove the brackets. However, the single quote characters surrounding each element will remain.

periodic_els = ['H', 'He', 'Li', 'Be', 'B', 'C', 'N', 'O', 'F', 'Ne']
print(str(periodic_els)[1:-1])

This code declares a List of the first 10 elements in the Periodic Table and saves them to periodic_els.

Next, the print() statement converts the List to a string and then uses slicing to extract the output as indicated by [1:-1].

Finally, the output is sent to the terminal as a String data type.

Output

‘H’, ‘He’, ‘Li’, ‘Be’, ‘B’, ‘C’, ‘N’, ‘O’, ‘F’, ‘Ne’

---

Method 4: Use join() and map()

If periodic_els contained integers instead of strings, it would need to be converted to a String data type first. Then, join() and map() are used to output the contents without brackets.

periodic_els = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
print(', '.join(map(str, periodic_els)))

This code declares a List of the first 10 elements in the Periodic Table and saves them to periodic_els.

Next, periodic_els is converted to a String, and the iterable map() object is accessed. Then, each element is evaluated, and a separator character
(‘, ‘) is placed between each element and concatenated.

Finally, the output is sent to the terminal as a String data type.

Output

1, 2, 3, 4, 5, 6, 7, 8, 9,10

---

Bonus: Strip quote characters from Method 3

This section expands on Method 3, where the resultant output contained quote characters (') surrounding each element. This can easily be removed by running the following code.

periodic_els = ['H', 'He', 'Li', 'Be', 'B', 'C', 'N', 'O', 'F', 'Ne']
print(str(periodic_els)[1:-1].replace("'", ""))

The only difference between this code from Method 3 and the code above is that replace("'", "") is appended to the end.

Output

H, He, Li, Be, B, C, N, O, F, Ne

---

Summary

These four (4) methods of how to print a List without Brackets should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!

---

What is Data Cleaning?

Data cleaning describes the process of turning messy data into clean datasets that can be used for research and data science purposes. For example, tidy data will be in a wide format: every column contains a variable, and every row contains one case. Also, data cleaning means getting rid of corrupt data with very little utility.

Related Tutorial: Data Preparation in Python

How to Clean Unstructured Data in Python?

Most data in the real world is messy and unstructured or semi-structured. Working in data science, most of your time will be spent on cleaning and structuring data.

In research, data from surveys or experiments is mostly already structured into Excel or CSV tables.

In companies, data can be stored in databases, Excel files, or distributed all over the company. This data can be in emails, documents, folders, images, and note apps. 

This study shows, that most companies are having problems handling unstructured or semi-structured data, and almost half of them don’t even know where their data is located.

Unstructured data includes videos, images, and text or speech messages. Unstructured data from the web is mainly acquired by web scraping.

Semi-structured data is data found in documents, emails, social media posts, and if acquired from the web, it can be in HTML, JSON, or any other web format. 

Is Web Scraping Legal?

Important: Even though web scraping is possible does not mean it is always legal!

If the data is publicly available and not copyrighted, it is mostly safe to scrape. But also pay attention to data privacy laws and do not scrape personal data.

Scraping data from social media websites, for example, is mostly illegal, as it is not publicly available without logging in and contains personal data.

There are also many services to get data via an API. To be safe, refer to this guide about what is legal when web scraping:

What is the Goal of Data Cleaning?

The goal of data cleaning and cleaning unstructured or semi-structured data is to create tidy data with which you can work. Tidy data will be in a wide format: every column contains a variable, and every row contains one case.

To demonstrate both perspectives, this article is divided into two parts:

• First, we will scrape, load, and wrangle some semi-structured data from the web.
• Second, we will clean this data. This second step is also valid for structured data as it is about finding missing, outliers and duplicates.

I recommend doing an exploratory data analysis before or during cleaning data to get a good feeling of the data you have. You can easily combine exploring and cleaning.

Easy Exploratory Data Analysis (EDA) in Python with Visualization

Python HTML scraping

First, we will scrape a table from Wikipedia of the largest cities in the world. Scraping from Wikipedia is legal because

1. The data is publicly available 
2. The data works under a creative commons deed, which means the content is free to copy, share and adapt

First, we load our packages:

• pandas as our go-to library for data wrangling and analysis,
• the standard library for HTTP requests, and
• BeautifulSoup for easily reading HTML data.

import pandas as pd # pandas for data wrangling, cleaning, and analysis
import requests # for http requests
from bs4 import BeautifulSoup # html reading

The wiki page that holds the table we are looking for can be found here:

• https://en.wikipedia.org/wiki/List_of_largest_cities

We pass this URL into requests. The table in the HTML page is within <table class> , and the tables in wiki pages are called wiki tables. We can check this by looking at the HTML page in our browser or the HTML text file later to confirm we pulled the right table. 

With requests.get(url).text, we pull the HTML from the page. 

url='https://en.wikipedia.org/wiki/List_of_largest_cities'
url_response=requests.get(url).text

BeautifulSoup will pull the data table from the HTML file and save us time. We will pass the url_response from our request into the html.parser. With soup.find() we can tell it to look exactly for the wikitable. The output also tells us the name of the table.

soup = BeautifulSoup(url_response.text, 'html.parser')
html_listofcities=soup.find('table',{'class':'wikitable'})
html_listofcities

Output: 

Then we use pandas to read the HTML file and turn it into a pandas data frame, just like we would load any other data into Python.

df=pd.read_html(str(listofcities))
cities=pd.DataFrame(df[0])
print(cities.head())

Output:

The data looks messy. Now we get to clean!

Python HTML Cleaning

Let us inspect our newfound data:

Cities

Output:

I will drop the columns of the metropolitan area and the urban area because I am just interested in the population of the actual city. This can be done in several ways.

Here are two:

cities.drop(cities.columns[[7, 8, 9, 10, 11, 12]], axis=1, inplace=True)
# or
cities= cities.drop(cities.iloc[:,7:], axis = 1)

Then we will drop the first header column as it does not contain any useful information and rename the remaining header column.

# Dropping the first index column
cities.columns = cities.columns.droplevel(0) 

Output:

# rename the header column
cities.columns = ['city', 'country', 'UN_2018_population_estimates', 'city_definition', 'city_population', 'city_area_km2', ' city_density/km2']

Output:

The heading looks clean. Now we explore the dataset to find information to clean.

With df.info() and df.describe() we get a quick overview of the data we scraped. 

cities.info()

Output:

cities.describe()

Output:

It is immediately clear that the city_density/km2 is not a float even though it is supposed to be numerical.

Inspecting the data frame, you might have already noticed that the columns contain numbers following numbers in brackets, like [12]. This turns this data into an object, so we will have to get rid of this.

There are different ways to remove characters from a string in Python. We could just remove the last three characters of each string.

However, this would not work if some of our data points do not have the brackets at the end or more than that. So we’ll use the slicing method str.partition() to cut the brackets from our numbers. 

First, we make sure our object type is a string that we can work string operations on. Then we apply the str.partition() method and advise the function to cut off at the first bracket [.

cities[' city_density/km2'] = cities['city_density/km2'].astype('string')
city_density_str = cities['city_density/km2'].str.partition('[')
print(city_density_str)

Output:

What we want is the first column, so we pick this one [0] and write it back on a variable. 

city_density_str_col_1 = city_density_str[0]
print(city_density_str_col_1)

Output:

The commas in the variable will prevent us from converting the string into a float, so we’ll remove the comma with str.replace() before turning the string to a float with s.astype('float') and assigning it back to our data frame.

city_density_str_col_1 = city_density_str_col_1.str.replace(',','')
cities['city_density/km2'] = city_density_str_col_1.astype('float')
print(cities['city_density/km2'])

Output:

The variable now shows up when we look at df.describe() and we’ll want the results rounded for better readability:

cities.describe().round(1)

Output:

Cleaning Structured Data in Python

Following the cleaning of the scraped data we can now use it like a structured data frame with data we collected or downloaded.

This also can be cleaned of missing data, outliers and duplicates but does not always need data wrangling. However, with a data frame with many strings the cleaning process also often involves a lot of string manipulation.

Important note:

If you want to apply machine learning algorithms to your data, do split your dataset before feature engineering and data transformation as this can create data leakage!

Dropping duplicates is easy: 

cities = cities.drop_duplicates()

There didn’t seem to be duplicates in our df, as the size remained the same.

When dealing with missing values, we must decide how to handle them based on our data. 

We can either

• Drop missing values
• Replace or impute the values
• Leave missing values in the dataset 
• Transform the information that they’re missing into a new variable

First, we inspect our missing data. The function df.isnull() is a boolean function, that tells us for the whole data frame if data is missing or not.

We can sum it up to determine, how many values are missing in each column.

cities.isnull().sum()

Output:

We can drop rows with missing values completely.

This will cause us to lose useful information in other columns. But as the first row is completely empty anyway, we can drop this one.

The df.dropna() function has useful features that help us pick what missing data we want to remove. So, I just want to remove the one row, or all of them if there are more, with all missing values.

cities = cities.dropna(how='all')

This will look like this:

What is left are the missing values for 8 cities for population, area and density. We will replace those. 

Of course, you can look up the data on Wikipedia and reinsert them. For the sake of the exercise and because most of the time it is not possible to look up missing data, we will not do this.

The dataset now contains the data of the 73 biggest cities in the world, using the average of these to impute the missing values in the other 8 is the only and closest guess we have. This does not create much more information but keeps us from losing other information from these 8 cities.

The alternative option would be to drop those 8 cities completely.

So, we’ll replace the missing values in the area column with the average area size of all the other cities. First, we create the mean of the city area sizes, then we fill the missing values in the column with this value.

Pandas has the right function for this: df.fillna()

city_area_mean = cities['city_area_km2'].mean()
cities['city_area_km2'] = cities['city_area_km2'].fillna(value=city_area_mean)
cities.isnull().sum()

Output:

The output shows that we now have replaced and eliminated several missing values.

For the population, we luckily have another column that shows the population estimates of the UN in 2018 for each city.

So, we can use these to impute the missing population data, as it is as close as we can get to replacing them accurately. 

cities['city_population'] = cities['city_population'].fillna(cities['UN_2018_population_estimates'])
cities['city_population'].describe().round(1)

Output:

Now we still have missing values in the city density column. This one we can calculate now by dividing the population by the area.

So, we create a new variable to calculate the density with our new imputed data. Then we fill the missing values with this calculated density.

city_density = cities['city_population']/cities['city_area_km2']
cities['city_density/km2'] = cities['city_density/km2'].fillna(value=city_density)
cities['city_density/km2'].describe().round(1)

Output:

We can check back our missing values and the description of our dataset.

cities.isnull().sum()

cities.describe().round(1)

There is still one value missing in our city definition. Let’s have a look at these categories. 

cities['city_definition'].value_counts()

Output:

As we don’t know if the missing city is a municipality or a capital, we could just replace the missing value with the generic description of “city”, as we know they all are cities.

If you’d want to calculate the differences between these categories, it would be useful to categorize and merge these single entries into bigger categories.

For now, we will just replace the missing value with “city”, as I am more interested in the size of the cities than the category.

cities['city_definition'] = cities['city_definition'].fillna('City')
cities.isnull().sum()

Great! We got rid of all the missing values.

Info: For many statistical operations, missing values will be dropped by default and don’t create a problem. For machine learning algorithms missing values must be removed before modelling.

We can also create dummy variables (information is missing/ not missing) as the fact that the data is missing might be useful information. This way, the fact that they’re missing can be included in the data analysis process. 

Visualization

Now we visualize our data and check for outliers with a seaborn scatterplot.

import seaborn as sns
sns.scatterplot(data=cities, x="city_population", y="city_area_km2", size="city_population")

Output:

The city in the right top corner is clearly an outlier, but not one we would want to remove or equalize as it is not a measurement error. It is just the biggest city (or metropolitan area) in the world!

Let’s find out which one it is with df.sort_values(), using ascending=False to sort the city population from high to low.

cities.sort_values('city_population', ascending=False)

Output:

The biggest city on earth by size and city population is the municipality Chongqing in China with over 32 million inhabitants!

To visualize our cities more beautifully, we can use a sns.relplot() with color and different sizing.

sns.relplot(x="city_population", y="city_area_km2", hue="country", size="city_population", sizes=(40, 400), alpha=.5, palette="muted", height=6, data=cities)

Output:

Now keep on cleaning!

Problem Formulation and Solution Overview

In this article, you’ll learn how to check a List for Duplicates in Python.

To make it more fun, we have the following running scenario:

The Finxter Academy has given you an extensive list of usernames. Somewhere along the line, duplicate entries were added. They need you to check if their List contains duplicates. For testing purposes, a small sampling of this List is used.

Question: How would we write Python code to check a List for duplicate elements?

We can accomplish this task by one of the following options:

• Method 1: Use set() and List to return a Duplicate-Free List
• Method 2: Use set(), For loop and List to return a List of Duplicates found.
• Method 3: Use a For loop to return Duplicates and Counts
• Method 4: Use any() to check for Duplicates and return a Boolean
• Method 5: Use List Comprehension to return a List of all Duplicates

---

Method 1: Use set() and List to return a Duplicate-Free List

This method uses set()which removes any duplicate values (set(users)) to produce a Duplicate-Free set(). This set is then converted to a List (list(set(users))).

users = ['AmyP', 'ollie3', 'shoeguy', 'kyliek', 'ollie3', 'stewieboy', 'csealker', 'shoeguy', 'cdriver', 'kyliek'] dup_free = list(set(users))
print(dup_free)

This code declares a small sampling of Finxter usernames and saves them to users.

Next, set() is called and users is passed as an argument to the same. Then, the new set is converted to a List and saved to dup_free.

If dup_free was output to the terminal before converting to a List, the result would be a set(), which is not subscriptable. Meaning the elements are inaccessible in this format.

Output

{'csealker', 'cdriver', 'shoeguy', 'ollie3', 'kyliek', 'stewieboy', 'AmyP'}

Note: Any attempt to access an element from a set will result in a not subscriptable error.

In this example, the set() was converted to a List, and displays a List of Duplicate-Free values.

Output

['csealker', 'cdriver', 'shoeguy', 'ollie3', 'kyliek', 'stewieboy', 'AmyP']

Note: An empty set will result if no argument is passed.

---

Method 2: Use set(), For loop, and List to return a List of Duplicates Found

This method uses set(), and a For loop to check for and return any Duplicates found (set(x for x in users if ((x in tmp) or tmp.add(x)))) to dups. The set() is then converted to a List (print(list(dups))).

Here’s an example:

users = ['AmyP', 'ollie3', 'shoeguy', 'kyliek', 'ollie3', 'stewieboy', 'csealker', 'shoeguy', 'cdriver', 'kyliek'] tmp = set()
dups = set(x for x in users if (x in tmp or tmp.add(x)))
print(list(dups))

This code declares a small sampling of Finxter usernames and saves them to users.

Next, a new empty set, tmp is declared. A For loop is then instantiated to check each element in users for duplicates. If a duplicate is found, it is appended to tmp. The results save to dups as a set().

Output

In this example, the set() was converted to a List and displays a List of Duplicates values found in the original List, users.

['kyliek', 'ollie3', 'shoeguy']

---

Method 3: Use a For loop to return Duplicates and Counts

This method uses a For loop to navigate through and check each element of users while keeping track of all usernames and the number of times they appear. A Dictionary of Duplicates, including the Usernames and Counts returns.

Here’s an example:

count = {}
dup_count = {}
for i in users: if i not in count: count[i] = 1 else: count[i] += 1 dup_count[i] = count[i]
print(dup_count)

This code declares two (2) empty sets, count and dup_count respectively.

A For loop is instantiated to loop through each element of users and does the following:

• If the element i is not in count, then the count element (count[i]=1) is set to one (1).
• If element i is found in count, it falls to else where one (1) is added to count (count[i]+=1) and then added to dup_count (dup_count[i]=count[i])

This code repeats until the end of users has been reached.

At this point, a Dictionary containing the Duplicates, and the number of times they appear displays.

Output

{'ollie3': 2, 'shoeguy': 2, 'kyliek': 2}

---

Method 4: Use Any to Check for Duplicate Values

This example uses any(), and passes the iterable users to iterate and locate Duplicates. If found, True returns. Otherwise, False returns. Best used on small Lists.

users = ['AmyP', 'ollie3', 'shoeguy', 'kyliek', 'ollie3', 'stewieboy', 'csealker', 'shoeguy', 'cdriver', 'kyliek'] dups = any(users.count(x) > 1 for x in users)
print(dups)

This code declares a small sampling of Finxter usernames and saves them to users.

Next, any() is called and loops through each element of users checking to see if the element is a duplicate. If found, True is assigned. Otherwise, False is assigned. The result saves to dups and the output displays as follows:

Output

True

---

Method 5: Use List Comprehension to return a List of all Duplicates

This method uses List Comprehension to loop through users, checking for duplicates. If found, the Duplicates are appended to dups.

Here’s an example:

users = ['AmyP', 'ollie3', 'shoeguy', 'kyliek', 'ollie3', 'stewieboy', 'csealker', 'shoeguy', 'cdriver', 'kyliek'] dups = [x for x in users if users.count(x) >= 2]
print(dups)

This code declares a small sampling of Finxter usernames and saves them to users.

Next, List Comprehension extracts and displays duplicate usernames and save them to a List. The duplicate values are output to the terminal

Output

['ollie3', 'shoeguy', 'kyliek', 'ollie3', 'shoeguy', 'kyliek']

---

Summary

These five (5) methods of checking a List for Duplicates should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!

---

Before we learn about the money, let’s get this question out of the way:

What is a Database Engineer?

A database engineer is responsible for providing the data infrastructure of a company or organization. This involves designing, creating, installing, configuring, debugging, optimizing, securing, and managing databases. Database engineers can either work as employees or as freelancers remotely or onsite.

What Does a Database Engineer Do?

As already indicated, a database engineer is responsible for providing the data infrastructure of a company or organization.

In particular, a database engineer has many responsibilities, such as the following 15 most popular activities performed by a database engineer today:

1. Creating a new database system.
2. Finding a database system tailored to the needs of an organization.
3. Designing the data models.
4. Accessing the data with scripting languages including SQL-like syntax.
5. Installing an existing database software system onsite.
6. Configuring a database system.
7. Optimizing a database management system for performance, speed, or reliability.
8. Consulting management regarding data management issues.
9. Keeping databases secure and providing proper access control to users.
10. Monitoring and managing an existing database system to keep it running smoothly.
11. Debugging potential bugs, errors, and security issues detected at runtime.
12. Testing and deploying a database system on a public cloud infrastructure such as AWS.
13. Handling distribution issues in the case of a distributed database management system.
14. Ensuring budget adherence when running on a public cloud and estimating costs for private database solutions.
15. Communicating and negotiating with salespeople (e.g., from Oracle).

These are only some of the most common activities frequently handled by database engineers.

Database Engineer vs Data Engineer

A database engineer is responsible for providing the data infrastructure of a company or organization. This involves designing, creating, installing, configuring, debugging, optimizing, securing, and managing databases. Database engineers can either work as employees or as freelancers remotely or onsite.

A data engineer prepares data to be used in data analytics and operations, essentially providing automated or semi-automated ways for data collection and creating pipelines that connect various data sources to database management systems such as the ones managed by a database engineer.

A data engineer focuses on filling data into a database system whereas a database engineer is focused on providing the database system in the first place. There are intersection points between data engineers and database engineers at the interface between data sources and data management.

Database Engineer vs Database Administrator

Database administrators perform a similar role to database engineers in that they are responsible for setting up, installing, configuring, securing, and managing a database management system.

The focus is more on the technical maintenance of existing systems than the theoretical development of new solutions.

But the lines between those two job descriptions are blurry and often overlap significantly.

Annual Income of Database Engineer (US)

How much does a Database Engineer make per year?

Question: How much does a Database Engineer in the US make per year?

The average annual income of a Database Engineer in the United States is between $72,536 and $135,000, with an average of $103,652 and a statistical median of $106,589 per year.

This data is based on our meta-study of ten (10) salary aggregators sources such as Glassdoor, ZipRecruiter, and PayScale.

Source	Average Income
Glassdoor.com	$91,541
ZipRecruiter.com	$107,844
BuiltIn.com	$120,961
Talent.com	$135,000
Indeed.com	$106,037
PayScale.com	$88,419
SalaryExpert.com	$107,141
Comparably.com	$110,987
Zippia.com	$96,058
Salary.com	$72,536

Note: This is the most comprehensive salary meta-study of database engineer income in the world, to the best of my knowledge!

Let’s have a look at the hourly rate of Database Engineers next!

Hourly Rate

Database Engineers are well-paid on freelancing platforms such as Upwork or Fiverr.

• Related Article: What’s the best freelancing platform?

If you decide to go the route as a freelance Database Developer, you can expect to make between $30 and $130 per hour on Upwork (source). Assuming an annual workload of 2000 hours, you can expect to make between $60,000 and $260,000 per year.

Note: Do you want to create your own thriving coding business online? Feel free to check out our freelance developer course — the world’s #1 best-selling freelance developer course that specifically shows you how to succeed on Upwork and Fiverr!

Industry Demand

But is there enough demand? Let’s have a look at Google trends to find out how interest evolves over time (source):

The interest in database engineering has remained relatively stable over the last two decades.

If you compare the interest with “database administration”, you can see that “database engineering” actually wins in relative importance (source):

Learning Path, Skills, and Education Requirements

Do you want to become a Database Endineer? Here’s a step-by-step learning path I’d propose to get started with Database :

• Step 1: Introduction to Computer Science (~40 hours)
• Step 2: Introduction to Distributed Systems (~20 hours)
• Step 3: Introduction to Databases (Free Video Courses) (~10 hours)
• Step 4: Introduction to MongoDB (~10 hours)
• Step 5: Introduction to SQL (~10 hours)
• Step 6: Introduction to MariaDB (~10 hours)

Here you can already start with the first step — do it now!

You can find many additional computer science courses on the Finxter Computer Science Academy (flatrate model).

But don’t wait too long to acquire practical experience!

Even if you have little skills, it’s best to get started as a freelance developer and learn as you work on real projects for clients — earning income as you learn and gaining motivation through real-world feedback.

Tip: An excellent start to turbo-charge your freelancing career (earning more in less time) is our Finxter Freelancer Course. The goal of the course is to pay for itself!

Related Video

You can find more job descriptions for coders, programmers, and computer scientists in our detailed overview guide:

• Get Your Job in IT [Overview]

Related Income of Professional Developers

The following statistic shows the self-reported income from 9,649 US-based professional developers (source).

The average annual income of professional developers in the US is between $70,000 and $177,500 for various programming languages.

Question: What is your current total compensation (salary, bonuses, and perks, before taxes and deductions)? Please enter a whole number in the box below, without any punctuation. If you are paid hourly, please estimate an equivalent weekly, monthly, or yearly salary. (source)

The following statistic compares the self-reported income from 46,693 professional programmers as conducted by StackOverflow.

The average annual income of professional developers worldwide (US and non-US) is between $33,000 and $95,000 for various programming languages.

Here’s a screenshot of a more detailed overview of each programming language considered in the report:

Here’s what different database professionals earn:

Here’s an overview of different cloud solutions experts:

Here’s what professionals in web frameworks earn:

There are many other interesting frameworks—that pay well!

Look at those tools:

Okay, but what do you need to do to get there? What are the skill requirements and qualifications to make you become a professional developer in the area you desire?

Let’s find out next!

General Qualifications of Professionals

StackOverflow performs an annual survey asking professionals, coders, developers, researchers, and engineers various questions about their background and job satisfaction on their website.

Interestingly, when aggregating the data of the developers’ educational background, a good three quarters have an academic background.

Here’s the question asked by StackOverflow (source):

Which of the following best describes the highest level of formal education that you’ve completed?

However, if you don’t have a formal degree, don’t fear! Many of the respondents with degrees don’t have a degree in their field—so it may not be of much value for their coding careers anyways.

Also, about one out of four don’t have a formal degree and still succeeds in their field! You certainly don’t need a degree if you’re committed to your own success!

Freelancing vs Employment Status

The percentage of freelance developers increases steadily. The fraction of freelance developers has already reached 11.21%!

This indicates that more and more work will be done in a more flexible work environment—and fewer and fewer companies and clients want to hire inflexible talent.

Here are the stats from the StackOverflow developer survey (source):

Do you want to become a professional freelance developer and earn some money on the side or as your primary source of income?

Resource: Check out our freelance developer course—it’s the best freelance developer course in the world with the highest student success rate in the industry!

Other Programming Languages Used by Professional Developers

The StackOverflow developer survey collected 58000 responses about the following question (source):

Which programming, scripting, and markup languages have you done extensive development work in over the past year, and which do you want to work in over the next year?

These are the languages you want to focus on when starting out as a coder:

And don’t worry—if you feel stuck or struggle with a nasty bug. We all go through it. Here’s what SO survey respondents and professional developers do when they’re stuck:

What do you do when you get stuck on a problem? Select all that apply. (source)

Related Tutorials

To get started with some of the fundamentals and industry concepts, feel free to check out these articles:

• Freelance Developer – How to Code From Home and Earn Six Figures [Industry Report]
• How to Become a Python Freelancer—and Earn $1,000 on the Side? [A Step-by-Step Tutorial]
• How Adam Earns $5000 per Month as a Python Freelancer on Upwork [Month 4]
• Database Wikipedia
• Learn Database Google

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

References

[1] The figure was generated using the following code snippet:

import matplotlib.pyplot as plt
import numpy as np
import math data = [91541, 107844, 120961, 135000, 106037, 88419, 107141, 110987, 96058, 72536] labels = ['Glassdoor.com', 'ZipRecruiter.com', 'BuiltIn.com', 'Talent.com', 'Indeed.com', 'PayScale.com', 'SalaryExpert.com', 'Comparably.com', 'Zippia.com', 'Salary.com'] median = np.median(data)
average = np.average(data)
print(median, average)
n = len(data) plt.plot(range(n), [median] * n, color='black', label='Median: $' + str(int(median)))
plt.plot(range(n), [average] * n, '--', color='red', label='Average: $' + str(int(average)))
plt.bar(range(len(data)), data)
plt.xticks(range(len(data)), labels, rotation='vertical', position = (0,0.45), color='white', weight='bold')
plt.ylabel('Average Income ($)')
plt.title('Database Engineer Annual Income - by Finxter')
plt.legend()
plt.show()

Question: Say you have an expression you want to execute using the exec() function. How to store the result of the expression in a Python variable my_result?

Before I show you the solution, let’s quickly recap the exec() function:

Recap Python exec()

Python’s exec() function executes the Python code you pass as a string or executable object argument.

This is called dynamic execution because, in contrast to normal static Python code, you can generate code and execute it at runtime.

This way, you can run programmatically-created Python code.

Related Tutorial: Python’s exec() built-in function

Without further ado, let’s learn how you can store the result of the exec() function in a Python variable:

Method 1: Define Variable in String Representation of Code

You can define a variable inside the exec() function that will then be added to the global namespace. Thus, you can access the defined variable in your code after the termination of the exec() expression!

Here’s how that works in a minimal example:

exec('my_result = 40 + 2')
print(my_result)
# 42

Variable my_result is only defined in the string expression passed into exec(). You can use it in the code like it was part of the original source code.

Method 2: How to Replace Value of a Variable in exec()

To replace a variable defined outside the exec() function call, you can simply do so using an f-string and pass the new value as the right-hand side of the assignment operator within the string representation of the code to be executed.

Here’s a minimal example where we overwrite the old_variable with the new string 'hello finxter' by using the expression:

exec(f'old_variable = "{new_string}"')

old_variable = 'hello world'
new_string = 'hello finxter' # Replace old_old variable with new string 'hello finxter'
exec(f'old_variable = "{new_string}"') print(old_variable)
# hello finxter

Method 3: How to Replace Global Variable in exec()?

If you use the previous approach to replace a global variable defined outside the exec() expression that is called in a local context, the assignment operator will work on a local alias variable.

The global variable to be replaced will remain the same!

This can be seen in the following code example, where the global old_variable is not replaced with the new string 'hello finxter' because of the local scope of the function f that creates its own namespace with its own old_variable that overshadows the global old_variable.

old_variable = 'hello world'
new_string = 'hello finxter' def f(): # Replace old_old variable with new string 'hello finxter' exec(f'old_variable = "{new_string}"') f() print(old_variable)
# hello world

To overwrite a global variable within the exec() function, use the global keyword, followed by an empty space, the name of the global variable, a semicolon ; to close the statement, and the expression to overwrite the global variable like so: exec(f'global old_variable; old_variable = "{new_string}"')

Here’s the full example:

old_variable = 'hello world'
new_string = 'hello finxter' def f(): # Replace old_old variable with new string 'hello finxter' exec(f'global old_variable; old_variable = "{new_string}"') f() print(old_variable)
# hello finxter

Related tutorial:

• How to Use Global Variables In a Python Function?

Method 4: Redirect Standard Output

Okay, let’s do something crazy, shall we?

Let’s redirect the standard output and print directly to the variable!

This method assumes you have a print() statement within the expression passed into the exec() function such as shown in the following three examples:

• exec('print(2+2)')
• exec('print([1, 2, 3, 4] + [5, 6])')
• exec('print(2+2*0)')

To get the output and store it in a variable my_result, you need to temporarily redirect the standard output to the variable.

The following code shows you how to accomplish exactly this:

# Step 1: Import libraries StringIO and sys
from io import StringIO
import sys # Step 2: Keep stdout in temporary variable
tmp = sys.stdout # Step 3: Capture standard output using a StringIO object
my_result = StringIO() # Step 4: Assign Standard Output Stream to StringIO object
sys.stdout = my_result # Step 5: Print to the standard output
expression = 'print(2+2)' # any exec() expression here
exec(expression) # Step 6: Clean up by redirecting stdout to Python shell
sys.stdout = tmp # Step 7: Get and print the string from stdout
print('VARIABLE:', my_result.getvalue())
# VARIABLE: 4

If you need some assistance understanding this whole idea of redirecting the standard output, have a look at our in-depth guide on the Finxter blog.

Related Article: 7 Easy Steps to Redirect Your Standard Output to a Variable (Python)

Note that this approach even works if you don’t have a print() statement in the original exec() expression because you can always artificially add the print() statement around the original expression like so:

• exec('2+2') becomes exec('print(2+2)')
• exec('2+2*0') becomes exec('print(2+2*0)')
• exec('[1, 2, 3] + [4, 5]') becomes exec('print([1, 2, 3] + [4, 5])')

Even if it’s a bit clunky, after applying this short trick, you can redirect the standard output and store the result of any exec() expression in a variable.

Related Article: How to Assign the Result of eval() to a Python Variable?

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Question: Say you have an expression you want to execute using the eval() function. How to store the result of the expression in a Python variable my_result?

Before I show you the solution, let’s quickly recap the eval() function:

Recap Python eval()

Python eval(s) parses the string argument s into a Python expression, runs it, and returns the result of the expression.

Related Tutorial: Python’s eval() built-in function

Without further ado, let’s learn how you can store the result of the eval() function in a Python variable:

Method 1: Simple Assignment

The most straightforward way to store the result of an eval() expression in a Python variable is to assign the whole return value to the variable. For example, the expression my_result = eval('2+2') stores the result 4 in the variable my_result.

Here’s a minimal example:

my_result = eval('2+2')
print(my_result)
# 4

This simple approach may not always work, for example, if you have a print() statement in the expression.

Read on to learn how to fix this issue next and learn something new!

Method 2: Redirect Standard Output

This method assumes you have a print() statement within the expression passed into the eval() function such as shown in the following three examples:

• eval('print(2+2)')
• eval('print([1, 2, 3, 4] + [5, 6])')
• eval('print(2+2*0)')

To get the output and store it in a variable my_result, you need to temporarily redirect the standard output to the variable.

The following code shows you how to accomplish exactly this:

# Step 1: Import libraries StringIO and sys
from io import StringIO
import sys # Step 2: Keep stdout in temporary variable
tmp = sys.stdout # Step 3: Capture standard output using a StringIO object
my_result = StringIO() # Step 4: Assign Standard Output Stream to StringIO object
sys.stdout = my_result # Step 5: Print to the standard output
expression = 'print(2+2)' # any eval() expression here
eval(expression) # Step 6: Clean up by redirecting stdout to Python shell
sys.stdout = tmp # Step 7: Get and print the string from stdout
print('VARIABLE:', my_result.getvalue())
# hello world

If you need some assistance understanding this whole idea of redirecting the standard output, have a look at our in-depth guide on the Finxter blog.

Related Article: 7 Easy Steps to Redirect Your Standard Output to a Variable (Python)

Note that this approach even works if you don’t have a print() statement in the original eval() expression because you can always artificially add the print() statement around the original expression like so:

• eval('2+2') becomes eval('print(2+2)')
• eval('2+2*0') becomes eval('print(2+2*0)')
• eval('[1, 2, 3] + [4, 5]') becomes eval('print([1, 2, 3] + [4, 5])')

Even if it’s a bit clunky, after applying this short trick, you can redirect the standard output and store the result of any eval() expression in a variable.

Method 3: Use exec()

Using only Python’s eval() function, you cannot define variables inside the expression to be evaluated. However, you can define a variable inside the exec() function that will then be added to the global namespace. Thus, you can access the defined variable in your code after termination of the exec() expression!

Here’s how that works in a minimal example:

exec('my_result = 40 + 2')
print(my_result)
# 42

Variable my_result is only defined in the string expression passed into exec(), but you can use it in the code like it was part of the original source code.

Recap exec() vs eval()

Python’s exec() function takes a Python program, as a string or executable object, and runs it. The eval() function evaluates an expression and returns the result of this expression. There are two main differences:

• exec() can execute all Python source code, whereas eval() can only evaluate expressions.
• exec() always returns None, whereas eval() returns the result of the evaluated expression.
• exec() can import modules, whereas eval() cannot.

You can learn more about the exec() function here:

• Related Article: Python exec() — A Hacker’s Guide to A Dangerous Function

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!