You can convert a list of lists to a Pandas DataFrame that provides you with powerful capabilities such as the to_csv() method. This is the easiest method and it allows you to avoid importing yet another library (I use Pandas in many Python projects anyways).
You create a Pandas DataFrame—which is Python’s default representation of tabular data. Think of it as an Excel spreadsheet within your code (with rows and columns).
The DataFrame is a very powerful data structure that allows you to perform various methods. One of those is the to_csv() method that allows you to write its contents into a CSV file.
You set the index and header arguments of the to_csv() method to False because Pandas, per default, adds integer row and column indices 0, 1, 2, …. Again, think of them as the row and column indices in your Excel spreadsheet. You don’t want them to appear in the CSV file so you set the arguments to False.
If you want to customize the CSV output, you’ve got a lot of special arguments to play with. Check out this article for a comprehensive list of all arguments.
Feel free to play with alternative methods to convert a list of lists to a CSV file in our interactive code shell. Simply click the “Run” button and find the generated CSV files in the “Files” tab.
Where to Go From Here?
Enough theory, let’s get some practice!
To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?
Practice projects is how you sharpen your saw in coding!
Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?
Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.
Pandas is an open-source Python library that is powerful and flexible for data analysis. If there is something you want to do with data, the chances are it will be possible in pandas. There are a vast number of possibilities within pandas, but most users find themselves using the same methods time after time. In this article, we compiled the best cheat sheets from across the web, which show you these core methods at a glance.
The primary data structure in pandas is the DataFrame used to store two-dimensional data, along with a label for each corresponding column and row. If you are familiar with Excel spreadsheets or SQL databases, you can think of the DataFrame as being the pandas equivalent. If we take a single column from a DataFrame, we have one-dimensional data. In pandas, this is called a Series. DataFrames can be created from scratch in your code, or loaded into Python from some external location, such as a CSV. This is often the first stage in any data analysis task. We can then do any number of things with our DataFrame in Pandas, including removing or editing values, filtering our data, or combining this DataFrame with another DataFrame. Each line of code in these cheat sheets lets you do something different with a DataFrame. Also, if you are coming from an Excel background, you will enjoy the performance pandas has to offer. After you get over the learning curve, you will be even more impressed with the functionality.
Whether you are already familiar with pandas and are looking for a handy reference you can print out, or you have never used pandas and are looking for a resource to help you get a feel for the library- there is a cheat sheet here for you!
This one is from the pandas guys, so it makes sense that this is a comprehensive and inclusive cheat sheet. It covers the vast majority of what most pandas users will ever need to do to a DataFrame. Have you already used pandas for a little while? And are you looking to up your game? This is your cheat sheet! However, if you are newer to pandas and this cheat sheet is a bit overwhelming, don’t worry! You definitely don’t need to understand everything in this cheat sheet to get started. Instead, check out the next cheat sheet on this list.
Dataquest is an online platform that teaches Data Science using interactive coding challenges. I love this cheat sheet they have put together. It has everything the pandas beginner needs to start using pandas right away in a friendly, neat list format. It covers the bare essentials of each stage in the data analysis process:
Importing and exporting your data from an Excel file, CSV, HTML table or SQL database
Cleaning your data of any empty rows, changing data formats to allow for further analysis or renaming columns
Filtering your data or removing anomalous values
Different ways to view the data and see it’s dimensions
Selecting any combination of columns and rows within the DataFrame using loc and iloc
Using the .apply method to apply a formula to a particular column in the DataFrame
Creating summary statistics for columns in the DataFrame. This includes the median, mean and standard deviation
Ok, this isn’t quite a cheat sheet, it’s more of an entire manifesto on the pandas DataFrame! If you have a little time on your hands, this will help you get your head around some of the theory behind DataFrames. It will take you all the way from loading in your trusty CSV from Microsoft Excel to viewing your data in Jupyter and handling the basics. The article finishes off by using the DataFrame to create a histogram and bar chart. For migrating your spreadsheet work from Excel to pandas, this is a fantastic guide. It will teach you how to perform many of the Excel basics in pandas. If you are also looking for how to perform the pandas equivalent of a VLOOKUP in Excel, check out Shane’s article on the merge method.
If you’re more of a visual learner, try this cheat sheet! Many common pandas tasks have intricate, color-coded illustrations showing how the operation works. On page 3, there is a fantastic section called ‘Computation with Series and DataFrames’, which provides an intuitive explanation for how DataFrames work and shows how the index is used to align data when DataFrames are combined and how element-wise operations work in contrast to operations which work on each row or column. At 8 pages long, it’s more of a booklet than a cheat sheet, but it can still make for a great resource!
Much like the other cheat sheets, there is comprehensive coverage of the pandas basic in here. So, that includes filtering, sorting, importing, exploring, and combining DataFrames. However, where this Cheat Sheet differs is that it finishes off with an excellent section on scikit-learn, Python’s machine learning library. In this section, the DataFrame is used to train a machine learning model. This cheat sheet will be perfect for anybody who is already familiar with machine learning and is transitioning from a different technology, such as R.
Data Camp is an online platform that teaches Data Science with videos and coding exercises. They have made cheat sheets on a bunch of the most popular Python libraries, which you can also check out here. This cheat sheet nicely introduces the DataFrame, and then gives a quick overview of the basics. Unfortunately, it doesn’t provide any information on the various ways you can combine DataFrames, but it does all fit on one page and looks great. So, if you are looking to stick a pandas cheat sheet on your bedroom wall and nail home the basics, this one might be for you! The cheat sheet finishes with a small section introducing NaN values, which come from NumPy. These indicate a null value and arise when the indices of two Series don’t quite match up in this case.
While there aren’t any pictures to be found in this sheet, it is an incredibly detailed set of notes on the pandas DataFrame. This cheat shines with its complete section on time series and statistics. There are methods for calculating covariance, correlation, and regression here. So, if you are using pandas for some advanced statistics or any kind of scientific work, this is going to be your cheat sheet.
Where to go from here?
For just automating a few tedious tasks at work, or using pandas to replace your crashing Excel spreadsheet, everything covered in these cheat sheets should be entirely sufficient for your purposes.
If you are looking to use pandas for Data Science, then you are only going to be limited by your knowledge of statistics and probability. This is the area that most people lack when they try to enter this field. I highly recommend checking out Think Stats by Allen B Downey, which provides an introduction to statistics using Python.
For those a little more advanced, looking to do some machine learning, you will want to start taking a look at the scikit-learn library. Data Camp has a great cheat sheet for this. You will also want to pick up a linear algebra textbook to understand the theory of machine learning. For something more practical, perhaps give the famous Kaggle Titanic machine learning competition.
Learning about pandas has many uses, and can be interesting simply for its own sake. However, Python is massively in demand right now, and for that reason, it is a high-income skill. At any given time, there are thousands of people searching for somebody to solve their problems with Python. So, if you are looking to use Python to work as a freelancer, then check out the Finxter Python Freelancer Course. This provides the step by step path to go from nothing to earning a full-time income with Python in a few months, and gives you the tools to become a six-figure developer!
Problem: You’re given a list of lists. Your goal is to convert it into a Pandas Dataframe.
Example: Say, you want to compare salary data of different companies and job descriptions. You’ve obtained the following salary data set as a list of list:
Solution: The straight-forward solution is to use the pandas.DataFrame() constructor that creates a new Dataframe object from different input types such as NumPy arrays or lists.
print(df) ''' 0 1 2
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''
Try It Yourself: Run this code in our interactive Python shell by clicking the “Run” button.
DataFrame.from_records()
An alternative is the pandas.DataFrame.from_records() method that generates the same output:
import pandas as pd salary = [['Company', 'Job', 'Salary($)'], ['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]] df = pd.DataFrame.from_records(salary)
print(df) ''' 0 1 2
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''
Try It Yourself: Run this code in our interactive Python shell by clicking the “Run” button.
Column Names
If you want to add column names to make the output prettier, you can also pass those as a separate argument:
import pandas as pd salary = [['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]] df = pd.DataFrame(salary, columns=['Company', 'Job', 'Salary($)'])
print(df) ''' Company Job Salary($)
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''
Try It Yourself: Run this code in our interactive Python shell by clicking the “Run” button.
If the first list of the list of lists contains the column name, use slicing to separate the first list from the other lists:
import pandas as pd salary = [['Company', 'Job', 'Salary($)'], ['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]] df = pd.DataFrame(salary[1:], columns=salary[0])
print(df) ''' Company Job Salary($)
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''
Summary: To convert a list of lists into a Pandas DataFrame, use the pd.DataFrame() constructor and pass the list of lists as an argument. An optional columns argument can help you structure the output.
Where to Go From Here?
Enough theory, let’s get some practice!
To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?
Practice projects is how you sharpen your saw in coding!
Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?
Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.
Short answer: Convert a list of lists—let’s call it l—to a NumPy array by using the standard np.array(l) function. This works even if the inner lists have a different number of elements.
Solution: Use the np.array(list) function to convert a list of lists into a two-dimensional NumPy array. Here’s the code:
# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5, 6]] # Convert it to a NumPy array
a = np.array(lst) # Print the resulting array
print(a) '''
[[1 2 3] [4 5 6]] '''
Try It Yourself: Here’s the same code in our interactive code interpreter:
Hint: The NumPy method np.array() takes an iterable as input and converts it into a NumPy array.
Convert a List of Lists With Different Number of Elements
Problem: Given a list of lists. The inner lists have a varying number of elements. How to convert them to a NumPy array?
Example: Say, you’ve got the following list of lists:
[[1, 2, 3], [4, 5], [6, 7, 8]]
What are the different approaches to convert this list of lists into a NumPy array?
Solution: There are three different strategies you can use. (source)
(1) Use the standard np.array() function.
# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5], [6, 7, 8]] # Convert it to a NumPy array
a = np.array(lst) # Print the resulting array
print(a) '''
[list([1, 2, 3]) list([4, 5]) list([6, 7, 8])] '''
This creates a NumPy array with three elements—each element is a list type. You can check the type of the output by using the built-in type() function:
>>> type(a)
<class 'numpy.ndarray'>
(2) Make an array of arrays.
# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5], [6, 7, 8]] # Convert it to a NumPy array
a = np.array([np.array(x) for x in lst]) # Print the resulting array
print(a) '''
[array([1, 2, 3]) array([4, 5]) array([6, 7, 8])] '''
This is more logical than the previous version because it creates a NumPy array of 1D NumPy arrays (rather than 1D Python lists).
(3) Make the lists equal in length.
# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5], [6, 7, 8, 9]] # Calculate length of maximal list
n = len(max(lst, key=len)) # Make the lists equal in length
lst_2 = [x + [None]*(n-len(x)) for x in lst]
print(lst_2)
# [[1, 2, 3, None], [4, 5, None, None], [6, 7, 8, 9]] # Convert it to a NumPy array
a = np.array(lst_2) # Print the resulting array
print(a) '''
[[1 2 3 None] [4 5 None None] [6 7 8 9]] '''
You use list comprehension to “pad” None values to each inner list with smaller than maximal length.
Where to Go From Here?
Enough theory, let’s get some practice!
To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?
Practice projects is how you sharpen your saw in coding!
Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?
Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.
For some applications, it’s quite useful to convert a list of lists into a dictionary.
Databases: List of list is table where the inner lists are the database rows and you want to assign each row to a primary key in a new dictionary.
Spreadsheet: List of list is two-dimensional spreadsheet data and you want to assign each row to a key (=row name).
Data Analytics: You’ve got a two-dimensional matrix (=NumPy array) that’s initially represented as a list of list and you want to obtain a dictionary to ease data access.
There are three main ways to convert a list of lists into a dictionary in Python (source):
Dictionary Comprehension
Generator Expression
For Loop
Let’s dive into each of those.
1. Dictionary Comprehension
Problem: Say, you’ve got a list of lists where each list represents a person and consists of three values for the person’s name, age, and hair color. For convenience, you want to create a dictionary where you use a person’s name as a dictionary key and the sublist consisting of the age and the hair color as the dictionary value.
Solution: You can achieve this by using the beautiful (but, surprisingly, little-known) feature of dictionary comprehension in Python.
Explanation: The dictionary comprehension statement consists of the expression x[0]: x[1:] that assigns a person’s name x[0] to the list x[1:] of the person’s age and hair color. Further, it consists of the context for x in persons that iterates over all “data rows”.
Exercise: Can you modify the code in our interactive code shell so that each hair color is used as a key and the name and age are used as the values?
Modify the code and click the “run” button to see if you were right!
2. Generator Expression
A similar way of achieving the same thing is to use a generator expression in combination with the dict() constructor to create the dictionary.
This code snippet is almost identical to the one used in the “list comprehension” part. The only difference is that you use tuples rather than direct mappings to fill the dictionary.
3. For Loop
Of course, there’s no need to get fancy here. You can also use a regular for loop and define the dictionary elements one by one within a simple for loop. Here’s the alternative code:
Again, you map each person’s name to the list consisting of its age and hair color.
Where to Go From Here?
Enough theory, let’s get some practice!
To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?
Practice projects is how you sharpen your saw in coding!
Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?
Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.
Problem: Given a Python tuple with n elements. How to convert it into a list with the same n elements?
Examples:
Convert tuple (1, 2, 3, 4, 5) into list [1, 2, 3, 4, 5].
Convert tuple ('Alice', 'Bob', 'Ann') into list ['Alice', 'Bob', 'Ann'].
Convert tuple (1,) into list [1].
NoteTuple: Tuples are similar to lists—with the difference that you cannot change the tuple values (tuples are immutable) and you use parentheses rather than square brackets.
Solution: Use the built-in Python list() function to convert a list into a tuple. You don’t need to import any external library.
Code: The following code converts the three given tuples into lists.
Try It Yourself: With our interactive code shell, you can try it yourself. As a small exercise, try to convert the empty tuple () into a list and see what happens.
Explanation: You can see that converting a tuple with one element leads to a list with one element. The list() function is the easiest way to convert a tuple into a list. Note that the values in the tuple are not copied—only a new reference to the same element is created:
The graphic also shows how to convert a tuple back to a list by using the tuple() function (that’s also a Python built-in function). Thus, calling list(tuple(lst)) on a list lst will result in a new list with the same elements.
Summing up a list of numbers appears everywhere in coding. Fortunately, Python provides the built-in sum() function to sum over all elements in a Python list—or any other iterable for that matter. (Official Docs)
The syntax is sum(iterable, start=0):
Argument
Description
iterable
Sum over all elements in the iterable. This can be a list, a tuple, a set, or any other data structure that allows you to iterate over the elements. Example: sum([1, 2, 3]) returns 1+2+3=6.
start
(Optional.) The default start value is 0. If you define another start value, the sum of all values in the iterable will be added to this start value. Example: sum([1, 2, 3], 9) returns 9+1+2+3=15.
Exercise: Try to modify the sequence so that the sum is 30 in our interactive Python shell:
Let’s explore some important details regarding the sum() function in Python.
Errors
A number of errors can happen if you use the sum() function in Python.
TypeError: Python will throw a TypeError if you try to sum over elements that are not numerical. Here’s an example:
# Demonstrate possible execeptions
lst = ['Bob', 'Alice', 'Ann'] # WRONG:
s = sum(lst)
If you run this code, you’ll get the following error message:
Traceback (most recent call last): File "C:\Users\xcent\Desktop\code.py", line 3, in <module> s = sum(lst)
TypeError: unsupported operand type(s) for +: 'int' and 'str'
Python tries to perform string concatenation using the default start value of 0 (an integer). Of course, this fails. The solution is simple: sum only over numerical values in the list.
If you try to “hack” Python by using an empty string as start value, you’ll get the following exception:
# Demonstrate possible execeptions
lst = ['Bob', 'Alice', 'Ann'] # WRONG:
s = sum(lst, '')
Output:
Traceback (most recent call last): File "C:\Users\xcent\Desktop\code.py", line 5, in <module> s = sum(lst, '')
TypeError: sum() can't sum strings [use ''.join(seq) instead]
You can get rid of all those errors by summing only over numerical elements in the list.
The time complexity of the sum() function is linear in the number of elements in the iterable (list, tuple, set, etc.). The reason is that you need to go over all elements in the iterable and add them to a sum variable. Thus, you need to “touch” every iterable element once.
Python Sum List of Strings
Problem: How can you sum a list of strings such as ['python', 'is', 'great']? This is called string concatenation.
Solution: Use the join() method of Python strings to concatenate all strings in a list. The sum() function works only on numerical input data.
Code: The following example shows how to “sum” up (i.e., concatenate) all elements in a given list of strings.
# List of strings
lst = ['Bob', 'Alice', 'Ann'] print(''.join(lst))
# BobAliceAnn print(' '.join(lst))
# Bob Alice Ann
Python Sum List of Lists
Problem: How can you sum a list of lists such as [[1, 2], [3, 4], [5, 6]] in Python?
Solution: Use a simple for loop with a helper variable to concatenate all lists.
Code: The following code concatenates all lists into a single list.
# List of lists
lst = [[1, 2], [3, 4], [5, 6]] s = []
for x in lst: s.extend(x)
print(s)
# [1, 2, 3, 4, 5, 6]
Problem: How can you sum over all list elements using a while loop (without sum())?
Solution: Create an aggregation variable and iteratively add another element from the list.
Code: The following code shows how to sum up all numerical values in a Python list without using the sum() function.
# list of integers
lst = [1, 2, 3, 4, 5] # aggregation variable
s = 0 # index variable
i = 0 # sum everything up
while i<len(lst): s += lst[i] i += 1 # print the result
print(s)
# 15
This is not the prettiest way but it’s readable and it works (and, you didn’t want to use the sum() function, right?).
Python Sum List For Loop
Problem: How can you sum over all list elements using a for loop (without sum())?
Solution: Create an aggregation variable and iteratively add another element from the list.
Code: The following code shows how to sum up all numerical values in a Python list without using the sum() function.
# list of integers
lst = [1, 2, 3, 4, 5] # aggregation variable
s = 0 # sum everything up
for x in lst: s += x # print the result
print(s)
# 15
This is a bit more readable than the previous version with the while loop because you don’t have to keep track about the current index.
Python Sum List with List Comprehension
List comprehension is a powerful Python features that allows you to create a new list based on an existing iterable. Can you sum up all values in a list using only list comprehension?
The answer is no. Why? Because list comprehension exists to create a new list. Summing up values is not about creating a new list. You want to get rid of the list and aggregate all values in the list into a single numerical “sum”.
Python Sum List of Tuples Element Wise
Problem: How to sum up a list of tuples, element-wise?
Example: Say, you’ve got list [(1, 1), (2, 0), (0, 3)] and you want to sum up the first and the second tuple values to obtain the “summed tuple” (1+2+0, 1+0+3)=(3, 4).
Solution: Unpack the tuples into the zip function to combine the first and second tuple values. Then, sum up those values separately. Here’s the code:
# list of tuples
lst = [(1, 1), (2, 0), (0, 3)] # aggregate first and second tuple values
zipped = list(zip(*lst))
# result: [(1, 2, 0), (1, 0, 3)] # calculate sum of first and second tuple values
res = (sum(zipped[0]), sum(zipped[1])) # print result to the shell
print(res)
# result: (3, 4)
Need a refresher of the zip() function and unpacking? Check out these articles on the Finxter blog:
Problem: Given a list. Sum up a slice of the original list between the start and the step indices (and assuming the given step size as well).
Example: Given is list [3, 4, 5, 6, 7, 8, 9, 10]. Sum up the slice lst[2:5:2] with start=2, stop=5, and step=2.
Solution: Use slicing to access the list. Then, apply the sum() function on the result.
Code: The following code computes the sum of a given slice.
# create the list
lst = [3, 4, 5, 6, 7, 8, 9, 10] # create the slice
slc = lst[2:5:2] # calculate the sum
s = sum(slc) # print the result
print(s)
# 12 (=5+7)
Let’s examine an interesting problem: to sum up conditionally!
Python Sum List Condition
Problem: Given is a list. How to sum over all values that meet a certain condition?
Example: Say, you’ve got the list lst = [5, 8, 12, 2, 1, 3] and you want to sum over all values that are larger than 4.
Solution: Use list comprehension to filter the list so that only the elements that satisfy the condition remain. Then, use the sum() function to sum over the remaining values.
Code: The following code sums over all values that satisfy a certain condition (e.g., x>4).
# create the list
lst = [5, 8, 12, 2, 1, 3] # filter the list
filtered = [x for x in lst if x>4]
# remaining list: [5, 8, 12] # sum over the filtered list
s = sum(filtered) # print everything
print(s)
# 25
Problem: Given is a list of numerical values that may contain some values None. How to sum over all values that are not the value None?
Example: Say, you’ve got the list lst = [5, None, None, 8, 12, None, 2, 1, None, 3] and you want to sum over all values that are not None.
Solution: Use list comprehension to filter the list so that only the elements that satisfy the condition remain (that are different from None). You see, that’s a special case of the previous paragraph that checks for a general condition. Then, use the sum() function to sum over the remaining values.
Code: The following code sums over all values that are not None.
# create the list
lst = [5, None, None, 8, 12, None, 2, 1, None, 3] # filter the list
filtered = [x for x in lst if x!=None]
# remaining list: [5, 8, 12, 2, 1, 3] # sum over the filtered list
s = sum(filtered) # print everything
print(s)
# 31
A similar thing can be done with the value Nan that can disturb your result if you aren’t careful.
Python Sum List Ignore Nan
Problem: Given is a list of numerical values that may contain some values nan (=”not a number”). How to sum over all values that are not the value nan?
Example: Say, you’ve got the list lst = [1, 2, 3, float("nan"), float("nan"), 4] and you want to sum over all values that are not nan.
Solution: Use list comprehension to filter the list so that only the elements that satisfy the condition remain (that are different from nan). You see, that’s a special case of the previous paragraph that checks for a general condition. Then, use the sum() function to sum over the remaining values.
Code: The following code sums over all values that are not nan.
# for checking isnan(x)
import math # create the list
lst = [1, 2, 3, float("nan"), float("nan"), 4] # forget to ignore 'nan'
print(sum(lst))
# nan # ignore 'nan'
print(sum([x for x in lst if not math.isnan(x)]))
# 10
Phew! Quite some stuff. Thanks for reading through this whole article! I hope you’ve learned something out of this tutorial and remain with the following recommendation:
Where to Go From Here?
Enough theory, let’s get some practice!
To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?
Practice projects is how you sharpen your saw in coding!
Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?
Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.
How to print a Python list in a beautiful and fully customizable way?
This article shows you six effective ways of doing it. By studying these alternatives, you’ll not only learn how to print lists in Python, you’ll become a better coder overall.
If you just want to know the best way to print a list in Python, here’s the short answer:
Pass a list as an input to the print() function in Python.
Use the asterisk operator * in front of the list to “unpack” the list into the print function.
Use the sep argument to define how to separate two list elements visually.
Here’s the code:
# Create the Python List
lst = [1, 2, 3, 4, 5] # Use three underscores as separator
print(*lst, sep='___')
# 1___2___3___4___5 # Use an arrow as separator
print(*lst, sep='-->')
# 1-->2-->3-->4-->5
Try It Yourself in Our Interactive Code Shell:
This is the best and most Pythonic way to print a Python list. If you still want to learn about alternatives—and improve your Python skills in the process of doing so—keep reading!
Method: Use Default print() Statement
The default print() statement converts the list into a string representation that encloses the list elements in the square brackets [ and ], and separates two subsequent elements with the comma and an empty space a, b. This is the standard list representation.
lst = [1, 2, 3, 4, 5]
print(lst)
The output is the following:
[1, 2, 3, 4, 5]
Advantages
Disadvantages
Easy to read and write
Non-customizable
Fast
Concise
Try It Yourself in Our Interactive Code Shell:
The next method overcomes the main disadvantage of being not very customizable.
Method: Iterate In a For Loop
If you want full control about the output of each list element, you can use the straightforward approach of using a for loop to iterate over each element x in the list. You can then decide for yourself how to print each element.
# Create the Python List
lst = [1, 2, 3, 4, 5] # Iterate over each element x
# in the list and customize printing
for x in lst: print('Element: ' + x)
If you’d rather print all elements in a single line, separated by three whitespace characters, you can do so by defining the end argument of the print() function that defines which character is added after each element that was printed to the shell (default: new-line character \n):
# Create the Python List
lst = [1, 2, 3, 4, 5] # Iterate over each element x
# in the list and customize printing
for x in lst: # Use the end argument to define # what to print after each element print(str(x), end=' ')
The output is:
1 2 3 4 5
You see that the end argument overwrites the default behavior of printing a new-line character at the end of each element. Instead, each two elements are separated by three empty spaces.
Advantages
Disadvantages
Fully customizable
Relatively slow
Simple
Less concise
Try It Yourself in Our Interactive Code Shell:
Let’s overcome the disadvantage of the for loop of being less concise!
Method: Unpacking With Separator Argument
The print() function works with an iterable as input. You can use the asterisk operator * in front of the list to “unpack” the list into the print function. Now, you can use the sep argument of the print() function to define how to separate two elements of the iterable.
# Create the Python List
lst = [1, 2, 3, 4, 5] # Use three underscores as separator
print(*lst, sep='___')
# 1___2___3___4___5 # Use an arrow as separator
print(*lst, sep='-->')
# 1-->2-->3-->4-->5
The sep argument allows you to define precisely what to put between each pair of elements in an iterable. This allows you full customization and keeps the code lean and concise.
Advantages
Disadvantages
Fully customizable
Harder to read for beginners
Fast
Concise
Try It Yourself in Our Interactive Code Shell:
This is the best and most Pythonic way to print a Python list. If you still want to learn about alternatives, keep reading.
Method: Use the string.join() Method
The string.join(iterable) method joins together all elements in the iterable, using the string as a separator between two elements. Thus, it works exactly like the sep argument of the print() function.
# Create the Python List
lst = ['1', '2', '3', '4', '5'] # Use three underscores as separator
print('___'.join(lst))
# 1___2___3___4___5 # Use arrow as separator
print('-->'.join(lst))
# 1-->2-->3-->4-->5
Note that you can only use this methods if the list elements are already strings. If they are integers, joining them together doesn’t work and Python throws an error:
TypeError: sequence item 0: expected str instance, int found
The string.join(iterable) method joins together all elements in the iterable, using the string as a separator between two elements. But it expects that all elements in the iterable are already strings. If they aren’t, you need to convert them first. To achieve this, you can use the built-in map() method in Python 3.x.
# Create the Python List
lst = [1, 2, 3, 4, 5] # Use three underscores as separator
print('___'.join(map(str, lst)))
# 1___2___3___4___5 # Use arrow as separator
print('-->'.join(map(str, lst)))
# 1-->2-->3-->4-->5
The map(str, lst) method applies the function str(x) to each element x in the list. In other words, it converts each integer element to a string. An alternative way without the map(str, lst) function would be list comprehension[str(x) for x in lst] that results in the same output.
Advantages
Disadvantages
Fully customizable
Harder to read for beginners
Concise
Slow
Works for all data types
Try It Yourself in Our Interactive Code Shell:
So, let’s finish this up!
Where to Go From Here?
Enough theory, let’s get some practice!
To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?
Practice projects is how you sharpen your saw in coding!
Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?
Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.
Note the dot character inside the character set. As you may know, the dot metacharacter matches an arbitrary character if it is used outside a character set.
The answer is that the dot inside the character set matches the dot symbol—and not an arbitrary character. The reason is that the character set removes the special meaning of the dot symbol.
If you’ve already learned how to make basic 3d plots in maptlotlib and want to take them to the next level, then look no further. In this article, I’ll teach you how to create the two most common 3D plots (surface and wireframe plots) and a step-by-step method you can use to create any shape you can imagine.
In addition to import matplotlib.pyplot as plt and calling plt.show(), to create a 3D plot in matplotlib, you need to:
Import the Axes3D object
Initialize your Figure and Axes3D object
Get some 3D data
Plot it using Axes notation
Here’s a wireframe plot:
# Standard import
import matplotlib.pyplot as plt # Import 3D Axes
from mpl_toolkits.mplot3d import axes3d # Set up Figure and 3D Axes
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d') # Get some data
X, Y, Z = axes3d.get_test_data(0.1) # Plot using Axes notation
ax.plot_wireframe(X, Y, Z)
plt.show()
Try It Yourself on our interactive Python shell (and check out the file 'plot.png'):
Changing the plot call to ax.plot_surface(X, Y, Z) gives
Great! You’ve just created your first 3D wireframe and surface plots. Don’t worry if that was a bit fast; let’s dive into a more detailed example.
But first, note that your plots may look different to mine because I use the seaborn style throughout. You can set this by installing the seaborn library and calling the set function at the top of your code.
import seaborn as sns; sns.set()
Matplotlib 3D Plot Example
The four steps needed to create advanced 3D plots are the same as those needed to create basic ones. If you don’t understand those steps, check out my article on how to make basic 3D plots first.
The most difficult part of creating surface and wireframe plots is step 3: getting 3D data. Matplotlib actually includes a helper function axes3d.get_test_data() to generate some data for you. It accepts a float and, for best results, choose a value between 0 and 1. It always produces the same plot, but different floats give you different sized data and thus impact how detailed the plot is.
However, the best way to learn 3D plotting is to create custom plots.
At the end of step 3, you want to have three numpy arraysX, Y and Z, which you will pass to ax.plot_wireframe() or ax.plot_surface(). You can break step 3 down into four steps:
Define the x-axis and y-axis limits
Create a grid of XY-points (to get X and Y)
Define a z-function
Apply the z-function to X and Y (to get Z)
In matplotlib, the z-axis is vertical by default. So, the ‘bottom’ of the Axes3D object is a grid of XY points. For surface or wireframe plots, each pair of XY points has a corresponding Z value. So, we can think of surface/wireframe plots as the result of applying some z-function to every XY-pair on the ‘bottom’ of the Axes3D object.
Since there are infinitely many numbers on the XY-plane, it is not possible to map every one to a Z-value. You just need an amount large enough to deceive humans – anything above 50 pairs usually works well.
To create your XY-points, you first need to define the x-axis and y-axis limits. Let’s say you want X-values ranging from -5 to +5 and Y-values from -2 to +2. You can create an array of numbers for each of these using the np.linspace() function. For reasons that will become clear later, I will make x have 100 points, and y have 70.
x = np.linspace(-5, 5, num=100)
y = np.linspace(-2, 2, num=70)
Both x and y are 1D arrays containing num equally spaced floats in the ranges [-5, 5] and [-2, 2] respectively.
Since the XY-plane is a 2D object, you now need to create a rectangular grid of all xy-pairs. To do this, use the numpy function np.meshgrid(). It takes n 1D arrays and turns them into an N-dimensional grid. In this case, it takes two 1D arrays and turns them into a 2D grid.
X, Y = np.meshgrid(x, y)
Now you’ve created X and Y, so let’s inspect them.
print(f'Type of X: {type(X)}')
print(f'Shape of X: {X.shape}\n')
print(f'Type of Y: {type(Y)}')
print(f'Shape of Y: {Y.shape}')
Type of X: <class 'numpy.ndarray'>
Shape of X: (70, 100) Type of Y: <class 'numpy.ndarray'>
Shape of Y: (70, 100)
Both X and Y are numpy arrays of the same shape: (70, 100). This corresponds to the size of y and x respectively. As you would expect, the size of y dictates the height of the array, i.e., the number of rows and the size of x dictates the width, i.e., the number of columns.
Note that I used lowercase x and y for the 1D arrays and uppercase X and Y for the 2D arrays. This is standard practice when making 3D plots, and I use it throughout the article.
Now you’ve created your grid of points; it’s time to define a function to apply to them all. Since this function outputs z-values, I call it a z-function. Common z-functions contain np.sin() and np.cos() because they create repeating, cyclical patterns that look interesting when plotted in 3D. Additionally, z-functions usually combine both X and Y variables as 3D plots look at how all the variables interact.
# Define z-function with 2 arguments: x and y
def z_func(x, y): return np.sin(np.cos(x) + y) # Apply to X and Y
Z = z_func(X, Y)
Here I defined a z-function that accepts 2 variables – x and y – and is a combination of np.sin() and np.cos() functions. Then I applied it to X and Y to get the Z array. Thanks to numpy broadcasting, python applies the z-function to every XY pair almost instantly and saves you from having to write a wildly inefficient for loop.
Note that Z is the same shape and type as both X and Y.
print(f'Type of Z: {type(Z)}')
print(f'Shape of Z: {Z.shape}')
Type of Z: <class 'numpy.ndarray'>
Shape of Z: (70, 100)
Now that you have got your data, all that is left to do is make the plots. Let’s put all the above code together:
# Set up Figure and 3D Axes
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d') # Create x and y 1D arrays
x = np.linspace(-5, 5, num=100)
y = np.linspace(-2, 2, num=70) # Create X and Y 2D arrays
X, Y = np.meshgrid(x, y) # Define Z-function
def z_func(x, y): return np.sin(np.cos(x) + y) # Create Z 2D array
Z = z_func(X, Y) # Plot using Axes notation
ax.plot_wireframe(X, Y, Z)
# Set axes lables
ax.set(xlabel='x', ylabel='y', zlabel='z')
plt.show()
Great, I found the above plot by playing around with different z-functions and think it looks pretty cool! Z-functions containing np.log(), np.exp(), np.sin(), np.cos() and combinations of x and y usually lead to interesting plots – I encourage you to experiment yourself.
Now I’ll create 3 different z-functions with the same X and Y as before and create a subplot of them so you can see the differences.
# Set up Figure and Axes
fig, axes = plt.subplots(1, 3, subplot_kw=dict(projection='3d'), figsize=plt.figaspect(1/3)) # Create 3 z-functions
def z_1(x, y): return np.exp(np.cos(x)*y)
def z_2(x, y): return np.log(x**2 + y**4)
def z_3(x, y): return np.sin(x * y) # Create 3 Z arrays Z_arrays = [z_1(X, Y), z_2(X, Y), z_3(X, Y)]
# Titles for the plots
z_func_names = ['np.exp(np.cos(x)*y)', 'np.log(x**2 + y**4)', 'np.sin(x * y)'] # Plot all 3 wireframes
for Z_array, z_name, ax in zip(Z_arrays, z_func_names, axes): ax.plot_wireframe(X, Y, Z_array) ax.set(title=z_name)
plt.show()
I think all of these images demonstrate the power of 3D plotting, and I hope they have encouraged you to create your own.
Now you know how to create any surface or wireframe plot with your data. But so far, you have only used the default settings. Let’s modify them using the available keyword arguments.
Matplotlib 3D Plot Wireframe
To make a wireframe plot, call ax.plot_wireframe(X, Y, Z). These plots give you an overview of the surface. Plus, you can see through them to more easily identify peaks and troughs that may otherwise be hidden.
A wireframe plot works by only plotting a sample of the data passed to it. You can modify how large the samples are with 4 keyword arguments:
rstride and cstride, or
rcount and ccount
The r and c stand for row and column respectively. The difference between them is similar to the difference between np.arange() and np.linspace().
The stride arguments default to 1 and set the step sizes between each sampled point. A stride of 1 means that every value is chosen, and a stride of 10 means that every 10th value is chosen. In this way, it is similar to np.arange() where you select the step size. A larger stride means fewer values are chosen, so your plot renders faster and is less detailed.
The count arguments default to 50 and set the number of (equally spaced) rows/columns sampled. A count of 1 means you use 1 row/column, and a count of 100 means you use 100. In this way, it is similar to np.linspace() where you select the total number of values with the num keyword argument. A larger count means more values are chosen, so your plot renders slower and is more detailed.
The matplotlib docs say that you should use the count arguments. However, both are still available, and it doesn’t look like the stride arguments will be depreciated any time soon. Note, though, that you cannot use both count and stride, and if you try to do so, it’s a ValueError.
By setting any of the keyword arguments to 0, you do not sample data along that axis. The result is then a 3D line plot rather than a wireframe.
To demonstrate the differences between different counts or strides, I’ll create a subplot with the same X, Y and Z arrays as the first example but with different stride and count values.
fig, axes = plt.subplots(nrows=1, ncols=3, subplot_kw=dict(projection='3d'), figsize=plt.figaspect(1/3))
# Same as first example
x = np.linspace(-5, 5, num=100)
y = np.linspace(-2, 2, num=70)
X, Y = np.meshgrid(x, y) def z_func(x, y): return np.sin(np.cos(x) + y)
Z = z_func(X, Y) # Define different strides
strides = [1, 5, 10] for stride, ax in zip(strides, axes.flat): ax.plot_wireframe(X, Y, Z, rstride=stride, cstride=stride) ax.set(title=f'stride={stride}') plt.show()
Here you can see that a larger stride produces a less detailed wireframe plot. Note that stride=1 is the default and is incredibly detailed for a plot that is supposed to give a general overview of the data.
fig, axes = plt.subplots(nrows=1, ncols=3, subplot_kw=dict(projection='3d'), figsize=plt.figaspect(1/3)) counts = [5, 20, 50] for count, ax in zip(counts, axes.flat): # Use same data as the above plots ax.plot_wireframe(X, Y, Z, rcount=count, ccount=count) ax.set(title=f'count={count}') plt.show()
Here you can see that a larger count produces a more detailed wireframe plot. Again note that the default count=50 produces a very detailed plot.
To make a surface plot call ax.plot_surface(X, Y, Z). Surface plots are the same as wireframe plots, except that spaces in between the lines are colored. Plus, there are some additional keyword arguments you can use, which can add a ton of value to the plot.
First, let’s make the same plots as above with the default surface plot settings and different rcount and ccount values.
fig, axes = plt.subplots(nrows=1, ncols=3, subplot_kw=dict(projection='3d'), figsize=plt.figaspect(1/3)) counts = [5, 20, 50] for count, ax in zip(counts, axes.flat): # Use same data as the above plots surf = ax.plot_surface(X, Y, Z, rcount=count, ccount=count) ax.set(title=f'count={count}') plt.show()
In contrast to wireframe plots, the space in between each line is filled with the color blue. Note that the plots get whiter as the count gets larger. This is because the lines are white, and, as the count increases, there are more lines on each plot. You can modify this by setting the linewidth or lw argument to a smaller number such, as 0.1 or even 0.
Much nicer! Now you can see the color of the plot rather than the color of the lines. It is possible to almost completely remove the lines by setting antialiased=False.
Antialiasing removes noise from data and smooths out images. By turning it off, the surface is less smooth, and so you can’t see the lines as easily.
Now the surface is slightly less smooth, and so you can’t see the lines.
Maptlotlib 3D Surface Plot Cmap
Arguably the most crucial keyword argument for surface plots is cmap which sets the colormap. When you look at a surface plot from different angles, having a colormap helps you understand which parts of the surface are where. Usually, you want high points to be one color (e.g., orange) and low points to be another (e.g., black). Having two distinct colors is especially helpful if you look at a plot from different angles (which I will show you how to do in a moment).
The colormap copper maps large z-values to orange and smaller ones to black.
Now I’ll use 3 different and commonly used colormaps for the same plot to give you an idea of how color can help and (massively) hinder your plots.
fig, axes = plt.subplots(nrows=1, ncols=3, subplot_kw=dict(projection='3d'), figsize=plt.figaspect(1/3)) cmaps = ['copper', 'coolwarm', 'jet'] for cmap, ax in zip(cmaps, axes): ax.plot_surface(X, Y, Z, lw=0, cmap=cmap) ax.set(title=f'{cmap}')
plt.show()
The coolwarm colormap works well if you want to highlight extremely high and extremely low points. This non-technical paper defines a colormap similar to coolwarm and argues it should be the default cmap for all data science work.
The jet colormap is well known and is a terrible choice for all of your plotting needs. It contains so many colors that it is hard for a human to know which corresponds to high, low, or middle points. I included it as an example here but urge you to never use it in any of your plots.
Now let’s look at how the count and stride arguments can affect the color of your surface plots. For brevity, I will just make one subplot demonstrating different rccount and ccount sizes and leave the reader to experiment with rstride and cstride.
fig, axes = plt.subplots(nrows=1, ncols=3, subplot_kw=dict(projection='3d'), figsize=plt.figaspect(1/3)) counts = [5, 20, 50] for count, ax in zip(counts, axes.flat): # Use same data as the above plots ax.plot_surface(X, Y, Z, rcount=count, ccount=count, cmap='copper', lw=0) ax.set(title=f'count={count}')
plt.show()
If you pass a lower value to the count keyword arguments, there are fewer areas that can be colored. As such, the colors have much more distinct bands when you set the count keyword arguments to smaller values. The change in color is much smoother in the plots that have large count arguments.
Matplotlib 3D Plot Colorbar
Adding a colorbar to a 3D surface plot is the same as adding them to other plots.
The simplest method is to save the output of ax.plot_surface() in a variable such as surf and pass that variable to plt.colorbar().
Here’s an example using the three different colormaps from before.
fig, axes = plt.subplots(nrows=1, ncols=3, subplot_kw=dict(projection='3d'), figsize=plt.figaspect(1/3)) cmaps = ['copper', 'coolwarm', 'jet'] for cmap, ax in zip(cmaps, axes): # Save surface in a variable: surf surf = ax.plot_surface(X, Y, Z, lw=0, cmap=cmap) # Plot colorbar on the correct Axes: ax fig.colorbar(surf, ax=ax) ax.set(title=f'{cmap}')
plt.show()
It’s essential to provide a colorbar for any colored plots you create, especially if you use different colormaps. Remember that colorbar() is a Figure (not Axes) method, and you must use the ax keyword argument to place it on the correct Axes.
Now, let’s see why colormaps are so crucial by rotating the surface plots and viewing them from different angles.
Matplotlib 3D Plot View_Init
One way to rotate your plots is by using the magic command %matplotlib notebook at the top of your Jupyter notebooks. If you do this, all your plots appear in interactive windows. If instead, you use %matplotlib inline (the default settings), you have to rotate your plots using code.
Two attributes that control the rotation of a 3D plot: ax.elev and ax.azim, which represent the elevation and azimuthal angles of the plot, respectively.
The elevation is the angle above the XY-plane and the azimuth (don’t worry, I hadn’t heard of it before either) is the counter-clockwise rotation about the z-axis. Note that they are properties of the Axes3D object and so you can happily create subplots where each has a different angle.
Let’s find the default values.
fig = plt.figure()
ax = plt.axes(projection='3d') print(f'The default elevation angle is: {ax.elev}')
print(f'The default azimuth angle is: {ax.azim}')
The default elevation angle is: 30
The default azimuth angle is: -60
You can see that the defaults are 30 and -60 degrees for the elevation and azimuth, respectively.
You can set them to any float you want, and there are two ways to do it:
Reassign the ax.azim and ax.elev attributes, or
Use the ax.view_init(elev, azim) method
Here’s an example with method 1.
# Same as usual
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.plot_surface(X, Y, Z, lw=0, cmap='copper')
# Set axis labels so you know what you are looking at
ax.set(xlabel='x', ylabel='y', zlabel='z') # Reassign rotation angles to 0
ax.azim, ax.elev = 0, 0
plt.show()
Here I set both angles to 0, and you can see the y-axis at the front, the x-axis at the side, and the z-axis as vertical.
I’ll now create the same plot using the ax.view_init() method, which accepts two floats: the elevation and azimuth.
# Same as usual
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.plot_surface(X, Y, Z, lw=0, cmap='copper')
# Set axis labels so you know what you are looking at
ax.set(xlabel='x', ylabel='y', zlabel='z') # Reassign rotation angles to 0
ax.view_init(elev=0, azim=0)
plt.show()
Excellent! This plot looks identical to the one above, but I used the ax.view_init() method instead. If you just want to change one of the angles, only pass one of the keyword arguments.
# Same as usual
fig = plt.figure()
ax = plt.axes(projection='3d')
ax.plot_surface(X, Y, Z, lw=0, cmap='copper')
# Set axis labels so you know what you are looking at
ax.set(xlabel='x', ylabel='y', zlabel='z') # Set elevation to 90 degrees
ax.view_init(elev=90)
plt.show()
Here I set the elevation to 90 degrees but left the azimuth with its default value. This demonstrates one more reason why colormaps are important: you can infer the shape of the surface from the color (black is low, light is high).
Conclusion
Now you know how to create the most critical 3D plots: wireframe and surface plots.
You’ve learned how to create custom 3D plot datasets using np.linspace(), np.meshgrid() and z-functions. Plus, you can create them with varying degrees of accuracy by modifying the count and stride keyword arguments.
You can make surface plots of any color and colormap and modify them so that the color of the lines doesn’t take over the plot. Finally, you can rotate them by setting the ax.azim or ax.elev attributes to a float of your choice and even use the ax.view_init() method to do the same thing.
Congratulations on mastering these plots! Creating other advanced ones such as contour, tri-surface, and quiver plots for you will be easy. You know all the high-level skills; you just need to go out there and practice.
Where To Go From Here?
Do you wish you could be a programmer full-time but don’t know how to start?
Check out the pure value-packed webinar where Chris – creator of Finxter.com – teaches you to become a Python freelancer in 60 days or your money back!
It doesn’t matter if you’re a Python novice or Python pro. If you are not making six figures/year with Python right now, you will learn something from this webinar.
These are proven, no-BS methods that get you results fast.
This webinar won’t be online forever. Click the link below before the seats fill up and learn how to become a Python freelancer, guaranteed.
