[Tut] How to Convert a List of Dicts to a CSV File in Python [4 Ways]

[Tut] How to Convert a List of Dicts to a CSV File in Python [4 Ways] - Printable Version

+- Sick Gaming (https://sickgaming.net)
+-- Forum: Programming (https://sickgaming.net/forum-76.html)
+--- Forum: Python (https://sickgaming.net/forum-83.html)
+--- Thread: [Tut] How to Convert a List of Dicts to a CSV File in Python [4 Ways] (/thread-99800.html)

[Tut] How to Convert a List of Dicts to a CSV File in Python [4 Ways] - xSicKxBot - 08-13-2022

How to Convert a List of Dicts to a CSV File in Python [4 Ways]

5/5 – (1 vote)

Problem: How to convert a list of dictionaries to a

Code:
csv

file?

Example: Given is a list of dicts—for example salary data of employees in a given company:

Code:
salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}]

Your goal is to write the content of the list of dicts into a comma-separated-values (CSV) file format. Your out file should look like this:

Code:
my_file.csv

Code:
Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Ann,Manager,119000

Solution: There are four simple ways to convert a list of dicts to a CSV file in Python.

Pandas: Import the pandas library, create a Pandas DataFrame, and write the DataFrame to a file using the DataFrame method
Code:
DataFrame.to_csv('my_file.csv')
.
CSV: Import the
Code:
csv
module in Python, create a CSV
Code:
DictWriter
object, and write the list of dicts to the file in using the
Code:
writerows()
method on the writer object.
Python: Use a pure Python implementation that doesn’t require any library by using the Python file I/O functionality.
Reduce Problem: You can first convert the list of dicts to a list of lists and then use our related tutorial’s methods to write the list of lists to the CSV.

My preference is Method 1 (Pandas) because it’s simplest to use, concise, and most robust for different input types (numerical or textual).

Method 1: Pandas DataFrame to_csv()

You can convert a list of lists to aPandas DataFrame that provides you with powerful capabilities such as the

Code:
to_csv()

method. This is the easiest method and it allows you to avoid importing yet another library (I use Pandas in many Python projects anyways).

Code:
salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 1
import pandas as pd
df = pd.DataFrame(salary)
df.to_csv('my_file.csv', index=False, header=True)

Output:

Code:
Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000

You create a Pandas DataFrame—which is Python’s default representation of tabular data. Think of it as an Excel spreadsheet within your code (with rows and columns).

The DataFrame is a very powerful data structure that allows you to perform various methods. One of those is the

Code:
to_csv()

method that allows you to write its contents into a CSV file.

You set the
Code:
index
argument of the
Code:
to_csv()
method to
Code:
False
because Pandas, per default, adds integer row and column indices 0, 1, 2, …. Again, think of them as the row and column indices in your Excel spreadsheet. You don’t want them to appear in the CSV file so you set the arguments to
Code:
False
.
You set the and
Code:
header
argument to
Code:
True
because you want the dict keys to be used as headers of the CSV.

If you want to customize the CSV output, you’ve got a lot of special arguments to play with. Check out thisarticle for a comprehensive list of all arguments.

Related article:Pandas Cheat Sheets to Pin to Your Wall

Method 2: Python CSV Module DictWriter

You can convert a list of dicts to a CSV file in Python easily—by using the

Code:
csv

library. This is the most customizable of all four methods.

Here are the six easy steps to convert a list of dicts to a CSV with header row:

Import the CSV library with import csv.
Open the CSV file using the expression
Code:
open('my_file.csv', 'w', newline='')
. You need the newline argument because otherwise, you may see blank lines between the rows in Windows.
Create a
Code:
csv.DictWriter()
object passing the file and the
Code:
fieldnames
argument.
Set the
Code:
fieldnames
argument to the first dictionary’s keys using the expression
Code:
salary[0].keys()
.
Write the header using
Code:
writer.writeheader()
Write the list of dicts using
Code:
writer.writerows()

Here’s the full code for copy&paste:

Code:
salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 2
import csv
with open('my_file.csv', 'w', newline='') as f: writer = csv.DictWriter(f, fieldnames=salary[0].keys()) writer.writeheader() writer.writerows(salary)

Output file named

Code:
'my_file.csv'

and located in the same folder:

Code:
Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000

You can customize the CSV writer in its constructor (e.g., by modifying the delimiter from a comma

Code:
','

to a whitespace

Code:
' '

character). Have a look at the specification to learn about advanced modifications.

Method 3: Pure Python Without External Dependencies

If you don’t want to import any library and still convert a list of dicts into a CSV file, you can use standard Python implementation as well: it’s not complicated and very efficient.

This method is best if you won’t or cannot use external dependencies.

Open the file
Code:
f
in writing mode using the standard
Code:
open()
function.
Write the first dictionary’s keys in the file using the one-liner expression
Code:
f.write(','.join(salary[0].keys()))
.
Iterate over the list of dicts and write the values in the CSV using the expression
Code:
f.write(','.join(str(x) for x in row.values()))
.

Here’s the concrete code example:

Code:
salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 3
with open('my_file.csv','w') as f: f.write(','.join(salary[0].keys())) f.write('\n') for row in salary: f.write(','.join(str(x) for x in row.values())) f.write('\n')

Output:

Code:
Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000

In the code, you first open the file object

Code:
f

. Then you iterate over each row and each element in the row and write the element to the file—one by one. After each element, you place the comma to generate the CSV file format. After each row, you place the newline character

Code:
'\n'

.

Note: to get rid of the trailing comma, you can check if the element

Code:
x

is the last element in the row within the loop body and skip writing the comma if it is.

Finxter Recommended: Join the Finxter community and download your 8+ Python cheat sheets to refresh your code understanding.

Method 4: Converting to List of Lists First

A simple approach to convert a list of dicts to a CSV file is to first convert the list of dicts to a list of lists and then use the approaches discussed in the following article (code block given).

Code:
salary = [['Alice', 'Data Scientist', 122000], ['Bob', 'Engineer', 77000], ['Ann', 'Manager', 119000]] # Method 1
import csv
with open('file.csv', 'w', newline='') as f: writer = csv.writer(f) writer.writerows(salary) # Method 2
import pandas as pd
df = pd.DataFrame(salary)
df.to_csv('file2.csv', index=False, header=False) # Method 3
a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] import numpy as np
a = np.array(a)
np.savetxt('file3.csv', a, delimiter=',') # Method 4
with open('file4.csv','w') as f: for row in salary: for x in row: f.write(str(x) + ',') f.write('\n')

Related Tutorial: How to Convert a List to a CSV File in Python [5 Ways]

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!