Create an account


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[Tut] How to Convert a List of Dicts to a CSV File in Python [4 Ways]

#1
How to Convert a List of Dicts to a CSV File in Python [4 Ways]

<div>
<div class="kk-star-ratings kksr-auto kksr-align-left kksr-valign-top" data-payload="{&quot;align&quot;:&quot;left&quot;,&quot;id&quot;:&quot;557892&quot;,&quot;slug&quot;:&quot;default&quot;,&quot;valign&quot;:&quot;top&quot;,&quot;reference&quot;:&quot;auto&quot;,&quot;class&quot;:&quot;&quot;,&quot;count&quot;:&quot;1&quot;,&quot;readonly&quot;:&quot;&quot;,&quot;score&quot;:&quot;5&quot;,&quot;best&quot;:&quot;5&quot;,&quot;gap&quot;:&quot;5&quot;,&quot;greet&quot;:&quot;Rate this post&quot;,&quot;legend&quot;:&quot;5\/5 - (1 vote)&quot;,&quot;size&quot;:&quot;24&quot;,&quot;width&quot;:&quot;142.5&quot;,&quot;_legend&quot;:&quot;{score}\/{best} - ({count} {votes})&quot;,&quot;font_factor&quot;:&quot;1.25&quot;}">
<div class="kksr-stars">
<div class="kksr-stars-inactive">
<div class="kksr-star" data-star="1" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="2" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="3" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="4" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="5" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
</p></div>
<div class="kksr-stars-active" style="width: 142.5px;">
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
</p></div>
</div>
<div class="kksr-legend" style="font-size: 19.2px;"> 5/5 – (1 vote) </div>
</div>
<p><strong>Problem</strong>: How to convert a <a href="https://blog.finxter.com/python-lists/" data-type="post" data-id="7332" target="_blank" rel="noreferrer noopener">list</a> of <a href="https://blog.finxter.com/python-dictionary/" data-type="post" data-id="5232" target="_blank" rel="noreferrer noopener">dictionaries</a> to a <code>csv</code> file? </p>
<p><strong>Example</strong>: Given is a list of dicts—for example salary data of employees in a given company:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}]</pre>
<p>Your goal is to write the content of the list of dicts into a <a rel="noreferrer noopener" href="https://en.wikipedia.org/wiki/Comma-separated_values" target="_blank">comma-separated-values</a> (CSV) file format. Your out file should look like this:</p>
<p><code><strong>my_file.csv</strong></code>:</p>
<pre class="wp-block-preformatted"><code>Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Ann,Manager,119000</code></pre>
<p><strong>Solution: </strong>There are four simple ways to convert a <a rel="noreferrer noopener" href="https://blog.finxter.com/python-lists/" target="_blank">list</a> of dicts to a CSV file in <a rel="noreferrer noopener" href="https://blog.finxter.com/python-cheat-sheets/" target="_blank">Python</a>.</p>
<ol class="has-global-color-8-background-color has-background">
<li><strong>Pandas</strong>: Import the <a rel="noreferrer noopener" href="https://blog.finxter.com/pandas-cheat-sheets/" target="_blank">pandas library</a>, create a Pandas DataFrame, and write the DataFrame to a file using the DataFrame method <code>DataFrame.to_csv('my_file.csv')</code>.</li>
<li><strong>CSV</strong>: Import the <code>csv</code> <a rel="noreferrer noopener" href="https://docs.python.org/3/library/csv.html" target="_blank">module</a> in Python, create a CSV <code>DictWriter</code> object, and write the list of dicts to the file in using the <code>writerows()</code> method on the writer object.</li>
<li><strong>Python</strong>: Use a pure <a rel="noreferrer noopener" href="https://blog.finxter.com/python-crash-course/" target="_blank">Python</a> implementation that doesn’t require any library by using the Python file I/O functionality.</li>
<li><strong>Reduce Problem</strong>: You can first convert the list of dicts to a list of lists and then use our <a href="https://blog.finxter.com/how-to-convert-a-list-of-lists-to-a-csv-file-in-python/" data-type="URL" data-id="https://blog.finxter.com/how-to-convert-a-list-of-lists-to-a-csv-file-in-python/" target="_blank" rel="noreferrer noopener">related tutorial’s methods</a> to write the list of lists to the CSV.</li>
</ol>
<p>My preference is Method 1 (<strong>Pandas</strong>) because it’s simplest to use, concise, and most robust for different input types (numerical or textual). </p>
<h2>Method 1: Pandas DataFrame to_csv()</h2>
<p>You can convert a list of lists to a<a rel="noreferrer noopener" href="https://pandas.pydata.org/" target="_blank"> Pandas</a> DataFrame that provides you with powerful capabilities such as the <a rel="noreferrer noopener" href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html" target="_blank"><code>to_csv()</code> method</a>. <strong>This is the easiest method and it allows you to avoid importing yet another library</strong> (I use Pandas in many Python projects anyways). </p>
</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 1
import pandas as pd
df = pd.DataFrame(salary)
df.to_csv('my_file.csv', index=False, header=True)
</pre>
<p>Output:</p>
<pre class="wp-block-preformatted"><code>Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000</code></pre>
<div class="wp-block-image">
<figure class="aligncenter size-full"><img loading="lazy" width="572" height="428" src="https://blog.finxter.com/wp-content/uploads/2022/08/image-9.png" alt="" class="wp-image-559621" srcset="https://blog.finxter.com/wp-content/uploads/2022/08/image-9.png 572w, https://blog.finxter.com/wp-content/uplo...00x224.png 300w" sizes="(max-width: 572px) 100vw, 572px" /></figure>
</div>
<p>You create a <a rel="noreferrer noopener" href="https://blog.finxter.com/how-to-create-a-dataframe-in-pandas/" data-type="post" data-id="16764" target="_blank">Pandas DataFrame</a>—which is Python’s default representation of tabular data. Think of it as an Excel spreadsheet within your code (with rows and columns). </p>
<p>The DataFrame is a very powerful data structure that allows you to perform various methods. One of those is the <code><a href="https://blog.finxter.com/pandas-to_csv/" data-type="post" data-id="8027" target="_blank" rel="noreferrer noopener">to_csv()</a></code> method that allows you to write its contents into a CSV file.</p>
<ul>
<li>You set the <code>index</code> argument of the <code>to_csv()</code> method to <code>False</code> because Pandas, per default, adds integer row and column indices 0, 1, 2, …. Again, think of them as the row and column indices in your Excel spreadsheet. You don’t want them to appear in the CSV file so you set the arguments to <code>False</code>.</li>
<li>You set the and <code>header</code> argument to <code>True</code> because you want the dict <a rel="noreferrer noopener" href="https://blog.finxter.com/python-dict-keys-method/" data-type="post" data-id="37711" target="_blank">keys</a> to be used as headers of the CSV. </li>
</ul>
<p>If you want to customize the CSV output, you’ve got a lot of special arguments to play with. Check out <a rel="noreferrer noopener" href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html" target="_blank">this </a>article for a comprehensive list of all arguments.</p>
<figure class="wp-block-image size-full"><a href="https://blog.finxter.com/pandas-cheat-sheets/" target="_blank" rel="noopener"><img loading="lazy" width="741" height="573" src="https://blog.finxter.com/wp-content/uploads/2022/04/image-257.png" alt="" class="wp-image-329079" srcset="https://blog.finxter.com/wp-content/uploads/2022/04/image-257.png 741w, https://blog.finxter.com/wp-content/uplo...00x232.png 300w" sizes="(max-width: 741px) 100vw, 741px" /></a></figure>
<p class="has-base-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f30d.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Related article</strong>:<a rel="noreferrer noopener" href="https://blog.finxter.com/pandas-cheat-sheets/" target="_blank"> Pandas Cheat Sheets to Pin to Your Wall</a></p>
<h2>Method 2: Python CSV Module DictWriter</h2>
<p>You can convert a list of dicts to a CSV file in Python easily—by using the <a rel="noreferrer noopener" href="https://docs.python.org/3/library/csv.html" target="_blank"><code>csv</code></a> library. <strong>This is the most customizable of all four methods.</strong></p>
<p><strong>Here are the six easy steps to convert a list of dicts to a CSV with header row:</strong></p>
<ol class="has-global-color-8-background-color has-background">
<li>Import the CSV library with import csv.</li>
<li>Open the CSV file using the expression <code>open('my_file.csv', 'w', newline='')</code>. You need the newline argument because otherwise, you may see blank lines between the rows in Windows.</li>
<li>Create a <code>csv.DictWriter()</code> object passing the file and the <code>fieldnames</code> argument.</li>
<li>Set the <code>fieldnames</code> argument to the first dictionary’s keys using the expression <code>salary[0].keys()</code>.</li>
<li>Write the header using <code>writer.writeheader()</code></li>
<li>Write the list of dicts using <code>writer.writerows()</code></li>
</ol>
<p>Here’s the full code for copy&amp;paste:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 2
import csv
with open('my_file.csv', 'w', newline='') as f: writer = csv.DictWriter(f, fieldnames=salary[0].keys()) writer.writeheader() writer.writerows(salary)
</pre>
<p>Output file named <code>'my_file.csv'</code> and located in the same folder:</p>
<pre class="wp-block-preformatted"><code>Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000</code> </pre>
<p>You can customize the CSV writer in its constructor (e.g., by modifying the delimiter from a comma <code>','</code> to a whitespace <code>' '</code> character). Have a look at the specification to learn about <a rel="noreferrer noopener" href="https://docs.python.org/3/library/csv.html" target="_blank">advanced modifications</a>. </p>
<h2>Method 3: Pure Python Without External Dependencies</h2>
<p>If you don’t want to import any library and still convert a list of dicts into a CSV file, you can use standard Python implementation as well: it’s not complicated and very efficient.</p>
<p><strong>This method is best if you won’t or cannot use external dependencies.</strong></p>
<ul>
<li>Open the file <code>f</code> in writing mode using the standard <code><a rel="noreferrer noopener" href="https://blog.finxter.com/python-open-function/" data-type="post" data-id="24793" target="_blank">open()</a></code> function.</li>
<li>Write the first dictionary’s keys in the file using the <a rel="noreferrer noopener" href="https://pythononeliners.com/" data-type="URL" data-id="https://pythononeliners.com/" target="_blank">one-liner</a> expression <code>f.write(','.join(salary[0].keys()))</code>.</li>
<li>Iterate over the <a href="https://blog.finxter.com/how-to-create-a-list-of-dictionaries-in-python/" data-type="post" data-id="10576" target="_blank" rel="noreferrer noopener">list of dicts</a> and write the values in the CSV using the expression <code>f.write(','.join(str(x) for x in row.values()))</code>.</li>
</ul>
<p>Here’s the concrete code example:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">salary = [{'Name':'Alice', 'Job':'Data Scientist', 'Salary':122000}, {'Name':'Bob', 'Job':'Engineer', 'Salary':77000}, {'Name':'Carl', 'Job':'Manager', 'Salary':119000}] # Method 3
with open('my_file.csv','w') as f: f.write(','.join(salary[0].keys())) f.write('\n') for row in salary: f.write(','.join(str(x) for x in row.values())) f.write('\n')</pre>
<p>Output:</p>
<pre class="wp-block-preformatted"><code>Name,Job,Salary
Alice,Data Scientist,122000
Bob,Engineer,77000
Carl,Manager,119000</code></pre>
<p>In the code, you first open the file object <code>f</code>. Then you iterate over each row and each element in the row and write the element to the file—one by one. After each element, you place the comma to generate the CSV file format. After each row, you place the newline character <code>'\n'</code>. </p>
<p><strong>Note</strong>: to get rid of the trailing comma, you can check if the element <code>x</code> is the last element in the row within the loop body and skip writing the comma if it is.</p>
<p class="has-base-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f30d.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Finxter Recommended</strong>: <a rel="noreferrer noopener" href="https://blog.finxter.com/subscribe/" target="_blank">Join the Finxter community and download your 8+ Python cheat sheets to refresh your code understanding.</a></p>
<h2>Method 4: Converting to List of Lists First</h2>
<p>A simple approach to convert a list of dicts to a CSV file is to first convert the list of dicts to a <a rel="noreferrer noopener" href="https://blog.finxter.com/python-list-of-lists/" data-type="post" data-id="7890" target="_blank">list of lists</a> and then use the approaches discussed in the following article (code block given).</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">salary = [['Alice', 'Data Scientist', 122000], ['Bob', 'Engineer', 77000], ['Ann', 'Manager', 119000]] # Method 1
import csv
with open('file.csv', 'w', newline='') as f: writer = csv.writer(f) writer.writerows(salary) # Method 2
import pandas as pd
df = pd.DataFrame(salary)
df.to_csv('file2.csv', index=False, header=False) # Method 3
a = [[1, 2, 3], [4, 5, 6], [7, 8, 9]] import numpy as np
a = np.array(a)
np.savetxt('file3.csv', a, delimiter=',') # Method 4
with open('file4.csv','w') as f: for row in salary: for x in row: f.write(str(x) + ',') f.write('\n')
</pre>
<p class="has-base-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f30d.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Related Tutorial</strong>: <a rel="noreferrer noopener" href="https://blog.finxter.com/how-to-convert-a-list-of-lists-to-a-csv-file-in-python/" data-type="URL" data-id="https://blog.finxter.com/how-to-convert-a-list-of-lists-to-a-csv-file-in-python/" target="_blank">How to Convert a List to a CSV File in Python [5 Ways]</a></p>
</p>
<h2>Where to Go From Here?</h2>
<p>Enough theory. Let’s get some practice!</p>
<p>Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation. </p>
<p>To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?</p>
<p><strong>You build high-value coding skills by working on practical coding projects!</strong></p>
<p>Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?</p>
<p class="has-global-color-8-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f680.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> If your answer is <strong><em>YES!</em></strong>, consider becoming a <a rel="noreferrer noopener" href="https://blog.finxter.com/become-python-freelancer-course/" data-type="page" data-id="2072" target="_blank">Python freelance developer</a>! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.</p>
<p>If you just want to learn about the freelancing opportunity, feel free to watch my free webinar <a rel="noreferrer noopener" href="https://blog.finxter.com/webinar-freelancer/" target="_blank">“How to Build Your High-Income Skill Python”</a> and learn how I grew my coding business online and how you can, too—from the comfort of your own home.</p>
<p><a href="https://blog.finxter.com/webinar-freelancer/" target="_blank" rel="noreferrer noopener">Join the free webinar now!</a></p></p>
</div>


https://www.sickgaming.net/blog/2022/08/...on-4-ways/
Reply



Forum Jump:


Users browsing this thread:
1 Guest(s)

Forum software by © MyBB Theme © iAndrew 2016