+- Sick Gaming (https://www.sickgaming.net)
+-- Forum: Programming (https://www.sickgaming.net/forum-76.html)
+--- Forum: Python (https://www.sickgaming.net/forum-83.html)
+--- Thread: [Tut] How to Convert a List of Objects to a CSV File in Python [5 Ways] (/thread-99753.html)
[Tut] How to Convert a List of Objects to a CSV File in Python [5 Ways] - xSicKxBot - 07-30-2022
How to Convert a List of Objects to a CSV File in Python [5 Ways]
<div>
<div class="kk-star-ratings kksr-auto kksr-align-left kksr-valign-top" data-payload="{"align":"left","id":"510694","slug":"default","valign":"top","reference":"auto","class":"","count":"1","readonly":"","score":"5","best":"5","gap":"5","greet":"Rate this post","legend":"5\/5 - (1 vote)","size":"24","width":"142.5","_legend":"{score}\/{best} - ({count} {votes})","font_factor":"1.25"}">
<div class="kksr-stars">
<div class="kksr-stars-inactive">
<div class="kksr-star" data-star="1" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="2" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="3" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="4" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" data-star="5" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
</p></div>
<div class="kksr-stars-active" style="width: 142.5px;">
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
<div class="kksr-star" style="padding-right: 5px">
<div class="kksr-icon" style="width: 24px; height: 24px;"></div>
</p></div>
</p></div>
</div>
<div class="kksr-legend" style="font-size: 19.2px;"> 5/5 – (1 vote) </div>
</div>
<p class="has-global-color-8-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f4ac.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Question</strong>: How to convert a list of custom objects to a <code>csv</code> file? </p>
<p><strong>Example</strong>: Given is a list of custom objects of, say, type <code>Employee</code> that holds the name, job description, and income like so:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">salary = [Employee('Alice', 'Data Scientist', 122000), Employee('Bob', 'Engineer', 77000), Employee('Ann', 'Manager', 119000)]</pre>
<p>Your goal is to write the content of the list of objects into a <a rel="noreferrer noopener" href="https://en.wikipedia.org/wiki/Comma-separated_values" target="_blank">comma-separated-values</a> (CSV) file format. </p>
<p>Your output file should look like this:</p>
<pre class="wp-block-preformatted"><code><strong><em># my_file.csv</em></strong>
Alice,Data Scientist,122000
Bob,Engineer,77000
Ann,Manager,119000</code></pre>
<p><strong>Solution: </strong>There are four simple ways to convert a <a href="https://blog.finxter.com/python-lists/" target="_blank" rel="noreferrer noopener">list</a> of lists to a CSV file in <a href="https://blog.finxter.com/python-cheat-sheets/" target="_blank" rel="noreferrer noopener">Python</a>.</p>
<ol>
<li><strong>CSV</strong>: Import the <code>csv</code> <a rel="noreferrer noopener" href="https://docs.python.org/3/library/csv.html" target="_blank">module</a> in Python, create a csv writer object, and find a list <code>lst</code> of elements representing each object as a row, that is then written into the CSV using <code>writer.writerow(lst)</code>.</li>
<li><strong>Pandas</strong>: Import the <a rel="noreferrer noopener" href="https://blog.finxter.com/pandas-cheat-sheets/" target="_blank">pandas library</a>, convert each object to a list to obtain a list of lists, create a Pandas DataFrame out of the list of lists, and write the DataFrame to a file using the DataFrame method <code>DataFrame.to_csv('file.csv')</code>.</li>
<li><strong>NumPy</strong>: Import the <a rel="noreferrer noopener" href="https://blog.finxter.com/collection-10-best-numpy-cheat-sheets-every-python-coder-must-own/" target="_blank">NumPy library</a>, convert each object to a list to obtain a list of lists, create a NumPy array, and write the output to a CSV file using the <code>numpy.savetxt('file.csv', array, delimiter=',')</code> method.</li>
<li><strong>Python</strong>: Use a pure <a rel="noreferrer noopener" href="https://blog.finxter.com/python-crash-course/" target="_blank">Python</a> implementation that doesn’t require any library by using the Python file I/O functionality.</li>
</ol>
<p class="has-global-color-8-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/2b50.png" alt="⭐" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Finxter Favorite</strong>: My preference is Method 4 (<strong>Vanilla Python</strong>) because it’s simplest to use, efficient, and most robust for different input types (numerical or textual) and doesn’t require external dependencies and data wrangling.</p>
<h2>Method 1: Python’s CSV Module</h2>
<p>You can convert a list of lists to a CSV file in Python easily—by using the <a rel="noreferrer noopener" href="https://docs.python.org/3/library/csv.html" target="_blank"><code>csv</code></a> library. <strong>This is the most customizable of all four methods.</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="13-18" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">class Employee(object): def __init__(self, name, description, salary): self.name = name self.description = description self.salary = salary employees = [Employee('Alice', 'Data Scientist', 122000), Employee('Bob', 'Engineer', 77000), Employee('Ann', 'Manager', 119000)] # Method 1
import csv
with open('my_file.csv', 'w', newline='') as f: writer = csv.writer(f) for x in employees: writer.writerow([x.name, x.description, x.salary]) </pre>
<p>Output:</p>
<pre class="wp-block-preformatted"><code><strong><em># my_file.csv</em></strong>
Alice,Data Scientist,122000
Bob,Engineer,77000
Ann,Manager,119000</code></pre>
<p>In the code, you first open the file using Python’s standard <code><a href="https://blog.finxter.com/python-open-function/" data-type="post" data-id="24793" target="_blank" rel="noreferrer noopener">open()</a></code> command. Now, you can write content to the file object <code>f</code>. </p>
<p>Next, you pass this file object to the constructor of the CSV writer that implements some additional helper method—and effectively wraps the file object providing you with new CSV-specific functionality such as the <code>writerow()</code> method. </p>
<p>You now iterate over the objects and convert each object to a list.</p>
<p>The list representing one row is then passed in the <code>writerow()</code> method of the CSV writer. This takes care of converting the list of objects to a CSV format. </p>
<p>You can customize the CSV writer in its constructor (e.g., by modifying the delimiter from a comma <code>','</code> to a whitespace <code>' '</code> character). Have a look at the specification to learn about <a rel="noreferrer noopener" href="https://docs.python.org/3/library/csv.html" target="_blank">advanced modifications</a>. </p>
<h2>Method 2: Pandas DataFrame to_csv()</h2>
<p>This method converts a list of objects to a CSV file in two steps:</p>
<ul>
<li>First, convert the list of objects to a <a href="https://blog.finxter.com/python-list-of-lists/" data-type="post" data-id="7890" target="_blank" rel="noreferrer noopener">list of lists</a>.</li>
<li>Second, <a href="https://blog.finxter.com/how-to-convert-a-list-of-lists-to-a-csv-file-in-python/" data-type="post" data-id="7999">convert the list of lists to a CSV</a> (e.g., using pandas <code>to_csv()</code>).</li>
</ul>
<div class="wp-block-image">
<figure class="aligncenter size-large is-resized"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/04/graphic-1024x576.jpg" alt="List of Lists to CSV" class="wp-image-8024" width="512" height="288" srcset="https://blog.finxter.com/wp-content/uploads/2020/04/graphic-scaled.jpg 1024w, https://blog.finxter.com/wp-content/uploads/2020/04/graphic-300x169.jpg 300w, https://blog.finxter.com/wp-content/uploads/2020/04/graphic-768x432.jpg 768w" sizes="(max-width: 512px) 100vw, 512px" /></figure>
</div>
<p>You can convert a list of lists to a<a rel="noreferrer noopener" href="https://pandas.pydata.org/" target="_blank"> Pandas</a> DataFrame that provides you with powerful capabilities such as the <a rel="noreferrer noopener" href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html" target="_blank"><code>to_csv()</code> method</a>. </p>
<p><strong>This is a super simple approach that avoids importing yet another library</strong> (I use Pandas in many Python projects anyways). </p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="13-21" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">class Employee(object): def __init__(self, name, description, salary): self.name = name self.description = description self.salary = salary employees = [Employee('Alice', 'Data Scientist', 122000), Employee('Bob', 'Engineer', 77000), Employee('Ann', 'Manager', 119000)] # Method 2
import pandas as pd # Step 1: Convert list of objects to list of lists
lst = [[x.name, x.description, x.salary] for x in employees] # Step 2: Convert list of lists to CSV
df = pd.DataFrame(lst)
df.to_csv('my_file.csv', index=False, header=False) </pre>
<p>Output:</p>
<pre class="wp-block-preformatted"><code><strong><em># my_file.csv</em></strong>
Alice,Data Scientist,122000
Bob,Engineer,77000
Ann,Manager,119000</code></pre>
<p><strong>Code Main Steps:</strong></p>
<ol>
<li><code>lst = [[x.name, x.description, x.salary] for x in employees]</code></li>
<li><code>df = pd.DataFrame(lst)</code></li>
<li><code>df.to_csv('my_file.csv', index=False, header=False)</code></li>
</ol>
<p>You convert a list of objects to a CSV file in three main steps.</p>
<ol>
<li>First, convert the list of objects to a list of lists by using <a rel="noreferrer noopener" href="https://blog.finxter.com/list-comprehension/" data-type="post" data-id="1171" target="_blank">list comprehension</a> to iterate over each object and convert each object to an inner list using your custom expression.</li>
<li>Second, create a <a rel="noreferrer noopener" href="https://blog.finxter.com/how-to-create-a-dataframe-in-pandas/" data-type="post" data-id="16764" target="_blank">Pandas DataFrame</a>, Python’s default representation of tabular data. </li>
<li>Third, the DataFrame is a very powerful data structure that allows you to perform various methods. One of those is the <code><a rel="noreferrer noopener" href="https://blog.finxter.com/pandas-dataframe-to_csv-method/" data-type="post" data-id="344277" target="_blank">to_csv()</a></code> method that allows you to write its contents into a CSV file.</li>
</ol>
<p>You set the <code>index</code> and <code>header</code> arguments of the <code>to_csv()</code> method to <code>False</code> because Pandas, per default, adds integer row and column indices 0, 1, 2, …. </p>
<p>Think of them as the row and column indices in your Excel spreadsheet. You don’t want them to appear in the CSV file so you set the arguments to <code>False</code>. </p>
<p>If you want to customize the CSV output, you’ve got a lot of special arguments to play with. Check out <a rel="noreferrer noopener" href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html" target="_blank">this </a>article for a comprehensive list of all arguments.</p>
<div class="wp-block-image">
<figure class="aligncenter size-full"><a href="https://blog.finxter.com/pandas-cheat-sheets/" target="_blank" rel="noopener"><img loading="lazy" width="741" height="573" src="https://blog.finxter.com/wp-content/uploads/2022/04/image-257.png" alt="" class="wp-image-329079" srcset="https://blog.finxter.com/wp-content/uploads/2022/04/image-257.png 741w, https://blog.finxter.com/wp-content/uploads/2022/04/image-257-300x232.png 300w" sizes="(max-width: 741px) 100vw, 741px" /></a></figure>
</div>
<p class="has-base-background-color has-background"><img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f30d.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /> <strong>Related article</strong>:<a rel="noreferrer noopener" href="https://blog.finxter.com/pandas-cheat-sheets/" target="_blank"> Pandas Cheat Sheets to Pin to Your Wall</a></p>
<h2>Method 3: NumPy savetext()</h2>
<p><a rel="noreferrer noopener" href="https://blog.finxter.com/numpy-tutorial/" target="_blank">NumPy </a>is at the core of Python’s <a rel="noreferrer noopener" href="https://blog.finxter.com/freelance-data-science/" data-type="post" data-id="16658" target="_blank">data science</a> and <a rel="noreferrer noopener" href="https://blog.finxter.com/machine-learning-engineer-income-and-opportunity/" data-type="post" data-id="306050" target="_blank">machine learning</a> functionality. Even <a href="https://blog.finxter.com/pandas-quickstart/" data-type="post" data-id="16511" target="_blank" rel="noreferrer noopener">Pandas</a> uses NumPy arrays to implement critical functionality.</p>
<p class="has-global-color-8-background-color has-background">You can convert a list of objects to a CSV file by first converting it to a list of lists which is then converted to a NumPy array, and then using <a rel="noreferrer noopener" href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.savetxt.html" target="_blank">NumPy’s <code>savetext()</code> function</a> by passing the NumPy array as an argument.</p>
<p><strong>This method is best if you can represent the numerical data only—otherwise, it’ll lead to complicated data type conversions which are not recommended.</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="13-23" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">class Employee(object): def __init__(self, name, description, salary): self.name = name self.description = description self.salary = salary employees = [Employee('Alice', 'Data Scientist', 122000), Employee('Bob', 'Engineer', 77000), Employee('Ann', 'Manager', 119000)] # Method 3
import numpy as np # Convert list of objects to list of lists
lst = [[hash(x.name), hash(x.description), x.salary] for x in employees] # Convert list of lists to NumPy array
a = np.array(lst) # Convert array to CSV
np.savetxt('my_file.csv', a, delimiter=',') </pre>
<p>In the code, we use the <code><a href="https://blog.finxter.com/python-hash-function/" data-type="post" data-id="24483" target="_blank" rel="noreferrer noopener">hash()</a></code> function to obtain a numerical value for the string attributes <code>name</code> and <code>description</code> of the <code>Employee</code> class.</p>
<p><strong>Output:</strong></p>
<pre class="wp-block-preformatted"><code><strong><em># my_file.csv</em></strong></code>
<code>-8.655249391637094400e+18,-4.821993523891147776e+18,1.220000000000000000e+05
7.826671284149683200e+18,-7.040934892515148800e+18,7.700000000000000000e+04
3.577554885237667328e+18,1.887669837421876992e+18,1.190000000000000000e+05</code></pre>
<p>The output doesn’t look pretty: it stores the values as floats. But no worries, you can reformat the output using the format argument <code>fmt</code> of the <code>savetxt()</code> method (<a rel="noreferrer noopener" href="https://numpy.org/doc/stable/reference/generated/numpy.savetxt.html?highlight=save#numpy.savetxt" target="_blank">more here</a>). However, I’d recommend you stick to method 2 (Pandas) to avoid unnecessary complexity in your code.</p>
<h2>Method 4: Pure Python Without External Dependencies</h2>
<p>If you don’t want to import any library and still convert a list of objects into a CSV file, you can use standard Python implementation as well: it’s not complicated but very efficient.</p>
<p class="has-global-color-8-background-color has-background">The idea is simple, iterate over the list of object and write a comma-separated representation of each object into the CSV file using a combination of the built-in <code><a rel="noreferrer noopener" href="https://blog.finxter.com/python-open-function/" data-type="post" data-id="24793" target="_blank">open()</a></code> function to create a file object and the <code>file.write()</code> method to write each row.</p>
<p><strong>This method is best if you won’t or cannot use external dependencies.</strong></p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="13-16" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">class Employee(object): def __init__(self, name, description, salary): self.name = name self.description = description self.salary = salary employees = [Employee('Alice', 'Data Scientist', 122000), Employee('Bob', 'Engineer', 77000), Employee('Ann', 'Manager', 119000)] # Method 4
with open('my_file.csv', 'w') as f: for x in employees: f.write(f'{x.name},{x.description},{x.salary}\n') </pre>
<p><strong>Output:</strong></p>
<pre class="wp-block-preformatted"><code><strong><em># my_file.csv</em></strong>
Alice,Data Scientist,122000,
Bob,Engineer,77000,
Ann,Manager,119000,</code></pre>
<p>In the code, you first open the file object <code>f</code>. Then you iterate over each object and write a custom comma-separated string representation of this object to the file using the file.write() method. </p>
<p>We use Python’s <a rel="noreferrer noopener" href="https://blog.finxter.com/string-formatting-vs-format-vs-formatted-string-literal/" data-type="post" data-id="13190" target="_blank">f-string</a> functionality to do that in a concise way. At the end of each row, you place the newline character <code>'\n'</code>. </p>
<h2>Method 5 – Bonus: Python One-Liner</h2>
<p>The previous method is a <a href="https://blog.finxter.com/python-one-liners/" data-type="post" data-id="13555" target="_blank" rel="noreferrer noopener">one-linerized</a> variant of <strong>Method 4</strong>. If you’re part of the Finxter community, you know how I love one-liners. <img src="https://s.w.org/images/core/emoji/14.0.0/72x72/1f609.png" alt="?" class="wp-smiley" style="height: 1em; max-height: 1em;" /></p>
<pre class="EnlighterJSRAW" data-enlighter-language="python" data-enlighter-theme="" data-enlighter-highlight="2" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group=""># Method 5
open('my_file.csv', 'w').writelines([f'{x.name},{x.description},{x.salary}\n' for x in employees])</pre>
<p>Concise, isn’t it? The output is the same as before.</p>
<p>If you’re interested in the art of crafting beautiful one-liners, check out my book on the topic!</p>
<h2>Python One-Liners Book: Master the Single Line First!</h2>
<p><strong>Python programmers will improve their computer science skills with these useful one-liners.</strong></p>
<div class="wp-block-image">
<figure class="aligncenter size-medium is-resized"><a href="https://www.amazon.com/gp/product/B07ZY7XMX8" target="_blank" rel="noopener noreferrer"><img loading="lazy" src="https://blog.finxter.com/wp-content/uploads/2020/06/3D_cover-1024x944.jpg" alt="Python One-Liners" class="wp-image-10007" width="512" height="472" srcset="https://blog.finxter.com/wp-content/uploads/2020/06/3D_cover-scaled.jpg 1024w, https://blog.finxter.com/wp-content/uploads/2020/06/3D_cover-300x277.jpg 300w, https://blog.finxter.com/wp-content/uploads/2020/06/3D_cover-768x708.jpg 768w" sizes="(max-width: 512px) 100vw, 512px" /></a></figure>
</div>
<p><a href="https://amzn.to/2WAYeJE" target="_blank" rel="noreferrer noopener" title="https://amzn.to/2WAYeJE"><em>Python One-Liners</em> </a>will teach you how to read and write “one-liners”: <strong><em>concise statements of useful functionality packed into a single line of code. </em></strong>You’ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.</p>
<p>The book’s five chapters cover (1) tips and tricks, (2) regular expressions, (3) machine learning, (4) core data science topics, and (5) useful algorithms. </p>
<p>Detailed explanations of one-liners introduce <strong><em>key computer science concepts </em></strong>and<strong><em> boost your coding and analytical skills</em></strong>. You’ll learn about advanced Python features such as <em><strong>list comprehension</strong></em>, <strong><em>slicing</em></strong>, <strong><em>lambda functions</em></strong>, <strong><em>regular expressions</em></strong>, <strong><em>map </em></strong>and <strong><em>reduce </em></strong>functions, and <strong><em>slice assignments</em></strong>. </p>
<p>You’ll also learn how to:</p>
<ul>
<li>Leverage data structures to <strong>solve real-world problems</strong>, like using Boolean indexing to find cities with above-average pollution</li>
<li>Use <strong>NumPy basics</strong> such as <em>array</em>, <em>shape</em>, <em>axis</em>, <em>type</em>, <em>broadcasting</em>, <em>advanced indexing</em>, <em>slicing</em>, <em>sorting</em>, <em>searching</em>, <em>aggregating</em>, and <em>statistics</em></li>
<li>Calculate basic <strong>statistics </strong>of multidimensional data arrays and the K-Means algorithms for unsupervised learning</li>
<li>Create more <strong>advanced regular expressions</strong> using <em>grouping </em>and <em>named groups</em>, <em>negative lookaheads</em>, <em>escaped characters</em>, <em>whitespaces, character sets</em> (and <em>negative characters sets</em>), and <em>greedy/nongreedy operators</em></li>
<li>Understand a wide range of <strong>computer science topics</strong>, including <em>anagrams</em>, <em>palindromes</em>, <em>supersets</em>, <em>permutations</em>, <em>factorials</em>, <em>prime numbers</em>, <em>Fibonacci </em>numbers, <em>obfuscation</em>, <em>searching</em>, and <em>algorithmic sorting</em></li>
</ul>
<p>By the end of the book, you’ll know how to <strong><em>write Python at its most refined</em></strong>, and create concise, beautiful pieces of “Python art” in merely a single line.</p>
<p><strong><a href="https://amzn.to/2WAYeJE" target="_blank" rel="noreferrer noopener" title="https://amzn.to/2WAYeJE"><em>Get your Python One-Liners on Amazon!!</em></a></strong></p>
</div>
https://www.sickgaming.net/blog/2022/07/25/how-to-convert-a-list-of-objects-to-a-csv-file-in-python-5-ways/