+- Sick Gaming (https://www.sickgaming.net)
+-- Forum: Programming (https://www.sickgaming.net/forum-76.html)
+--- Forum: Python (https://www.sickgaming.net/forum-83.html)
+--- Thread: [Tut] How to Split a List Into Evenly-Sized Chunks? (/thread-94539.html)
[Tut] How to Split a List Into Evenly-Sized Chunks? - xSicKxBot - 04-14-2020
How to Split a List Into Evenly-Sized Chunks?
<div><p>In this article, I’ll show you how to divide a list into equally-sized chunks in Python. Step-by-step, you’ll arrive at the following great code that accomplishes exactly that:</p>
<p> <iframe height="800px" width="100%" src="https://repl.it/@finxter/listchunking2?lite=true" scrolling="no" frameborder="no" allowtransparency="true" allowfullscreen="true" sandbox="allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals"></iframe> </p>
<p>You can play around with the code yourself but if you need some explanations, read on because I’ll explain it to you in much detail:</p>
<h2>Chunking Your List</h2>
<p>Let’s make this question more palpable by transforming it into a practical problem: </p>
<p><strong>Problem</strong>: Imagine that you have a temperature sensor that sends data every 6 minutes, which makes 10 data points per hour. All these data points are stored in one list for each day.</p>
<p>Now, we want to have a list of hourly average temperatures for each day—this is why we need to split the list of data for one day into evenly sized chunks.</p>
<p><strong>Solution</strong>: To achieve this, we use a for-loop and Python’s built-in function <code>range()</code> which we have to examine in depth.</p>
<p>The <code>range()</code> function can be used either with one, two or three arguments.</p>
<ul>
<li>If you use it with one single argument, e.g., <code>range(10)</code>, we get a range object containing the numbers 0 to 9. So, if you call range with one argument, this argument will be interpreted as the max or stop value of the range, but it is excluded from the range.</li>
<li>You can also call the <code>range()</code> function with two arguments, e.g., <code>range(5, 10)</code>. This call with two arguments returns a range object containing the numbers 5 to 9. So, now we have a lower and an upper bound for the range. Contrary to the stop value, the start value is included in the range.</li>
<li>In a call of the function <code>range()</code> with three parameters, the first parameter is the start value, the second one is the stop value and the third value is the step size. For example, <code>range(5, 15, 2)</code> returns a range object containing the following values: 5, 7, 9, 11, 13. As you can see, the range starts with the start and then it adds the step value as long as the values are less than the stop value.</li>
</ul>
<p>In our problem, our chunks have a length of 10, the start value is 0 and the max value is the end of the list of data. </p>
<p>Putting all together: Calling <code>range(0, len(data), 10)</code> will give us exactly what we need to iterate over the chunks. Let’s put some numbers there to visualize it.</p>
<p>For one single day, we have a data length of 24 * 10 = 240, so the call of the range function would be this: <code>range(0, 240, 10)</code> and the resulting range would be 0, 10, 20, 30, …, 230. Pause a moment and consider these values: they represent the indices of the first element of each chunk.</p>
<p>So what do we have now? The start indices of each chunk and also the length – and that’s all we need to slice the input data into the chunks we need.</p>
<p>The slicing operator takes two or three arguments separated by the colon <code>:</code> symbol. They have the same meaning as in the range function. </p>
<p><a rel="noreferrer noopener" href="https://blog.finxter.com/introduction-to-slicing-in-python/" target="_blank">If you want to know more about slicing read our detailed article here</a>.</p>
<p>A first draft of our code could be this:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">data = [15.7, 16.2, 16.5, 15.9, ..., 27.3, 26.4, 26.1, 27.2]
chunk_length = 10 for i in range(0, len(data), chunk_length): print(data[i:i+chunk_length])</pre>
<p>Play with this code in our interactive Python shell:</p>
<p> <iframe height="600px" width="100%" src="https://repl.it/@finxter/chunklist?lite=true" scrolling="no" frameborder="no" allowtransparency="true" allowfullscreen="true" sandbox="allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals"></iframe> </p>
<p>However, we can still improve this code and make it reusable by creating a <a rel="noreferrer noopener" href="https://blog.finxter.com/how-to-use-generator-expressions-in-python-dictionaries/" target="_blank">generator </a>out of it. </p>
<h2>Chunking With Generator Expressions</h2>
<p>A generator is a function but instead of a return statement it uses the keyword <code>yield</code>.</p>
<p>The keyword <code>yield</code> interrupts the function and returns a value. The next time the function gets called, the next value is returned and the function’s execution stops again. This behavior can be used in a for-loop, where we want to get a value from the generator, work with this value inside the loop and then repeat it with the next value. Now, let’s take a look at the improved version of our code:</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">data = [15.7, 16.2, 16.5, 15.9, ..., 27.3, 26.4, 26.1, 27.2]
chunk_length = 10 def make_chunks(data, length): for i in range(0, len(data), length): yield data[i:i + length] for chunk in make_chunks(data, chunk_length): print(chunk)</pre>
<p>That looks already pretty pythonic and we can reuse the function <code>make_chunks()</code> for all the other data we need to process.</p>
<p>Let’s finish the code so that we get a list of hourly average temperatures as result.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import random def make_chunks(data, length): for i in range(0, len(data), length): yield data[i:i + length] def process(chunk): return round(sum(chunk)/len(chunk), 2) n = 10
# generate random temperature values
day_temperatures = [random.random() * 20 for x in range(24 * n)]
avg_per_hour = [] for chunk in make_chunks(day_temperatures, n): r = process(batch) avg_per_hour.append® print(avg_per_hour)</pre>
<p>And that’s it, this cool pythonic code solves our problem. We can make the code even a bit shorter but I consider this code less readable because you need to know really advanced Python concepts.</p>
<pre class="EnlighterJSRAW" data-enlighter-language="generic" data-enlighter-theme="" data-enlighter-highlight="" data-enlighter-linenumbers="" data-enlighter-lineoffset="" data-enlighter-title="" data-enlighter-group="">import random make_chunks = lambda data, n: (data[i:i + n] for i in range(0, len(data), n))
process = lambda data: round(sum(data)/len(data), 2) n = 10
# generate random temperature values
day_temperatures = [random.random() * 20 for x in range(24 * n)]
avg_per_hour = [] for chunk in make_chunks(day_temperatures, n): r = process(batch) avg_per_hour.append® print(avg_per_hour)</pre>
<p>So, what did we do? We reduced the helper functions to <a rel="noreferrer noopener" href="https://blog.finxter.com/how-to-filter-in-python-using-lambda-functions/" target="_blank">lambda expressions</a> and for the generator function we use a special shorthand – the parenthesis.</p>
<h2>Summary</h2>
<p><strong>To sum up the solution</strong>: We used the range function with three arguments, the start value, the stop value and the step value. By setting the step value to our desired chunk length, the start value to 0 and the stop value to the total data length, we get a range object containing all the start indices of our chunks. With the help of slicing we can access exactly the chunk we need in each iteration step.</p>
<h2>Where to Go From Here? </h2>
<p>Want to start earning a full-time income with Python—while working only part-time hours? Then join our free <a rel="noreferrer noopener" href="https://www.digistore24.com/content/261950/26036/lukasrieger/" target="_blank">Python Freelancer Webinar</a>. </p>
<p>It shows you exactly how you can grow your business and Python skills to a point where you can work comfortable for 3-4 hours from home and enjoy the rest of the day (=20 hours) spending time with the persons you love doing things you enjoy to do. </p>
<p><a href="https://www.digistore24.com/content/261950/26036/lukasrieger/" target="_blank" rel="noreferrer noopener">Become a Python freelancer now!</a></p>
</div>
https://www.sickgaming.net/blog/2020/04/13/how-to-split-a-list-into-evenly-sized-chunks/