Posted on Leave a comment

How to Convert List of Lists to a Pandas Dataframe

Problem: You’re given a list of lists. Your goal is to convert it into a Pandas Dataframe.

Example: Say, you want to compare salary data of different companies and job descriptions. You’ve obtained the following salary data set as a list of list:

salary = [['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]]

How can you convert this into a Pandas Dataframe?

DataFrame()

Solution: The straight-forward solution is to use the pandas.DataFrame() constructor that creates a new Dataframe object from different input types such as NumPy arrays or lists.

Here’s how to do it for the given example:

import pandas as pd salary = [['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]] df = pd.DataFrame(salary)

This results in the following Dataframe:

print(df) ''' 0 1 2
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''

Try It Yourself: Run this code in our interactive Python shell by clicking the “Run” button.

DataFrame.from_records()

An alternative is the pandas.DataFrame.from_records() method that generates the same output:

import pandas as pd salary = [['Company', 'Job', 'Salary($)'], ['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]] df = pd.DataFrame.from_records(salary)
print(df) ''' 0 1 2
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''

Try It Yourself: Run this code in our interactive Python shell by clicking the “Run” button.

Column Names

If you want to add column names to make the output prettier, you can also pass those as a separate argument:

import pandas as pd salary = [['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]] df = pd.DataFrame(salary, columns=['Company', 'Job', 'Salary($)'])
print(df) ''' Company Job Salary($)
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''

Try It Yourself: Run this code in our interactive Python shell by clicking the “Run” button.

If the first list of the list of lists contains the column name, use slicing to separate the first list from the other lists:

import pandas as pd salary = [['Company', 'Job', 'Salary($)'], ['Google', 'Machine Learning Engineer', 121000], ['Google', 'Data Scientist', 109000], ['Google', 'Tech Lead', 129000], ['Facebook', 'Data Scientist', 103000]] df = pd.DataFrame(salary[1:], columns=salary[0])
print(df) ''' Company Job Salary($)
0 Google Machine Learning Engineer 121000
1 Google Data Scientist 109000
2 Google Tech Lead 129000
3 Facebook Data Scientist 103000 '''

Slicing is a powerful Python feature and before you can master Pandas, you need to master slicing. To refresh your Python slicing skills, download my ebook “Coffee Break Python Slicing” for free.

Summary: To convert a list of lists into a Pandas DataFrame, use the pd.DataFrame() constructor and pass the list of lists as an argument. An optional columns argument can help you structure the output.

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

How to Convert List of Lists to NumPy Array?

Short answer: Convert a list of lists—let’s call it l—to a NumPy array by using the standard np.array(l) function. This works even if the inner lists have a different number of elements.

Convert List of Lists to 2D Array

Problem: Given a list of lists in Python. How to convert it to a 2D NumPy array?

Example: Convert the following list of lists

[[1, 2, 3], [4, 5, 6]]

into a NumPy array

[[1 2 3] [4 5 6]]

Solution: Use the np.array(list) function to convert a list of lists into a two-dimensional NumPy array. Here’s the code:

# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5, 6]] # Convert it to a NumPy array
a = np.array(lst) # Print the resulting array
print(a) '''
[[1 2 3] [4 5 6]] '''

Try It Yourself: Here’s the same code in our interactive code interpreter:

<iframe height="700px" width="100%" src="https://repl.it/@finxter/numpylistoflists?lite=true" scrolling="no" frameborder="no" allowtransparency="true" allowfullscreen="true" sandbox="allow-forms allow-pointer-lock allow-popups allow-same-origin allow-scripts allow-modals"></iframe>

Hint: The NumPy method np.array() takes an iterable as input and converts it into a NumPy array.

Convert a List of Lists With Different Number of Elements

Problem: Given a list of lists. The inner lists have a varying number of elements. How to convert them to a NumPy array?

Example: Say, you’ve got the following list of lists:

[[1, 2, 3], [4, 5], [6, 7, 8]]

What are the different approaches to convert this list of lists into a NumPy array?

Solution: There are three different strategies you can use. (source)

(1) Use the standard np.array() function.

# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5], [6, 7, 8]] # Convert it to a NumPy array
a = np.array(lst) # Print the resulting array
print(a) '''
[list([1, 2, 3]) list([4, 5]) list([6, 7, 8])] '''

This creates a NumPy array with three elements—each element is a list type. You can check the type of the output by using the built-in type() function:

>>> type(a)
<class 'numpy.ndarray'>

(2) Make an array of arrays.

# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5], [6, 7, 8]] # Convert it to a NumPy array
a = np.array([np.array(x) for x in lst]) # Print the resulting array
print(a) '''
[array([1, 2, 3]) array([4, 5]) array([6, 7, 8])] '''

This is more logical than the previous version because it creates a NumPy array of 1D NumPy arrays (rather than 1D Python lists).

(3) Make the lists equal in length.

# Import the NumPy library
import numpy as np # Create the list of lists
lst = [[1, 2, 3], [4, 5], [6, 7, 8, 9]] # Calculate length of maximal list
n = len(max(lst, key=len)) # Make the lists equal in length
lst_2 = [x + [None]*(n-len(x)) for x in lst]
print(lst_2)
# [[1, 2, 3, None], [4, 5, None, None], [6, 7, 8, 9]] # Convert it to a NumPy array
a = np.array(lst_2) # Print the resulting array
print(a) '''
[[1 2 3 None] [4 5 None None] [6 7 8 9]] '''

You use list comprehension to “pad” None values to each inner list with smaller than maximal length.

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

How to Convert a List of List to a Dictionary in Python?

For some applications, it’s quite useful to convert a list of lists into a dictionary.

  • Databases: List of list is table where the inner lists are the database rows and you want to assign each row to a primary key in a new dictionary.
  • Spreadsheet: List of list is two-dimensional spreadsheet data and you want to assign each row to a key (=row name).
  • Data Analytics: You’ve got a two-dimensional matrix (=NumPy array) that’s initially represented as a list of list and you want to obtain a dictionary to ease data access.

There are three main ways to convert a list of lists into a dictionary in Python (source):

  1. Dictionary Comprehension
  2. Generator Expression
  3. For Loop

Let’s dive into each of those.

1. Dictionary Comprehension

Problem: Say, you’ve got a list of lists where each list represents a person and consists of three values for the person’s name, age, and hair color. For convenience, you want to create a dictionary where you use a person’s name as a dictionary key and the sublist consisting of the age and the hair color as the dictionary value.

Solution: You can achieve this by using the beautiful (but, surprisingly, little-known) feature of dictionary comprehension in Python.

persons = [['Alice', 25, 'blonde'], ['Bob', 33, 'black'], ['Ann', 18, 'purple']] persons_dict = {x[0]: x[1:] for x in persons}
print(persons_dict)
# {'Alice': [25, 'blonde'],
# 'Bob': [33, 'black'],
# 'Ann': [18, 'purple']}

Explanation: The dictionary comprehension statement consists of the expression x[0]: x[1:] that assigns a person’s name x[0] to the list x[1:] of the person’s age and hair color. Further, it consists of the context for x in persons that iterates over all “data rows”.

Exercise: Can you modify the code in our interactive code shell so that each hair color is used as a key and the name and age are used as the values?

Modify the code and click the “run” button to see if you were right!

2. Generator Expression

A similar way of achieving the same thing is to use a generator expression in combination with the dict() constructor to create the dictionary.

persons = [['Alice', 25, 'blonde'], ['Bob', 33, 'black'], ['Ann', 18, 'purple']] persons_dict = dict((x[0], x[1:]) for x in persons)
print(persons_dict)
# {'Alice': [25, 'blonde'],
# 'Bob': [33, 'black'],
# 'Ann': [18, 'purple']}

This code snippet is almost identical to the one used in the “list comprehension” part. The only difference is that you use tuples rather than direct mappings to fill the dictionary.

3. For Loop

Of course, there’s no need to get fancy here. You can also use a regular for loop and define the dictionary elements one by one within a simple for loop. Here’s the alternative code:

persons = [['Alice', 25, 'blonde'], ['Bob', 33, 'black'], ['Ann', 18, 'purple']] persons_dict = {}
for x in persons: persons_dict[x[0]] = x[1:] print(persons_dict)
# {'Alice': [25, 'blonde'],
# 'Bob': [33, 'black'],
# 'Ann': [18, 'purple']}

Again, you map each person’s name to the list consisting of its age and hair color.

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

Beef Programming Language

Beef is an in development programming language designed specifically for games and similar performance critical applications.  This comment from Hacker News best sums up the intentions of the BEEF language:

Author here. I’m the engineering co-founder of PopCap Games. I left PopCap after the EA acquisition, and I’ve been working on this project mostly full-time for the last five years.

Before Beef, I was developing game code in C# and engine code in C++ and I felt C# was just much more pleasant to work with – faster compile times, better IDE tooling, better errors, etc. Then it struck me that none of the things I liked about C# really had anything to do with the JIT or the GC, and it may be possible to create a “best of” merging between C# and C++.

I know there are other “C replacement” contenders out there – the differences are probably best explained through Beef’s specific design goals listed at https://www.beeflang.org/docs/foreward/

Beef consists of a complete compiler tool chain built on an LLVM backend, as well as a full IDE with modern features such as refactoring and code completion as well as a complete debugger and profiler.  It is available as a small (>100MB) download for Windows, or can be built from sources on Mac and Linux environments.

The Beef homepage is available here.

The Beef documentation is available here.

The move recent versions release notes are available here.

You can learn more about the Beef language and see the IDE in action in the video below.

GameDev News Programming


Posted on Leave a comment

Convert Tuple to List

Problem: Given a Python tuple with n elements. How to convert it into a list with the same n elements?

Examples:

  • Convert tuple (1, 2, 3, 4, 5) into list [1, 2, 3, 4, 5].
  • Convert tuple ('Alice', 'Bob', 'Ann') into list ['Alice', 'Bob', 'Ann'].
  • Convert tuple (1,) into list [1].

Note Tuple: Tuples are similar to lists—with the difference that you cannot change the tuple values (tuples are immutable) and you use parentheses rather than square brackets.

Solution: Use the built-in Python list() function to convert a list into a tuple. You don’t need to import any external library.

Code: The following code converts the three given tuples into lists.

tuple_1 = (1, 2, 3, 4, 5)
print(list(tuple_1))
# [1, 2, 3, 4, 5] tuple_2 = ('Alice', 'Bob', 'Ann')
print(list(tuple_2))
# ['Alice', 'Bob', 'Ann'] tuple_3 = (1,)
print(list(tuple_3))
# [1]

Try It Yourself: With our interactive code shell, you can try it yourself. As a small exercise, try to convert the empty tuple () into a list and see what happens.

Explanation: You can see that converting a tuple with one element leads to a list with one element. The list() function is the easiest way to convert a tuple into a list. Note that the values in the tuple are not copied—only a new reference to the same element is created:

The graphic also shows how to convert a tuple back to a list by using the tuple() function (that’s also a Python built-in function). Thus, calling list(tuple(lst)) on a list lst will result in a new list with the same elements.

Related articles:

Try to execute this code with the interactive Python tutor:

Posted on Leave a comment

Blazor WebAssembly 3.2.0 Preview 5 release now available

Daniel Roth

Daniel

A new preview update of Blazor WebAssembly is now available! Here’s what’s new in this release:

  • Read configuration during startup
  • Configure HTTP fetch request options
  • Honor existing web.config when publishing
  • Attach tokens to outgoing requests
  • Support for time zones

Get started

To get started with Blazor WebAssembly 3.2.0 Preview 5 install the latest .NET Core 3.1 SDK.

NOTE: Version 3.1.201 or later of the .NET Core SDK is required to use this Blazor WebAssembly release! Make sure you have the correct .NET Core SDK version by running dotnet --version from a command prompt.

Once you have the appropriate .NET Core SDK installed, run the following command to install the updated Blazor WebAssembly template:

dotnet new -i Microsoft.AspNetCore.Components.WebAssembly.Templates::3.2.0-preview5.20216.8

If you’re on Windows using Visual Studio, we recommend installing the latest preview of Visual Studio 2019 16.6. For this preview, you should still install the template from the command-line as described above to ensure that the Blazor WebAssembly template shows up correctly in Visual Studio and on the command-line.

That’s it! You can find additional docs and samples on https://blazor.net.

Upgrade an existing project

To upgrade an existing Blazor WebAssembly app from 3.2.0 Preview 4 to 3.2.0 Preview 5:

  • Update all Microsoft.AspNetCore.Components.WebAssembly.* package references to version 3.2.0-preview5.20216.8.
  • Update any Microsoft.AspNetCore.Components.WebAssembly.Runtime package references to version 3.2.0-preview5.20216.1.
  • Remove any calls to set WebAssemblyHttpMessageHandlerOptions.DefaultCredentials and instead call SetBrowserRequestCredentials on individual requests (see “Configure HTTP fetch request options” section below).
  • Remove the redirect parameter from calls to TryGetToken on AccessTokenResult.

You’re all set!

Read configuration during startup

Configuration data is now available during app startup in Program.Main using the Configuration property on WebAssemblyHostBuilder. This property can now be used both to add configuration sources and to access the current configuration data.

You can see this feature in action in the project templates when you enable authentication with Azure AD, Azure AD B2C, or an OpenID Connect provider of your choice. The authentication settings are stored in appsettings.json and then read from configuration when the app starts up:

Program.cs

public class Program
{ public static async Task Main(string[] args) { var builder = WebAssemblyHostBuilder.CreateDefault(args); builder.RootComponents.Add<App>("app"); builder.Services.AddTransient(sp => new HttpClient { BaseAddress = new Uri(builder.HostEnvironment.BaseAddress) }); builder.Services.AddOidcAuthentication(options => { // Configure your authentication provider options here. // For more information, see https://aka.ms/blazor-standalone-auth builder.Configuration.Bind("Local", options.ProviderOptions); }); await builder.Build().RunAsync(); }
}

appsettings.json

{ "Local": { "Authority": "https:login.microsoftonline.com/", "ClientId": "33333333-3333-3333-33333333333333333" }
}

Configure HTTP fetch request options

HTTP requests issued from a Blazor WebAssembly app using HttpClient are handled using the browser fetch API. In this release, we’ve added a set of extension methods for HttpRequestMessage that configure various fetch related options. These extension methods live in the Microsoft.AspNetCore.Components.WebAssembly.Http namespace:

HttpRequestMessage extension method Fetch request property
SetBrowserRequestCredentials credentials
SetBrowserRequestCache cache
SetBrowserRequestMode mode
SetBrowserRequestIntegrity integrity

You can set additional options using the more generic SetBrowserRequestOption extension method.

The HTTP response is typically buffered in a Blazor WebAssembly app to enable support for sync reads on the response content. To enable support for response streaming, use the SetBrowserResponseStreamingEnabled extension method on the request.

Honor existing web.config when publishing

When publishing a standalone Blazor WebAssembly app, a web.config is automatically generated for the app that handles configuring IIS appropriately. You can now specify your own web.config in the project, which will get used instead of the generated one.

Attach tokens to outgoing requests

Configuring authentication now adds an AuthorizationMessageHandler as a service that can be used with HttpClient to attach access tokens to outgoing requests. Tokens are acquired using the existing IAccessTokenProvider service. If a token cannot be acquired, an AccessTokenNotAvailableException is thrown. This exception has a Redirect method that can be used to navigate the user to the identity provider to acquire a new token. The AuthorizationMessageHandler can be configured with the authorized URLs, scopes, and return URL using the ConfigureHandler method.

For example, you can configure an HttpClient to use the AuthorizationMessageHandler like this:

builder.Services.AddSingleton(sp =>
{ return new HttpClient(sp.GetRequiredService<AuthorizationMessageHandler>() .ConfigureHandler( new [] { "https://www.example.com/base" }, scopes: new[] {"example.read", "example.write"})) { BaseAddress = new Uri("https://www.example.com/base") };
});

For convenience, a BaseAddressAuthorizationMessageHandler is also included that is preconfigured with the app base address as an authorized URL. The authentication enabled Blazor WebAssembly templates now use IHttpClientFactory to set up an HttpClient with the BaseAddressAuthorizationMessageHandler:

builder.Services.AddHttpClient("BlazorWithIdentityApp1.ServerAPI", client => client.BaseAddress = new Uri(builder.HostEnvironment.BaseAddress)) .AddHttpMessageHandler<BaseAddressAuthorizationMessageHandler>(); // Supply HttpClient instances that include access tokens when making requests to the server project
builder.Services.AddTransient(sp => sp.GetRequiredService<IHttpClientFactory>().CreateClient("BlazorWithIdentityApp1.ServerAPI"));

You can use the configured HttpClient to make authorized requests using a simple try-catch pattern. For example, here’s the updated code in the FetchData component for requesting the weather forecast data:

protected override async Task OnInitializedAsync()
{ try { forecasts = await Http.GetFromJsonAsync<WeatherForecast[]>("WeatherForecast"); } catch (AccessTokenNotAvailableException exception) { exception.Redirect(); }
}

Alternatively, you can simplify things even further by defining a strongly-typed client that handles all of the HTTP and token acquisition concerns within a single class:

WeatherClient.cs

public class WeatherClient
{ private readonly HttpClient httpClient; public WeatherClient(HttpClient httpClient) { this.httpClient = httpClient; } public async Task<IEnumerable<WeatherForecast>> GetWeatherForeacasts() { IEnumerable<WeatherForecast> forecasts = new WeatherForecast[0]; try { forecasts = await httpClient.GetFromJsonAsync<WeatherForecast[]>("WeatherForecast"); } catch (AccessTokenNotAvailableException exception) { exception.Redirect(); } return forecasts; }
}

Program.cs

builder.Services.AddHttpClient<WeatherClient>(client => client.BaseAddress = new Uri(builder.HostEnvironment.BaseAddress)) .AddHttpMessageHandler<BaseAddressAuthorizationMessageHandler>();

FetchData.razor

protected override async Task OnInitializedAsync()
{ forecasts = await WeatherClient.GetWeatherForeacasts();
}

Support for time zones

Blazor now infers the user’s time zone and uses it in date and time calculations. In addition, APIs on System.TimeZoneInfo that previously returned incomplete results now report correct results.

Help improve the Blazor docs!

Thank you everyone who has taken the time to give feedback on how we can best improve the Blazor docs!

If you haven’t already, please join in with helping us improve the docs by doing the following:

  • As you read the Blazor docs, let us know where we should focus our efforts by telling us if you find a topic helpful or not using the helpfulness widget at the top of each doc page:

    Doc helpfulness

  • Use the Feedback section at the bottom of each doc page to let us know when a particular topic is unclear, inaccurate, or incomplete.

    Doc feedback

  • Comment on our Improve the Blazor docs GitHub issue with your suggestions for new content and ways to improve the existing content.

Feedback

We hope you enjoy the new features in this preview release of Blazor WebAssembly! Please let us know what you think by filing issues on GitHub.

Thanks for trying out Blazor!

Posted on Leave a comment

ASP.NET Core updates in .NET 5 Preview 3

Avatar

Sourabh

.NET 5 Preview3 is now available and is ready for evaluation! .NET 5 will be a current release.

Get started

To get started with ASP.NET Core in .NET 5.0 Preview3 install the .NET 5.0 SDK.

If you’re on Windows using Visual Studio, we recommend installing the latest preview of Visual Studio 2019 16.6.

If you’re on macOS, we recommend installing the latest preview of Visual Studio 2019 for Mac 8.6.

Upgrade an existing project

To upgrade an existing ASP.NET Core 5.0 preview2 app to ASP.NET Core 5.0 preview3:

  • Change the TFM in your *.csproj file from netcoreapp5.0 to net5.0
  • Update all Microsoft.AspNetCore.* package references to 5.0.0-preview.3.20215.14.
  • Update all Microsoft.Extensions.* package references to 5.0.0-preview.3.20215.2.

See the full list of breaking changes in ASP.NET Core 5.0.

That’s it! You should now be all set to use .NET 5 Preview3.

What’s new?

Performance Improvements to HTTP/2

By significantly reducing allocations in the HTTP/2 code path and adding support for HPack static compression of HTTP/2 response headers in Kestrel, the 5.0.0-prevew3 release improves the performance of HTTP/2.

We expect to announce additional features in upcoming preview releases.
See the release notes for additional details and known issues.

Give feedback

We hope you enjoy this release of ASP.NET Core in .NET 5! We are eager to hear about your experiences with this latest .NET 5 release. Let us know what you think by filing issues on GitHub.

Thanks for trying out ASP.NET Core!

Posted on Leave a comment

Python sum() List – A Simple Illustrated Guide

Summing up a list of numbers appears everywhere in coding. Fortunately, Python provides the built-in sum() function to sum over all elements in a Python list—or any other iterable for that matter. (Official Docs)

The syntax is sum(iterable, start=0):

Argument Description
iterable Sum over all elements in the iterable. This can be a list, a tuple, a set, or any other data structure that allows you to iterate over the elements.
Example: sum([1, 2, 3]) returns 1+2+3=6.
start (Optional.) The default start value is 0. If you define another start value, the sum of all values in the iterable will be added to this start value.
Example: sum([1, 2, 3], 9) returns 9+1+2+3=15.

Check out the Python Freelancer Webinar and KICKSTART your coding career!

Code: Let’s check out a practical example!

lst = [1, 2, 3, 4, 5, 6] print(sum(lst))
# 21 print(sum(lst, 10))
# 31

Exercise: Try to modify the sequence so that the sum is 30 in our interactive Python shell:

Let’s explore some important details regarding the sum() function in Python.

Errors

A number of errors can happen if you use the sum() function in Python.

TypeError: Python will throw a TypeError if you try to sum over elements that are not numerical. Here’s an example:

# Demonstrate possible execeptions
lst = ['Bob', 'Alice', 'Ann'] # WRONG:
s = sum(lst)

If you run this code, you’ll get the following error message:

Traceback (most recent call last): File "C:\Users\xcent\Desktop\code.py", line 3, in <module> s = sum(lst)
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Python tries to perform string concatenation using the default start value of 0 (an integer). Of course, this fails. The solution is simple: sum only over numerical values in the list.

If you try to “hack” Python by using an empty string as start value, you’ll get the following exception:

# Demonstrate possible execeptions
lst = ['Bob', 'Alice', 'Ann'] # WRONG:
s = sum(lst, '')

Output:

Traceback (most recent call last): File "C:\Users\xcent\Desktop\code.py", line 5, in <module> s = sum(lst, '')
TypeError: sum() can't sum strings [use ''.join(seq) instead]

You can get rid of all those errors by summing only over numerical elements in the list.

(For more information about the join() method, check out this blog article.)

Python Sum List Time Complexity

The time complexity of the sum() function is linear in the number of elements in the iterable (list, tuple, set, etc.). The reason is that you need to go over all elements in the iterable and add them to a sum variable. Thus, you need to “touch” every iterable element once.

Python Sum List of Strings

Problem: How can you sum a list of strings such as ['python', 'is', 'great']? This is called string concatenation.

Solution: Use the join() method of Python strings to concatenate all strings in a list. The sum() function works only on numerical input data.

Code: The following example shows how to “sum” up (i.e., concatenate) all elements in a given list of strings.

# List of strings
lst = ['Bob', 'Alice', 'Ann'] print(''.join(lst))
# BobAliceAnn print(' '.join(lst))
# Bob Alice Ann

Python Sum List of Lists

Problem: How can you sum a list of lists such as [[1, 2], [3, 4], [5, 6]] in Python?

Solution: Use a simple for loop with a helper variable to concatenate all lists.

Code: The following code concatenates all lists into a single list.

# List of lists
lst = [[1, 2], [3, 4], [5, 6]] s = []
for x in lst: s.extend(x)
print(s)
# [1, 2, 3, 4, 5, 6]

The extend() method is little-known in Python—but it’s very effective to add a number of elements to a Python list at once. Check out my detailed tutorial on this Finxter blog.

Python Sum List While Loop

Problem: How can you sum over all list elements using a while loop (without sum())?

Solution: Create an aggregation variable and iteratively add another element from the list.

Code: The following code shows how to sum up all numerical values in a Python list without using the sum() function.

# list of integers
lst = [1, 2, 3, 4, 5] # aggregation variable
s = 0 # index variable
i = 0 # sum everything up
while i<len(lst): s += lst[i] i += 1 # print the result
print(s)
# 15

This is not the prettiest way but it’s readable and it works (and, you didn’t want to use the sum() function, right?).

Python Sum List For Loop

Problem: How can you sum over all list elements using a for loop (without sum())?

Solution: Create an aggregation variable and iteratively add another element from the list.

Code: The following code shows how to sum up all numerical values in a Python list without using the sum() function.

# list of integers
lst = [1, 2, 3, 4, 5] # aggregation variable
s = 0 # sum everything up
for x in lst: s += x # print the result
print(s)
# 15

This is a bit more readable than the previous version with the while loop because you don’t have to keep track about the current index.

Python Sum List with List Comprehension

List comprehension is a powerful Python features that allows you to create a new list based on an existing iterable. Can you sum up all values in a list using only list comprehension?

The answer is no. Why? Because list comprehension exists to create a new list. Summing up values is not about creating a new list. You want to get rid of the list and aggregate all values in the list into a single numerical “sum”.

Python Sum List of Tuples Element Wise

Problem: How to sum up a list of tuples, element-wise?

Example: Say, you’ve got list [(1, 1), (2, 0), (0, 3)] and you want to sum up the first and the second tuple values to obtain the “summed tuple” (1+2+0, 1+0+3)=(3, 4).

Solution: Unpack the tuples into the zip function to combine the first and second tuple values. Then, sum up those values separately. Here’s the code:

# list of tuples
lst = [(1, 1), (2, 0), (0, 3)] # aggregate first and second tuple values
zipped = list(zip(*lst))
# result: [(1, 2, 0), (1, 0, 3)] # calculate sum of first and second tuple values
res = (sum(zipped[0]), sum(zipped[1])) # print result to the shell
print(res)
# result: (3, 4)

Need a refresher of the zip() function and unpacking? Check out these articles on the Finxter blog:

Python Sum List Slice

Problem: Given a list. Sum up a slice of the original list between the start and the step indices (and assuming the given step size as well).

Example: Given is list [3, 4, 5, 6, 7, 8, 9, 10]. Sum up the slice lst[2:5:2] with start=2, stop=5, and step=2.

Solution: Use slicing to access the list. Then, apply the sum() function on the result.

Code: The following code computes the sum of a given slice.

# create the list
lst = [3, 4, 5, 6, 7, 8, 9, 10] # create the slice
slc = lst[2:5:2] # calculate the sum
s = sum(slc) # print the result
print(s)
# 12 (=5+7)

Let’s examine an interesting problem: to sum up conditionally!

Python Sum List Condition

Problem: Given is a list. How to sum over all values that meet a certain condition?

Example: Say, you’ve got the list lst = [5, 8, 12, 2, 1, 3] and you want to sum over all values that are larger than 4.

Solution: Use list comprehension to filter the list so that only the elements that satisfy the condition remain. Then, use the sum() function to sum over the remaining values.

Code: The following code sums over all values that satisfy a certain condition (e.g., x>4).

# create the list
lst = [5, 8, 12, 2, 1, 3] # filter the list
filtered = [x for x in lst if x>4]
# remaining list: [5, 8, 12] # sum over the filtered list
s = sum(filtered) # print everything
print(s)
# 25

Need a refresher on list comprehension? Check out my in-depth tutorial on the Finxter blog.

Python Sum List Ignore None

Problem: Given is a list of numerical values that may contain some values None. How to sum over all values that are not the value None?

Example: Say, you’ve got the list lst = [5, None, None, 8, 12, None, 2, 1, None, 3] and you want to sum over all values that are not None.

Solution: Use list comprehension to filter the list so that only the elements that satisfy the condition remain (that are different from None). You see, that’s a special case of the previous paragraph that checks for a general condition. Then, use the sum() function to sum over the remaining values.

Code: The following code sums over all values that are not None.

# create the list
lst = [5, None, None, 8, 12, None, 2, 1, None, 3] # filter the list
filtered = [x for x in lst if x!=None]
# remaining list: [5, 8, 12, 2, 1, 3] # sum over the filtered list
s = sum(filtered) # print everything
print(s)
# 31

A similar thing can be done with the value Nan that can disturb your result if you aren’t careful.

Python Sum List Ignore Nan

Problem: Given is a list of numerical values that may contain some values nan (=”not a number”). How to sum over all values that are not the value nan?

Example: Say, you’ve got the list lst = [1, 2, 3, float("nan"), float("nan"), 4] and you want to sum over all values that are not nan.

Solution: Use list comprehension to filter the list so that only the elements that satisfy the condition remain (that are different from nan). You see, that’s a special case of the previous paragraph that checks for a general condition. Then, use the sum() function to sum over the remaining values.

Code: The following code sums over all values that are not nan.

# for checking isnan(x)
import math # create the list
lst = [1, 2, 3, float("nan"), float("nan"), 4] # forget to ignore 'nan'
print(sum(lst))
# nan # ignore 'nan'
print(sum([x for x in lst if not math.isnan(x)]))
# 10

Phew! Quite some stuff. Thanks for reading through this whole article! I hope you’ve learned something out of this tutorial and remain with the following recommendation:

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

Print a Python List Beautifully [Click & Run Code]

How to print a Python list in a beautiful and fully customizable way?

This article shows you six effective ways of doing it. By studying these alternatives, you’ll not only learn how to print lists in Python, you’ll become a better coder overall.

If you just want to know the best way to print a list in Python, here’s the short answer:

  • Pass a list as an input to the print() function in Python.
  • Use the asterisk operator * in front of the list to “unpack” the list into the print function.
  • Use the sep argument to define how to separate two list elements visually.

Here’s the code:

# Create the Python List
lst = [1, 2, 3, 4, 5] # Use three underscores as separator
print(*lst, sep='___')
# 1___2___3___4___5 # Use an arrow as separator
print(*lst, sep='-->')
# 1-->2-->3-->4-->5

Try It Yourself in Our Interactive Code Shell:

This is the best and most Pythonic way to print a Python list. If you still want to learn about alternatives—and improve your Python skills in the process of doing so—keep reading!

Method: Use Default print() Statement

The default print() statement converts the list into a string representation that encloses the list elements in the square brackets [ and ], and separates two subsequent elements with the comma and an empty space a, b. This is the standard list representation.

lst = [1, 2, 3, 4, 5]
print(lst)

The output is the following:

[1, 2, 3, 4, 5]
Advantages Disadvantages
Easy to read and write Non-customizable
Fast
Concise

Try It Yourself in Our Interactive Code Shell:

The next method overcomes the main disadvantage of being not very customizable.

Method: Iterate In a For Loop

If you want full control about the output of each list element, you can use the straightforward approach of using a for loop to iterate over each element x in the list. You can then decide for yourself how to print each element.

# Create the Python List
lst = [1, 2, 3, 4, 5] # Iterate over each element x
# in the list and customize printing
for x in lst: print('Element: ' + x)

The output is the following:

Element: 1
Element: 2
Element: 3
Element: 4
Element: 5
Advantages Disadvantages
Fully customizable Relatively slow
Simple Less concise
Newline after each element

Try It Yourself in Our Interactive Python Shell:

Method: Iterate in For Loop with End Argument

If you’d rather print all elements in a single line, separated by three whitespace characters, you can do so by defining the end argument of the print() function that defines which character is added after each element that was printed to the shell (default: new-line character \n):

# Create the Python List
lst = [1, 2, 3, 4, 5] # Iterate over each element x
# in the list and customize printing
for x in lst: # Use the end argument to define # what to print after each element print(str(x), end=' ')

The output is:

1 2 3 4 5 

You see that the end argument overwrites the default behavior of printing a new-line character at the end of each element. Instead, each two elements are separated by three empty spaces.

Advantages Disadvantages
Fully customizable Relatively slow
Simple Less concise

Try It Yourself in Our Interactive Code Shell:

Let’s overcome the disadvantage of the for loop of being less concise!

Method: Unpacking With Separator Argument

The print() function works with an iterable as input. You can use the asterisk operator * in front of the list to “unpack” the list into the print function. Now, you can use the sep argument of the print() function to define how to separate two elements of the iterable.

# Create the Python List
lst = [1, 2, 3, 4, 5] # Use three underscores as separator
print(*lst, sep='___')
# 1___2___3___4___5 # Use an arrow as separator
print(*lst, sep='-->')
# 1-->2-->3-->4-->5

The sep argument allows you to define precisely what to put between each pair of elements in an iterable. This allows you full customization and keeps the code lean and concise.

Advantages Disadvantages
Fully customizable Harder to read for beginners
Fast
Concise

Try It Yourself in Our Interactive Code Shell:

This is the best and most Pythonic way to print a Python list. If you still want to learn about alternatives, keep reading.

Method: Use the string.join() Method

The string.join(iterable) method joins together all elements in the iterable, using the string as a separator between two elements. Thus, it works exactly like the sep argument of the print() function.

# Create the Python List
lst = ['1', '2', '3', '4', '5'] # Use three underscores as separator
print('___'.join(lst))
# 1___2___3___4___5 # Use arrow as separator
print('-->'.join(lst))
# 1-->2-->3-->4-->5

Note that you can only use this methods if the list elements are already strings. If they are integers, joining them together doesn’t work and Python throws an error:

TypeError: sequence item 0: expected str instance, int found
Advantages Disadvantages
Fully customizable Harder to read for beginners
Concise Slow
Works only for string elements

Try It Yourself in Our Interactive Code Shell:

So how do you apply this method to integer lists?

Method: Use the string.join() Method with Map()

The string.join(iterable) method joins together all elements in the iterable, using the string as a separator between two elements. But it expects that all elements in the iterable are already strings. If they aren’t, you need to convert them first. To achieve this, you can use the built-in map() method in Python 3.x.

# Create the Python List
lst = [1, 2, 3, 4, 5] # Use three underscores as separator
print('___'.join(map(str, lst)))
# 1___2___3___4___5 # Use arrow as separator
print('-->'.join(map(str, lst)))
# 1-->2-->3-->4-->5

The map(str, lst) method applies the function str(x) to each element x in the list. In other words, it converts each integer element to a string. An alternative way without the map(str, lst) function would be list comprehension [str(x) for x in lst] that results in the same output.

Advantages Disadvantages
Fully customizable Harder to read for beginners
Concise Slow
Works for all data types

Try It Yourself in Our Interactive Code Shell:

So, let’s finish this up!

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

The Dot Character in a Character Set – What Does It Match?

Given is the following regular expression:

regex = '[a-z.]+'

Note the dot character inside the character set. As you may know, the dot metacharacter matches an arbitrary character if it is used outside a character set.

But what does it match if you place the dot character inside a regex character set?

The answer is that the dot inside the character set matches the dot symbol—and not an arbitrary character. The reason is that the character set removes the special meaning of the dot symbol.

Here’s a simple example:

import re regex = '[a-z.]+' text_1 = 'hello.world'
text_2 = 'HELLO.WORLD' print(re.match(regex, text_1))
# <re.Match object; span=(0, 11), match='hello.world'> print(re.match(regex, text_2))
# None

The first text will be matched in both cases (the dot character matches an arbitrary character or the dot symbol).

But the second text will only match if the dot has the meaning: “match an arbitrary character”. Otherwise, the character set cannot match the text.

As the result is None, the text could not have been matched. This proves that the dot metacharacter loses its special meaning inside a character set.