Posted on Leave a comment

Top 21 Developer Jobs and Career Paths in 2023

5/5 – (1 vote)

This article will go over the top 21 most attractive developer jobs in the decade to come. Note that the purpose of this article is to look forward to the future rather than looking backward into the past. The future is inherently uncertain but we did everything we could (as you’ll see) to remain objective and use data to guide our predictions.

Here’s a quick tabular overview on the most relevant career paths and jobs you could pursue in the decade to come – if you’re a developer optimizing for the future rather than the past:

Career Path (Job) Annual Income (Low) Annual Income (High)
Blockchain Developer $105,180 $108,560
Data Scientist $97,294 $135,924
Machine Learning Engineer $112,000 $157,000
Deep Learning Engineer $124,000 $148,000
Computer Science Researcher $74,210 $208,000
Distributed Systems Developer $97,000 $169,656
AWS Developer $100,000 $130,000
Android App Developer $85,000 $126,577
C++ Developer $45,000 $140,000
Full-Stack Developer $79,584 $108,984
Solidity Developer $60,000 $180,000
Python Developer $65,000 $82,000
Crypto Bot Developer $70,000 $110,000
Azure Developer $107,000 $127,000
Game Developer $64,053 $115,846
JavaScript Developer $62,000 $118,000
Quant Developer $86,528 $170,000
Security Engineer $75,732 $144,874
Test Automation Engineer $74,821 $120,000
VHDL Developer $32,000 $169,500
Frontend Developer $61,194 $119,224

Okay, let’s dive into the meat—the most promising developer role for the upcoming decade(s)!

Blockchain Developer

A blockchain engineer operates, designs, develops, analyzes, implements, and supports a distributed blockchain network. Blockchain engineers manage specific business models dealing with blockchain technology.

The average annual income of a Blockchain engineer is between $105,180 and $108,560 according to Glassdoor (source):

But is there enough demand? Let’s have a look at Google trends to find out how interest evolves over time (source):

Do you want to become a Blockchain engineer? Here’s a learning path I’d propose in five steps to get started:

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Data Scientist

Data scientists use data to find quantifiable answers to questions that often need to be found as well!

For example, they not only find the answer to the question “How can company ABC make more money?” Instead, they may find that a better question to ask would be: “Who are the top 20% of the clients that bring 80% of the revenue, and what do they want in the first place?”

How much does a Data Scientist make per year?

Average Income of a Data Scientist in the US by Source
Figure: Average Income of a Data Scientist in the US by Source. [1]

The average annual income of a Data Scientist in the United States is between $97,294 and $135,924 with an average of $116,505 and a median of $119,413 per year.

But is there enough demand? Let’s have a look at Google trends to find out how interest evolves over time (source):

Clearly, this is a long-term trend you can build your whole career on!

Do you want to become a Data Scientist? Here’s a step-by-step learning path I’d propose to get started with Data :

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Machine Learning Engineer

A Machine Learning Engineer creates, edits, analyzes, debugs, models, and supervises the development of machine learning models using programming languages such as Python or C++ and machine learning libraries such as Keras or TensorFlow.

How much does a Machine Learning Engineer make per year?

Figure: Average Machine Learning Engineer Income. [1]

The average annual income of a Machine Learning Engineer in the United States is between $112,000 and $157,000 with a median of $131,000 per year according to multiple data sources such as Indeed, Glassdoor, Salary.com, and Payscale.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Deep Learning Engineer

Deep learning is a subset of machine learning using artificial neural network (ANN) models with more than three layers. ANNs are inspired by the behavior of the human brain to enable machines to learn — with the idea to connect neurons with each other via artificial “synapses” and learning is modeled as the collective weights and magnitude of the neural connections.

The average annual income of a Deep Learning Engineer in the United States is between $124,000 and $148,000 based on multiple sources such as Indeed, Ziprecruiter, and Salary.com.

A Deep Learning Engineer creates, edits, analyzes, debugs, and supervises the development of artificial neural networks (ANN) with multiple layers written in programming environments such as Python, TensorFlow, or Keras.

Do you want to become a Deep Learning Engineer? Here’s a step-by-step learning path I’d propose to get started with Deep Learning :

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Computer Science Researcher

A computer science researcher and scientist identifies and answers open research questions in computer science. They apply scientific reasoning and research techniques to push the state-of-the-art forward in various fields such as machine learning, distributed systems, databases, algorithms, and data science.

Six of the most common activities of computer science researchers, based on my own experience:

  • reading research papers,
  • thinking about research questions and problems,
  • identifying research gaps and discussing them with their peers,
  • creating code and software systems for evaluation purposes,
  • writing research papers, and
  • presenting those scientific results at conferences and in journals.

The median annual income (=50th percentile) of a computer science researcher was $131,490 in May 2021. The bottom 10% (=10th percentile) of computer science researchers earned less than $74,210 and the top 10% (=90th percentile) earned more than $208,000.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Distributed Systems Developer

A distributed system is a computer system spread across multiple computing devices connected via a communication network. Each participating device takes over part of the overall work performed by the system. By means of the collaboration of individual units, the system can provide services that each individual system component couldn’t provide on its own.

Some examples of distributed systems are:

  • Ethereum is a distributed system of Ethereum nodes connected via the Internet and a specific communication protocol.
  • Bitcoin is a distributed system of Bitcoin nodes connected via the Internet and a specific communication protocol as defined by the open-source Bitcoin protocol.
  • The World Wide Web is a distributed system of servers connected via IP to provide a coherent web experience via browsers and an HTML-like web experience.

A distributed systems engineer designs, implements, and debugs distributed systems for data storage, crypto & web3, or analytics. The idea is to design a distributed system that can provide a service to users that couldn’t be provided by a centralized system (e.g., providing a decentralized, censorship-free monetary network).

Average Income of a Distributed Systems Engineer in the US by Source.
Figure: Average Income of a Distributed Systems Engineer in the US by Source. [1]

The average annual income of a Distributed Systems Engineer in the United States is between $97,000 and $169,656, with an average of $126,894 and a statistical median of $130,000 per year.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

AWS Developer

An AWS Developer creates, edits, analyzes, debugs, and supervises the development of software written for the AWS cloud services that support many programming languages such as Python or Java.

Learning AWS is one of the most important, most sought-after, and most profitable things you can do as a developer!

The average annual income of an AWS Developer in the United States is $122,799 per year according to ZipRecruiter. Another estimate is provided by Glassdoor that provides data supporting an annual income of AWS Developers of $115,000 per year.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Android App Developer

An Android app developer is a programmer who focuses on software creation for mobile devices such as smartphones or wearables using the Android operating system.

As an Android app developer, your skill set varies depending on the concrete set of applications you’re working on. However, these skills will proof useful no matter what, and most successful Android app developers have these seven skills:

  1. General programming skills (e.g., Java, C++, Python)
  2. Specific app framework skills (e.g., Flutter)
  3. Distributed systems skills
  4. Web development skills (e.g., HTML, CSS, JavaScript)
  5. Design skills (e.g., Photoshop)
  6. Security skills (e.g., SSL encryption)
  7. Soft skills (e.g., communication, presentation, marketing)
Figure: Average Income of an Android App Developer in the US by Source. [1]

The average annual income of an Android App Developer in the United States is between $85,000 and $126,577 with an average of $106,923 and a statistical median of $107,343 per year.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

C++ Developer

As a C++ developer, you create software in the programming language C++ which is among the most widely used programming languages. For example, Google, Amazon, Facebook all employ a large number of C++ developers.

The average annual income of a C++ Developer is between $45,000 and $140,000 according to PayScale with an average of $67,473 in the US based on 31 salary reports (source). But Indeed.com reports an even higher annual C++ developer income of $116,925 based on 480 salaries reported (source).

Do you want to become a C++ Developer? Here’s a step-by-step learning path I’d propose to get started with C++:

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Full-Stack Developer

A full-stack web developer works both with back-end and front-end web technologies.

  • The back-end consists of the webserver infrastructure, databases, and code function integration to facilitate a smooth and secure serving of user requests.
  • The front-end focuses on the graphical user interface (GUI) of the website using HTML, CSS, and JavaScript with the goal of setting up the whole technology stack to enable users to view and interact with the website.

Full-stack developers have skills in all those fields so they often take crucial roles in overseeing the technical implementation of large web projects.

The average annual income of a Full-Stack Web Developer in the United States is between $79,584 and $108,984 with an average income of $98,454 and a median income of $99,274 per year according to our meta-study of 7 aggregated data sources such as Glassdoor and Indeed.

Figure: Average Income of a Full-Stack Developer in the US by Source.
Figure: Average Income of a Full-Stack Developer in the US by Source. [1]

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Solidity Developer

The average annual income of a Solidity Developer is between $60,000 and $180,000 with an average of $100,000 per year (source).

Here are a couple of example jobs for Solidity engineers:

Let’s have a look at Google trends to find out how interest evolves over time (source):

A Solidity developer creates, edits, analyzes, and debugs code in the Solidity programming language used to develop smart contracts for modern Blockchain ecosystems such as Ethereum.

Do you want to become a Solidity Developer? Here’s a step-by-step learning path I’d propose to get started with Solidity:

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Python Developer

A Python developer is a programmer who creates software in the Python programming language. Python developers are often involved in data science, web development, and machine learning applications.

A Python developer earns $65,000 (entry-level), $82,000 (mid-level), or $114,000 (experienced) per year in the US according to Indeed. (source)

Do you want to become a Python Developer? Here’s a step-by-step learning path I’d propose to get started with Python:

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Crypto Bot Developer

The average annual income of a Crypto Trading Bot Developer is similar to algorithmic traders of $104,422 (source). However, due to the novelty of the industry, there’s little official data. If you assume an hourly rate of $50 and an annual 2000 hours worked, the annual income of a crypto trading bot developer would be $100,000.

Let’s have a look at Google trends to find out how interest evolves over time (source):

Yes, this definitely is an interesting industry for programmers to make $50 per hour and more!

Trading bots are software programs that talk directly to financial exchanges. Crypto trading bots are programs that talk to crypto exchanges. A crypto bot developer develops those programs. Crypto trading bot developers tend to be very proficient in trading, financial algorithms, APIs, and web services.

Do you want to become a Crypto Trading Bot Developer? Here’s a learning path I’d propose in five steps to get started:

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Azure Developer

An Azure Developer creates, designs, edits, analyzes, debugs, deploys, and supervises the development of cloud applications written for the Azure cloud and development ecosystem.

The average annual income of an Azure Developer in the United States is $122,031 per year according to Talent.com. Entry-level Azure Developers start with $107,250 per year. Top earners make $127,000 and more in the US according to Glassdoor!

Let’s have a look at Google trends to find out how interest evolves over time (source):

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Game Developer

Game Development is the art of creating games that involves multiple development stages such as game concept generation, game design, game development, game testing, game building, game deployment, and game release.

Game developers and video game developers create code for games in a variety of formats such as desktop-based games (Windows, macOS, Linux), consoles (PS2), web browsers (Chrome, Safari), crypto (Ethereum, Polygon/Optimism, Solana), and mobile phone (iOS, Android).

Figure: Average Income of a Game Developer in the US by Source. [1]

The average annual income of a Game Developer in the United States is between $64,053 and $115,846 with an average of $89,889 and a median of $92,061 per year.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

JavaScript Developer

A JavaScript developer creates dynamic web applications focusing mostly on the front-end logic—but recently some back-end JavaScript frameworks emerged as well. If you like web development and programming user interfaces, you’ll love the work as a JavaScript developer.

The average annual income of a JavaScript Developer is between $62,000 and $118,000 with an average of $84,000 per year according to Daxx.com and PayScale (source).

Do you want to become a JavaScript Developer? Here’s a step-by-step learning path I’d propose to get started with JavaScript and web development:

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Quant Developer

A quantitative developer (i.e., Quant) is a financial programmer focused on financial modeling and quantitative finance and trading.

Quants use their profound knowledge of

  • statistics and math,
  • finance,
  • data structures,
  • algorithms,
  • machine learning,
  • scientific computing,
  • data science,
  • chart technique, and
  • data visualization

to create models for financial prediction, backtesting, analysis, and implementation of trading and financial applications (e.g., for risk management).

Figure: Average Income of a Quant Developer in the US by Source.
Figure: Average Income of a Quant Developer in the US by Source. [1]

The expected annual income of a Quantitative Developer (Quant) in the United States is between $86,528 and $170,000 per year, with an average annual income of $127,375 per year and a median income of $136,321 per year.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Security Engineer

A security engineer is a “white-hat hacker”, i.e., an IT professional who analyzes computer systems and computer networks to ensure they are running securely. This involves proactive analysis and understanding of possible security threats and attack vectors and designing the system to minimize the exposure to these threats.

Figure: Average Income of a Security Engineer in the US by Source. [1]

The average annual income of a Security Engineer in the United States is between $75,732 and $144,874, with an average of $108,851 and a statistical median of $105,928 per year.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Test Automation Engineer

A Test Automation Engineer is a software developer who creates automated software tests for existing or new applications. Testing is a crucial phase in the software development cycle to learn about bugs, usability, and security issues and fix them before deploying an application in the real world.

Average Income of a Test Automation Engineer in the US by Source.
Figure: Average Income of a Test Automation Engineer in the US by Source. [1]

The expected annual income of a Test Automation Engineer in the United States is between $74,821 and $120,000 per year, with an average annual income of $95,285 per year and a median income of $93,657 per year.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

VHDL Developer

A VHDL Developer developer creates, edits, analyzes, debugs, and supervises the development of software written in the VHDL programming language. VHDL is the other popular hardware description language. In fact, most professionals who know VHDL also know Verilog.

The annual income of a VHDL developer in the US can be as high as $169,500 and as low as $32,000 according to Ziprecruiter (source). Most VHDL developer make between $65,000 (25th percentile) and $135,000 (75th percentile) with top earners (90th percentile) making $157,500 annually across the United States.

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Frontend Developer

A web developer is a programmer who specializes in the development of websites or applications viewed on web browsers, mobile devices, and large desktop screens that are transported over private or public networks such as the Internet.

A front-end web developer focuses on the graphical user interface (GUI) of the website using HTML, CSS, and JavaScript with the goal of setting up the whole technology stack to enable users to view and interact with the website.

The average annual income of a Front-end Web Developer in the United States is between $61,194 and $119,224 with an average income of $89,683 and a median income of $90,499 per year according to our meta-study of 8 aggregated data sources such as Glassdoor and Indeed.

The following graphic shows the individual data sources, as well as the average and median income level of a web developer in the US:

Average Income of a Front-End Web Developer in the US by Source
Figure: Average Income of a Front-End Web Developer in the US by Source. [1]

🌎 Learn More: You can find out more about this career path in my full article on the Finxter blog.

Summary

This article has shown you the top career choices for developers in the upcoming decade:

Feel free to join our free email academy to stay tuned and informed in many of those fields!

Programmer Humor – Blockchain

“Blockchains are like grappling hooks, in that it’s extremely cool when you encounter a problem for which they’re the right solution, but it happens way too rarely in real life.” source xkcd

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

🚀 If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

8 Best Ways to Check the Package Version in Python

5/5 – (1 vote)

In this article, I’ll show you how to check the version of a Python module (package, library).

These are the eight best ways to check the version of a Python module:

  • Method 1: pip show my_package
  • Method 2: pip list
  • Method 3: pip list | findstr my_package
  • Method 4: my_package.__version__
  • Method 5: importlib.metadata.version
  • Method 6: conda list
  • Method 7: pip freeze
  • Method 8: pip freeze | grep my_package

Let’s dive into some examples for each of those next!

Method 1: pip show

To check which version of a given Python library, say xyz, is installed, use pip show xyz or pip3 show xyz. For example, to check the version of your NumPy installation, run pip show numpy in your CMD/Powershell (Windows), or terminal (macOS/Linux/Ubuntu).

This will work if your pip installation is version 1.3 or higher—which is likely to hold in your case because pip 1.3 was released a decade ago in 2013!!

Here’s an example in my Windows Powershell for NumPy: I’ve highlighted the line that shows that my package version is 1.21.0:

PS C:\Users\xcent> pip show numpy
Name: numpy
Version: 1.21.0
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: c:\users\xcent\appdata\local\programs\python\python39\lib\site-packages
Requires:
Required-by: pandas, matplotlib

In some instances, this will not work—depending on your environment. In this case, try those commands before giving up:

python -m pip show numpy
python3 -m pip show numpy
py -m pip show numpy
pip3 show numpy

Of course, replace “numpy” with your particular package name.

Method 2: pip list

To check the versions of all installed packages, use pip list and locate the version of your particular package in the output list of package versions sorted alphabetically.

This will work if your pip installation is version 1.3 or higher.

Here’s an example in my Windows Powershell, I’ve highlighted the line that shows that my package version is 1.21.0:

PS C:\Users\xcent> pip list
Package Version
--------------- ---------
beautifulsoup4 4.9.3
bs4 0.0.1
certifi 2021.5.30
chardet 4.0.0
cycler 0.10.0
idna 2.10
kiwisolver 1.3.1
matplotlib 3.4.2
mss 6.1.0
numpy 1.21.0
pandas 1.3.1
Pillow 8.3.0
pip 21.1.1
pyparsing 2.4.7
python-dateutil 2.8.1
pytz 2021.1
requests 2.25.1
setuptools 56.0.0
six 1.16.0
soupsieve 2.2.1
urllib3 1.26.6

In some instances, this will not work—depending on your environment. Then try those commands before giving up:

python -m pip list
python3 -m pip list
py -m pip list
pip3 list 

Method 3: pip list + findstr on Windows

To check the versions of a single package on Windows, you can chain pip list with findstr xyz using the CMD or Powershell command: pip3 list | findstr numpy to locate the version of your particular package xyz in the output list of package versions automatically.

Here’s an example for numpy:

pip3 list | findstr numpy 1.21.0

Method 4: Library.__version__ Attribute

To check your package installation in your Python script, you can also use the xyz.__version__ attribute of the particular library xyz. Not all packages provide this attribute but as it is recommended by PEP, it’ll work for most libraries.

Here’s the code:

import numpy
print(numpy.__version__)
# 1.21.0

Here’s an excerpt from the PEP 8 docs mentioning the __version__ attribute.

PEP 8 describes the use of a module attribute called __version__ for recording “Subversion, CVS, or RCS” version strings using keyword expansion. In the PEP author’s own email archives, the earliest example of the use of an __version__ module attribute by independent module developers dates back to 1995.”

Method 5: importlib.metadata.version

The importlib.metadata library provides a general way to check the package version in your Python script via importlib.metadata.version('xyz') for library xyz. This returns a string representation of the specific version. For example, importlib.metadata.version('numpy') returns 1.21.0 in my current environment.

Here’s the code:

import importlib.metadata
print(importlib.metadata.version('numpy'))
# 1.21.0

Method 6: conda list

If you have created your Python environment with Anaconda, you can use conda list to list all packages installed in your (virtual) environment. Optionally, you can add a regular expression using the syntax conda list regex to list only packages matching a certain pattern.

How to list all packages in the current environment?

conda list

How to list all packages installed into the environment 'xyz'?

conda list -n xyz

Regex: How to list all packages starting with 'py'?

conda list '^py'

Regex: How to list all packages starting with 'py' or 'code'?

conda list '^(py|code)'

Method 7: pip freeze

The pip freeze command without any option lists all installed Python packages in your environment in alphabetically order (ignoring UPPERCASE or lowercase). You can spot your specific package if it is installed in the environment.

pip freeze

Output from my local Windows environment with PowerShell (strange packages I know) ;):

PS C:\Users\xcent> pip freeze
asn1crypto==1.5.1
et-xmlfile==1.1.0
openpyxl==3.0.10

For example, I have the Python package openpyxl installed with version 3.0.10.


You can modify or exclude specific packages using the options provided in this screenshot:

Method 8: pip freeze + grep on Linux/Ubuntu/macOS

To check the versions of a single package on Linux/Ubuntu/macOS, you can chain pip freeze with grep xyz using the CMD or Powershell command: pip freeze | grep xyz to programmatically locate the version of your particular package xyz in the output list of package versions.

Here’s an example for numpy:

pip freeze | grep scikit-learn
scikit-learn==0.17.1

Related Questions

Check Package Version Python

How to check package version in Python?

To check which version of a given Python package is installed, use pip show my_package. For example, to check the version of your NumPy installation, run pip show numpy in your CMD/Powershell (Windows), or terminal (macOS/Linux/Ubuntu).

pip show my_package

Check Package Version Linux

How to check my package version in Linux?

To check which version of a given Python package is installed, use pip show my_package. For example, to check the version of your NumPy installation, run pip show numpy in your Linux terminal.

pip show my_package

Check Package Version Ubuntu

How to check my package version in Ubuntu?

To check which version of a given Python package is installed, use pip show my_package. For example, to check the version of your NumPy installation, run pip show numpy in your Ubuntu terminal/shall/bash.

pip show my_package

Check Package Version Windows

How to check package version on Windows?

To check which version of a given Python package is installed, use pip show my_package. For example, to check the version of your NumPy installation, run pip show numpy in your Windows CMD, command line, or PowerShell.

pip show my_package

Check Package Version Mac

How to check package version on macOS?

To check which version of a given Python package is installed, use pip show my_package. For example, to check the version of your NumPy installation, run pip show numpy in your macOS terminal.

pip show my_package

Check Package Version Jupyter Notebook

How to check package version in your Jupyter Notebook?

To check which version of a given Python package is installed, add the line !pip show my_package to your notebook cell where you want to check. Notice the exclamation mark prefix ! that allows you to run commands in your Python script cell. For example, to check the version of your NumPy installation, run !pip show numpy in your macOS terminal.

!pip show my_package

For example, this is a screenshot on how this looks for numpy in a Jupyter Notebook:

Check Package Version Terminal

How to check package version in my terminal?

To check which version of a given Python package is installed, use pip show my_package. For example, to check the version of your NumPy installation, run pip show numpy in your terminal.

pip show my_package

Check Package Version Conda/Anaconda

How to check package version in my conda installation?

Use conda list 'my_package' to list version information about the specific package installed in your (virtual) environment.

conda list 'my_package'

Check Package Version with PIP

How to check package version with pip?

You can use multiple commands to check the package version with PIP such as pip show my_package, pip list, pip freeze, and pip list.

pip show my_package
pip list
pip freeze
pip list

Check Package Version in VSCode or PyCharm

How to check package version in VSCode or PyCharm?

Integrated Development Environments (IDEs) such as VSCode or PyCharm provide a built-in terminal where you can run pip show my_package to check the current version of my_package in the specific environment you’re running the command in.

pip show my_package
pip list
pip freeze

You can type any of those commands in your IDE terminal like so:

pip IDE check package version

Summary

In this article, you’ve learned those best ways to check a Python package version:

  • Method 1: pip show my_package
  • Method 2: pip list
  • Method 3: pip list | findstr my_package
  • Method 4: my_package.__version__
  • Method 5: importlib.metadata.version
  • Method 6: conda list
  • Method 7: pip freeze
  • Method 8: pip freeze | grep my_package

Thanks for giving us your valued attention — we’re grateful to have you here! 🙂


Programmer Humor

There are only 10 kinds of people in this world: those who know binary and those who don’t.
👩🧔‍♂️
~~~

There are 10 types of people in the world. Those who understand trinary, those who don’t, and those who mistake it for binary.
👩🧔‍♂️👱‍♀️

Posted on Leave a comment

How to Multiply List Elements by a Number – Top 5 Ways

5/5 – (1 vote)

Problem Formulation and Solution Overview

In this article, you’ll learn how to multiply List Elements by a Number in Python.

This example multiples the first five (5) Prime Numbers by two (2) and return the result.


💬 Question: How would we write Python code to multiply the list elements?

We can accomplish this task by one of the following options:


Method 1: Use List Comprehension

This method uses List Comprehension to apply a mathematical operation to each element and return the result.

prime_nums = [2, 3, 5, 7, 11]
mult_result = [x * 2 for x in prime_nums]
print(mult_result)

Above declares the first (5) Prime Numbers and saves this List to prime_nums. Next, List Comprehension loops through each element and applies the multiplication operation to each. The output saves to mult_result and is output to the terminal.

[4, 6, 10, 14, 22]

Method 2: Use Pandas tolist()

This method requires an additional library to be imported, Pandas, to use the tolist() function.

import pandas as pd prime_nums = [2, 3, 5, 7, 11]
mult_result = pd.Series(prime_nums)
mult_result = (mult_result*2).tolist()
print(mult_result)

Above, imports the Pandas Library. Click here if this requires installation. Then, the first (5) Prime Numbers are declared and saved to prime_nums.

Next, prime_nums is passed as an argument to the pd.Series() function and returns mult_result. The output of mult_result at this point is shown below.

0 2
1 3
2 5
3 7
4 11
dtype: int64

Now, we need to convert this output to a list (tolist()) and apply the multiplication operation to each element. The results save to mult_result and are output to the terminal.

[4, 6, 10, 14, 22]

Method 3: Use map and lambda Functions

This method wraps the map(), and lambda functions inside a Python List and calculates the results.

prime_nums = [2, 3, 5, 7, 11]
mult_result = list(map(lambda x: x*2, prime_nums))
print(mult_result)

Above declares the first (5) Prime Numbers and saves them to prime_nums. The next line does the following:

  • The map() function is passed the lambda() function as an argument (map(lambda x: x*2, prime_nums)).
  • The lambda performs the multiplication operation to each element of prime_nums and saves it to map() as an object similar to below.
    <map object at 0x000001DC99CBBBB0>
  • The map() object is then converted to a List.
  • The results save to mult_result.

Then, mult_result is output to the terminal.

[4, 6, 10, 14, 22]

Method 4: Use Numpy Array()

This method requires an additional library to be imported, NumPy, to use the np.array() function.

import numpy as np prime_nums = [2, 3, 5, 7, 11]
the_result = list(np.array(prime_nums) * 2)
print(the_result)

Above, imports the NumPy Library. Click here if this requires installation. Then the first (5) Prime Numbers are declared and saved to prime_nums.

Next, prime_nums is passed as an argument to np.array() where the multiplication operation is applied to each element. Then, this is converted to a List, saved to the_result and output to the terminal.

[4, 6, 10, 14, 22]

Method 5: Use Slicing

This method uses Python’s infamous Slicing! No overhead, and a very pythonic way to resolve the issue.

prime_nums = [2, 3, 5, 7, 11]
prime_nums[:] = [x * 2 for x in prime_nums]
print(prime_nums)

Above declares the first (5) Prime Numbers and saves them to prime_nums.

Then slicing is applied and used in conjunction with List Comprehension to apply the multiplication operation to each element. The results save back to prime_nums and are output to the terminal.

[4, 6, 10, 14, 22]

🌟A Finxter Favorite!


Summary

These methods of multiplying list elements by a number should give you enough information to select the best one for your coding requirements.

Good Luck & Happy Coding!


Programmer Humor

👱‍♀️ Programmer 1: We have a problem
🧔‍♂️ Programmer 2: Let’s use RegEx!
👱‍♀️ Programmer 1: Now we have two problems

… yet – you can easily reduce the two problems to zero as you polish your “RegEx Superpower in Python“. 🙂

Posted on Leave a comment

How to Fix TypeError: unhashable type: ‘list’

5/5 – (2 votes)

The TypeError: unhashable type: 'list' usually occurs when you try to use a list object as a set element or dictionary key and Python internally passes the unhashable list into the hash() function. But as lists are mutable objects, they do not have a fixed hash value. The easiest way to fix this error is to use a hashable tuple instead of a non-hashable list as a dictionary key or set element.

We’ll show how this is done in the remaining article. The last method is a unique way to still use lists in sets or dictionary keys that you likely won’t find anywhere else, so keep reading and learn something new! 🐝

Problem Formulation and Explanation

💬 Question: How to fix the TypeError: unhashable type: 'list' in your Python script?

There are two common reasons for this error:

  • You try to use a list as a dictionary key, or
  • You try to use a list as a set element.

This is not a trivial problem because Python lists are mutable and, therefore, not hashable.

Here’s the first example:

my_list = [1, 2, 3]
my_dict = {}
my_dict[my_list] = 'hello Finxters!' '''
Traceback (most recent call last): File "C:\Users\xcent\Desktop\code.py", line 3, in <module> my_dict[my_list] = 'hello Finxters!'
TypeError: unhashable type: 'list' '''

And here’s the second example:

my_list = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
my_set = set(my_list) '''
Traceback (most recent call last): File "C:\Users\xcent\Desktop\code.py", line 2, in <module> my_set = set(my_list)
TypeError: unhashable type: 'list' '''

As you’ve seen in the previous two code snippets, the TypeError: unhashable type: 'list' usually occurs when you try to use a list object as a set element or dictionary key.

But let’s dive deeper to find the real reason for the error:

🪲 Minimal Reproducible Error Example: Lists are mutable objects so they do not have a fixed hash value. In fact, the error can be reproduced most easily when calling hash(lst) on a list object lst.

This is shown in the following minimal example that causes the error:

hash([1, 2, 3])

The output is the error message:

Traceback (most recent call last): File "C:\Users\xcent\Desktop\code.py", line 1, in <module> hash([1, 2, 3])
TypeError: unhashable type: 'list'

Because you cannot successfully pass a list into the hash() function, you cannot directly use lists as set elements or dictionary keys.

But let’s dive into some solutions to this problem!

Method 1: Use Tuple Instead of List as Dictionary Key

The easiest way to fix the TypeError: unhashable type: 'list' is to use a hashable tuple instead of a non-hashable list as a dictionary key. For example, whereas d[my_list] will raise the error, you can simply use d[tuple(my_list)] to fix the error.

Here’s an example of what you can do instead:

my_list = [1, 2, 3]
my_dict = {}
my_dict[tuple(my_list)] = 'hello Finxters!' print(my_dict)
# {(1, 2, 3): 'hello Finxters!'}

The error may also occur when you try to use a list as a set element. Next, you’ll learn what to do in that case:

Method 2: Use Tuple Instead of List as Set Element

To fix the TypeError: unhashable type: 'list' when trying to use a list as a set element is to use a hashable tuple instead of a non-hashable list. For example, whereas set.add([1, 2]) will raise the error, you can simply use set.add((1, 2)) or set.add(tuple([1, 2])) to fix the error.

Here’s a minimal example:

my_set = set() # Error: my_set.add([1, 2]) # This is how to resolve the error:
my_set.add((1, 2))
# Or: my_set.add(tuple([1, 2])) print(my_set)
# {(1, 2)}

If you want to convert a list of lists to a set, you can check out my detailed tutorial on the Finxter blog:

🌎 Related Tutorial: How to convert a list of lists to a set?

Method 3: Use String Representation of List as Set Element or Dict Key

To fix the TypeError: unhashable type: 'list', you can also use a string representation of the list obtained with str(my_list) as a set element or dictionary key. Strings are hashable and immutable, so Python won’t raise the error when using this approach.

Here’s an example:

my_list = [1, 2, 3] # 1. Use str repr of list as dict key:
d = {}
d[str(my_list)] = 'hello Finxters' # 2. Use str repr of list as set element:
s = set()
s.add(str(my_list))

In both cases, we used the string representation of the list instead of the list itself. The string is immutable and hashable and it fixes the error.

But what if you really need a mutable set or dictionary key? Well, you shouldn’t but you can by using this approach:

Method 4: Create Hashable Wrapper List Class

You can still use a mutable list as a dictionary key, set element, or argument of the hash() function by defining a wrapper class, say HackedList, that overrides the __hash__() dunder method.

Python’s built-in hash(object) function takes one object as an argument and returns its hash value as an integer. You can view this hash value as a unique fingerprint of this object.

The Python __hash__() method implements the built-in hash() function.

Here’s the minimal code example that creates a wrapper class HackedList that overrides the __hash__() dunder method so you can use an instance of HackedList as a dictionary key, set element, or just as input to the hash() function:

my_list = [1, 2, 3] class HackedList: def __init__(self, lst): self.lst = lst def __hash__(self): return len(self.lst) my_hacked_list = HackedList(my_list) # 1. Pass hacked list into hash() function:
print(hash(my_hacked_list)) # Output: 3 # 2. Use hacked list as dictionary key:
d = dict()
d[my_hacked_list] = 'hello Finxters' # 3: Use hacked list as set element:
s = set()
s.add(my_hacked_list)

Here’s the content of the dictionary and set defined previously:

{<__main__.HackedList object at 0x0000016CFB0BDFA0>: 'hello Finxters'}
{<__main__.HackedList object at 0x0000016CFB0BDFA0>}

If you want to fix the ugly output, you can additionally define the __str__() and __repr__() magic methods like so:

my_list = [1, 2, 3] class HackedList: def __init__(self, lst): self.lst = lst def __hash__(self): return len(self.lst) def __str__(self): return str(self.lst) def __repr__(self): return str(self.lst) my_hacked_list = HackedList(my_list) # 1. Pass hacked list into hash() function:
print(hash(my_hacked_list)) # Output: 3 # 2. Use hacked list as dictionary key:
d = dict()
d[my_hacked_list] = 'hello Finxters' # 3: Use hacked list as set element:
s = set()
s.add(my_hacked_list) print(d)
print(s)

Beautiful output:

{[1, 2, 3]: 'hello Finxters'}
{[1, 2, 3]}

Summary

The five most Pythonic ways to convert a list of lists to a set in Python are:

Feel free to check out more free tutorials and cheat sheets for learning and improving your Python skills in our free email academy:


Nerd Humor

Oh yeah, I didn’t even know they renamed it the Willis Tower in 2009, because I know a normal amount about skyscrapers.xkcd (source)
Posted on Leave a comment

A Beginner’s Guide to Forex Trading Bots and Python – Practical Projects

Rate this post

🌎 Full Course: Check out the full beginner course on Forex trading on this Finxter page (5 video lessons).

As a Python beginner, or anything else new that we dive into, everything is fresh and exciting for a while and we have no problem staying motivated to do the work and move ahead.

It’s no wonder you can stay fired up when you are learning the most popular language, in a field that looks promising for years to come, and its innovations will shape the future. That’s exciting!

There’s a book that summarizes the next step in your journey, whether it be Python, Forex, business, freelancing, or anything else. It deals with what most people call, “being at the intermediate level.”

It’s called “The Dip”, by Seth Godin. Like most “self-help” type books, even though this one is only around 100 pages, it could have been done in 10 or 15. In this case though, the author gets an “A+” for the concept.

The idea that after the honeymoon, there will be a period of uncertain struggle on where to go next. Python has the mother of all dips.

To wrap up this beginner’s guide, I want to help you find your way through the dip and come out the other side a success.  That “way” is in the title – “Practical Projects.”

Freelance and Get Some Work that Uses Your Python Skills

Doing a project for someone who doesn’t know how, or have the time to do it themselves, is a great way to put your Python skills to the test.

The great thing about freelancing your skills, is you never know what someone is going to need, and this can give you a great variety of projects.

PRO TIP: Don’t wait until you “feel” ready.  You will never feel ready – what you need is confidence – by doing some real work, learning from your mistakes, and not making them again.

Getting started on a platform like Upwork is simple and you will know which projects you can handle, and which ones you can’t – besides, it’s good for you to take a couple that will push your skills and require you to learn how to complete them.

Here are a few more Ideas for some real-world projects:

Data Analysis Projects with Python and Its Libraries

We went through some simple examples of what you can do with data earlier in the series.  Let’s break down a sample in detail:

  • Think about a subject that interests you, and where you can find data collections for that topic.
  • Do a search and find some downloadable files from their collections.
  • Pick a file that suits your project, download the CSV, (I hope you’re using Anaconda and Jupyter), clean it up and organize it, then see what types of patterns, if any, you can identify.  I grabbed historical data on interest rates from the Fed’s website for my last analysis.  There is so much information out there for free that we will never be able to cover a tiny percentage of it.  So narrow it down to your specific needs.

Projects and Tests for Forex Trading and Python

  • Form a hypothesis – “Is there a correlation between the EUR/USD and WTI?”  In light of recent global events, one would be safe in questioning crude oil’s affect on the entire world.
  • Do a comparison – Do you remember in a previous lesson when I demonstrated how to overlay one instrument with another on your charts? This is a simple way to look for correlation. Remember, correlation can be positive or negative.
  • Look to see if one or the other seems to “lead” its partner.  This can be a great way to see into the future – so to speak.
  • If your theory looks promising, question if there is a way to quantify and automate the information using Python.  This would also be a good time to start digging into machine learning.  Use Python to streamline the process and set alerts.

This is a hypothetical situation I created as an example.  Do not trade any theory from anyone without thoroughly testing it yourself.

Sources for Datasets

  • Governments collect data and make it available to the public on their websites.  Records of everything from NFP to GDP, and weather events can be found with a little effort.
  • Central banks, the IMF, and the World Bank also issue reports and data on a variety of economic indicators and predictions created by their own experts.
  • Be wary of “advice” sites that are trying to sell you something – look for facts gleaned from statistics and research instead.

Get on Board with a Broker and Get a Robot

We have already discussed how to choose a broker, and did some analysis together on the subject.  With the regulations in place these days, it’s really easy to find one that is legit.  It will boil down to personal preference in the end.  Make sure you feel comfortable with your choice, and that they have responsive customer service so you can communicate easily.

As a beginner, just like with Python, it’s important to start getting some experience while you’re learning to code your own bots. Using a ready-made bot on a demo account is the best way to get going and see if automated trading is right for you.

REMEMBER: Don’t make it all about the money just yet – the knowledge you’re getting in the process is the real value.  If you have followed the steps in this series, you should already be on your way to safely making money with Python.

Bonus for Finishing the Beginner Series on Forex Bots and Python

For all of you who have stuck it out until the end of our beginner series, I’m going to give you some analysis that will demonstrate the many different ways to go about your trade planning – they’re endless, which is what makes Forex so interesting.  No matter your style, you can find a system that fits.

In the accompanying video, I’m going to give some high-level tips and analysis on the EUR/USD pair that we have been using in the series, and explain what actually makes currency values change.

Check out the video, and it has been a pleasure sharing this information with you.

Here’s to good trading and good luck!

Posted on Leave a comment

How to Get the Last N Rows of a Pandas DataFrame?

Rate this post

In this tutorial we will unearth the solutions to three commonly asked questions that users come across while dealing with huge sets of data.

Problem Formulation

Given: Consider the following csv file (Note: You need to use it as a Pandas DataFrame).

import pandas as pd df = pd.read_csv('countries.csv')
print(df)
 Country Capital Population Area
0 Germany Berlin 84,267,549 348,560
1 France Paris 65,534,239 547,557
2 Spain Madrid 46,787,468 498,800
3 Italy Rome 60,301,346 294,140
4 India Delhi 1,404,495,187 2,973,190
5 USA Washington 334,506,463 9,147,420
6 China Beijing 1,449,357,022 9,388,211
7 Poland Warsaw 37,771,789 306,230
8 Russia Moscow 146,047,418 16,376,870
9 England London 68,529,747 241,930

Here’s the list of the questions that we will be focusing upon in this article:

  • How to get the last N rows of a Pandas DataFrame?
  • How to get last N rows from last N columns of a Pandas DataFrame?
  • How to read last N rows of a large csv file in Pandas?

Recommended Read: How to Select Rows From a DataFrame Based on Column Values?

Without further delay, let us dive into the solutions to the first question and learn how to get the last N rows of a Pandas DataFrame.

Method 1: Using iloc

Approach: Use the iloc property as pandas.DataFrame.iloc[-n:].

The iloc property is used to get or set the values of specified indices. Select the last n rows using the square bracket notation syntax [-n:] with the iloc property. Here, -n represents the index of the last n rows of the given pandas DataFrame.

Code:

import pandas as pd df = pd.read_csv('countries.csv')
rows = df.iloc[-5:]
print(rows)

Output:

 Country Capital Population Area
5 USA Washington 334,506,463 9,147,420
6 China Beijing 1,449,357,022 9,388,211
7 Poland Warsaw 37,771,789 306,230
8 Russia Moscow 146,047,418 16,376,870
9 England London 68,529,747 241,930

Method 2: Using tail()

Approach: Use the pandas.DataFrame.tail(n) to select the last n rows of the given DataFrame.

The tail(n) method returns n number of methods from the bottom end of the DataFrame. Here, n represents an integer that denotes the number of rows you want to fetch from the bottom end of the DataFrame.

Code:

import pandas as pd df = pd.read_csv('countries.csv')
rows = df.tail(5)
print(rows)

Output:

 Country Capital Population Area
5 USA Washington 334,506,463 9,147,420
6 China Beijing 1,449,357,022 9,388,211
7 Poland Warsaw 37,771,789 306,230
8 Russia Moscow 146,047,418 16,376,870
9 England London 68,529,747 241,930

Well, that brings us to the next question in line – “How to get the last N rows from last N columns of a Pandas DataFrame?”

Method 1: Integer Based Indexing

Approach: Call pandas.DataFrame.iloc[-n:, -m:] to display last n rows from the last m columns of the given DataFrame.

Code: In the following code snippet we will fetch the last 5 rows from the last 2 columns, i.e., Population and Area.

import pandas as pd df = pd.read_csv('countries.csv')
rows = df.iloc[-5:, -2:]
print(rows)

Output:

 Population Area
5 334,506,463 9,147,420
6 1,449,357,022 9,388,211
7 37,771,789 306,230
8 146,047,418 16,376,870
9 68,529,747 241,930

Method 2: Name Based Indexing

In case, you happen to know the names of the specific columns and you want to get the last N records from the DataFrame from those columns then you can follow a two step process.

  • Call the Pandas.DataFrame.loc(:, 'start_column_name':'end_column_name') selector. It allows you to use slicing on column names instead of integer identifiers which can be more comfortable.
  • .loc is for label based indexing. Hence, the negative indices are not found and reindexed to NaN. Thus, to deal with this you have to use the tail() method to extract the last N records from the selected columns.

Code: The following code snippet shows how you can use the column names and fetch the corresponding values from the last 5 rows of the given Dataframe.

import pandas as pd df = pd.read_csv('countries.csv')
rows = df.loc[:, 'Population':'Area']
print(rows.tail(5))

Output:

 Population Area
5 334,506,463 9,147,420
6 1,449,357,022 9,388,211
7 37,771,789 306,230
8 146,047,418 16,376,870
9 68,529,747 241,930

Last but not least, let us solve the third and final problem of today’s tutorial – “How to read last N rows of a large csv file in Pandas?

Unfortunately, read_csv() does not facilitate us with any parameter that allows you to directly read the last N lines from a file. This can be a troublesome issue to handle when you are dealing with large datasets.

Thus, a workaround to this problem is to first find out the total number of lines/records in the file. Then use the skiprows parameter to directly jump to the row/line from which you want to select the records.

Code: In the following code snippet we will fetch the first 5 rows from the csv file into our DataFrame.

import pandas as pd def num_of_lines(fname): with open(fname) as f: for i, _ in enumerate(f): pass return i + 1 num_lines = num_of_lines("countries.csv")
n = 5
df = pd.read_csv("countries.csv", skiprows=range(1, num_lines - n))
print(df)

Output:

 Country Capital Population Area
0 USA Washington 334,506,463 9,147,420
1 China Beijing 1,449,357,022 9,388,211
2 Poland Warsaw 37,771,789 306,230
3 Russia Moscow 146,047,418 16,376,870
4 England London 68,529,747 241,930

Conclusion

Phew! We have successfully solved all the problems that were presented to us in this tutorial.  I hope this tutorial helped you to sharpen your coding skills. Please stay tuned and subscribe for more interesting coding problems.

Recommended Reads:


Learn Pandas the Fun Way by Solving Code Puzzles

If you want to boost your Pandas skills, consider checking out my puzzle-based learning book Coffee Break Pandas (Amazon Link).

Coffee Break Pandas Book

It contains 74 hand-crafted Pandas puzzles including explanations. By solving each puzzle, you’ll get a score representing your skill level in Pandas. Can you become a Pandas Grandmaster?

Coffee Break Pandas offers a fun-based approach to data science mastery—and a truly gamified learning experience.

Posted on Leave a comment

How to Convert .blf (CAN) to .csv in Python

5/5 – (1 vote)

Problem Formulation

💬 How to convert .blf from a CAN bus to .csv in Python?

💡 What is BLF? The Binary Logging Format (BLF) is a proprietary
CAN log format from the automative company Vector Informatik GmbH.

💡 What is CAN? The Controller Area Network (CAN bus) is a message-based protocol standard for microcontrollers in vehicles to communicate without a host computer.

Method 1: Using BLF Reader and CSV Writer

To convert the BLF file 'my_file.blf' to the CSV file 'my_file.csv', you can first iterate over the bus messages using can.BLFReader('my_file.csv') and add the data to a list of lists. Then, you can use the csv.writer() approach to write the list of lists to a CSV file.

Here’s an example that improves upon this SO thread:

import can
import csv log = [] for msg in list(can.BLFReader("my_file.blf")): msg = str(msg) row = [msg[18:26], msg[38:40], msg[40:42], msg[46], msg[62], msg[67:90]] log.append(row) with open("my_file.csv", "w", newline='') as f: writer = csv.writer(f, delimiter=',', quotechar='\"', quoting=csv.QUOTE_ALL) writer.writerows(log)

A more sophisticated version of this code is provided in this Github repository. Here’s a screenshot of the code — notice the more advanced processing of a single message compared to our solution:

Method 2: Using the candas Library

The candas library provides utility functions to work with .blf files and the CAN bus. Among other things, it helps you with the conversion from BLF to CSV as outlined here.

This is the provided example:

import candas as cd db = cd.load_dbc("dbc_folder") # This is the BLF file 'my_file.blf':
log = cd.from_file("my_file") # This prints a signal from the messages in the BLF:
print(log["AVGcellTemperature"])

Method 3: Using Custom Solution from python-can Library

You can use your tailor-made solutions by combining the Readers and Writers provided in the python-can library.

It provides multiple utility functions such as:

  • Listener
  • BufferedReader
  • RedirectReader
  • Logger
  • Printer
  • CSVWriter
  • SqliteWriter
  • ASC
  • Log
  • BLF

Chances are you’ll find what you’re looking for when going over those functions!

Related Video

Still not satisfied? I found the following relevant video when searching for a solution to this problem. I think you’ll find some nice tricks in the video!

Posted on Leave a comment

How to Convert a CSV to NumPy Array in Python?

5/5 – (1 vote)

Problem Formulation

Given a CSV file (e.g., stored in the file with name 'my_file.csv').

INPUT: file 'my_file.csv'
9,8,7
6,5,4
3,2,1
How to Convert a CSV to NumPy Array in Python?

Challenge: How to convert it to a NumPy Array?

OUTPUT: 2D NumPy Array
[[9. 8. 7.] [6. 5. 4.] [3. 2. 1.]]

Method 1: np.loadtxt()

np.loadtxt()

You can convert a CSV file to a NumPy array simply by calling np.loadtxt() with two arguments: the filename and the delimiter string. For example, the expression np.loadtxt('my_file.csv', delimiter=',') returns a NumPy array from the 'my_file.csv' with delimiter symbols ','.

Here’s an example:

import numpy as np array = np.loadtxt('my_file.csv', delimiter=',')
print(array)

Output:

[[9. 8. 7.] [6. 5. 4.] [3. 2. 1.]]

Method 2: np.loadtxt() with Header

np.loadtxt() + header

You can convert a CSV file with first-line header to a NumPy array by calling np.loadtxt() with three arguments: the filename, skiprows=1 to skip the first line (header), and the delimiter string. For example, the expression np.loadtxt('my_file.csv', skiprows=1, delimiter=',') returns a NumPy array from the 'my_file.csv' with delimiter symbols ',' while skipping the first line.

Figure: Skip the first header line in the CSV using the skiprows argument of the np.loadtxt() function.

Here’s an example:

import numpy as np array = np.loadtxt('my_file.csv', skiprows=1, delimiter=',')
print(array)

Output:

[[9. 8. 7.] [6. 5. 4.] [3. 2. 1.]]

Method 3: CSV Reader

CSV Reader

To convert a CSV file 'my_file.csv' into a list of lists in Python, use the csv.reader(file_obj) method to create a CSV file reader. Then convert the resulting object to a list using the list() constructor. As a final step, you can convert the nested list to a NumPy array by using the np.array(list) constructor.

Here’s an example:

import numpy as np
import csv csv_filename = 'my_file.csv' with open(csv_filename) as f: reader = csv.reader(f) lst = list(reader) print(lst)

The output is the list of lists:

[['9', '8', '7'], ['6', '5', '4'], ['3', '2', '1']]

Now, if you need to convert it to a NumPy array, you can simply use the np.array() function on the newly-created list like so:

array = np.array(lst)
print(array)

Output:

[['9' '8' '7'] ['6' '5' '4'] ['3' '2' '1']]

🌎 Related Tutorial: How to Convert CSV to List of Lists in Python

Method 4: np.genfromtxt()

np.genfromtxt()

You can convert a CSV file to a NumPy array simply by calling np.genfromtxt() with two arguments: the filename and the delimiter string. For example, the expression np.genfromtxt('my_file.csv', delimiter=',') returns a NumPy array from the 'my_file.csv' with delimiter symbol ','.

Here’s an example:

import numpy as np array = np.loadtxt('my_file.csv', delimiter=',')
print(array)

Output:

[[9. 8. 7.] [6. 5. 4.] [3. 2. 1.]]

Method 5: Pandas read_csv() and df.to_numpy()

read_csv() and df.to_numpy()

A quick and efficient way to read a CSV to a NumPy array is to combine Pandas’ pd.read_csv() function to read a given CSV file to a DataFrame with the df.to_numpy() function to convert the Pandas DataFrame to a NumPy array.

Here’s an example:

import pandas as pd df = pd.read_csv('my_file.csv', header=None)
array = df.to_numpy() print(array)

Output:

[[9 8 7] [6 5 4] [3 2 1]]

🌎 Related Tutorial: 17 Ways to Read a CSV File to a Pandas DataFrame

Summary

We have seen five ways to convert a CSV file to a 2D NumPy array:

  • Method 1: np.loadtxt()
  • Method 2: np.loadtxt() with Header
  • Method 3: CSV Reader
  • Method 4: np.genfromtxt()
  • Method 5: Pandas read_csv() and df.to_numpy()

Our preferred way is np.loadtxt() for its simplicity and Pandas for its extensibility.

Posted on Leave a comment

How to Install psycopg2 in Python?

5/5 – (1 vote)
pip install psycopg2

The Python psycopg2 library is among the top 100 Python libraries, with more than 15,749,750 downloads. This article will show you everything you need to get this installed in your Python environment.

How to Install psycopg2 on Windows?

  1. Type "cmd" in the search bar and hit Enter to open the command line.
  2. Type “pip install psycopg2” (without quotes) in the command line and hit Enter again. This installs psycopg2 for your default Python installation.
  3. The previous command may not work if you have both Python versions 2 and 3 on your computer. In this case, try "pip3 install psycopg2" or “python -m pip install psycopg2“.
  4. Wait for the installation to terminate successfully. It is now installed on your Windows machine.

Here’s how to open the command line on a (German) Windows machine:

Open CMD in Windows

First, try the following command to install psycopg2 on your system:

pip install psycopg2

Second, if this leads to an error message, try this command to install psycopg2 on your system:

pip3 install psycopg2

Third, if both do not work, use the following long-form command:

python -m pip install psycopg2

The difference between pip and pip3 is that pip3 is an updated version of pip for Python version 3. Depending on what’s first in the PATH variable, pip will refer to your Python 2 or Python 3 installation—and you cannot know which without checking the environment variables. To resolve this uncertainty, you can use pip3, which will always refer to your default Python 3 installation.

How to Install psycopg2 on Linux?

You can install psycopg2 on Linux in four steps:

  1. Open your Linux terminal or shell
  2. Type “pip install psycopg2” (without quotes), hit Enter.
  3. If it doesn’t work, try "pip3 install psycopg2" or “python -m pip install psycopg2“.
  4. Wait for the installation to terminate successfully.

The package is now installed on your Linux operating system.

How to Install psycopg2 on macOS?

Similarly, you can install psycopg2 on macOS in four steps:

  1. Open your macOS terminal.
  2. Type “pip install psycopg2” without quotes and hit Enter.
  3. If it doesn’t work, try "pip3 install psycopg2" or “python -m pip install psycopg2“.
  4. Wait for the installation to terminate successfully.

The package is now installed on your macOS.

How to Install psycopg2 in PyCharm?

Given a PyCharm project. How to install the psycopg2 library in your project within a virtual environment or globally? Here’s a solution that always works:

  • Open File > Settings > Project from the PyCharm menu.
  • Select your current project.
  • Click the Python Interpreter tab within your project tab.
  • Click the small + symbol to add a new library to the project.
  • Now type in the library to be installed, in your example "psycopg2" without quotes, and click Install Package.
  • Wait for the installation to terminate and close all pop-ups.

Here’s the general package installation process as a short animated video—it works analogously for psycopg2 if you type in “psycopg2” in the search field instead:

Make sure to select only “psycopg2” because there may be other packages that are not required but also contain the same term (false positives):

How to Install psycopg2 in a Jupyter Notebook?

To install any package in a Jupyter notebook, you can prefix the !pip install my_package statement with the exclamation mark "!". This works for the psycopg2 library too:

!pip install my_package

This automatically installs the psycopg2 library when the cell is first executed.

How to Resolve ModuleNotFoundError: No module named ‘psycopg2’?

Say you try to import the psycopg2 package into your Python script without installing it first:

import psycopg2
# ... ModuleNotFoundError: No module named 'psycopg2'

Because you haven’t installed the package, Python raises a ModuleNotFoundError: No module named 'psycopg2'.

To fix the error, install the psycopg2 library using “pip install psycopg2” or “pip3 install psycopg2” in your operating system’s shell or terminal first.

See above for the different ways to install psycopg2 in your environment.

Improve Your Python Skills

If you want to keep improving your Python skills and learn about new and exciting technologies such as Blockchain development, machine learning, and data science, check out the Finxter free email academy with cheat sheets, regular tutorials, and programming puzzles.

Join us, it’s fun! 🙂

Posted on Leave a comment

Layout of a Solidity Source File

5/5 – (1 vote)

In this article, I am going to explain the fundamentals of a Solidity source file structure. In a way, a rookie (like me and you) can understand the basics of Ethereum programming.

First things first – the Solidity file that stores your code will have a .sol extension.

Take action.

  1. In your favorite browser go to https://remix.ethereum.org/ 
  2. Using Create New File create a new finxter.sol file

Well done. Your file – empty so far – is ready for more actions!

In this article, we’ll add (i) license identifier, (ii) pragma, (iii) another file via import, and (iv) add some comments.

The actual smart contract code is out of scope here but check out other Finxter tutorials – there is plenty of that.

License Identifier

I know you are eager to get to the meat, but before you jump there, bear with me.  The so-called SPDX license identifier is the first element you need to jot down.

What the heck is that? SPDX, or the Software Package Data Exchange, is an international, open standard for communicating software information including licenses or copyrights. 

Being a standard means that many companies and organizations have agreed to do some things in a certain way. And Solidity has also adopted that standard.

Why bother, you ask?

Well, your code will be transparent in a blockchain and that transparency triggers copyright issues. The SPDX identifier hence allows you to specify what you allow others to do with your code. And vice versa, you learn what you can do with other people’s code too.

An example comment line with an identifier would be:

// SPDX-License-Identifier: MIT

What this means is:

👩‍⚖️ Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, [etc etc…]

SPDX license list has over 450 various license identifiers!

But don’t worry too much now, we are here to learn Solidity, not the legal twists. So for now, take my word and use MIT as your default one. Or if you do not want to specify a license or if the source code is not open-source, please use the special value UNLICENSED.

🛑 Caution here folks – UNLICENSE (without ‘d’ at the end) is a completely different license! Open door type of one. It offers free and unencumbered software released into the public domain.

By the way, a good rule of thumb is to use the OSI-approved ones when browsing https://spdx.org/licenses/. “Only” about 120+ options for consideration.

Does that identifier do anything technically to your code? No, it won’t break how it works. After all, it is a comment.

But from Solidity >=0.6.8 (so 0.6.8 and any higher), that comment must be part of your code. Otherwise expect a Warning in your compiler.

The compiler checks the existence but does not know if your identifier is the right one (if it exists in the SPDX list). Starting in Solidity 0.8.8 it checks for multiple SPDX license identifiers next to each other and validates them. It still allows you to play with it though 🙂

In 0.8.7 you could easily get away with some crazy identifier.

// SPDX-License-Identifier: cheating_on_you pragma solidity 0.8.7;

From 0.8.8 onwards it actually starts to pay attention and throws errors. Note the red error icon on the left in line 1.

Same happens when you add multiple licenses inappropriately. 

This one below goes unnoticed though – compiler says OK.

// SPDX-License-Identifier: cheatingonyou pragma solidity 0.8.15;

Since SPDX info is a comment, it is recognized by the compiler anywhere in the file at the file level, but for clarity put it at the very top of the file.

Lastly, the identifier will become part of your metadata once it’s compiled. And that is machine-readable so others will find it easy to query.

Take action.

1. Add the license-related comment to your code (finxter.sol)

// SPDX-License-Identifier: MIT

Pragmas

The second important keyword is pragma. It comes in several shapes and forms – as a version pragma, ABI Coder pragma or experimental pragma.

⭐ Note: A pragma directive is always local to a source file. So you must add it to all your files if you want to enable it in your whole project.

Remember also that if you import another file, the pragma from that file does not automatically apply to the importing file.

1. Version pragma defines for which versions of Solidity the code is written.

In the example:

pragma solidity >= 0.8.7;

we can expect that no compiler on version 0.8.7 or higher will throw any pragma errors.

Other examples – for illustration and education – how to define pragma versions:

pragma solidity 0.6.8; //single instance
pragma solidity >= 0.6.8; //0.6.8 and any above
pragma solidity ^0.6.8; //0.6.8 and any above but less than 0.7.0
pragma solidity 0.6.8 ^0.7.5; // single instance AND any above 0.7.5 but less than 0.8 -> this AND condition cannot be met here
pragma solidity 0.8.1 || ^0.8.10; // one instance OR any from 0.8.10 but less than 0.9.0

For practical reasons, the version pragma is the only type you should really care about when you start your Solidity journey.

2. ABI Coder pragma

As per Solidity documentation, you have two options to choose from:

pragma abicoder v1;
pragma abicoder v2;

However as of Solidity 0.8.0 the ABIEncoder is activated by default so for a rookie like you and me, there’s nothing to worry about anymore.

With version 0.8.0+ you can already enjoy the benefits of working more effectively with arrays and structs. These are just some of data types but explaining this goes way beyond this tutorial.

And no need to call this pragma additionally, as you had to do in the past, e.g.:

// SPDX-License-Identifier: GPL-3.0
pragma solidity ^0.4.16;
pragma experimental ABIEncoderV2;

3. Experimental pragma

Getting here is a risky business so better do not try it yourself at home 🙂

Solidity might be offering features that are – as labeled – experimental.

So if you have some technical appetite and skills and want to play, showcase to your potential clients or whatever the purpose, go ahead. But if again, you are still early in the game, just park for now. 

Take Action.

1. Add a version pragma directive to your code

pragma solidity ^0.8.15;

Importing other Source Files

You can import files in Solidity. That sounds obvious but let’s say this upfront.

Importing other files is important since you can break down your code into multiple files, which makes it more modular, easier to manage and control, and – best of all – re-usable.

The simplest way to import is using this line of code:

import "filename";

In our quick Remix exercise, imagine we have another file called “helloWorld.sol” located in the same directory. In order to import it to our finxter.sol file, one would use:

import "./helloWorld.sol";

Note: Pythonic  import "helloWorld.sol" would not work here.

For education purposes and in very simple implementations, this is the shortest way to import. Its disadvantage is that it pollutes the namespace by importing all global symbols from the imported file into the current global system.

That approach carries also a risk of importing symbols that are imported into the file we are importing. That file can contain symbols imported for yet another file and so on. Such subsequent importing may lead to confusion about where the symbols come from and where actually they are defined.

Solidity recommends using a variant, which may look more complex at first. But it only adds one new global symbol to the namespace, here symbolName, whose members are symbols from the imported file.

Makes sense?

import * as symbolName from "filename";

The best approach however would be to import relevant symbols explicitly.

So for instance, if the imported file “helloWorld.sol” would have a contract named “sayHello”, one could use only that. Rule of thumb here: import only the things you will use. 

import {something} from "filename";

Take action:

1. Add a new file named “helloWorld.sol” that contains this code

// SPDX-License-Identifier: MIT pragma solidity ^0.8.0; contract sayHello { // empty contract
}

2. In the “finxter.sol” file, add import 

import {sayHello} from "./helloWorld.sol";

Comments

Commenting the code is possible in the two following ways:

1. Regular comments

1.1 Single-line comment, e.g.

// this is a single-line regular comment

1.2 Multi-line comment, e.g.

/*
This
comment
spans
many
lines
*/

2. NatSpec comments

NatSpec stands for Ethereum Natural Language Specification. It is a special form of comments to provide rich documentation for functions, return variables, and more. 

The recommendation is that Solidity contracts are fully annotated using NatSpec for all public interfaces (everything in the ABI).

Use NatSpecs comments directly above function declarations or statements.e.g.

2.1 Single-line

/// single-line NatSpec comment

2.2 Multi-line

/**
multi-line
NatSpec
comment
*/

CODE example

// SPDX-License-Identifier: GPL-3.0
pragma solidity >=0.4.16 <0.9.0; /// @author The Solidity Team
/// @title A simple storage example
contract SimpleStorage { uint storedData; /// Store `x`. /// @param x the new value to store /// @dev stores the number in the state variable `storedData` function set(uint x) public { storedData = x; } /** Return the stored value. @dev retrieves the value of the state variable `storedData` @return the stored value */ function get() public view returns (uint) { return storedData; }
}

Take action:

1. Add a regular multi-line comment to your finxter.sol file

/*
This
tutorial
comes
from
finxter.com
*/

Reference: This article is based on some contents from the documentation. Check out this awesome resource too!


Learn Solidity Course

Solidity is the programming language of the future.

It gives you the rare and sought-after superpower to program against the “Internet Computer”, i.e., against decentralized Blockchains such as Ethereum, Binance Smart Chain, Ethereum Classic, Tron, and Avalanche – to mention just a few Blockchain infrastructures that support Solidity.

In particular, Solidity allows you to create smart contracts, i.e., pieces of code that automatically execute on specific conditions in a completely decentralized environment. For example, smart contracts empower you to create your own decentralized autonomous organizations (DAOs) that run on Blockchains without being subject to centralized control.

NFTs, DeFi, DAOs, and Blockchain-based games are all based on smart contracts.

This course is a simple, low-friction introduction to creating your first smart contract using the Remix IDE on the Ethereum testnet – without fluff, significant upfront costs to purchase ETH, or unnecessary complexity.