Category: Programming

A not-so-fictious problem: Say, you’ve created a web application that runs on a dedicated Linux server in the cloud. Thousands of users visit your web app and suddenly … it crashes. Your users start complaining, and you lose revenue. More importantly, you bleed credibility by the hour. Your server is down, so what do you do?

First, don’t panic.

Let’s analyze your server logs!

This article shows you how to convert your log file to a CSV file in Python, that you can use for further processing (e.g., in Pandas or Excel).

Problem Formulation by Example

Given a file my_file.log like this one I pulled from a real IBM server log example:

03/22 08:51:01 INFO :.main: *************** RSVP Agent started ***************
03/22 08:51:01 INFO :...locate_configFile: Specified configuration file: /u/user10/rsvpd1.conf
03/22 08:51:01 INFO :.main: Using log level 511
03/22 08:51:01 INFO :..settcpimage: Get TCP images rc - EDC8112I Operation not supported on socket.
03/22 08:51:01 INFO :..settcpimage: Associate with TCP/IP image name = TCPCS

How to convert this log file to a CSV file of the following standard comma-separated values format:

03/22,08:51:01,INFO,:.main: *************** RSVP Agent started ***************
03/22,08:51:01,INFO,:...locate_configFile: Specified configuration file: /u/user10/rsvpd1.conf
03/22,08:51:01,INFO,:.main: Using log level 511
03/22,08:51:01,INFO,:..settcpimage: Get TCP images rc - EDC8112I Operation not supported on socket.
03/22,08:51:01,INFO,:..settcpimage: Associate with TCP/IP image name = TCPCS

Or, here’s how that would look if you opened it with Excel:

Prettier, isn’t it? Unlike the first representation (log file), this CSV representation is easier to read for (most) human beings.

Convert Server Log to CSV with Pandas

You can convert a .log file to a CSV file in Python in four simple steps: (1) Install the Pandas library, (2) import the Pandas library, (3) read the log file as DataFrame, and (4) write the DataFrame to the CSV file.

1. (Optional in shell) pip install pandas
2. import pandas as pd
3. df = pd.read_csv('my_file.log', sep='\s\s+', engine='python')
4. df.to_csv('my_file.csv', index=None)

Here’s a minimal example:

import pandas as pd
df = pd.read_csv('my_file.log', sep='\s\s+', engine='python')
df.to_csv('my_file.csv', index=None)

Note: The regular expression sep='\s\s+' specifies more than one single whitespace as a separator between two CSV values. If you have a different separator string, you can define it here.

You specify the engine='python' to tell Pandas that we want the Python regular expression engine to process the separator regular expression.

The result of the code is the following CSV file:

You can use this CSV file as input for, say, an Excel sheet or Google Spreadsheet for further processing and analysis.

This is what your log file looks converted to a CSV and imported to Excel:

And this is how your log file looks as a Pandas DataFrame:

 03/22 ... :.main: *************** RSVP Agent started ***************
0 03/22 ... :...locate_configFile: Specified configuration... 1 03/22 ... :.main: Using log level 511 2 03/22 ... :..settcpimage: Get TCP images rc - EDC8112I O... 3 03/22 ... :..settcpimage: Associate with TCP/IP image na... [4 rows x 4 columns]

Related Tutorial: Python Pandas DataFrame to_csv()

In this article, we will introduce one of the core elements describing the mathematics of neural networks: tensors.

Although typically, you won’t work directly with tensors (usually they operate under the hood), it is important to understand what’s going on behind the scenes. In addition, you may often wish to examine tensors so that you can look directly at the data, or look at the arrays of weights and biases, so it’s important to be able to work with tensors.

Note: This article assumes you are familiar with how neural networks work. To review those basics, see the article The Magic of Neural Networks: History and Concepts. It also assumes you have some familiarity with Python’s object oriented programming.

Theoretically, we could use pure Python to implement neural networks.

• We could use Python lists to represent data in the network;
• We could use other lists representing weights and biases in the network; and
• We could use nested for loops to perform the operations of multiplying the inputs by the connection weights.

There are a few issues with this, however: Python, especially the list data type, performs rather slowly. Also, the code would not be very readable with nested for loops.

Instead, the libraries that implement neural networks in software packages such as PyTorch use tensors, and they run much more quickly than pure Python. Also, as you will see, tensors allow much more readable descriptions of networks and their data.

Tensors

Tensors are essentially arrays of values. Since neural networks are essentially arrays of neurons, tensors are a natural fit for describing them. They can be used for describing the data, describing the network connection weights, and other things.

A one-dimensional tensor is known as a vector. Here is an example:

Vectors can also be written horizontally. Here’s the same vector written horizontally:

Switching a vector from vertical to horizontal, or vice versa, is called transposing, and is sometimes needed depending on the math specifics. We will not go into detail on this in this article (see here for more).

Vectors are typically used to represent data in the network. For example, each individual element in a vector can represent the input value for each individual input neuron in the network.

2D Tensor Matrix

A two-dimensional tensor is known as a matrix. Here’s an example:

For a fully connected network, where each neuron in one layer connects to every neuron in the next layer, a matrix is typically used to represent all the connection weights. If there are m neurons connected to n neurons you would need an n x m matrix to describe all the connection weights.

Here’s an example of two neurons connected to three neurons. Here is the network, with connection weights included:

And here is the connection weights matrix:

Why We Use Tensors

Before we finish introducing tensors, let’s use what we’ve seen so far to see why they’re so important to use when modeling neural networks.

Let’s introduce a two-element vector of data and run it through the network we just showed.

Info: Recall neurons add together their weighted inputs, then run the result through an activation function.

In this example, we are ignoring the activation function to keep things simple for the demonstration.

Here is our data vector:

Here’s a diagram depicting the operation:

Let’s calculate the operation (the neuron computations) by hand:

The final result is a 3 element vector:

If you have learned about matrices in grade school and remember doing matrix multiplication, you may note that what we just calculated is identical to matrix multiplication:

Note: Recall matrix multiplication involves multiplying first matrix rows by second matrix columns element-wise, then adding elements together.

This is why tensors are so important for neural networks: tensor math precisely describes neural network operation.

As an added benefit, the equation above showing matrix multiplication is so much more a succinct description than nested for loops would be.

If we introduce the nomenclature of bold lower case for a vector and bold upper case for a matrix, then the operation of vector data running through a neural network weight matrix is described by this very compact equation:

We will see later that matrix multiplication within PyTorch is a similarly compact code equation.

Higher Dimensional Tensors

A three-dimensional (3D) tensor is known simply as a tensor. As you can see, the term tensor generically refers to any dimensional array of numbers. It’s just one-dimensional and two-dimensional tensors that have the unique names “vector” and “matrix” respectively.

You might not think that there is a need for three-dimensional and larger tensors, but that’s not quite true.

A grayscale image is clearly a two-dimensional tensor, in other words, a matrix. But a color image is actually three two-dimensional arrays, one each for red, green, and blue color channels. So a color image is essentially a three-dimensional tensor.

In addition, typically we process data in mini-batches. So if we’re processing a mini-batch of color images we have the three-dimensional aspect already noted, plus one more dimension of the list of images in the mini-batch. So a mini-batch of color images can be represented by a four-dimensional tensor.

Tensors in Neural Network Libraries

One Python library that is well suited to working with arrays is NumPy. In fact, NumPy is used by some users for implementing neural networks. One example is the scikit-learn machine learning library which works with NumPy.

However, the PyTorch implementation of tensors is more powerful than NumPy arrays. PyTorch tensors are designed with neural networks in mind. PyTorch tensors have these advantages:

1. PyTorch tensors include gradient calculations integrated into them.
2. PyTorch tensors also support GPU calculations, substantially speeding up neural network calculations.

However, if you are used to working with NumPy, you should feel fairly at home with PyTorch tensors. Though the commands to create PyTorch tensors are slightly different, they will feel fairly familiar. For the rest of this article, we will focus exclusively on PyTorch tensors.

Tensors in PyTorch: Creating Them, and Doing Math

OK, let’s finally do some coding!

First, make sure that you have PyTorch available, either by installing on your system or by accessing it through online Jupyter notebook servers.

Reference: See PyTorch’s website for instructions on how to install it on your own system.

See this Finxter article for a review of available online Jupyter notebook services:

Recommended Tutorial: Top 4 Jupyter Notebook Alternatives for Machine Learning

For this article, we will use the online Jupyter notebook service provided by Google called Colab. PyTorch is already installed in Colab; we simply have to import it as a module to use it:

import torch

There are a number of ways of creating tensors in PyTorch.

Typically you would be creating tensors by importing data from data sets available through PyTorch, or by converting your own data into tensors.

For now, since we simply want to demonstrate the use of tensors we will use basic commands to create very simple tensors.

You can create a tensor from a list:

t_list = torch.tensor([[1,2], [3,4]])
t_list

Output:

tensor([[1, 2], [3, 4]])

Note that when we evaluate the tensor variable, the output is labeled to indicate it as a tensor. This means that it is a PyTorch tensor object, so an object within PyTorch that performs just like math tensors, plus has various features provided by PyTorch (such as supporting gradient calculations, and supporting GPU processing).

You can create tensors filled with zeros, filled with ones, or filled with random numbers:

t_zeros = torch.zeros(2,3)
t_zeros

Output:

tensor([[0., 0., 0.], [0., 0., 0.]])

t_ones = torch.ones(3,2)
t_ones

Output:

tensor([[1., 1.], [1., 1.], [1., 1.]])

t_rand = torch.rand(3,2,4)
t_rand

Output:

tensor([[[0.9661, 0.3915, 0.0263, 0.2753], [0.7866, 0.0503, 0.3963, 0.1334]], [[0.4085, 0.1816, 0.2827, 0.3428], [0.9923, 0.4543, 0.0872, 0.0771]], [[0.2451, 0.6048, 0.8686, 0.8148], [0.7930, 0.4150, 0.6125, 0.3401]]])

An important attribute to be familiar with to understand the shape of a tensor is the appropriately named shape attribute:

t_rand.shape
# Output: torch.Size([3, 2, 4])

This shows you that tensor “t_rand” is a three-dimensional tensor composed of three elements of two rows by four columns.

Note: The dimensions of a tensor is referred to as its rank. A one-dimensional tensor, or vector, is a rank-1 tensor; a two-dimensional tensor, or matrix, is a rank-2 tensor; a three-dimensional tensor is a rank-3 tensor, and so on.

Let’s do some math with tensors – let’s add two tensors together:

Note the tensors are added together element-wise. Now here it is in PyTorch:

t_first = torch.tensor([[1,2], [3,4]])
t_second = torch.tensor([[5,6],[7,8]])
t_sum = t_first + t_second
t_sum

Output:

tensor([[ 6, 8], [10, 12]])

Let’s add a scalar, that is, an independent number (or a rank-0 tensor!) to a tensor:

t_add3 = t_first + 3
t_add3

Output:

tensor([[4, 5], [6, 7]])

Note that the scalar is added to each element of the tensor. The same applies when multiplying a scalar by a tensor:

t_times3 = t_first * 3
t_times3

Output:

tensor([[ 3, 6], [ 9, 12]])

The same kind of thing applies to raising a tensor to a power, that is the power operation is applied element-wise:

t_squared = t_first ** 2
t_squared

Output:

tensor([[ 1, 4], [ 9, 16]])

Recall that after summing weighted inputs, the neuron processes the result through an activation function. Note that the same performance applies here as well: when a vector is processed through an activation function, the operation is applied to the vector element-wise.

Earlier, we pointed out that matrix multiplication is an important part of neural network calculations.

There are two ways to do this in PyTorch: you can use the matmul function:

t_matmul1 = torch.matmul(t_first, t_second)
t_matmul1

Output:

tensor([[19, 22], [43, 50]])

Or you can use the matrix multiplication symbol “@“:

t_matmul2 = t_first @ t_second
t_matmul2


Output:

tensor([[19, 22], [43, 50]])

Recall previously, we showed running an input signal through a neural network, where a vector of input signals was multiplied by a matrix of connection weights.

Here is that in PyTorch:

x = torch.tensor([[7],[8]])
x

Output:

tensor([[7], [8]])

W = torch.tensor([[1,4], [2,5], [3,6]])
W

Output:

tensor([[1, 4], [2, 5], [3, 6]])

y = W @ x
y


Output:

tensor([[39], [54], [69]])

Note how compact and readable that is instead of doing nested for loops.

Other math can be done with tensors as well, but we have covered most situations that are relevant to neural networks. If you find you need to do additional math with your tensors, check PyTorch documentation or do a web search.

Indexing and Slicing Tensors

Slicing allows you to examine subsets of your data and better understand how the dataset is constructed. You may find you will use this a lot.

Indexing Slicing PyTorch vs NumPy vs Python Lists

Indexing and slicing tensors work the same way it does with NumPy arrays. Note that the syntax is different from Python lists. With Python lists, a separate pair of brackets are used for each level of nested lists. Instead, with Pytorch one pair of brackets contains all dimensions, separated by commas.

Let’s find the item in tensor “t_rand” that is 2nd element, first row, third column. First here is “t_rand” again:

t_rand

Output:

tensor([[[0.9661, 0.3915, 0.0263, 0.2753], [0.7866, 0.0503, 0.3963, 0.1334]], [[0.4085, 0.1816, 0.2827, 0.3428], [0.9923, 0.4543, 0.0872, 0.0771]], [[0.2451, 0.6048, 0.8686, 0.8148], [0.7930, 0.4150, 0.6125, 0.3401]]])

And here is the item at the 2nd element, first row, and third column (don’t forget indexing starts at zero):

t_rand[1, 0, 2]
# Output: tensor(0.2827)

Let’s look at the slice second element, first row, second through third columns:

t_rand[1, 0, 1:3]
# tensor([0.1816, 0.2827])

Let’s look at the entire 3rd column:

t_rand[:, :, 2]

Output:

tensor([[0.0263, 0.3963], [0.2827, 0.0872], [0.8686, 0.6125]])

Important Slicing Tip: In the above, we use the standard Python convention that a blank before a “:” means “start from the beginning”, and a blank after a “:” means “go all the way to the end”. So a “:” alone means “include everything from beginning to end”.

A likely use for slicing would be to look at a full array (i.e. a matrix) within a set of arrays, i.e. one image out of a set of images.

Let’s pretend our “t_rand” tensor is a list of images. We may wish to sample just a few “images” to get an idea of what they are like.

Let’s examine the first “image” in our tensor (“list of images”):

t_rand[0]

Output:

tensor([[0.9661, 0.3915, 0.0263, 0.2753], [0.7866, 0.0503, 0.3963, 0.1334]])

And here is the last array (“image”) in tensor “t_rand”:

t_rand[-1]

Output:

tensor([[0.2451, 0.6048, 0.8686, 0.8148], [0.7930, 0.4150, 0.6125, 0.3401]])

Using small tensors to demonstrate indexing can be instructive, but let’s see it in action for real. Let’s examine some real datasets with real images.

Real Example

We won’t describe the following in detail, except to note that we are importing various libraries that allow us to download and work with a dataset. The last line creates a function that converts tensors into PIL images:

import torch
from torch.utils.data import Dataset
from torchvision import datasets
from torchvision.transforms import ToTensor
import matplotlib.pyplot as plt import torchvision.transforms as T conv_to_PIL = T.ToPILImage()

The following downloads the Caltech 101 dataset, which is a collection of over 8000 images in 101 categories:

caltech101_data = datasets.Caltech101( root="data", download=True, transform=ToTensor()
)

Extracting data/caltech101/101_ObjectCategories.tar.gz to data/caltech101
Extracting data/caltech101/Annotations.tar to data/caltech101

This has created a dataset object which is a container for the data. These objects can be indexed like lists:

len(caltech101_data)
# 8677 type(caltech101_data[0])
# tuple len(caltech101_data[0])
# 2

The above code shows the dataset contains 8677 items. Looking at the first item of the set we can see they are tuples of 2 items each. Here are the kinds of items in the tuples:

type(caltech101_data[0][0])
# torch.Tensor type(caltech101_data[0][1])
# int

The two items in the tuple are the image as a tensor, and an integer code corresponding to the image’s category.

Colab has a convenient function display() which will display images. First, we use the conversion function we created earlier to convert our tensors to a PIL image, then we display the images.

img = conv_to_PIL(caltech101_data[0][0])
display(img)

We can use indexing to sample and display a few other images from the set:

img = conv_to_PIL(caltech101_data[1234][0])
display(img)

img = conv_to_PIL(caltech101_data[4321][0])
display(img)

Summary

We have learned a number of things:

1. What tensors are
2. Why tensors are key mathematical objects for describing and implementing neural networks
3. Creating tensors in PyTorch
4. Doing math with tensors in PyTorch
5. Doing indexing and slicing of tensors in PyTorch, especially to examine images in datasets

We hope you have found this article informative. We wish you happy coding!

---

Programmer Humor

In this article, we continue building on our previous topic, the Solidity compiler installation:

Previous Topic: Solidity Compiler Installation (NPM)

The previous article was focused on an installation via npm, and in this article, we’ll go through the installation and use of the Solidity compiler via Docker.

Our goal is to get more familiar with the possibilities of this approach, as well as to get introduced to the technology that “runs the show”. This knowledge and experience will enable us to recognize the reasons behind choosing any of the approaches in the future, depending on the real-world needs of our projects.

What is Docker?

Before we go into details about the Docker installation of solc, let’s first get introduced to what Docker is.

Docker is an open platform for developing, shipping, and running applications… Docker provides the ability to package and run an application in a loosely isolated environment called a container… Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is currently installed on the host.

Source: https://docs.docker.com/get-started/overview/

There are some parts of the description I’ve deliberately left out (separated by the symbol …) because they’re not essential to our understanding of the technology.

Now, let’s dissect the Docker description: the keywords of our interest are platform, isolated environment, and container. Let’s quickly dive into each of those next

Platform

A platform is a software framework that supports a specific function or a goal.

The goal Docker supports is enabling a piece of software (application, service, etc.) to correctly run, regardless of the target environment.

For us, this means running the Solidity compiler, i.e. feeding it with the input source code and producing the output bytecode in the form of .abi and .bin files.

Isolated Environment

By mentioning an isolated environment, we remember the concept of virtualization learned about earlier, meaning that Docker enables our software to run as intended by providing it with the resources in form of software libraries, network access, remote services, and other dependencies.

Container

Docker ensures the resources are provided without additional intervention by arranging them in a package called a container. Containers begin their lifecycle as images that we most commonly download and run.

We can also create a Docker image, but that’s another story.

Running an image creates a live instance of it, a container. Before it can be used, a Docker image has to be prepared, meaning that someone should install and configure all the required resources needed for the software to run.

Preparation of a Docker image falls in the domain of DevOps, i.e. Development and Operations:

“DevOps engineers manage the operations of software development, implementing engineering tools and knowledge of the software development process to streamline software updates and creation.”

Source: https://www.indeed.com/hire/c/info/devops-engineer

Also, read our article:

Recommended Article: Top 20 Skills Every DevOps Engineer Ought to Have

Using Solidity Compiler via Docker

Now that we have introduced Docker in general, we are continuing with the installation of the Solidity compiler via Docker.

First, we have to check if Docker is present on our system by simultaneously checking the Docker version:

$ docker version
bash: /usr/bin/docker: No such file or directory

As our check shows, we have to install Docker on our system before we can use it. The installation process via the Ubuntu repository is made of several steps (https://docs.docker.com/engine/install/ubuntu/):

Step 1: Update the apt package index

$ sudo apt update
…
Reading package lists... Done
Building dependency tree Reading state information... Done
All packages are up to date.

Step 2: Install packages

Installation of additional packages; we need these packages to enable the installation process accessing the repository over the secure HTTPS connection (note the backslash symbol \ for the multiline command):

$ sudo apt install \
ca-certificates \
curl gnupg lsb-release
...
The following additional packages will be installed: gnupg-l10n gnupg-utils gpg-wks-server
Suggested packages: parcimonie xloadimage
The following NEW packages will be installed: ca-certificates curl gnupg gnupg-l10n gnupg-utils gpg-wks-server lsb-release
...
Do you want to continue? [Y/n] y
...

Step 3: Add Docker GPG key

Adding the Docker’s official GPG key:

$ sudo mkdir \
-p /etc/apt/keyrings
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
| sudo gpg – dearmor -o /etc/apt/keyrings/docker.gpg

Info: “GPG, or GNU Privacy Guard, is a public key cryptography implementation. This allows for the secure transmission of information between parties and can be used to verify that the origin of a message is genuine.”

Source: https://www.digitalocean.com/community/tutorials/how-to-use-gpg-to-encrypt-and-sign-messages

Step 4: Set up repository

Setting up the repository by writing to docker.list file.

The echo command evaluates the text inside the $( ), populates it with the command outputs (in parentheses), and sends it via stdin to system utility sudo tee with root privileges, which in turn overwrites the docker.list file and omits the output by redirecting it to /dev/null:

$ echo \ "deb [arch=$(dpkg – print-architecture) \
signed-by=/etc/apt/keyrings/docker.gpg] \
https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee \
/etc/apt/sources.list.d/docker.list > /dev/null

Info: Repositories added by mistake can be removed from Ubuntu 20.04 by selectively deleting them in /etc/apt/sources.list.d/ directory.

Step 5: Update apt package index

Updating the apt package index (once again):

$ sudo apt update
...
Reading package lists... Done
Building dependency tree Reading state information... Done
All packages are up to date.

Step 6: Install Docker

Installing Docker (the latest stable version) and its components:

$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
Reading package lists... Done
Building dependency tree Reading state information... Done
The following additional packages will be installed: docker-ce-rootless-extras docker-scan-plugin pigz slirp4netns
Suggested packages: aufs-tools cgroupfs-mount | cgroup-lite
The following NEW packages will be installed: containerd.io docker-ce docker-ce-cli docker-ce-rootless-extras docker-compose-plugin docker-scan-plugin pigz slirp4netns
0 upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 108 MB of archives.
After this operation, 449 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
...

Let’s check the Docker version once again:

$ docker version
Client: Docker Engine - Community Version: 20.10.17 API version: 1.41 Go version: go1.17.11 Git commit: 100c701 Built: Mon Jun 6 23:02:57 2022 OS/Arch: linux/amd64 Context: default Experimental: true Server: Docker Engine - Community Engine: Version: 20.10.17 API version: 1.41 (minimum version 1.12) Go version: go1.17.11 Git commit: a89b842 Built: Mon Jun 6 23:01:03 2022 OS/Arch: linux/amd64 Experimental: false containerd: Version: 1.6.7 GitCommit: 0197261a30bf81f1ee8e6a4dd2dea0ef95d67ccb runc: Version: 1.1.3 GitCommit: v1.1.3-0-g6724737 docker-init: Version: 0.19.0 GitCommit: de40ad0

Now that we’re sure that our Docker installation went through and the Docker Engine version we have is 20.20.17 (at the time of writing this article). The next step is getting the Docker image with the Solidity compiler.

Docker images are identified by their release organization, image name (shorter, images), and tag, i.e. label that makes them unique. In general, we can download a Docker image by referencing it with its organization/image:tag marker.

We will download a Docker image of the Solidity compiler by specifying its marker as ethereum/solc:stable for a stable version, and ethereum/solc:nightly for the bleeding edge, potentially unstable version.

We can also specify a distinct version of the Solidity compiler by setting a tag to a specific version, e.g. ethereum/solc:0.5.4.

We will do three things with one Docker command: we’ll download the image, instantiate (run) a container from the image and print the container usage (flag – help):

docker run ethereum/solc:stable – help

Sure enough, we’d like to compile our Solidity files, so we’ll make three preparations (First, Second, Third):

First: Create a local directory containing our Solidity source code (I’ll use 1_Storage.sol from the Remix contracts folder by creating an empty file and pasting the content into it):

$ mkdir ~/solidity_src/ && cd ~/solidity_src/
$ touch 1_Storage.sol

Second: You can write your own contract for testing purposes or just open the 1_Storage.sol with your favorite text editor and paste the contents from 1_Storage.sol example in Remix.

Third: Run a Docker container (we already have the image so the download procedure will be skipped); command flag -v mounts our local ~/solidity_src directory to the container’s path /sources, path ethereum/solc:stable selects the Docker image to run a container, command flag -o sets the output location for the compiled files, --abi and --bin activate the generation of both .abi and .bin files, and the path /sources/1_Storage.sol selects the source file for compilation:

$ docker run -v ~/solidity_src:/sources ethereum/solc:stable -o /sources/output – abi – bin /sources/1_Storage.sol
Compiler run successful. Artifact(s) can be found in directory "/sources/output".

When checking our solidity_src directory, we’ll discover a new directory output, created by the Solidity compiler, containing both .abi and .bin files.

Docker also enables us to use the standard JSON interface, and it is a recommended approach when using the compiler with a toolchain. This interface doesn’t require mounted directories if the JSON input is self-contained, in other words, all the code is already contained in the source files and there are no references to external, imported files:

docker run ethereum/solc:stable – standard-json < input.json > output.json

Since we haven’t done any examples using the JSON interface, we’ll suspend this approach until a later time.

Conclusion

This article introduced us to a Solidity-supporting technology called Docker.

Of course, our main focus is on an ecosystem consisting of Solidity, Ethereum, blockchain technology, etc., but I recognized an opportunity of making a detour and walking us through the process of setting up and using the Solidity compiler via the Docker platform. Therefore, although initially unplanned, we’re also gaining some DevOps skills.

In the first and only chapter (yeah, I’m a bit surprised as well) we’ve set the mining charges by getting to know what Docker is. Then we blew a big piece of rock away by discovering how to install Docker on Ubuntu Linux (and by extension, some other operating systems). I believe this article will prove useful and provide multiple tips and tricks in terms of setting your development environment for Solidity on Ubuntu Linux. Besides that and personally speaking, it was always useful to gain secondary knowledge whenever I learned a specific topic, and I’m sure you’ll have the same experience.

Recommended Tutorial: Solidity Crash Course (by Matija)

---

Learn Solidity Course

Solidity is the programming language of the future.

It gives you the rare and sought-after superpower to program against the “Internet Computer”, i.e., against decentralized Blockchains such as Ethereum, Binance Smart Chain, Ethereum Classic, Tron, and Avalanche – to mention just a few Blockchain infrastructures that support Solidity.

In particular, Solidity allows you to create smart contracts, i.e., pieces of code that automatically execute on specific conditions in a completely decentralized environment. For example, smart contracts empower you to create your own decentralized autonomous organizations (DAOs) that run on Blockchains without being subject to centralized control.

NFTs, DeFi, DAOs, and Blockchain-based games are all based on smart contracts.

This course is a simple, low-friction introduction to creating your first smart contract using the Remix IDE on the Ethereum testnet – without fluff, significant upfront costs to purchase ETH, or unnecessary complexity.

Problem Formulation

Given a Python list of elements.

If you print the list to the shell using print([1, 2, 3]), the output is enclosed in square brackets and separated by commas like so: "[1, 2, 3]".

But you want the list without brackets and commas like so: 1 2 3.

print([1, 2, 3])
# Output: [1, 2, 3]
# Desired: 1 2 3

How to print the list without enclosing brackets and without separating commas in Python?

Recommended Tutorial: How to Print a List Without Brackets in Python?

Method 1: Unpacking Multiple Values into Print Function

The asterisk operator * is used to unpack an iterable into the argument list of a given function.

You can unpack all list elements into the print() function to print all values individually, separated by an empty space per default (that you can override using the sep argument). For example, the expression print(*my_list) prints the elements in my_list, empty space separated, without the enclosing square brackets and without the separating commas!

Here’s an example:

my_list = [1, 2, 3]
print(*my_list)
# Output: 1 2 3

Note: If you want a different separating character, you can set the sep argument of the print() function. For example, print(*my_list, sep='|') will use the vertical bar '|' as a separating character.

Here’s an example:

my_list = [1, 2, 3]
print(*my_list, sep='|')
# Output: 1|2|3

You can learn about the ins and outs of the built-in print() function in the following video:

To master the basics of unpacking, feel free to check out this video on the asterisk operator:

Method 2: String Replace Method

A simple way to print a list without commas and square brackets is to first convert the list to a string using the built-in str() function. Then modify the resulting string representation of the list by using the string.replace() method until you get the desired result.

Here’s an example:

my_list = [1, 2, 3] # Convert List to String
s = str(my_list)
print(s)
# [1, 2, 3] # Replace Separating Commas and Square Brackets
s = s.replace(', ', '\n').replace('[', '').replace(']', '') # Print List Without Commas and Brackets
print(s)

The result is a string without commas and without brackets:

1
2
3

Method 3: String Join With Generator Expression

You can print a list without brackets and without commas. Use the string.join() method on any separator string such as ' ' or '\t'. Pass a generator expression to convert each list element to a string using the str() built-in function. For example, the expression print(' '.join(str(x) for x in my_list)) prints my_list to the shell without enclosing brackets and commas.

my_list = [1, 2, 3]
print(' '.join(str(x) for x in my_list))
# Output: 1 2 3

You can modify the separator string on which you join to customize the appearance of the list:

my_list = [1, 2, 3]
print('xxx'.join(str(x) for x in my_list))
# Output: 1xxx2xxx3

• The string.join(iterable) method concatenates the elements in the given iterable.
• The str(object) built-in function converts a given object to its string representation.
• Generator expressions or list comprehensions are concise one-liner ways to create a new iterable based by reusing elements from another iterable.

You can dive deeper into generators in the following video:

Where to Go From Here?

Enough theory. Let’s get some practice!

Coders get paid six figures and more because they can solve problems more effectively using machine intelligence and automation.

To become more successful in coding, solve more real problems for real people. That’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

You build high-value coding skills by working on practical coding projects!

Do you want to stop learning with toy projects and focus on practical code projects that earn you money and solve real problems for people?

If your answer is YES!, consider becoming a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

If you just want to learn about the freelancing opportunity, feel free to watch my free webinar “How to Build Your High-Income Skill Python” and learn how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Programmer Humor

Question: How did the programmer die in the shower? ☠

❗ Answer: They read the shampoo bottle instructions:
Lather. Rinse. Repeat.

Use the tuple concatenation operation * with a tuple with one element (42,) as a right operand and the number of repetitions of this element as a left operand. For example, the expression (42,) * n creates the tuple (42, 42, 42, 42, 42) for n=5.

Let’s play with an interactive code shell before you’ll dive into the detailed solution!

Exercise: Initialize the tuple with n=20 placeholder elements -1 and run the code.

---

Problem Formulation

Next, you’ll learn about the more formal problem and dive into the step-by-step solution.

Problem: Given an integer n. How to initialize a tuple with n placeholder elements (e.g., 42)?

# n=0 --> ()
# n=1 --> (42,)
# n=5 --> (42, 42, 42, 42, 42)

Example 1 – Tuple Concatenation

Use the tuple concatenation operation * with a tuple with one element (42,) as right operand and the number of repetitions of this element as left operand. For example, the expression (42,) * n creates the tuple (42, 42, 42, 42, 42) for n=5.

n = 5
t = (42,) * n
print(t)
# (42, 42, 42, 42, 42)

Note that you cannot change the values of a tuple, once created, because unlike lists tuples are immutable. For example, trying to overwrite the third tuple value will yield a TypeError: 'tuple' object does not support item assignment.

>>> x = (42,) * 5
>>> x[0] = 'Alice'
Traceback (most recent call last): File "<pyshell#6>", line 1, in <module> x[0] = 'Alice'
TypeError: 'tuple' object does not support item assignment

Example 2 – N-Ary Tuple Concatenation

You can also use a generalization of the unary tuple concatenation — I call it n-ary tuple concatenation — to create a tuple of size n. For example, given a tuple t of size 3, you can create a tuple of size 9 by multiplying it with the integer 3 like so: t * 3.

Here’s an example:

simple_tuple = ('Alice', 42, 3.14)
complex_tuple = simple_tuple * 3 print(complex_tuple)
# ('Alice', 42, 3.14, 'Alice', 42, 3.14, 'Alice', 42, 3.14)

Example 3 – Tuple From List

This approach is simple: First, create a list of size n. Second, pass that list into the tuple() function to create a tuple of size n.

n = 100 # 1. Create list of size n
lst = [42] * n # 2. Change value in (mutable) list
lst[2] = 'Alice' # 3. Create tuple from list AFTER modification
t = tuple(lst) # 4. Print tuple
print(t)
# (42, 42, 'Alice', 42, 42, ...)

Recommended Tutorial: Create a List of Size n

Example 4 – Generator Expression (List Comprehension)

You can pass a generator expression into Python’s built-in tuple() function to dynamically create a tuple of elements, given another iterable. For example, the expression tuple(i**2 for i in range(10)) creates a tuple with ten square numbers.

Here’s the code snippet for copy&paste:

x = tuple(i**2 for i in range(10))
print(x)
# (0, 1, 4, 9, 16, 25, 36, 49, 64, 81)

In case you need some background on this terrific Python feature, check out my article on List Comprehension and my best-selling Python textbook on writing super condensed and concise Python code:

Python One-Liners Book: Master the Single Line First!

Python programmers will improve their computer science skills with these useful one-liners.

Python One-Liners will teach you how to read and write “one-liners”: concise statements of useful functionality packed into a single line of code. You’ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.

The book’s five chapters cover (1) tips and tricks, (2) regular expressions, (3) machine learning, (4) core data science topics, and (5) useful algorithms.

Detailed explanations of one-liners introduce key computer science concepts and boost your coding and analytical skills. You’ll learn about advanced Python features such as list comprehension, slicing, lambda functions, regular expressions, map and reduce functions, and slice assignments.

You’ll also learn how to:

• Leverage data structures to solve real-world problems, like using Boolean indexing to find cities with above-average pollution
• Use NumPy basics such as array, shape, axis, type, broadcasting, advanced indexing, slicing, sorting, searching, aggregating, and statistics
• Calculate basic statistics of multidimensional data arrays and the K-Means algorithms for unsupervised learning
• Create more advanced regular expressions using grouping and named groups, negative lookaheads, escaped characters, whitespaces, character sets (and negative characters sets), and greedy/nongreedy operators
• Understand a wide range of computer science topics, including anagrams, palindromes, supersets, permutations, factorials, prime numbers, Fibonacci numbers, obfuscation, searching, and algorithmic sorting

By the end of the book, you’ll know how to write Python at its most refined, and create concise, beautiful pieces of “Python art” in merely a single line.

Get your Python One-Liners on Amazon!!