Posted on Leave a comment

How to Solve Python “TypeError: ‘int’ object is not iterable”?

It’s quite common for your code to throw a typeerror, especially if you’re just starting out with Python. The reason for this is that the interpreter expects variables of certain types in certain places in the code.

We’ll look at a specific example of such an error: "typeerror: 'int' object is not iterable".

Exercise: Run this minimal example and reproduce the error in your online Python shell!

Let’s start decomposing this error step-by-step!

Background Integer & Iterable

First, it’s worth understanding what int and iterable are.

The int type in Python, as in almost all programming languages, is a type for storing integers such as 1, 2, 3, -324, 0. There can be many variables of type int in our program. We can assign values ​​to them ourselves directly:

a = 5

In this case, we will most often understand what is a type of our variable. But the value can, for example, be returned from a function. Python uses implicit typing. Implicit typing means that when declaring a variable, you do not need to specify its type; when explicitly, you must. Therefore, by assigning the result of a function to a variable, you may not be clearly aware of what type your variable will be.

s1 = sum([1, 2, 3])
print(s1)
print(type(s1))

Output:

6
<class 'int'>

Here’s another example:

s2 = iter([1, 2, 3])
print(s2)
print(type(s2))

Output:

<list_iterator object at 0x7fdcf416eac8>
<class 'list_iterator'>

In this example, s1 is an integer and is of type int. This number is returned by the sum function with an argument in the form of a list of 3 elements. And the variable s2 is of type list_iterator, an object of this type is returned by the iter function, whose argument is the same list of 3 elements. We’ll talk about iteration now.

Iteration is a general term that describes the procedure for taking the elements of something in turn.

More generally, it is a sequence of instructions that is repeated a specified number of times or until a specified condition is met.

An iterable is an object that is capable of returning elements one at a time. It is also an object from which to get an iterator.

Examples of iterable objects:

It seems that the easiest way to find out what exactly our function is returning is to look at the documentation.

So we see for the iter: iter(object[, sentinel]) Return an iterator object.

But for the sum we have nothing about a type of returning value. Check it out by yourself!

So, the typeerror: ‘int’ object is not iterable error occurs when the interpreter expects an iterable object and receives just an integer. Let’s consider the most common examples of such cases.

Invalid ‘sum’ Argument

We already wrote about the sum function. It returns the int value. The sum function takes at most two arguments. The first argument must be an object that is iterable. If it’s a collection of some sort, then it’s probably a safe assumption that it’s iterable. The second argument to the sum function is optional. It’s a number that represents the first number you’ll start adding to. If you omit the second argument, then you’ll start adding to 0. For novice Python programmers, it seems common sense that a function should return the sum of its arguments. Often they try to apply it like this:

a = 4
b = 3
sum(a, b)

Output:

TypeError Traceback (most recent call last) <ipython-input-12-35b280174f65> in <module>() 1 a = 4 2 b = 3
----> 3 sum(a, b) TypeError: 'int' object is not iterable

But we see that this leads to an error. We can fix this situation by pre-writing our variables for summation in an iterable object, in a list or a tuple, or a set, for example:

a = 4
b = 3 tuple_sum = (a, b)
list_sum = [a, b]
set_sum = {a, b}
dict_sum = {a: 0, b: 1} print(sum(tuple_sum))
print(sum(list_sum))
print(sum(set_sum))
print(sum(dict_sum))

Output:

7
7
7
7

As you can see, the result remains the same. Whether we are using pre-entry into a tuple, list, set, or even a dictionary. Note that for dictionaries, the sum function sums key values by default.

You can even write one variable to a list and calculate the sum of this list. As a search on stackoverflow shows, newbies in programming often try to calculate the sum of one element, which of course leads to an error.

a = 2
sum(a)

Output:

TypeError Traceback (most recent call last) <ipython-input-21-5db7366faaa2> in <module>() 1 a = 2
----> 2 sum(a) TypeError: 'int' object is not iterable

But if we pass an iterable object for example a list (even if it consists of one element) to the function then the calculation is successful.

a = 2
list_sum = [a]
print(sum(list_sum))

Output:

2

Another way to form such a list is to use the list.append method:

a = 2
list_sum = []
list_sum.append(a)
print('Sum of "a":', sum(list_sum))
b = 5
list_sum.append(b)
print('Sum of "a" and "b":',sum(list_sum))

Output:

'''
Sum of "a": 2
Sum of "a" and "b": 7 '''

Let’s consider a more complex version of the same error. We have a function that should calculate the sum of the elements of the list including the elements of the nested lists.

def nested_sum(list_): total = 0 for item in list_: item = sum(item) total = total + item return total
list1 = [1, 2, 3, [4, 5]]
print(nested_sum(list1))

Output:

TypeError Traceback (most recent call last) <ipython-input-35-c30be059e3a4> in <module>() 6 return total 7 list1 = [1, 2, 3, [4, 5]]
----> 8 nested_sum(list1) <ipython-input-35-c30be059e3a4> in nested_sum(list_) 2 total = 0 3 for item in list_:
----> 4 item = sum(item) 5 total = total + item 6 return total TypeError: 'int' object is not iterable

You can probably already see what the problem is here. The loop parses the list into its elements and goes through them. The items in our list are numbers 1, 2, 3 and a list [4, 5]. You can compute a sum of the list but you can’t get the sum of one number in Python. So we have to rewrite code.

def nested_sum(list_): total = 0 for item in list_: if type(item) == list: item = sum(item) total = total + item return total
list1 = [1, 2, 3, [4, 5]]
print(nested_sum(list1))

Output:

15

Now, in the loop, we first of all check the type of our local variable 'item' and if it is a list, then with a clear conscience we calculate its sum and rewrite the variable 'item' with the resulting value. If it’s just a single element, then we add its value to the 'total'.

Incorrect use of ‘for’ loop

Let’s consider another common case of this error. Can you see right away where the problem is?

n = 10
for i in n: print(i)

Output:

TypeError Traceback (most recent call last) <ipython-input-24-7bedb9f8cc4c> in <module>() 1 n = 10
----> 2 for i in n: 3 print(i) TypeError: 'int' object is not iterable

Perhaps the error in this construction is associated with the tradition of teaching children the Pascal language at school. There you can actually write something similar: for i:=1 to n do.

But in Python ‘for’ loops are used for sequential traversal. Their construction assumes the presence of an iterable object. In other languages, a ‘for each’ construct is usually used for such a traversal.

Thus, the ‘for’ construct in Python expects an iterable object which to be traversed, and cannot interpret an integer. This error can be easily corrected using the function ‘range’. Let’s see how our example would look in this case.

n = 10
for i in range(n): print(i)

Output:

0
1
2
3
4
5
6
7
8
9

The ‘range’ function can take 3 arguments like this: range(start, stop[, step]). The ‘start’ is the first number from which the loop will begin, ‘stop’ is the number at which the loop will end. Please note that the number ‘stop’ will not be included in the cycle. The ‘step’ is how much the number will differ at each next iteration from the previous one. By default, ‘start’ has a value of 0, ‘step’=1, and the stop parameter must be passed compulsory. More details with examples can be found in the documentation. https://docs.python.org/3.3/library/stdtypes.html?highlight=range#range

for i in range(4, 18, 3): print(i)

Output:

4
7
10
13
16

Here is a small example of using all three parameters of the ‘range’ function. In the loop, the variable ‘i’ in the first step will be equal to 4, ‘i’ will never be greater than or equal to 18, and will increase in increments of 3.

Problems With Tuples

The next example where an error "typeerror: ‘int’ object is not iterable" can occur is multiple assignment of values using a tuple. Let’s take a look at an example.

a, b = 0

Output:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-6-6ffc3a683bb5> in <module>()
----> 1 a, b = 0 TypeError: 'int' object is not iterable

It’s a very pythonic way of assignment but you should be careful with it. On the left we see a tuple of two elements ‘a’ and ‘b’, so to the right of the equal sign there must also be a tuple (or any other iterable object) of two elements. Don’t be intimidated by writing a tuple without parentheses, this is an allowable way in Python.

So to fix this error, we can write the assignment like this:

a, b = 0, 0
print(a)
print(b)

Output:

0
0

And a few more examples of how you can assign values to several variables at once:

a, b = (1, 2)
c, d = {3, 4}
e, f = [5, 6]
print(a, b, c ,d ,e, f)

Output:

1 2 3 4 5 6

A similar problem can arise if you use a function that returns multiple values as a tuple. Consider, for example, a function that returns the sum, product, and result of division of two numbers.

def sum_product_division(a, b): if b != 0: return a + b, a * b, a / b else: return -1 sum_, product, division = sum_product_division(6, 2)
print("The sum of numbers is:", sum_)
print("The product of numbers is:", product)
print("The division of numbers is:", division)

Output:

The sum of numbers is: 8
The product of numbers is: 12
The division of numbers is: 3.0

Note that I have added an underscore to the variable name ‘sum_’. This is because the word ‘sum’ is the name of the built-in function that we discussed above. As you can see, in the case when ‘b’ is not equal to zero, our code works correctly, the variables take the appropriate values. Now let’s try to pass the value ‘b’ equal to 0 to the function. Division by zero will not occur, since we provided for this in the function and return -1 as an error code.

sum_, product, division = sum_product_division(6, 0)
print("The sum of numbers is:", sum_)
print("The product of numbers is:", product)
print("The division of numbers is:", division)

Output:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-9-6c197be50200> in <module>()
----> 1 sum_, product, division = sum_product_division(6, 0) 2 print("The sum of numbers is:", sum_) 3 print("The product of numbers is:", product) 4 print("The division of numbers is:", division) TypeError: 'int' object is not iterable

The error “TypeError: 'int' object is not iterable” occurs again. What’s the matter? As I already said, this situation is similar to the previous one. Here we also try to assign values to several variables using a tuple. But our function when there is a danger of division by zero returns not a tuple but the only value of the error code ‘-1’.

How to fix it? For example, we can check the type of a returning value. And depending on this type, already output the result. Let’s do it!

result = sum_product_division(6, 0)
if type(result) == int: print("Error, b should not be zero!")
else: sum_, product, division = result print("The sum of numbers is:", sum_) print("The product of numbers is:", product) print("The division of numbers is:", division)

Output:

Error, b should not be zero!

Here’s another example:

result = sum_product_division(6, 3)
if type(result) == int: print("Error, b should not be zero!")
else: sum_, product, division = result print("The sum of numbers is:", sum_) print("The product of numbers is:", product) print("The division of numbers is:", division)

Output:

The sum of numbers is: 9
The product of numbers is: 18
The division of numbers is: 2.0

We can also redesign our function to return the result of the operation from the beginning of the tuple. And use some trick when assigning variables. Take a look at this:

def sum_product_division(a, b): if b != 0: return "Ok", a + b, a * b, a / b else: return ("Error",) status, *results = sum_product_division(6, 0)
print(status, results) status, *results = sum_product_division(6, 2)
print(status, results)

Output:

Error []
Ok [8, 12, 3.0]

If division by zero is possible we return a tuple with a single element – the string ‘Error’. If everything is correct then we return a tuple where the first element is a status message – the string ‘Ok’ and then the results of the calculations follow sequentially: sum, product, result of division.

There can be many options here, because this is a function that we wrote ourselves, so we can fix it as we please. But it so happens that we use functions from libraries. For example here is an error from a topic on stackoverflow.

import subprocess
data = subprocess.call(["echo", '''Hello World!
Hello!'''])
sum_lines = 0
for line in data: print(line) sum_lines += 1
print(sum_lines)

Output:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-32-8d4cf2cb9ee0> in <module>() 3 Hello!''']) 4 sum_lines = 0
----> 5 for line in data: 6 print(line) 7 sum_lines += 1 TypeError: 'int' object is not iterable

I rewrote the code a bit so that the essence is clear. We want to run a command on the command line and count the number of lines printed on the screen. In our case, it will be just a command to display a little message to the World.

We should read a documentation to figure it out. The subprocess module allows you to spawn new processes, connect to their input /output /error pipes, and obtain their return codes. We see that the ‘call’ function starts the process on the command line, then waits for its execution and returns the execution result code! That’s it! The function returned the code of execution. It’s integer and we are trying to traverse this integer in a loop. Which is impossible, as I described above.

What to do? Explore the documentation for the module further. And so we find what we need. The ‘check_output’ function. It returns everything that should be displayed in the console when the command being passed is executed. See how it works:

import subprocess
data=subprocess.check_output(["echo", '''Hello World!
Hello!'''])
sum_lines = 0
for line in data.splitlines(): print(line) sum_lines +=1
print(sum_lines)

Output:

b'Hello World!'
b'Hello!'
2

Great! We got a byte string separated by newline symbols ‘\n’ at the output. And we can traverse over it as shown with a ‘splitlines’ function. It returns a list of the lines in the string, breaking at line boundaries. This method uses the universal newlines approach to splitting lines. Line breaks are not included in the resulting list unless ‘keepends’ parameter is given and true.

Thus, we fixed the error and got what we needed, but had to do a little research in the documentation. This study of documentation is one of the most effective ways to improve your programming skills.

The snag with lists

Often the error "TypeError: 'int' object is not iterable" appears when using various functions related to lists. For example I have a list of my exam grades. I want to add to it a grade in physical education which I passed perfectly in contrast to math. I am trying to do it like this:

grades = [74, 85, 61]
physical_education_mark = 100
grades += physical_education_mark

I’m using the most conventional method to perform the list concatenation, the use of “+” operator. It can easily add the whole of one list behind the other list and hence perform the concatenation. But it doesn’t work here. Because list concatenation is only possible for two lists. We cannot combine list and number. The most obvious way to solve this problem is to use the ‘append’ function. It is designed just to do that. It adds an item to the list. The argument can also be an integer.

grades = [74, 85, 61]
physical_education_mark = 100
grades.append(physical_education_mark)
print(grades)

Output:

[74, 85, 61, 100]

Voila! We did it! Of course, if we really want to use the ‘+’ operator, we can pre-write our physical education grade in a list with one element, for example like this:

grades = [74, 85, 61]
physical_education_mark = 100
grades += [physical_education_mark]
print(grades)

Output:

[74, 85, 61, 100]

The result is expectedly the same as the previous one. Move on.

Another list-related problem is when you’re trying to add element with ‘extend‘ method. This method can be very useful to concatenate lists. Unlike the ‘+’ operator, it changes the list from which it is called. For example, I need to add new semester grades to the grades list. It’s easy to do with the method ‘extend’:

grades = [74, 85, 61]
new_semestr_grades = [85, 79]
physical_education_mark = 100
grades.extend(new_semestr_grades)
print(grades)

Output:

[74, 85, 61, 85, 79]

So we did it easily but wait! We forgot our perfect physical education score!

grades = [74, 85, 61]
new_semestr_grades = [85, 79]
physical_education_mark = 100
grades.extend(new_semestr_grades)
grades.extend(physical_education_mark)
print(grades)

Output:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-48-6d49503fc731> in <module>() 3 physical_education_mark = 100 4 grades.extend(new_semestr_grades)
----> 5 grades.extend(physical_education_mark) 6 print(grades) TypeError: 'int' object is not iterable

And we can’t do it like this. ‘extend’ is waiting for iterable object as an argument. We can use ‘append’ method or pre-writing manner.

grades = [74, 85, 61]
new_semestr_grades = [85, 79]
physical_education_mark = [100]
grades.extend(new_semestr_grades)
grades.extend(physical_education_mark)
print(grades)

Output:

[74, 85, 61, 85, 79, 100]

Did you notice the difference? I originally defined the variable ‘physical_education_mark’ as a list with one item. And this works perfect!

Now suppose we need a function that will find the location of variables in the formula “A + C = D – 6”. If you know that each variable in the formula is denoted by one capital letter. We’re trying to write it:

def return_variable_indexes(formula): for element in formula: if element.isupper(): indexes = list(formula.index(element)) return indexes print(return_variable_indexes("A + C = D - 6"))

Output:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-44-5a9b17ff47ae> in <module>() 5 return indexes 6 ----> 7 print(return_variable_indexes("A + C = D - 6")) <ipython-input-44-5a9b17ff47ae> in return_variable_indexes(formula) 2 for element in formula: 3 if element.isupper():
----> 4 indexes = list(formula.index(element)) 5 return indexes 6 TypeError: 'int' object is not iterable

Yes, we got the same error again. Let’s try to understand what’s the matter. We go through the elements of the string ‘formula’. And if this element is a upper-case letter then we use the ‘index’ function to find its position in the string. And try to write it into a list ‘indexes’. So we have two functions ‘index’ and ‘list’. What is returning value of the ‘index’ function? It is an integer number the position at the first occurrence of the specified value. So we’re trying to add this to the list ‘indexes’ with a ‘list’ function. And stop here! The ‘list’ constructor takes one argument. It should be an iterable object so that could be a sequence (string, tuples) or collection (set, dictionary) or any iterator object. Not an integer number of course. So we can use ‘append’ method again and get the result we need:

def return_variable_indexes(formula): indexes = [] for element in formula: if element.isupper(): indexes.append(formula.index(element)) return indexes print(return_variable_indexes("A + C = D - 6"))

Output:

[0, 4, 8]

And just for fun you can do it as a one-liner using a list comprehension and the ‘enumerate’ method. It takes iterable object as an argument and returns its elements with index as tuples (index, element) one tuple by another:

def return_variable_indexes(formula): return [index_ for index_, element in enumerate(formula) if element.isupper()]
print(return_variable_indexes("A + C = D - 6"))

Output:

[0, 4, 8]

Conclusion

We have considered some cases in which an error “TypeError: ‘int’ object is not iterable” occurs. This is always a situation where the interpreter expects an iterable object, and we provide it an integer.

The most common cases of such errors:

  • incorrect sum argument;
  • incorrect handling of tuples;
  • related to various functions and methods of lists

I hope that after reading this article you will never have a similar problem. And if it suddenly arises then you can easily solve it. You may need to read the documentation for this though =)

Posted on Leave a comment

Python One Line Generator

A generator function is a Pythonic way to create an iterable without explicitly storing it in memory. This reduces memory usage of your code without incurring any additional costs.

The following code shows a function get_numbers(n) that returns a list of n random numbers.

import random # NOT A GENERATOR!
# Create and return a list of numbers
def get_numbers(n): numbers = [] for i in range(n): numbers.append(random.random()) # List of n elements exists in memory return numbers # Sum up 1000 random numbers
s = 0
for x in get_numbers(1000): s += x
print(s)

However, this is not very efficient code because you create a list in advance without need. What if you had 1,000,000,000 numbers? Your memory would quickly fill up!

A better way is to use a generator function with the yield keyword that creates the random numbers dynamically as they are iterated over:

import random # GENERATOR
# Generate numbers one by one
def generate_numbers(n): for i in range(n): yield random.random() # Sum up 1000 random numbers
s = 0
for x in generate_numbers(1000): s += x
print(s)

There are two big advantages to using a generator:

  • (1) You don’t have to create a huge list first and store it in memory but generate the next element as you iterate over it.
  • (2) It’s shorter and more concise.

However, it may not be concise enough for you! 😉 So, here’s the problem addressed in this article:


Problem: Can we write a one-line generator?

Let’s dive into different methods to accomplish this!

Method 1: One-Liner Generator Function

print(sum(random.random() for i in range(1000)))

The code consists of the following parts:

  • The print() function prints the result of the expression to the shell.
  • The sum() function sums over all values in the following iterable.
  • The generator expression random.random() for i in range(1000) generates 1000 random numbers and feeds them into the outer sum() function without creating all of them at once.

This way, we still don’t store the whole list of 1000 numbers in memory but create them dynamically.

Method 2: exec()

The following method is not pretty—but it solves the problem to create a generator in a single line of code.

exec('def g(n):\n for i in range(n):\n yield random.random()')

The exec() function can be used to one-linerize every Python code snippet under the sun. Just pass the code you want to run as a string and replace all newlines with the newline character '\n'. This way, you can create a generator function g(n) that dynamically creates n random numbers. You can now iterate them using the standard code snippet:

s = 0
for x in g(1000): s += x
print(s)
# 488.318368852096

Because the numbers are random, the output will be different for you. You can try it yourself in our interactive shell:

Exercise: What’s the output for you? Why is it different than ours?

Python One-Liners Book

Python programmers will improve their computer science skills with these useful one-liners.

Python One-Liners

Python One-Liners will teach you how to read and write “one-liners”: concise statements of useful functionality packed into a single line of code. You’ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.

The book’s five chapters cover tips and tricks, regular expressions, machine learning, core data science topics, and useful algorithms. Detailed explanations of one-liners introduce key computer science concepts and boost your coding and analytical skills. You’ll learn about advanced Python features such as list comprehension, slicing, lambda functions, regular expressions, map and reduce functions, and slice assignments. You’ll also learn how to:

  Leverage data structures to solve real-world problems, like using Boolean indexing to find cities with above-average pollution
  Use NumPy basics such as array, shape, axis, type, broadcasting, advanced indexing, slicing, sorting, searching, aggregating, and statistics
  Calculate basic statistics of multidimensional data arrays and the K-Means algorithms for unsupervised learning
  Create more advanced regular expressions using grouping and named groups, negative lookaheads, escaped characters, whitespaces, character sets (and negative characters sets), and greedy/nongreedy operators
  Understand a wide range of computer science topics, including anagrams, palindromes, supersets, permutations, factorials, prime numbers, Fibonacci numbers, obfuscation, searching, and algorithmic sorting

By the end of the book, you’ll know how to write Python at its most refined, and create concise, beautiful pieces of “Python art” in merely a single line.

Get your Python One-Liners Now!!

Posted on Leave a comment

Calculating Entropy with SciPy

Problem: How to calculate the entropy with the SciPy library?

Solution: Import the entropy() function from the scipy.stats module and pass the probability and the base of the logarithm into it.

from scipy.stats import entropy p = [0.5, 0.25, 0.125, 0.125]
e = entropy(p, base=2) print(e)
# 1.75

Try It Yourself: Run this code in the interactive code shell!

Exercise: Change the probabilities. How does the entropy change?

Let’s start slowly! You’ll going to learn the most relevant background about entropy next.

Entropy Introduction

In thermodynamics, entropy is explained as a state of uncertainty or randomness.

In statistics, we borrow this concept as it easily applies to calculating probabilities.

When we calculate statistical entropy, we are quantifying the amount of information in an event, variable, or distribution. Understanding this measurement is useful in machine learning in many cases, such as building decision trees or choosing the best classifier model.

We will discuss the applications of entropy later in this article, but first we will dig into the theory of entropy and how to calculate it with the use of SciPy.

Calculating the Entropy

Calculating the information of a variable was developed by Claude Shannon, whose approach answers the question, how many “yes” or “no” questions would you expect to ask to get the correct answer?

Consider flipping a coin. Assuming the coin is fair, you have 1 in 2 chance of predicting the outcome. You would guess either heads or tails, and whether you are correct or incorrect, you need just one question to determine the outcome.

Now, say we have a bag with four equally sized disks, but each is a different color:

To guess which disk has been drawn from the bag, one of the better strategies is to eliminate half of the colors. For example, start by asking if it is Blue or Red. If the answer is yes, then only one more question is required since the answer must be Blue or Red. If the answer is no, then you can assume it is Green or Gray, so only one more question is needed to correctly predict the outcome, bringing our total to two questions regardless if the answer to our question is Green of Gray.

We can see that when an event is less likely to occur, choosing 1 in 4 compared to 1 in 2, there is more information to learn, i.e., two questions needed versus one.

Shannon wrote his calculation this way:

Information(x) = -log(p(x))

In this formula log() is a base-2 algorithm (because the result is either true or false), and p(x) is the probability of x.

As the higher the information value grows, the less predictable the outcome becomes.

When a probability is certain (e.g., a two-headed coin flip coming up heads), the probability is 1.0, which yields an information calculation of 0.

We can run Shannon’s calculation in python using the math library shown here:

When we change the probability to 0.25, as in the case of choosing the correct color of the disk, we get this result:

While it appears that the increase in information is linear, what happens when we calculate the roll of a single die, or ask someone to guess a number between 1 and 10? Here is a visual of the information calculations for a list of probabilities from less certain (p = 0.1) to certain (p = 1.0):

The graph shows that with greater uncertainty, the information growth is sub-linear, not linear.

Unequal Probabilities

Going back to the colored disks example, what if we now have 8 disks in the bag, and they are not equally distributed? Look at this breakout by color:

Color Quantity
Blue 1
Green 1
Red 2
Gray 4
Total 8

If we use the original strategy of eliminating half of the colors by asking if the disk Blue or Green, we become less efficient since there is a combined 0.25 probability of either color being correct in this scenario.

We know that gray has the highest probability. Using a slightly different strategy, we first ask if Gray is correct (1 question), then move on to the next highest probability, Red (2nd question), and then to check if it is Blue or Green (3rd question).

In this new scenario, weighting our guesses will lead to less information required. The tables below show the comparison of the two methods. The info column is the product of the Probability and Questions columns.

Equal Guesses
Color Prob Q’s Info
Blue 0.25 2 0.50
Green 0.25 2 0.50
Red 0.25 2 0.50
Gray 0.25 2 0.50
Total 1 8 2.00
Weighted Guesses
Color Prob Q’s Info
Blue 0.125 3 0.375
Green 0.125 3 0.375
Red 0.25 2 0.50
Gray 0.5 1 0.50
Total 1 9 1.75

The Equal guess method takes an average of 2 questions, but the weighted guess method takes an average of 1.75.

We can use the Scipy library to perform the entropy calculation. Scipy’s “stats” sub-library has an entropy calculation that we can use. Here is the code to calculate the entropy for the scenario where the four disks have different probabilities:

The entropy method takes two entries: the list of probabilities and your base. Base=2 is the choice here since we are using a binary log for the calculation.

We get the same result as in the table shown above. With minimal code, the Scipy library allows us to quickly calculate Shannon’s entropy.

Further Uses

Entropy calculation is successfully used in real-world application in Machine Learning. Here are some examples.

Decision Trees

A Decision Tree is based on a set of binary decisions (True or False, Yes or No). It is constructed with a series of nodes where each node is question: Does color == blue? Is the test score > 90? Each node splits into two and decomposes into smaller and smaller subsets as you move through the tree.

 Accuracy with your Decision Tree is maximized by reducing your loss. Using entropy as your loss function is a good choice here. At each step moving through the branches, entropy is calculated before and after each step. If the entropy decreases, the step is validated. Otherwise you must try another branch.

Classification with Logistic Regression

The key to a logistic regression is minimizing the loss or error for the best model fit. Entropy is the standard loss function for logistic regression and neural networks.

Code Sample

While there are several choices for using entropy as your loss function in machine learning, here is a snippet of code to show how the selection is made during model compilation:

Conclusion

The purpose of this article was to shed some light on the use of entropy with Machine Learning and how it can be calculated with Python.

Posted on Leave a comment

How to Fix “ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()”

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

If you run the following code, you’ll experience a special ValueError:

import numpy as np
a = np.array([1, 2, 3])
b = bool(a)
print(b)

The output will be this error message:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Try It Yourself in our interactive browser shell:

Solution: Using the Numpy functions called logical_and() and logical_or() instead of Python’s logical operators (“and” and “or”) is the solution.

Why does the ValueError occur at all?

Many programmers who start learning Numpy think they can use Python’s logical operators while writing code, but the creators of this module have decided that there is not one commonly understood way to rate an array in a boolean context.

It might mean True if any element is True, or True if all elements are True, or True if the array is of non-zero length. And we only mentioned three possibilities—there are more!

Because different users may have various needs and goals, developers refused to speculate and decided to raise the ValueError each time someone tries to evaluate an array in a Boolean context, so what did they give in exchange?

Logical_and() function – the equivalent for “and”

The logical_and() function is equivalent to the Python built-in “and” logical operator. When we use this function, the program will return an array with True and False values. 

This function has two crucial parameters, i.e. our input arrays, which we put after the comma (in this example arr1 < 3 and arr_2 > 3). Let’s take a look at the example:

import numpy as np arr_1 = np.arange(5)
arr_2 = np.arange(6, 10)
arr_3 = np.array(['First', 'Second', 'Third', 'Fourth', 'Fifth']) mask = np.logical_and(arr_1 < 3, arr_2 > 3)
print(arr_3[mask])

Output:

['First' 'Second' 'Third']

The code printed the first, second and third element of the array arr_3, because it checked our conditions and it came out that the first three numbers of our arrays meet the conditions at the same time. 

Logical_or() function – the equivalent for “or”

The functionality is the same as the previous one. It also has two most important parameters—input arrays. The only difference is in the behavior of the code, after all we want to achieve something different:

import numpy as np arr_1 = np.arange(5)
arr_2 = np.arange(5, 10)
arr_3 = np.array(['First', 'Second', 'Third', 'Fourth', 'Fifth']) mask = np.logical_or(arr_1 >= 3, arr_2 < 3)
print(arr_3[mask])

As at least one of the elements in positions 4 and 5 of our arrays meets our condition, the result is as follows:

['Fourth' 'Fifth']

Logical And with “&” and Logical Or with “|”

Instead of writing logical_and() or logical_or() we can use & and | symbols. Take a look at this code: 

import numpy
arr_1 = np.arange(5)
arr_2 = np.arange(5, 10)
arr_3 = np.array(['First', 'Second', 'Third', 'Fourth', 'Fifth']) # Same functionality as logical_and
mask = np.array((arr_1 < 3) & (arr_2 > 3))
print(arr_3[mask]) # Same functionality as logical_or
mask = np.array((arr_1 >= 3) | (arr_2 < 3))
print(arr_3[mask])

Output:

['Fourth' 'Fifth']
['First' 'Second' 'Third']

any() and all()

As these two functions appear in the topic, here is a quick explanation of what they do at all!

The function any() checks if any of the elements are non-zero and all() checks if all elements are non-zero.These functions takes several parameters, but two are the most important:

  • a -> Input array or object that can be converted to an array.
  • axis -> Axis or axes along which a logical OR reduction is performed. The default (axis=None) is to perform a logical OR over all the dimensions of the input array. axis may be negative, in which case it counts from the last to the first axis.
arr_1 = np.array([[1, 2, 3, 0],[0, 1, 2, 3]]) print('Outputs of function any')
print(np.any(arr_1))
print(np.any(arr_1, axis=0))
print(np.any(arr_1, axis=1)) print('\nOutputs of function all')
print(np.all(arr_1))
print(np.all(arr_1, axis=0))
print(np.all(arr_1, axis=1))

Output:

Outputs of function any:
True
[ True True True True]
[ True True] Outputs of function all:
False
[False True True False]
[False False]

As you can see, our script checked at the beginning if any values along the axis are not zero.

Note: axis=0 is a vertical axis and axis=1 is a horizontal axis.

Summary

We learned why there is a ValueError error when we want to use the logical operators built into Python (“and” and “or”) in logical operations while using arrays. Next, there were 2 equivalents of these logical operators (“logical_and” and “logical_or”) and an even faster way to achieve the same. Finally, the function any() and all() in the Numpy module was explained.

References

Posted on Leave a comment

Python Keyboard Errors

When you’re learning and working with Python, you’re going to encounter errors and there’s no getting around that fact. So how do you get around them? It helps to learn about the errors. We’re going to take the opportunity to learn one of them today.

What is “Keyboard Interrupt”?

This is an error that you’re not likely to run across often unless you are running a Python program for more than two seconds. If your program has loops, a while loop specifically, then there might come a point when you need to stop the program. Because while loops, when not written with an ending in mind, keep going, like the Energizer Bunny.

In this example, we created a simple while loop that is loaded with print statements. If you run it, then the script will keep printing the same three statements.

Impressive, isn’t it? You can keep it going for a long time, but you might want to think about stopping it. When you press Ctrl and C, you’ll get this Exception that pops up, otherwise known as the Keyboard Interrupt.

This is one of the built-in exceptions that you’ll come across when programming in Python. Based on the hierarchy, the Keyboard Interruption exception is right towards the top, underneath Base Exception and System Exit. You can find the full hierarchy here.

Now you might be wondering if there is any way you can keep that error from popping up? Not really. It is there for a reason and it’s to stop the script from running.

Unless you want to keep it going forever. We hope your computer is built for that. However, we do know of a good way for you to clean it up a little bit.

The Try/Except method

If you’re hoping to avoid those awkward error messages that pop up when running your Python code, then this is the best route for you to go. The Try/Except method is another solid way for you to run your Python code. And you can do so without dealing with specific errors in your code

This is the simplest example that we can provide for how it works.

You put the code that you want to run underneath the try. And underneath your except is what you can enter to deal with any errors you might encounter. This can help you with larger projects. If you’re building a Twitter bot, for example, you could set it up so that it runs your code and if there’s a problem with getting the tweet out, you’ll be able to catch the error.

You might not think you’ll need it, but once you start catching errors when running your code, you’ll want to use it.

To get it to work, we’re going to make a few adjustments.

At the top of our script, we imported the Sys module, which is built-in for Python. You don’t need to install it.

Inside of our while loop, we enter our Try and Except block. Underneath Try, we put in three print statements. You’re free to put as many print statements as you want in this. If you want to make it 10, then go for it! We want you to be ambitious with your infinite time loop.

Underneath except, we just have one print statement in there. Obviously, you can do more, but that would defeat the purpose. Please don’t go crazy with your print statements.  Put them all underneath your try statement.

What we put in next, underneath our print statement, is what you would consider an exit command. And there’s more than one you can use. However, for this instance, we just chose sys.exit(). You can also import os and use the exit command for that one.

In fact, you don’t need to import any Python modules. You can just use quit() and it works just as well. But we like to be fancy sometimes.

Works pretty good, don’t you think?

Let’s Build A Time Loop

What we have now makes for a pretty good time loop. But now we can try to have a little more fun with our Python script. Let’s build it differently and see how that can work. And we’ll set it up so you can’t escape from the loop.

Now when we say you won’t escape, we mostly mean it won’t be as simple as pressing Ctrl + C on your keyboard. If you’re worried about stopping it, all you would theoretically need to do is just exit out of your command line. It would stop at that point. Of course, you’d have to start all over by re-opening your line. But let’s have some fun.

First, you’ll need to import the Time module, which is built-in already for Python. We’ll be making some sleep functions later on in our code. But first, we’re going to create the time loop function.

It will be simple. Just one print statement involved. However, you can create as many print statements as your heart desires.

The function will look like this:

Once that is done, we can make our while loop. Embedded in it will be our try and except blocks. Underneath try, we’ll include the time_loop() function, as well as our sleep function. Inside the parenthesis, you’ll want to put in how long you want the program to sleep for. It’s done in seconds. You could have it at 1, 100, 1000, 10000, whatever you want. For our example, we chose five seconds. It should be a little easier on your eyes, instead of having it go non-stop. Gives you that breath of fresh air!

While under the except one, we add another print statement. It might seem cruel to you, but hey it’s a time loop. You’re stuck! Isn’t that how time loops work, at least? We don’t know for sure. We watched Palm Springs recently and that fills up about 98% of our knowledge on the topic.

But your time loop should end up looking like this:

Pretty cool, right? You’ll have a fun and frustrating time trying to escape this time loop. Of course, like we said, you can always exit out of command line. But why take the risk? 😊

If nothing else, now is your chance to play around with the script. Have some fun. Maybe try to do something different with your time loop. Get a little creative!

Posted on Leave a comment

Replacements For Switch Statement In Python?

Summary: Since switch-case statements are not a part of Python, you can use one of the following methods to create a switch-case construct in your code :

Problem: Given a selection control switch case scenario in python; how to find a replacement for switch statement since python does not have a provision for switch-case statements.

Overview: Before we discuss the solutions to our problem, we must have a clear picture of the switch-case statement itself. To understand the switch statements let us answer a few questions :

What is a switch statement?

In programming languages like C, C++, C#, Java, etc switch is a conditional statement used to test the value of a variable and then compare this value with several cases. Once the match case is found, the block of code defined within the matched case is executed. Let us have a look at the following flowchart to visualize the working principle of switch case:

fig: switch-case statement depicting program flow when nth case is TRUE

switch vs if-else

In a program that needs a complex set of nested if statements, a switch statement is always a better and more efficient alternative in such situations. Following are some of the key factors which should be taken into consideration while deciding whether to use switch case or if-else statements in the code:

  1. If the number of cases is more, then switch-case statements are always more efficient and faster than if-else.
  2. Switch case statements are more suited for fixed data values whereas if-else conditionals should be used in case of boolean values.
  3. Switch statements check expressions based only on a single parameter / variable value whereas if-else conditional statements can be used to test expressions on a range of values.

There are a few other factors governing the proper usage of the two statements. However, that is beyond the scope of this article, as we are going to focus on the replacements for switch statements in python in this article. Now that brings us to the next and probably the most important question.

Why does Python Not Have A “switch” Statement?

Thinking Face on Facebook 2.0

You might have been wondering all the while, why are we even proposing a solution for replacing the switch statement in python. Does python not have a “switch” statement?

The answer is NO!!! Python does not need one.

  • Most programming languages have switch-case statements because they lack proper mapping constructs. While in Python, we have well-defined mapping constructs and you can easily have a mapping table in the form of a python dictionary.
  • Furthermore, Python does not have a switch statement because none of the proposals that have been suggested so far have been deemed acceptable.

#Note

Guido van Rossum says, “Some kind of switch statement is found in many languages and it is not unreasonable to expect that its addition to Python will allow us to write up certain code more cleanly and efficiently than before.” So this might indicate that in the future we might find that the switch statement is implemented in Python, though it is quite unlikely to happen anytime soon. (But the future is full of unpredictabilities!!!)

Now that we have an overview of the switch-statement, let us find out the alternatives for switch statements that can be used in python.

Solutions

Let us discuss how we can create alternatives to switch cases and ask the user to select an option for the operation they want to perform. Based on the users’ choice the output shall be displayed. Here’s a sample output of what we want to achieve:

Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division Enter your choice: 1
Enter 1st Number: 500
Enter 2nd Number: 2000
Answer = 2500

So without further delay, let the games begin!

Method 1: Using A Dictionary

A dictionary in python stores items in the form of key-value pairs such that a certain value maps to a specific key. Let us have a look at the following code to understand how we can use a dictionary as an alternative to switch case and solve our problem (Please follow the comments to get a better grip on the code):

# The default case when no case matches
def default(a, b): return "Invalid Entry!" def switch(option, a, b): # create the dictionary and switch-cases / choices return { '1': lambda a, b: a + b, '2': lambda a, b: a - b, '3': lambda a, b: a * b, '4': lambda a, b: a / b, }.get(option, default)(a, b) # User Menu Prompt
print('''
Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division ''')
# Accept User Entry
choice = input("Enter your choice: ")
x = int(input("Enter 1st Number: "))
y = int(input("Enter 2nd Number: "))
# Pass Values Entered by User to evaluate and then Print the output
print(switch(choice, x, y))

Output:

Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division Enter your choice: 1
Enter 1st Number: 15
Enter 2nd Number: 50
Answer = 65

Things to remember:

  • get() is an in-built function in python which returns the value for a specified key.
  • lambda is a keyword in python which is used to define an anonymous function inside another function.

I would strongly recommend you to go through these articles for an in-depth understanding of lambda and python dictionary functions and their usage:

  1. Link to learn about Python Dictionaries.
  2. Link to learn about Python Lambda Function.

If you are not comfortable with the usage of get() and lambda, I have a solution for you too. Have a look at the following code which might be a little lengthier and more complex in terms of time and memory usage but is certainly easier to grasp :

# Addition
def choice1(x, y): return x + y # Subtraction
def choice2(x, y): return x - y # Multiplication
def choice3(x, y): return x * y # Division
def choice4(x, y): return x / y def switch(option, a, b): try: # create the dictionary and switch-cases / choices return { '1': choice1, '2': choice2, '3': choice3, '4': choice4, }[choice](x, y) except KeyError: # Default Case return 'Invalid Entry!' # User Menu Prompt
print('''
Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division ''')
# Accept User Entry
choice = input("Enter your choice: ")
x = int(input("Enter 1st Number: "))
y = int(input("Enter 2nd Number: "))
# Pass Values Entered by User to evaluate and then Print the output
print("Answer = ", switch(choice, x, y))

Output

Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division Enter your choice: 2
Enter 1st Number: 20
Enter 2nd Number: 10
Answer = 10

Method 2: Creating A Switch Class

Another workaround to use switch cases in our programs is to create a class with basic switch-case functionality defined within it. We can define each case within separate functions inside the class.

Let us have a look at the following program to understand how we can create a switch-case class (As always, I request you to read the comments within the code to get a better grip):

class SwitchCase: def case(self, option, x, y): default = "Invalid Entry!" # lambda invokes the default case # getattr() is used to invoke the function choice that the user wants to evaluate return getattr(self, 'choice' + str(option), lambda x, y: default)(x, y) # Addition def choice1(self, x, y): return x + y # Subtraction def choice2(self, x, y): return x - y # Multiplication def choice3(self, x, y): return x * y # Division def choice4(self, x, y): return x / y # Default Case def default(self, x, y): return "Invalid Entry!" # creating the SwitchCase class object
switch = SwitchCase()
# User Menu Prompt
print('''
Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division ''')
# Accept User Entry
option = input("Enter your choice: ")
x = int(input("Enter 1st Number: "))
y = int(input("Enter 2nd Number: "))
# Pass Values Entered by User to evaluate and then Print the output
print("Answer = ", switch.case(option, x, y))

Output

Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division Enter your choice: 4
Enter 1st Number: 24
Enter 2nd Number: 4
Answer = 6.0

Method 3: A Quick Fix Using Lambda Function

Though this method might not be the best in terms of code complexities, however, it might come in handy in situations where you want to use a function once and then throw it away after the purpose is served. Also, this method displays how long pieces of codes can be minimized, hence this method makes a worthy entry into the list of our proposed solutions.

var = (lambda x, y, c: x + y if c == '1' else x - y if c == '2' else x * y if c == '3' else x / y if c == '4' else 'Invalid Entry!') # User Menu Prompt
print('''
Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division ''')
# Accept User Entry
option = input("Enter your choice: ")
x = int(input("Enter 1st Number: "))
y = int(input("Enter 2nd Number: "))
print(var(x, y, option))

Output

Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division Enter your choice: 4
Enter 1st Number: 500
Enter 2nd Number: 25
20.0

Method 4: Using if-elif-else

Sometimes it is always better to follow the standard programming constructs because they provide us the simplest solutions in most cases. Using the if-elif-else might not be the exact replacement for a switch case but it provides a simpler construct and structure. Also, it avoids the problem of KeyError if the matching case is not found.

Let us have a look at the following code to understand how we can use the if-elif-else conditionals to solve our problem:

def func(choice, a, b): if choice == '1': return a + b elif choice == '2': return a - b elif choice == '3': return a * b elif choice == '4': return a / b else: return 'Invalid Entry!' # User Menu Prompt
print('''
Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division ''')
# Accept User Entry
option = input("Enter your choice: ")
x = int(input("Enter 1st Number: "))
y = int(input("Enter 2nd Number: "))
print("Answer = ", func(option, x, y))

Output

Press 1: Addition
Press 2: Subtraction
Press 3: Multiplication
Press 4: Division Enter your choice: 1
Enter 1st Number: 500
Enter 2nd Number: 2000
Answer = 2500

Conclusion

In this article, we discussed how we can replace the switch case statements in our python code using python dictionary mapping or by creating a class or by applying a quick fix using a python lambda function. Having said that, the best way to implement switches in python is to use a dictionary that maps the key and value pairs.

I hope you enjoyed reading this article and after reading it you can use an alternative to switch-case in your code with ease. If you find these discussions and solutions interesting then please subscribe and stay tuned for more interesting articles in the future!

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

Python raw_input() vs input()

Summary: The key differences between raw_input() and input() functions are :

  • raw_input() can be used only in Python 2.x and is obsolete in python 3.x and above and has been renamed input()
  • In Python 2.x, raw_input() returns a string whereas input() returns result of an evaluation. While in Python 3.x input() returns a string but can be converted to another type like a number.

Overview

Before looking at the differences between raw_input() and input(), let us understand why we need them?

A user-friendly code is one that is interactive. To make a code interactive instead of hard coding values, a developer/programmer must aim to allow the user to input their own values into the program. We use the raw_input() and input() functions to accept user inputs.

Example: The following program is an example to accept user input in Python:

name = input("Please enter your full name: ")
age = input("Please enter your age: ")
# In Python2.x use raw_input() instead print("Name: ", name)
print("Age: ", age)

Output

Please enter your full name: FINXTER
Please enter your age: 25
Name: FINXTER
Age: 25

In this article, we shall be discussing the key differences between the input() and raw_input() functions. So let us jump into the mission-critical question:

Problem: What is difference between raw_input() and input() in python?

Let us have an in-depth look at each difference one by one:

Existential Difference

raw_input() input()
Inbuilt function present only in Python 2.x and is not a part of Python 3.x Inbuilt function present in both, Python 2.x and Python 3.x

Functional Difference Based on Python Versions

  Python 2.x Python 3.x
raw_input() raw_input() accepts input as it is, i.e. exactly as the input has been entered by the user and returns a string.

◆ Since it accepts the input as it is, it does not expect the input to be syntactically correct.  

raw_input() is obsolete and no longer a part of Python 3.x and above.
input() input() accepts the input from the user as a statement or expression and returns the output after evaluating the input. In other words, it accepts the user entry as raw_input(), performs an eval() on it, and then returns the result as output.

◆ It expects a syntactically correct input (statement/expression) from the user.

◆ In Python 3.x, raw_input() has been replaced by input(). This means that the input() function performs the same operation in Python 3.x as raw_input() used to do in Python 2.  

Thus input() accepts and returns a string in Python 3.x and above.   

Examples

Python 2.x

input() function

a = raw_input("What is your name? ")
print "Name: %s" %a)
b = raw_input(" Enter a mathematical expression: ")
print Output": %d", %b

Output

What is your name? Finxter
Name: Finxter Enter a mathematical expression: 2+5
Output: 2+5

raw_input() function

a = input("Enter Your Full Name: ")
print "Name: %s " %a
b = input("Enter a Mathematical Expression: ")
print "Output: %d" %b

Output

Enter Your Full Name: 'Finxter Shubham'
Name: Finxter Shubham
Enter a Mathematical Expression: 5**2
Output: 25

Python 3.x And Above

input() function

a = input("What is your name? ")
print("Name: ", a)
b = input("Enter a mathematical expression: ")
print("Output: ", b)

Output

What is your name? Finxter Shubham
Name: Finxter Shubham
Enter a mathematical expression: 3+5
Output: 3+5

Trivia

If you want to implement or leverage the functionality of input() of Python 2.x in Python 3.x and evaluate the statement entered by the user, you can use one of the following procedures:

  • Type Conversion : int(input(“Enter value”))
  • Using eval(input(“Enter Value”))

Example

a = int(input("Enter first number: "))
b = int(input("Enter second number: "))
print("Addition: ", a+b)
x = eval(input("Enter a mathematical expression: "))
print("Result: ", x)

Output:

Enter first number: 25
Enter second number: 75
Addition: 100
Enter a mathematical expression: 10**2
Result: 100

But you must avoid the usage of eval() function unless necessary because it has a severe drawback.

I would strongly recommend you to read this article in connection with this topic. It will help you have a broader understanding of this concept. Also, if you are wondering about the version of python installed in your system, you may want to have a look at this article.

Conclusion

In this article, we discussed the key differences between input() and raw_input() in terms of their functionality and existence in different versions of python along with their examples. I hope all your doubts regarding the difference between input() and raw_input() have been clarified after reading this article.

Please stay tuned and Subscribe for more interesting articles!

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

What Is Python Output Buffering and How to Disable It?

Summary: Python output buffering is the process of storing the output of your code in buffer memory. Once the buffer is full, the output gets displayed on the standard output screen. Buffering is enabled by default and you can use one of the following methods to disable it:

  • Execute the Python code with the -u command line switch, for example python -u code.py
  • Use the flush keyword
  • Use sys.stdout.flush()
  • Wrap sys.stdout in TextIOWrapper and set the buffered size to 0
  • Set PYTHONUNBUFFERED to a non-empty string

Problem: Given a Python program; how to disable output buffering?

Overview Of Python Output Buffering

Before we learn buffering in the context of Python, we should ponder upon what buffer means in real life? Let me answer this question with another question. When you are in a stressful situation, whom or what do you look up to as an option to lessen your stress? On a personal level, I look up to my friends and family. So they are my “buffer” against stress.

According to the dictionary definition, a buffer is a person or thing that reduces a shock or that forms a barrier between incompatible or antagonistic people or things. This is what exactly a buffer does when it comes to a programming language like Python (though it is not a person in this case).

In terms of computer science, you may consider buffer as a middle layer which ensures a more efficient way for data to be transferred from one location to another in the system. For example, if you are downloading a 100 MB file and 1 MB gets written at a time to the hard disk would mean that it would require a total of 100 disk writes. However, a buffer could ensure that 10 MB gets written at a time to the disk, and this way only 10 disk writes will be needed.

Advantages and Disadvantages:

From the above scenario, we learned how buffer can act as an intermediate resource and reduce the number of disk writes thereby enhancing write efficiency and memory management. However, when it comes to a programming language, buffering has a downside too. We cannot see the output in real-time. This does not make much of a difference in case of simple or short programs but in complex and long programs that require a lot of computation, this might become a major drawback since we might need a more granular output rather than the entire output being displayed at once for debugging and testing purposes.

Example: Consider the following program for which we shall compare the buffered and unbuffered outputs:

for buffer in range(20): print(buffer, end=" ")

We know that the Output is a range of numbers from 0 to 19, that is

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 

But we are more interested in understanding the nature of the output. If buffering is enabled you will get the output at once on the screen after waiting for 19 seconds. However, when buffering is disabled the numbers will be displayed one by one every 2 seconds.

You can try this in our interactive shell:

Exercise: Which behavior do you observe?

How To Disable Output Buffering In Python?

By default output buffering is enabled in Python. Let us have a look at the different ways in which we can disable output buffering in python.

Method 1: Running Python With -u Command Line Arguement

If you want to skip buffering for the entire program then the simplest solution to do this is to run your program using the command line interface and use the -u switch to bypass buffering. The syntax to disable buffering through the command line is:

python -u filename.py

Output

Method 2: Using The Flush Keyword

If you want to skip buffering for specific lines of your code, passing the flush keyword as an argument to the print statement is a good option. By default, the flush argument is set as False. To ensure that output buffering is disabled, the flush argument should be set to True.

Let us have a look at the following example to understand the syntax and usage of the flush keyword:

import time
for buffer in range(20): print(buffer, end=" ", flush=True) time.sleep(2)

Output

This image has an empty alt attribute; its file name is Untitled1.gif

Method 3: Using sys.stdout.flush()

Using the sys.stdout.flush() forces the program to flush the buffer. Therefore everything in the buffer will be displayed on the standard output without delay.

Let us have a look at how sys.stdout.flush() works in the following program:

import time
import sys for buffer in range(20): print(buffer, end=" ") sys.stdout.flush() time.sleep(2)

Output

This image has an empty alt attribute; its file name is Untitled1.gif

Method 4: Wrapping sys.stdout Using TextIOWrapper And Setting Buffered Size = 0

Another approach to disable the output buffering in python is to open up the stdout in write mode with buffer size set to 0 (unbuffered) and then wrapping it using TextIOWrapper to ensure that you get a TextIO stream and not a binary file object. Also, you must ensure to enable the write_through to eliminate all buffers.

Let us have a look at the following code to get a better grip on the above explanation:

# open the stdout file descriptor in write mode and set 0 as the buffer size: unbuffered
import io
import os
import sys
import time try: # open stdout in binary mode, then wrap it in a TextIOWrapper and enable write_through sys.stdout = io.TextIOWrapper(open(sys.stdout.fileno(), 'wb', 0), write_through=True) # for flushing on newlines only use : # sys.stdout.reconfigure(line_buffering=True)
except TypeError: # In case you are on Python 2 sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0) print("Enter a Line: ")
line = sys.stdin.readline()
for i in line: print(i, end=" ") time.sleep(2)

Output

Method 5: Setting Python Environment Variable PYTHONUNBUFFERED

When the PYTHONUNBUFFERED variable is set to a non-empty string, it is equivalent to the -u command line option.

Conclusion

I hope this article helped you to get an overview of Python output buffering and the methods to disable it. Please stay tuned for more interesting articles.

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

Posted on Leave a comment

Python One Line If Not None

To assign the result of a function get_value() to variable x if it is different from None, use the Walrus operator if tmp := get_value(): x = tmp within a single-line if block. The Walrus operator assigns the function’s return value to the variable tmp and returns it at the same time, so that you can check and assign it to variable x subsequently.

Problem: How to assign a value to a variable if it is not equal to None—using only a single line of Python code?

Example: Say, you want to assign the return value of a function get_value(), but only if it doesn’t return None. Otherwise, you want to leave the value as it is. Here’s a code example:

import random def get_value(): if random.random()>0.5: return None return 1 # Naive approach:
x = 42
tmp = get_value()
if tmp != None: x = tmp
print(tmp)

While this works, you need to execute the function get_value() twice which is not optimal. An alternative would be to assign the result of the get_value() function to a temporary variable to avoid repeated function execution:

x = 42
temp = get_value()
if temp != None: x = temp
print(x)

However, this seems clunky and ineffective. Is there a better way?

Let’s have an overview of the one-liners that conditionally assign a value to a given variable:

Exercise: Run the code. Does it always generate the same result?

Method 1: Ternary Operator + Semicolon

The most basic ternary operator x if c else y consists of three operands x, c, and y. It is an expression with a return value. The ternary operator returns x if the Boolean expression c evaluates to True. Otherwise, if the expression c evaluates to False, the ternary operator returns the alternative y.

You can use the ternary operator to solve this problem in combination with the semicolon to write multiple lines of code as a Python one-liner.

# Method 1
tmp = get_value(); x = tmp if tmp else x

You cannot run the get_value() function twice—to check whether it returns True and to assign the return value to the variable x. Why? Because it’s nondeterministic and may return different values for different executions.

Therefore, the following code would be a blunt mistake:

x = get_value() if get_value() else x

The variable x may still be None—even after the ternary operator has seemingly checked the condition.

Related articles:

Method 2: Walrus + One-Line-If

A beautiful extension of Python 3.8 is the Walrus operator. The Walrus operator := is an assignment operator with return value. Thus, it allows you to check a condition and assign a value at the same time:

# Method 2
if tmp := get_value(): x = tmp

This is a very clean, readable, and Pythonic way. Also, you don’t have the redundant identity assignment in case the if condition is not fulfilled.

Related Article: The Walrus Operator in Python 3.8

Python One-Liners Book

Python programmers will improve their computer science skills with these useful one-liners.

Python One-Liners

Python One-Liners will teach you how to read and write “one-liners”: concise statements of useful functionality packed into a single line of code. You’ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.

The book’s five chapters cover tips and tricks, regular expressions, machine learning, core data science topics, and useful algorithms. Detailed explanations of one-liners introduce key computer science concepts and boost your coding and analytical skills. You’ll learn about advanced Python features such as list comprehension, slicing, lambda functions, regular expressions, map and reduce functions, and slice assignments. You’ll also learn how to:

  Leverage data structures to solve real-world problems, like using Boolean indexing to find cities with above-average pollution
  Use NumPy basics such as array, shape, axis, type, broadcasting, advanced indexing, slicing, sorting, searching, aggregating, and statistics
  Calculate basic statistics of multidimensional data arrays and the K-Means algorithms for unsupervised learning
  Create more advanced regular expressions using grouping and named groups, negative lookaheads, escaped characters, whitespaces, character sets (and negative characters sets), and greedy/nongreedy operators
  Understand a wide range of computer science topics, including anagrams, palindromes, supersets, permutations, factorials, prime numbers, Fibonacci numbers, obfuscation, searching, and algorithmic sorting

By the end of the book, you’ll know how to write Python at its most refined, and create concise, beautiful pieces of “Python art” in merely a single line.

Get your Python One-Liners Now!!

Posted on Leave a comment

Python One Line HTTP Get

You may already know about Python’s capability to create a simple web server in a single line of Python code. Old news. Besides, what’s the point of creating a webserver that only runs on your machine? It would be far more interesting to learn how to access existing websites in a single line of code. Surprisingly, nobody talks about this in the Python One-Liners community. Time to change it!

This tutorial shows you how to perform simple HTTP get and post requests to an existing webserver!

Problem: Given the URL location of a webserver serving websites via HTTP. How to access the webserver’s response in a single line of Python code?

Example: Say, you want to accomplish the following:

url = 'https://google.com'
# ... Magic One-Liner Here...
print(result)
# ... Google HTML file: '''
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="de"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title>... '''

You can try it yourself in our interactive Python shell:

Exercise: Does this script download the complete source code of the Google.com website?

Let’s learn about the three most important methods to access a website in a single line of Python code—and how they work!

Method 1: requests.get(url)

The simplest one-liner solution is the following:

import requests; print(requests.get(url = 'https://google.com').text)

Here’s how this one-liner works:

  • Import the Python library requests that handles the details of requesting the websites from the server in an easy-to-process format.
  • Use the requests.get(...) method to access the website and pass the URL 'https://google.com' as an argument so that the function knows which location to access.
  • Access the actual body of the get request (the return value is a request object that also contains some useful meta information like the file type, etc.).
  • Print the result to the shell.

Note that the semicolon is used to one-linerize this method. This is useful if you want to run this command from your operating system with the following command:

python -r "import requests; print(requests.get(url = 'https://google.com').text)"

The output is the desired Google website:

'''
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="de"><head><meta content="text/html; charset=UTF-8" http-equiv="Content-Type"><meta content="/images/branding/googleg/1x/googleg_standard_color_128dp.png" itemprop="image"><title>Google</title>... '''

Note that you may have to install the requests library with the following command in your operating system terminal:

pip install requests

A similar approach can be taken if you want to issue a post request:

Method 2: requests.post(url)

What if you want to post some data to a web resource? Use the post method of the requests module! Here’s a minimal one-liner example of the request.post() method:

import requests as r; print(r.post('https://example.com', {'key': 'val'}).text)

The approach is similar to the first one:

  • Import the requests module.
  • Call the r.post(...) method.
  • Pass the URL 'https://example.com' as the first parameter into the function.
  • Pass the value to be posted to the URL—in our case a simple key-value pair in a dictionary data structure.
  • Access the body via the text attribute of the request object.
  • Print it to the shell.

Method 3: urllib.request

A recommended way to fetch web resources from a website is the urllib.request() function. This also works to create a simple one-liner to access the Google website in Python 3 as before:

import urllib.request as r; print(r.urlopen('https://google.com').read())

It works similarly than before by returning a Request object that can be accessed to read the server’s response. We’re cramming everything into a single line so that you can run it from your OS’s terminal:

python -r "import urllib.request as r; print(r.urlopen('https://google.com').read())"

Congrats! You now have mastered the art of accessing websites in a single line of Python code. If you’re interested in boosting your one-liner power, have a look at my new book:

Python One-Liners Book

Python programmers will improve their computer science skills with these useful one-liners.

Python One-Liners

Python One-Liners will teach you how to read and write “one-liners”: concise statements of useful functionality packed into a single line of code. You’ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.

The book’s five chapters cover tips and tricks, regular expressions, machine learning, core data science topics, and useful algorithms. Detailed explanations of one-liners introduce key computer science concepts and boost your coding and analytical skills. You’ll learn about advanced Python features such as list comprehension, slicing, lambda functions, regular expressions, map and reduce functions, and slice assignments. You’ll also learn how to:

  Leverage data structures to solve real-world problems, like using Boolean indexing to find cities with above-average pollution
  Use NumPy basics such as array, shape, axis, type, broadcasting, advanced indexing, slicing, sorting, searching, aggregating, and statistics
  Calculate basic statistics of multidimensional data arrays and the K-Means algorithms for unsupervised learning
  Create more advanced regular expressions using grouping and named groups, negative lookaheads, escaped characters, whitespaces, character sets (and negative characters sets), and greedy/nongreedy operators
  Understand a wide range of computer science topics, including anagrams, palindromes, supersets, permutations, factorials, prime numbers, Fibonacci numbers, obfuscation, searching, and algorithmic sorting

By the end of the book, you’ll know how to write Python at its most refined, and create concise, beautiful pieces of “Python art” in merely a single line.

Get your Python One-Liners Now!!