Category: Programming

McKinsey Authors: James Manyika | San FranciscoSusan Lund | WashingtonJacques Bughin | BrusselsKelsey Robinson | San FranciscoJan Mischke | ZurichDeepa Mahajan | San Francisco

Download Link (PDF)

Anyone who has ever felt trapped in a cubicle, annoyed by a micromanaging boss, or fed up with office politics has probably dreamed of leaving it all behind and going it alone.

The powerful opening line from the 148 page report (published in 2014) sets the tone of the report and shows one of the underlying reasons the freelancer market has grown double-digits since then.

In this article, I’ll summarize the most important points and statistics of the report. I’ll also give some quotes. You can see the original document here. The report is based on 8000 participants of a McKinsey survey.

Statistics

The report states the following statistics (highlights by me):

• 20 to 30 percent of the working-age population in the United States and the EU-15, or up to 162 million individuals, engage in independent work
• At least 15 percent of workers use online platforms such as Uber and Upwork (in 2015)
• 30 percent are “free agents,” who actively choose independent work and derive their primary income from it.
• Approximately 40 percent are “casual earners,” who use independent work for supplemental income and do so by choice
• “Reluctants,” who make their primary living from independent work but would prefer traditional jobs, make up 14 percent.
• The “financially strapped,” who do supplemental independent work out of necessity, account for 16 percent.

Quotes

Interesting Nuggets

numpy.argpartition(a, kth, axis=-1, kind='introselect', order=None)

The NumPy argpatition function performs an indirect partition along the given axis using the algorithm specified by the kind keyword. It returns an array of indices of the same shape as a that index data along the given axis in partitioned order.

The following table shows the return value of the function:

Related: See partition for notes on the different selection algorithms.

Let’s dive into some examples to show how the function is used in practice:

Examples

One-dimensional array:

import numpy as np x = np.array([3, 4, 2, 1]) print(x[np.argpartition(x, 3)])
# [2 1 3 4] print(x[np.argpartition(x, (1, 3))])
# [1 2 3 4]

Multi-dimensional array:

import numpy as np x = np.array([3, 4, 2, 1]) print(x[np.argpartition(x, 3)])
# [2 1 3 4] print(x[np.argpartition(x, (1, 3))])
# [1 2 3 4] x = [3, 4, 2, 1]
print(np.array(x)[np.argpartition(x, 3)])
# [2 1 3 4]

Any master coder has a “hands-on” mentality with a bias towards action. Try it yourself—play with the function in the following interactive code shell:

Exercise: Change the parameters of your polynomials and print them without the comparisons. Do you understand where they come from?

Master NumPy—and become a data science pro:

Related Video

This tutorial will teach you how to write one-line for loops in Python using the popular expert feature of list comprehension. After you’ve learned the basics of list comprehension, you’ll learn how to restrict list comprehensions so that you can write custom filters quickly and effectively.

Are you ready? Let’s roll up your sleeves and learn about list comprehension in Python!

List Comprehension Basics

The following section is based on my detailed article List Comprehension [Ultimate Guide]. Read the shorter version here or the longer version on the website—you decide!

This overview graphic shows how to use list comprehension statement to create Python lists programmatically:

List comprehension is a compact way of creating lists. The simple formula is [expression + context].

• Expression: What to do with each list element?
• Context: What elements to select? The context consists of an arbitrary number of for and if statements.

The example [x for x in range(3)] creates the list [0, 1, 2].

Have a look at the following interactive code snippet—can you figure out what’s printed to the shell? Go ahead and click “Run” to see what happens in the code:

Exercise: Run the code snippet and compare your guessed result with the actual one. Were you correct?

Now, that you know about the basics of list comprehension (expression + context!), let’s dive into a more advanced example where list comprehension is used for filtering by adding an if clause to the context part.

List Comprehension for Filtering (using If Clauses)

You can also modify the list comprehension statement by restricting the context with another if statement:

Problem: Say, we want to create a list of squared numbers—but you only consider even and ignore odd numbers.

Example: The multi-liner way would be the following.

squares = [] for i in range(10): if i%2==0: squares.append(i**2) print(squares)
# [0, 4, 16, 36, 64]

You create an empty list squares and successively add another square number starting from 0**2 and ending in 8**2—but only considering the even numbers 0, 2, 4, 6, 8. Thus, the result is the list [0, 4, 16, 36, 64].

Again, you can use list comprehension [i**2 for i in range(10) if i%2==0] with a restrictive if clause (in bold) in the context part to compress this in a single line of Python code:

print([i**2 for i in range(10) if i%2==0])
# [0, 4, 16, 36, 64]

This line accomplishes the same output with much less bits.

Related Article: Python One Line For Loop

Python One-Liners Book

Python programmers will improve their computer science skills with these useful one-liners.

Python One-Liners will teach you how to read and write “one-liners”: concise statements of useful functionality packed into a single line of code. You’ll learn how to systematically unpack and understand any line of Python code, and write eloquent, powerfully compressed Python like an expert.

The book’s five chapters cover tips and tricks, regular expressions, machine learning, core data science topics, and useful algorithms. Detailed explanations of one-liners introduce key computer science concepts and boost your coding and analytical skills. You’ll learn about advanced Python features such as list comprehension, slicing, lambda functions, regular expressions, map and reduce functions, and slice assignments. You’ll also learn how to:

•  Leverage data structures to solve real-world problems, like using Boolean indexing to find cities with above-average pollution
•  Use NumPy basics such as array, shape, axis, type, broadcasting, advanced indexing, slicing, sorting, searching, aggregating, and statistics
•  Calculate basic statistics of multidimensional data arrays and the K-Means algorithms for unsupervised learning
•  Create more advanced regular expressions using grouping and named groups, negative lookaheads, escaped characters, whitespaces, character sets (and negative characters sets), and greedy/nongreedy operators
•  Understand a wide range of computer science topics, including anagrams, palindromes, supersets, permutations, factorials, prime numbers, Fibonacci numbers, obfuscation, searching, and algorithmic sorting

By the end of the book, you’ll know how to write Python at its most refined, and create concise, beautiful pieces of “Python art” in merely a single line.

Get your Python One-Liners Now!!

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

In this article, I’ll show you five free resources you can study to reach an intermediate Python level being an absolute beginner. Studying all the resources will take 20-40 hours. But after you’ve completed these steps, you’ll have created yourself a new high-income skill Python development. So, let’s get started!

Step 1: Complete Python Crash Course

The purpose of this tutorial is to help you refresh your knowledge of all the basic Python keywords, data structures, and fundamentals. I wrote it for the intermediate Python programmer who wants to reach the next level of programming expertise.

The way of achieving an expert-level is through studying the basics. Computer science professors usually have an extremely profound knowledge of the basics in their field. This enables them to argue from “first principles” rather than from the state-of-the-art—it’s easier for them to identify research gaps because they know about the ground rules in their field rather than being blinded by the latest technology and state-of-the-art. If you want to reach the next level in coding, take your time and study the basics carefully.

This tutorial provides you with the most important Python basics which serve as a foundation for more advanced topics.

Table of Contents:

Python Keywords
Python Basic Data Structures
Python Container Data Structures
Membership Operator
List and Set Comprehension
Summary

Complete full tutorial now: Python Programming Tutorial [+Cheat Sheets]

Step 2: Study Cheat Sheets

Don’t have much time to learn Python? Cheat sheets to the rescue!

Cheat sheets are among the most efficient ways to acquire knowledge. A great cheat sheet focuses on the key learning material and skips the rest. If you read over them every day, you’ll quickly learn all the basics you need to know to master Python.

In this tutorial, we’ll share the top 11 Python cheat sheets with you. Download them, print them, put them on your wall and watch your Python skills grow!

Complete full tutorial now: [Collection] 11 Python Cheat Sheets Every Python Coder Must Own

Step 3: Solve Python Puzzles to Discover Your True Skill Level

Do you belong to the top 50% of Python coders? This article poses you 10 intermediate-level Python puzzles to test yourself. By solving all 10 Python puzzles and tracking your success rate, you will discover your exact Python skill level.

How to do this? The idea of solving rated Python puzzles to measure your skill level is inspired by the Elo rating system in chess. Two chess players play against each other fighting for Elo rating points. If you have collected enough points, you can become a chess grandmaster. We transformed this idea to Python coding: you play against the Python puzzle. If you solve it (=you win), you get points – but if you don’t, you lose points.

The Python puzzles and the test originate from our experience of teaching 23,464 online students at Finxter.com. (It’s free.)

You have read this far – you have skin in the game. Hereby, I officially challenge you to take the test and prove to yourself that you, indeed, belong to the top-50% of Python coders.

But first things first:

What is a Python Puzzle?

A Python puzzle is an educative snippet of Python source code that teaches a single computer science concept by activating the learner’s curiosity and involving them in the learning process.

Complete full tutorial now: Solve 10 Python Puzzles to Discover Your True Skill Level

Step 4: Master Python Data Structures

Data structures are at the heart of every algorithm. Understanding data structures is the secret weapon of master coders. So, without further ado, let’s dive into the three most important Python data structures!

Complete full tutorials now:

• Python Lists [Ultimate Guide]
• Python Set [Ultimate Guide]
• Python Dictionary [Ultimate Guide]

Step 5: Master the Single Line of Code

You cannot understand any advanced code base without understanding the single line. Mastering the single line is at the heart of any non-trivial Python project.

You may ask: what is a Pyhton one-liner anyway?

A Python one-liner is a snippet of code that solves a problem in a single line.

In this resource, you will find 50 (!) one-liner videos. Watch all of them so that you truly have mastered everything there is to know about the single line of Python code. As a result, you’ll truly master an essential skill that will prove valuable throughout your career!

Complete full tutorial now: PythonOneLiners.com

Where to Go From Here

If you’ve studied all the tutorials I proposed in this article, you should now be able to solve practical code projects by yourself. This is how you reach expert level. Close the room, turn off your smartphone, and dive into your dream practical code project in Python!

You can join our free email academy to gradually polish your skills over the upcoming months and access free Python resources for continuous improvement:

In this article, you’ll learn the basics of parsing an HTML document using Python and the LXML library.

Introduction

Data is the most important ingredient in programming. It comes in all shapes and forms. Sometimes it is placed inside documents such as CSV or JSON, but sometimes it is stored on the internet or in databases. Some of it is stored/transferred or processed through the XML format, which is in many ways similar to HTML format, yet its purpose is to transfer and store data, unlike HTML, whose main purpose is to display the data. On top of that, the way of writing HTML and XML is similar. Despite the differences and similarities, they supplement each other very well.

Both Xpath and XML are engineered by the same company W3C, which imposes that Xpath is the most compatible Python module to be used for parsing the XML documents. Since one of the programing principals which would push you towards the programming success is to “not reinvent the wheel”, we are going to refer to the W3C (https://www.w3.org/) consortium document and sources in regarding the syntax and operators on our examples to bring the concept of XPath closer to the people wishing to understand it better and use it on real-life problems. 

The IT industry has accepted the XML way of transferring data as one of its principles. Imagine if one of your tasks was to gather information from the internet? Copying and pasting are one of the simplest tools to use (as it is regularly used by programmers as well); it might only lead us to gather some simple data from the web, although the process might get painfully repetitive. Yet, in case if we have more robust data, or more web pages to gather the data from, we might be inclined to use more advanced Python packages to automate our data gathering.

Before we start looking into scraping tools and strategies, it is good to know that scraping might not be legal in all cases, therefore it is highly suggested that we look at the terms of service of a particular web site, or copyright law regarding the region in which the web site operates.

For purposes of harvesting the web data, we will be using several Python libraries that allow us to do just that. The first of them is the requests module. What it does is that it sends the HTTP requests, which returns us the response object. It only used if urge to scrape the content from the internet. If we try to parse the static XML file it would not be necessary.

There are many parsing modules. LXML, Scrapy and BeautifulSoup are some of them. To tell which one is better is often neglected since their size and functionality differs from one another. For example, BeautifulSoup is more complex and serves you with more functionality, but LXML and Scrapy comes lightweight and can help you traversing through the documents using XPath and CSS selectors.

There are certain pitfalls when trying to travel through the document using XPath. Common mistake when trying to parse the XML by using XPath notation is that many people try to use the BeautifulSoup library. In fact that is not possible since it does not contain the XPath traversing methods. For those purposes we shall use the LXML library.

The requests library is used in case we want to download a HTML mark-up from the particular web site.

The first step would be to install the necessary packages. Trough pip install notation all of the modules above could be installed rather easily.

Necessary steps:

1. pip install lxml (xpath module is a part of lxml library)
2. pip install requests (in case the content is on a web page)

The best way to explain the XML parsing is to picture it through the examples.

The first step would be to install the necessary modules. Trough pip install notation all of the modules above could be installed rather easily.

What is the XPath?

The structure of XML and HTML documents is structurally composed of the nodes (or knots of some sort), which is a broader picture that represents the family tree-like structure. The roof instance, or the original ancestor in each tree, is called the root node, and it has no superior nodes to itself. Subordinate nodes are in that sense respectively called children or siblings, which are the elements at the same level as the children. The other terms used in navigating and traversing trough the tree are the ancestors and descendants, which in essence reflect the node relationship the same way we reflect it in real-world family tree examples. 

XPath is a query language that helps us navigate and select the node elements within a node tree. In essence, it is a step map that we need to make to reach certain elements in the tree.  The single parts of this step map are called the location steps, and each of these steps would lead us to a certain part of the document.

The terminology used for orientation along the axis (with regards to the current node) is very intuitive since it uses regular English expressions related to real-life family tree relationships.

XPath Selector

XPath selector is the condition using which we could navigate through an XML document. It describes relationships as a hierarchical order of the instances included in our path.  By combining different segment of XML syntax it helps us traverse through to the desired parts of the document. The selector is a part of the XPath query language. By simply adding different criteria, the XPath selector would lead us to different elements in the document tree.  The best way to learn the XPath selector syntax and operators is to implement it on an example. In order to know how to configure the XPath selector, it is essential to know the XPath syntax. XPath selector is compiled using an etree or HTML module which is included within the LXML package. The difference is only if we are parsing the XML document or HTML.

The selector works similarly as a find method with where it allows you to select a relative path of the element rather than the absolute one, which makes the whole traversing less prone to errors in case the absolute path gets too complicated.

XPath Syntax

XPath syntax could be divided into several groups. To have an exact grasp of the material presented we are going to apply further listed expressions and functions on our sample document, which would be listed below. In this learning session, we are going to use a web site dedicated to scraping exercises.

Node selection:

Using “..”  and “.” we can direct and switch levels as we desire. Two dot notations would lead us from wherever we are to our parent element, whereas the one dot notations would point us to the current node. 

The way that we travel from the “context node” (our reference node), which is the milestone of our search, is called “axes”, and it is noted with double slash //. What it does is that it starts traversing from the first instance of the given node. This way of path selection is called the “relative path selection”. To be certain that the // (empty tag) expression would work, it must precede an asterisk (*) or the name tag. Trough inspecting the element and copying its XPath value we are getting the absolute path.

XPath Functions and Operators

here are 6 common operators which are used inside the XPath query. Operators are noted the same way as in plain Python and serve the same purpose. The functions are meant to aid the search of desired elements or their content.

To add more functionality to our XPath expression we can use some of LXML library functions. Everything that is written in-between the “[]” is called a predicate and it is used to closer describe the search path. Most frequently used functions are contains() and starts-with(). Those functions and their results would be displayed in the table below.

Going Up and Down the Axis

The conventional syntax used to traverse up and down the XPath axes is ElementName::axis.

To reach the elements placed above or below our current axes, we might use some of the following axes.

A Simple Example

The goal of this scraping exercise is to scrape all the book genres placed at the left-hand side of the web site. It almost necessary to see the page source and to inspect some of the elements which are we aiming to scrape.

from lxml import html
import requests url = 'http://books.toscrape.com/' # downloading the web page by making a request objec
res = requests.get(url) # making a tree object
tree = html.fromstring(res.text) # navingating the tree object using the XPath
book_genres = tree.xpath("//ul/li/a[contains(@href, 'categ')]/text()")[0:60] # since the result is the list object, we iterate the elements,
# of the list by making a simple for loop
for book_genre in book_genres: print (book_genre.strip())

Resources:

1. https://lxml.de/
2. https://scrapinghub.github.io/xpath-playground/
3. https://2.python-requests.org/en/master/
4. ‘http://books.toscrape.com/

Crafting beautiful Python one-liners is as much an art as it is a science. In this tutorial, you’ll learn how to compress an if statement without an else branch into a single line of Python code.

Problem: What’s the one-liner equivalent of the simple if statement without an else branch?

Here’s an example:

condition = True if condition: print('hi') # hi

You may want to (i) print something, (ii) assign a value to a variable, or (iii) append an element to a list if the condition holds.

In this article, I’ll show you four methods of how to accomplish this goal. All four methods are generally applicable—and you can easily customize them to your specific application.

Let’s have a quick overview of the four methods in our interactive code shell:

Exercise: Run the code for both True and False conditions. Are all methods semantically equivalent?

Method 1: One-Liner If Statement

The first is also the most straightforward method: if you want a one-liner without an else statement, just write the if statement in a single line! There are many tricks (like using the semicolon) that help you create one-liner statements. But for an if body with only one statement, it’s just as simple as avoiding the line break.

condition = True # Method 1: One-Liner If
if condition: print('hi')
# hi

This method is perfectly valid and you could see it in practice. Yet, I have to mention that it “violates” the PEP8 standard (multiple statements in a single line). Therefore, you shouldn’t consider this to be Pythonic code (there are worse things under the sun though).

Method 2: Ternary with Throw-Away Else Branch

Sure, you can also use the ternary operator:

If you need a quick refresher on the ternary operator, check out my detailed blog article. The ternary operator is commonly used to conditionally assign values. But you can also throw away the return value by not assigning the result to any variable. In this case, it doesn’t matter if you use None or any other “dummy” return value as the result of the else branch:

condition = True # Method 2: Ternary with Dummy
print('hi') if condition else None

It’s readable, short, and concise and I like this (well, I may be a bit biased as author of the book Python One-Liners). Sure, people will ask why you didn’t write it in multiple lines. But where is the fun there?

Method 3: Ternary with Default Value for Assignment

If you need to assign a value conditionally to a variable, but you want to do so without an else branch, you can do the following:

condition = True # Method 3: Ternary with Dummy for Assignment
x = 42 if condition else None

If the condition does not hold, the “dummy” value None is assigned to the variable.

Method 4: Short Circuiting

This method I like most. It uses a Python optimization called “short circuiting” for Boolean operators: the logical and operator simply returns the second operand if the first is True. There’s no Boolean conversion of the second operand, it’s just returned as is.

If the first operand is False, the second operand is not even evaluated.

You can use this to conditionally execute the if branch print('hi') or any other code function.

condition = True # Method 4: Short circuiting
condition and print('hi')

There are two options:

• condition == True: As the first operand is True, the second operand is returned. Thus, the statement print('hi') is executed and the string hi appears on the screen.
• condition == False: As the first operand is False, the second operand is not even evaluated because the result of the logical and operation is False anyway. Thus, the statement print('hi') is never executed.

Where to Go From Here?

Enough theory, let’s get some practice!

To become successful in coding, you need to get out there and solve real problems for real people. That’s how you can become a six-figure earner easily. And that’s how you polish the skills you really need in practice. After all, what’s the use of learning theory that nobody ever needs?

Practice projects is how you sharpen your saw in coding!

Do you want to become a code master by focusing on practical code projects that actually earn you money and solve problems for people?

Then become a Python freelance developer! It’s the best way of approaching the task of improving your Python skills—even if you are a complete beginner.

Join my free webinar “How to Build Your High-Income Skill Python” and watch how I grew my coding business online and how you can, too—from the comfort of your own home.

Join the free webinar now!

numpy.polynomial.polynomial.polymulx(c)

The numpy.polymulx function multiplies the polynomial c with a value x which is the independent variable.

The following table shows the return value of the function:

Let’s dive into some examples to show how the function is used in practice:

Examples

import numpy as np
import numpy.polynomial.polynomial as poly print(poly.polymulx([0]) == [0])
print(poly.polymulx([1]) == [0, 1])
for i in range(1, 5): ser = [0]*i + [1] tgt = [0]*(i + 1) + [1] print(poly.polymulx(ser) == tgt) '''
[ True]
[ True True]
[ True True True]
[ True True True True]
[ True True True True True]
[ True True True True True True] '''

This function is inspired from this Github repository.

Any master coder has a “hands-on” mentality with a bias towards action. Try it yourself—play with the function in the following interactive code shell:

Exercise: Change the parameters of your polynomials and print them without the comparisons. Do you understand where they come from?

Master NumPy—and become a data science pro:

Related Video