This tutorial teaches you two advanced Python skills: closures and decorators. Mastering them will make you a better coder today—so, let’s dive right into them!
Closures
Every function in Python is first class, because they can be passed around like any other object. Usually, when a programming language creates a function just like other data types, that programming language supports something called Closures.
A closure is a nested function. It is defined within an outer function.
def outer_hello_fn(): def hello(): print("Hello Finxter!") hello()
Here, we have an outer function called outer_ hello_ fn, it has no input arguments. The function hello is a nested function defined within the outer function. The hello function is a closure.
Try It Yourself:
Exercise: What’s the output of this code snippet? Run the code to test if you’re correct.
When the outer function is called, the hello function within it will be defined and then invoked. Here is the function call and output:
outer_hello_fn()
Output:
Hello Finxter!
hello has been defined within outer_hello_fn, which means if you try and invoke the hello function, it will not work.
hello()
Output:
NameError: name 'hello' is not defined
If you want access to a function that is defined within another function, return the function object itself. Here is how.
def get_hello_fn(): def hello(): print("Hello Finxter!") return hello
The outer function is called get_hello_fn. hello, is an inner function, or closure. Instead of invoking this hello function, simply return the hello function to whoever calls get_hello_fn. For example:
hello_fn = get_hello_fn()
Invoking get_hello_fn stores the return function object in the hello_fn variable. If you explore the contents of this hello_fn variable, you will see that it is a function object.
hello_fn
Output:
<function __main__.get_hello_fn.<locals>.hello>
As you can see in the structure, it is a locally defined function within get_hello_fn, that is, a function defined within another function, that is a closure. Now, this closure can be invoked by using the hello_fn variable.
hello_fn()
Output:
Hello Finxter!
Invoke hello_fn() will print out Hello Finxter! to screen. A closure is something more than just an inner function defined within an outer function. There is more to it. Here is another example:
def hello_by_name(name): def hello(): print("Hello!", name) hello() return hello
Here, the outer function is called hello_by_name, which takes in one input argument, the name of an individual. Within this outer function, there is the hello inner function. It prints to the screen Hello!, and the value of the name.
The name variable is an input argument to the outer function. It is also accessible within the inner hello function. The name variable here can be thought of as a variable that is local to the outer function. Local variables in the outer function can be accessed by closures. Here is an example of passing an argument to the outside function:
greet_hello_fn = hello_by_name("Chris")
The function hello is returned and it is stored in the greet_hello_fn variable.
Executing this prints out Hello! Chris to screen. That is because we invoked the closure from within the outer function. We have a reference to the closure that was defined by the outer function.
greet_hello_fn()
Output:
Hello! Chris
Notice something interesting here. Chris is available in the variable name which is local to the hello_by_name function.
Now, we have already invoked and exited hello_by_name but the value in the name variable is still available to our closure. And this is another important concept about closures in Python. They hold the reference to the local state even after the outer function that has defined the local state has executed and no longer exists. Here is another slightly different example illustrating this concept.
def greet_by_name(name): greeting_msg = "Hi there!" def greeting(): print(greeting_msg, name) return greeting
The outer function, greet_by_name, takes in one input argument, name. Within the outer function, a local variable called greeting_msg is defined which says, “Hi there!”. A closure called greeting is defined within the outer function. It accesses the local variable greeting_msg as well as the input argument name. A reference to this greeting closure is returned from the outer greet_by_name function.
Let’s go ahead and invoke greet_by_name and store the function object that it returns in the greet_fn variable. We will use this function object to greet Ray by name. Go ahead and invoke the greet_fn() by specifying parentheses. And it should say, Hi there! Ray. Observe how the closure has access not just to the name Ray but also to the greeting message, even after we have executed and exited the outer function.
greet_fn = greet_by_name("Ray")
greet_fn()
Output:
Hi there! Ray
Closures carry around information about the local state. Let’s see what happens when the greet_by_name function is deleted, so you no longer have access to the outer function.
del greet_by_name
Now, remember that name and greeting message are both variables that were defined in the outer function. What happens to them? Now if you try to invoke greet by name.
greet_by_name("Ray")
Output:
NameError: name 'greet_by_name' is not defined
What about the greet_fn?
Remember that greet_fn is a reference to our closure. Does this still work?
greet_fn()
Output:
Hi there! Ray
Not only does it work, but it still has access to the local variables that were defined in the outer function. The outer function no longer exists in Python memory, but the local variables are still available along with our closure.
Decorators – Code Modification
Decorators help to add functionality to existing code without having to modify the code itself. Decorators are so-called because they decorate code, they do not modify the code, but they make the code do different things using decoration. Now that we have understood closures, we can work our way step by step to understanding and using decorators.
def print_message(): print("Decorators are cool!")
Here is a simple function that prints a message to screen.
print_message()
Output:
Decorators are cool!
Each time you invoke this function it will always print the same message. I want to use a few characters to decorate the original message, and I do this using the highlight function.
import random def highlight(): annotations = ['-', '*', '+', ':', '^'] annotate = random.choice(annotations) print(annotate * 50) print_message() print(annotate * 50)
The outer function highlight has no input arguments. Within the highlight function, a random choice of annotations is used to decorate the original message. The message will be highlighted with a random choice between the dash, the asterisk, the plus, the colon, and the caret. The output will have an annotation of 50 characters before and after the message which is inside the print_message function.
Try It Yourself:
Exercise: What’s the output of this code snippet? Run the code to test your understanding!
highlight()
Output:
::::::::::::::::::::::::::::::::::::::::::::::::::
Decorators are cool!
::::::::::::::::::::::::::::::::::::::::::::::::::
Here is another function with a different message, print_another_message.
def print_another_message(): print("Decorators use closures.")
Now if I want to highlight this message as well, the existing highlight function will not work because it has been hardcoded to invoke the print_message function. So how do I change this highlight function so that it is capable of highlighting any message that I want printed out to screen? Remember that functions are first-class citizens in Python, which means whatever print function you have, you can pass it as an input argument to the highlight function. Here is a redefined highlight function, make_highlighted.
def make_highlighted(func): annotations = ['-', '*', '+', ':', '^'] annotate = random.choice(annotations) def highlight(): print(annotate * 50) func() print(annotate * 50) return highlight
The only difference here is that make_highlighted takes in an input argument that is a function. This function is what will print out the message to be displayed. The next change is that within the highlight closure, the function object that was passed in is invoked. That is the function object that will print out the message. Now we have two print functions so far.
print_message()
print_another_message()
And now with the help of make_highlighted function, any printed message can be highlighted. For example:
highlight_and_print_message = make_highlighted(print_message) highlight_and_print_message()
Output:
++++++++++++++++++++++++++++++++++++++++++++++++++
Decorators are cool!
++++++++++++++++++++++++++++++++++++++++++++++++++
To print a different message and have it highlighted, simply pass a different function object to the make_highlighted function.
highlight_and_print_another_message = make_highlighted(print_another_message) highlight_and_print_another_message()
Output:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Decorators use closures.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
It is clear that the make_highlighted function is very generic, you can use it to highlight any message that you want printed to screen. The function make_highlighted is a decorator.
Why is it a decorator? Well, it takes in a function object and decorates it and changes it. In this example, it highlights the function with random characters. Decorators are a standard design pattern, and in Python, you can use decorators more easily. Instead of passing in a function object to make_highlighted, accessing the closure, and then invoking the closure, you can simply decorate any function by using @ and placing the decorator before the function to decorate.
@make_highlighted
def print_a_third_message(): print("This is how decorators are used")
The use of the decorator @make_highlighted will automatically pass the function print_a_third_message as an input to make_highlighted and highlight the message.
print_a_third_message()
Output:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This is how decorators are used
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Now you can use the decorator to highlight any messages.
@make_highlighted
def print_any_message(): print("This message is highlighted!")
And now if you invoke print_any_message, you will find that the result that is displayed to screen is highlighted.
print_any_message()
Output:
++++++++++++++++++++++++++++++++++++++++++++++++++
This message is highlighted!
++++++++++++++++++++++++++++++++++++++++++++++++++
Decorators – Customization
Let’s see another example of a Decorator that will do some work. It will do some error checking for us.
Here are two functions that will be the input to our decorator
def square_area(length): return length**2 def square_perimeter(length): return 4 * length
We assume that the value of the radius passed in is positive and correct.
square_area(5)
Output:
25
What if I invoke the square_area and pass in -1?
square_area(-1)
Output:
-4
The input -1 doesn’t make sense as a value for the length. The function should have thrown an error or told us in some way that negative values of length are not valid. Now, if you were to perform an error check for each of these functions, we would have to do it individually. We would have to have an if statement within the area function as well as the perimeter function. Instead of that, let’s write a decorator that will perform this error checking for us. The decorator safe_calculate takes in one input argument that is a function object.
def safe_calculate(func): def calculate(length): if length <= 0: raise ValueError("Length cannot be negative or zero") return func(length) return calculate
This is the function object that will perform the calculation. Within the safe_calculate outer function, the inner function called calculate is the closure. calculate takes in one input argument, the length. It checks to see whether length is less than or equal to 0. If yes, it throws an error. And the way it throws an error is by simply calling a raise ValueError, “Length cannot be negative or zero”. Once we raise this error, Python will stop the execution. But if length is positive, it will invoke func and pass in length as an input argument. The safe_calculate is our decorator, which takes as its input a function object and returns a closure that will perform the safe calculation.
square_area_safe = safe_calculate(square_area)
Let’s test it first:
square_area_safe(5)
This is safe and I get the result here on the screen.
25
Invoking it with a negative number will raise an error
square_area_safe(-1)
Output:
ValueError: Length cannot be negative or zero
Let’s decorate the perimeter function as well with the safe_calculate.
square_perimeter_safe = safe_calculate(square_perimeter) square_perimeter(10)
Output:
40
But if you were to call square_perimeter_safe with a negative value for length well, that is a ValueError.
square_perimeter_safe(-10)
Output:
ValueError: Length cannot be negative or zero
Now that you have a decorator, you should decorate your functions rather than use the way that we have been using so far.
@safe_calculate
def square_area(length): return length**2 @safe_calculate
def square_perimeter(length): return 4 * length
Now, the next time square_area or the square_perimeter is called, the safety check will be performed.
square_perimeter(3)
Output:
12
If you try to calculate the perimeter for a negative value of the length, you will get a ValueError. The safe_calculate function that we set up earlier has a limitation, and you will see what it in a future example.
square_perimeter(-3)
Output:
ValueError: Length cannot be negative or zero
What happens when you have more than one input? Here is a function that calculates the area of a rectangle.
@safe_calculate
def rectangle_area(length, width): return length * width
Within our safe_calculate function, we had invoked the func object which performs the calculation with just one input argument, with just the variable length. This is going to cause a problem when we use the safe_calculate decorator for the rectangle_area function.
Once I have decorated this function, I’m going to invoke it with 4, 5.
rectangle_area(4, 5)
Output:
TypeError: calculate() takes 1 positional argument but 2 were given
The problem is with the way we had defined the closure inside the safe_calculate function.
The calculate closure takes in just one input argument. If a function has multiple input arguments, then safe_calculate cannot be used. A redefined safe_calculate_all function is shown below:
def safe_calculate_all(func): def calculate(*args): for arg in args: if arg <= 0: raise ValueError("Argument cannot be negative or zero") return func(*args) return calculate.
It takes in one input argument that is the function object that is to be decorated. The main change is in the input arguments that are passed into the calculate closure. The function calculate now takes in variable length arguments, *args. The function iterates over all of the arguments that were passed in, and checks to see whether the argument is less than or equal to 0. If any of the arguments are less than or equal to 0, a ValueError will be raised. Remember, *args will unpack the original arguments so that the elements of the tuple are passed in individually to the function object, func. You can now use this safe_calculate_all decorator with functions that have any number of arguments.
@safe_calculate_all
def rectangle_area(length, width): return length * width
rectangle_area(10, 3)
Output:
30
Let’s try invoking the same function, but this time one of the arguments is negative. Width is negative and that gives me a ValueError, thanks to our safe_calculate_all decorator.
rectangle_area(10, -3)
When you invoke this function, it will check all arguments.
ValueError: Argument cannot be negative or zero
It doesn’t matter which argument is negative, you still get the ValueError. Here the length is negative:
rectangle_area(-10, 3)
Output:
ValueError: Argument cannot be negative or zero
Chaining Decorators
You can have a function decorated using multiple decorators. And these decorators will be chained together.
Here are two decorators, one prints asterisks and the other plus signs
def asterisk_highlight(func): def highlight(): print("*" * 50) func() print("*" * 50) return highlight def plus_highlight(func): def highlight(): print("+" * 50) func() print("+" * 50) return highlight
The print_message_one is decorated with the asterisk_highlight.
@asterisk_highlight
def print_message_one(): print("Decorators are cool!") print_message_one()
Output:
**************************************************
Decorators are cool!
**************************************************
Now let’s define another print function, but this time we will decorate it using two decorators, the plus_highlight and the asterisk_highlight.
@plus_highlight
@asterisk_highlight
def print_message_one(): print("Decorators are cool!")
What you see here is an example of chaining decorators together. But how are they chained? Which decoration comes first, the asterisk_highlight, or the plus_highlight? Whichever decorator is the closest to the function definition is what is executed first, and then the decorator which is further away from the function definition. This means that the message will be first highlighted with the asterisk, then the plus.
print_message_one()
Output:
++++++++++++++++++++++++++++++++++++++++++++++++++
**************************************************
Decorators are cool!
**************************************************
++++++++++++++++++++++++++++++++++++++++++++++++++
If you change the order of the decorators, the decorations order will change as well.
@asterisk_highlight
@plus_highlight
def print_message_one(): print("Decorators are cool!")
You will have the same function print_message_one, but the decorator that is closest to the function definition is the plus_highlight and then the asterisk_highlight.
print_message_one()
Output:
**************************************************
++++++++++++++++++++++++++++++++++++++++++++++++++
Decorators are cool!
++++++++++++++++++++++++++++++++++++++++++++++++++
**************************************************
Use of kwargs in Decorators
In this example we are using kwargs to display different messages for a decorator that times the execution of a function
def timeit(func): def timed(*args, **kw): if 'start_timeit_desc' in kw: print(kw.get('start_timeit_desc')) ts = time.time() result = func(*args, **kw) te = time.time() if 'end_timeit_desc' in kw: print('Running time for {} is {} ms'.format(kw.get('end_timeit_desc'), (te - ts) * 1000)) return result return timed
The timeit decorator is used for the test function. Three parameters are passed to the function test: a, b and, **kwargs. The parameters a and b are handled in the decorator with *args as we have seen before. The **kwargs parameter is used to pass descriptions for the function. These parameters are start_timeit_desc and end_timeit_desc. These two parameters are checked inside the timed closure and will display the messages that are in them.
@timeit
def test(a,b, **kwargs): return a * b result = test(10,20, start_timeit_desc = "Start of test(10,20)...", end_timeit_desc = "End of test(10,20)")
print("result of test(10,20) = " + str(result))
Output:
Start of test(10,20)...
Running time for End of test(10,20) is 0.0 ms
result of test(10,20) = 200
