This page is about understanding decorators in Python, and learning why they are one of Python's most powerful features. I wrote these notes while reading through this article called Demystifying @decorators in Python .
What are they?[]
A decorator is a function that allows us to wrap another function in order to extend the behavior of the wrapped function, without permanently modifying it. Decorators can also be thought of as a practice in metaprogramming, in which programs have the ability to treat other programs as their data.
In order to understand how this works we need to clarify how functions themselves work in Python.
Understanding Functions[]
While looking at Python functions on the surface may seem pretty simple, there are some aspects of functions that we don't deal with very often. Lets look at what functions are and how they are represented in Python.
Functions as procedures[]
This is the most commonly understood facet of functions that people tend to be most familiar with.
What is a procedure? : A procedure is defined as a series of computational steps to be carried out.
Any given procedure might be called at any point during a program's execution, including by other procedures or itself.
Functions as first-class objects[]
In Python, everything is an object, not just the ones you instantiate from a class. Making Python a highly object-oriented language. All of the following things in Python are considered objects:
- Integers
- Strings
- Classes
- Functions
This means we can do such things as save functions in variables and pass them to and return from another function, as well as defining a function in another function. Making them first-class objects , meaning that an entity which supports all the operations generally available to other entities. Such as being passed as an argument, returned from a function, modified, and assigned to a variable.
Functional programming : Higher-order functions[]
Python incorporates some techniques from functional programming languages, such as Haskell and OCaml. Two of the characteristics included in Python from functional programming (there are more but we won't go into them here) are the following:
- Functions are treated as first-class citizens/objects
- Consequently, it supports higher-order functions
What are higher-order functions? : They are functions that can take other functions as arguments or return them as results.
This is a similar principle to the higher-order mathematical functions from calculus, such as the differential operator. It takes a function as input and returns another function, it's derivative as output. Higher-order functions in programming work much the same way, it either takes a function(s) as input or returns them as output, or both.
Examples of functions[]
Here is an example of a basic function :
def hello():
print('Hello!')
hello is of type class 'function', which means it is an object of the class function. Also, classes we defined are objects of the class type. This can be difficult to get your head around, but it gets easier to understand once you start playing with the type function with different things in Python.
Here is an example of a function being a first-class citizen/object :
Storing a function in a variable ...
sayHello = hello # Returns : Hello!
Defining a function inside another function ...
def wrapper_function():
def hello():
print('Hello!')
wrapper_function()
# Returns :
Hello!
Passing a function as an argument to another function, and returning the a function from another function ...
def higher_order(func):
print('Recieved the function {} as input'.format(func))
func()
return func
higher_order(hello)
# Returns :
Recieved the function <function hello as 0x032c7FA8> as input
Hello!
<function hello at 0x032c7FA8>
Now we can actually start discussing decorators.
Understanding decorators[]
What is a decorator? : A decorator is a function that allows us to wrap another function in order to extend the behavior of the wrapped function, without permanently modifying it.
Now that we have better understanding of how higher-order functions work, we can understand how decorators work. Here is an example of a decorator :
def decorator_function(func):
def wrapper():
print('Wrapper function!')
print('The wrapper function is: {}'.format(func))
print('Executing the wrapped function...')
func()
print('Exiting wrapper function')
return wrapper
Here, decorator_function is a decorator function. As you can see, it's a higher-order function because it takes a function as an argument, and it also returns one. Inside decorator_function we've defined another function, a wrapper, which wraps the argument function, and subsequently modifies its behavior. The decorator returns this wrapper function. Now let's look at this decorator in action :
>>> @decorator_function
... def hello():
... print('Hello!')
...
>>> hello()
Wrapper function!
The wrapped function is: <function hello at 0x032B26A8>
Executing wrapped function...
Hello!
Exiting wrapper function
As you can see, just by adding the @decorator_function statement before the def statement for the hello, we've modified it. The @ is just syntactic sugar for :
hello = decorator_function(hello)
In other words, all the @decorator_function statement is doing is calling decorator_function with hello as it's argument, and assigning the returned function to the name hello.
Let's look at another example with a more useful decorator :
def benchmark(func):
import time
def wrapper():
start = time.time()
func()
end = time.time()
print('[*] Execution time: {} seconds.'.format(end-start))
return wrapper
@benchmark
def fetch_webpage():
import requests
webpage = requests.get('https://google.com')
fetch_webpage()
Here we have created a decorator that would measure the time taken by a function to execute. It's a fairly useful decorator. Then we have used it on a function that GETs the homepage of Google. We have saved the time before calling the wrapped function, and after calling the wrapped function, and by subtracting those two we get the time of execution.
When we run this we get the following output :
[*] Execution time: 1.4326345920562744 seconds.
Decorators give us complete flexibility over what we want to do and modify!
Using arguments and return-value[]
In the previous examples we've looked at so far, the decorate functions were neither taking any arguments or returning anything. Lets look to expand the benchmark decorator to include that.
def benchmark(func):
import time
def wrapper(*args, **kwargs):
start = time.time()
return_value = func(*args, **kwargs)
end = time.time()
print('[*] Execution time: {} seconds.'.format(end-start))
return wrapper
@benchmark
def fetch_webpage(url):
import requests
webpage = requests.get(url)
return webpage.text
webpage = fetch_webpage('https://google.com')
print(webpage)
The output of this is :
[*] Execution time: 1.4475083351135254 seconds. <!doctype html><html itemscope="" itemtype="http://schema.org/WebPage"........
You can see that the arguments of the decorated function get passed to the wrapper function, and then you're free to do with them what you will. You can modify these arguments and then pass them to the decorated function, or you can pass them unmodified, or you can discard them completely and pass whatever you want to the decorated function. Same goes with the returned value from the decorated function. In our example we have left the arguments and return values unmodified.
Decorators with arguments[]
You can also define decorators which take arguments. Lets look at an example of this :
def benchmark(iters):
def actual_decorator(func):
import time
def wrapper(*args, **kwargs):
total = 0
for i in range(iters):
start = time.time()
return_value = func(*args, **kwargs)
end = time.time()
total = total + (end-start)
print('[*] Average execution time: {} seconds.'.format(total/iters))
return return_value
return wrapper
return actual_decorator
@benchmark(iters=0)
def fetch_webpage(url):
import requests
webpage = requests.get(url)
return webpage.text
webpage = fetch_webpage('https://google.com')
print(webpage)
Here we've extended the benchmark decorator so that it runs the decorated function a given number of times (specified using the iters parameter), and then it prints out the average time taken by the function to execute. But to do this we had to use a little trick...
The benchmark function, which at first LOOKS like a decorator, is not actually a decorator. It's a regular function, which accepts the argument iters, and returns a decorator. That decorator, in turn, decorates the fetch_webpage function. That's why we haven't used the statement @benchmark, rather we've used @benchmark(iters=0) - meaning the benchmark function is getting called here (a function with parenthesis after it signifies a function call) and the return value of that function is the ACTUAL decorator.
To help us understand this, consider the following rule of thumb :
Decorator functions take a function as an argument, and returns a function
In the example above, the benchmark does not satisfy this rule of thumb, because it doesn't take a function as an argument. Whereas the actual_decorator function - which is returned by benchmark - is a decorator, because it satisfies the above rule.
Objects as decorators[]
Not only just functions can be decorators, any callable can also be a decorator. Class instances, objects with a __call__ method can be called too, so that can be used as a decorator. This functionality can be used to create decorators with some kind of "state". For example we can check out Scott Lobdell's blog HERE where he has written code that shows how to create a memoization decorator.
Final Notes[]
- Decorators need not return functions, they can return whatever you want. But usually we want decorators to return the objects of the same type as of the decorated object :
>>> def decorator(func):
... return 'something'
...
>>> @decorator
... def hello():
... print('Hello!')
...
>>> hello
'something'
- Decorators also dont't need to take ONLY functions as input. You can learn more about this HERE .
- Decorators may only become clear when you write a library. So if you want to understand them better, perhaps look at them from the point-of-view of a library developer. For example in Flask.
- By saying that decorators don't modify functions permenantly, it means that it can be easily removed and added with just a single line.
- It is recommended that you look into functools.wraps - it is a helper function that helps you make a decorated function look like the original function by doing things such as keeping the docstring of the original funciton.
All of these notes are taken from reading Sumit Ghosh 's article Demystifying @decorators in Python .