Understanding Decorators

This page is about understanding decorators in Python, and learning why they are one of Python's most powerful features. I wrote these notes while reading through this article called Demystifying @decorators in Python.

What are they?
A decorator is a function that allows us to wrap another function in order to extend the behavior of the wrapped function, without permanently modifying it. Decorators can also be thought of as a practice in metaprogramming, in which programs have the ability to treat other programs as their data.

In order to understand how this works we need to clarify how functions themselves work in Python.

Understanding Functions
While looking at Python functions on the surface may seem pretty simple, there are some aspects of functions that we don't deal with very often. Lets look at what functions are and how they are represented in Python.

Functions as procedures
This is the most commonly understood facet of functions that people tend to be most familiar with.

 What is a procedure? : A procedure is defined as a series of computational steps to be carried out.

Any given procedure might be called at any point during a program's execution, including by other procedures or itself.

Functions as first-class objects
In Python, everything is an object, not just the ones you instantiate from a class. Making Python a highly object-oriented language. All of the following things in Python are considered objects: This means we can do such things as save functions in variables and pass them to and return from another function, as well as defining a function in another function. Making them first-class objects, meaning that an entity which supports all the operations generally available to other entities. Such as being passed as an argument, returned from a function, modified, and assigned to a variable.
 * Integers
 * Strings
 * Classes
 * Functions

Functional programming : Higher-order functions
Python incorporates some techniques from functional programming languages, such as Haskell and OCaml. Two of the characteristics included in Python from functional programming (there are more but we won't go into them here) are the following:
 * Functions are treated as first-class citizens/objects
 * Consequently, it supports higher-order functions

 What are higher-order functions? : They are functions that can take other functions as arguments or return them as results.

This is a similar principle to the higher-order mathematical functions from calculus, such as the differential operator. It takes a function as input and returns another function, it's derivative as output. Higher-order functions in programming work much the same way, it either takes a function(s) as input or returns them as output, or both.

Examples of functions
Here is an example of a basic function : def hello: print('Hello!') hello is of type class 'function', which means it is an object of the class function. Also, classes we defined are objects of the class type. This can be difficult to get your head around, but it gets easier to understand once you start playing with the type function with different things in Python.

Here is an example of a function being a first-class citizen/object :

Storing a function in a variable ... sayHello = hello Hello!
 * 1) Returns :

Defining a function inside another function ... def wrapper_function: def hello: print('Hello!') wrapper_function Hello!
 * 1) Returns :

Passing a function as an argument to another function, and returning the a function from another function ... def higher_order(func): print('Recieved the function {} as input'.format(func)) func return func higher_order(hello) Recieved the function  as input Hello! 
 * 1) Returns :

Now we can actually start discussing decorators.

Understanding decorators
 What is a decorator? : A decorator is a function that allows us to wrap another function in order to extend the behavior of the wrapped function, without permanently modifying it.

Now that we have better understanding of how higher-order functions work, we can understand how decorators work. Here is an example of a decorator : def decorator_function(func): def wrapper: print('Wrapper function!') print('The wrapper function is: {}'.format(func)) print('Executing the wrapped function...') func print('Exiting wrapper function') return wrapper Here, decorator_function is a decorator function. As you can see, it's a higher-order function because it takes a function as an argument, and it also returns one. Inside decorator_function we've defined another function, a wrapper, which wraps the argument function, and subsequently modifies its behavior. The decorator returns this wrapper function. Now let's look at this decorator in action : >>> @decorator_function ... def hello: ...    print('Hello!') ... >>> hello Wrapper function! The wrapped function is:  Executing wrapped function... Hello! Exiting wrapper function As you can see, just by adding the @decorator_function statement before the def statement for the hello, we've modified it. The <span style="color:rgb(102,255,102);">@ is just syntactic sugar for : hello = decorator_function(hello) In other words, all the <span style="color:rgb(102,255,102);">@decorator_function statement is doing is <span style="color:rgb(255,51,153);">calling decorator_function with hello as it's argument, and assigning the returned function to the name hello.

Let's look at another example with a more useful decorator : def benchmark(func): import time def wrapper: start = time.time func end = time.time print('[*] Execution time: {} seconds.'.format(end-start)) return wrapper @benchmark def fetch_webpage: import requests webpage = requests.get('https://google.com') fetch_webpage Here we have created a decorator that would measure the time taken by a function to execute. It's a fairly useful decorator. Then we have used it on a function that <span style="color:rgb(102,255,102);">GET s the homepage of Google. We have saved the time before calling the wrapped function, and after calling the wrapped function, and by subtracting those two we get the time of execution.

When we run this we get the following output : [*] Execution time: 1.4326345920562744 seconds. Decorators give us complete flexibility over what we want to do and modify!

Using arguments and return-value
In the previous examples we've looked at so far, the decorate functions were neither taking any arguments or returning anything. Lets look to expand the <span style="color:rgb(102,255,102);">benchmark decorator to include that. def benchmark(func): import time def wrapper(*args, **kwargs): start = time.time return_value = func(*args, **kwargs) end = time.time print('[*] Execution time: {} seconds.'.format(end-start)) return wrapper @benchmark def fetch_webpage(url): import requests webpage = requests.get(url) return webpage.text webpage = fetch_webpage('https://google.com') print(webpage) The output of this is : [*] Execution time: 1.4475083351135254 seconds. <!doctype html><html itemscope="" itemtype="http://schema.org/WebPage"........ You can see that the arguments of the decorated function get passed to the wrapper function, and then you're free to do with them what you will. You can modify these arguments and then pass them to the decorated function, or you can pass them unmodified, or you can discard them completely and pass whatever you want to the decorated function. Same goes with the returned value from the decorated function. In our example we have left the arguments and return values unmodified.

Decorators with arguments
You can also define decorators which take arguments. Lets look at an example of this : def benchmark(iters): def actual_decorator(func): import time def wrapper(*args, **kwargs): total = 0 for i in range(iters): start = time.time return_value = func(*args, **kwargs) end = time.time total = total + (end-start) print('[*] Average execution time: {} seconds.'.format(total/iters)) return return_value return wrapper return actual_decorator @benchmark(iters=0) def fetch_webpage(url): import requests webpage = requests.get(url) return webpage.text webpage = fetch_webpage('https://google.com') print(webpage) Here we've extended the benchmark decorator so that it runs the decorated function a given number of times (specified using the <span style="color:rgb(102,255,102);">iters parameter), and then it prints out the average time taken by the function to execute. But to do this we had to use a little trick...

The benchmark function, which at first LOOKS like a decorator, is not actually a decorator. It's a regular function, which accepts the argument <span style="color:rgb(102,255,102);">iters, and returns a decorator. That decorator, in turn, decorates the <span style="color:rgb(102,255,102);">fetch_webpage function. That's why we haven't used the statement <span style="color:rgb(102,255,102);">@benchmark, rather we've used <span style="color:rgb(102,255,102);">@benchmark(iters=0) - meaning the <span style="color:rgb(102,255,102);">benchmark function is getting called here (a function with parenthesis after it signifies a function call) and the return value of that function is the ACTUAL decorator.

To help us understand this, consider the following rule of thumb :

<span style="color:rgb(255,51,153);">Decorator functions take a function as an argument, and returns a function

In the example above, the <span style="color:rgb(102,255,102);">benchmark does not satisfy this rule of thumb, because it doesn't take a function as an argument. Whereas the <span style="color:rgb(102,255,102);">actual_decorator function - which is returned by <span style="color:rgb(102,255,102);">benchmark - is a decorator, because it satisfies the above rule.

Objects as decorators
Not only just functions can be decorators, any callable can also be a decorator. Class instances, objects with a <span style="color:rgb(102,255,102);">__call__ method can be called too, so that can be used as a decorator. This functionality can be used to create decorators with some kind of "state". For example we can check out Scott Lobdell's blog HERE where he has written code that shows how to create a memoization decorator.

Final Notes
>>> def decorator(func): ...    return 'something' ... >>> @decorator ... def hello: ...    print('Hello!') ... >>> hello 'something'
 * Decorators need not return functions, they can return whatever you want. But usually we want decorators to return the objects of the same type as of the decorated object :


 * Decorators also dont't need to take ONLY functions as input. You can learn more about this HERE.


 * Decorators may only become clear when you write a library. So if you want to understand them better, perhaps look at them from the point-of-view of a library developer. For example in Flask.


 * By saying that decorators don't modify functions permenantly, it means that it can be easily removed and added with just a single line.


 * It is recommended that you look into <span style="color:rgb(102,255,102);">functools.wraps - it is a helper function that helps you make a decorated function look like the original function by doing things such as keeping the docstring of the original funciton.

All of these notes are taken from reading Sumit Ghosh 's article Demystifying @decorators in Python.