Python generators are a simple way of creating iterators. By using generators all of the overhead is automatically handled. So basically a generator is a function that returns an object (iterator) which we can iterate over one value at a time.
What are they?Edit
Building an iterator in Python requires a lot of overhead, we need to create a class with __iter__() and __next__() methods, keep track of internal states, raise StopIteration when there was no values to be returned, etc. All this takes time and is counter intuitive. This is where generators come in.
How to create themEdit
This is a fairly simple task. All you have to do is define a function with the yield statement instead of the usual return statement. The difference between these two statements is that a return statement terminates a function entirely, whereas the yield statement pauses the function saving all its states and later continues from there on successive calls.
The difference between a Generator function and a Normal functionEdit
- A generator function contains one or more yield statements.
- When called, it returns an object (iterator) but does not start execution immediately.
- Methods such as __iter__() and __next__() are implemented automatically. So we can iterate through the items using next().
- Once the function yields, the function is paused and the control is transferred to the caller.
- Local variables and their states are remembered between successive calls.
- When the function terminates, StopIteration is raised automatically on further calls.
Here we have created a generator function named my_gen() with several yield statements.
def my_gen(): n = 1 print('This is printed first') yield n n += 1 printed('This is printed second') yield n n += 1 print('This is printed last') yield n # It returns an object but does not start execution immediately. a = my_gen() # We can iterate through the items using next(). next(a) next(a) next(a)
This would return:
This is printed first This is printed second This is printed at last [Finished in 0.1s]
Something to take note of is that the value of n is remembered between each call. Unlike normal functions, the local variables are not destroyed when the function yields. Also, the generator object can only be iterated over ONCE. To restart the process we would need to create another instance of the generator (e.g b = my_gen()).
Generators with a loopEdit
We can also iterate through a generator using a for-loop, like the following:
for each in my_gen(): print(each)
This would return:
This is printed first 1 This is printed second 2 This is printed at last 3 [Finished in 0.1s]
Lets take a look at a look at an example of a generator the reverses a given string:
def rev_str(my_str): length = len(my_str) for i in range(length - 1, -1, -1): yield my_str[i] for char in rev_str("hello"): print(char)
A simple generator can be easily created on the fly using generator expressions, which makes building generators easier.
Same an lambda funcitons create an anonymous one-time function, a generator expression creates an anonymous generator function.
The syntax is similar to list comprehension but the square brackets are replaced with round parentheses.
# Initialize the list myList = [6, 2, 7, 9, 12] # Square each term using list comprehension. [x**2 for x in myList] # Do the same as above but with a generator expression. a = (x**2 for x in myList) # Generator expression output: print(next(a)) print(next(a)) print(next(a)) print(next(a)) print(next(a)) # StopIteration: next(a)
Python Generators - Progamiz
Introduction to Python Generators - Real Python