Python Cheat Sheet - Closure (闭包) and Decorator (装饰器)

Python functions

1. Python function can be assigned to a variable.

>>> slogan = "Life is short, I use Python"
\
>>> print
>>> <function print>
>>> print(slogan)
>>> Life is short, I use Python
\
>>> len
>>> <function len(obj, /)>
>>> len(slogan)
>>> 27
\
>>> original_print = print    # assign print to original_print
>>> original_print    # original_print now becomes print
>>> <function print>
\
>>> original_print(slogan)
>>> Life is short, I use Python
\
>>> print = len    # assign len to print
>>> print    # print now becomes len
>>> <function len(obj, /)>
\
>>> print(slogan)
>>> 27

2. A Python function can be passed as a argument to a function.

• Custom example

def smart_add(x, y, f):
    return f(x) + f(y)

>>> smart_add(-3, 7, abs)    # abs(-3) + abs(7)
>>> 10

• Built-in example: map map（）函数语法：map(func, *iterables) --> map object # help(map)，map(f, iterable)基本上等于：[f(x) for x in iterable]即x in iterables |--map to--> func(x)，Python 2.x 返回列表，Python 3.x 返回迭代器（通过list(map对象)获得输出）。

>>> names = ["Tom", "Jerry", "Bugs Bunny"]
>>> mapped_obj = map(len, names)
>>> mapped_obj
>>> <map at 0x1615a4da5f8>
\
>>> print(list(mapped_obj))
>>> [3, 5, 10]

3. Return a function in a Python function:

def outer():
    print('call outer() ...')
    # define an inner function within the outer function
    def inner():
        print('call inner() ...')
    # return the inner function
    return inner

没有括号的话只是return了function，没有调用return的function

In	Out
r = outer()	call outer() ...
r	<function __main__.outer.<locals>.inner()>
r()	call inner() ...
r2=outer
r2	<function __main__.outer()>
type(r2)	function
r2()	call outer() ...
type(r2())	call outer() ... function
r2()()	call outer() ... call inner() ...
type(r2()())	call outer() ... call inner() ... NoneType

4. Bonus: higher-order function and first-class function

Closures

A Closure is a function object that remembers values in enclosing scopes even if they are not present in memory.

1. Free variable

• If a variable in a function is neither a local variable nor a parameter of that function, this variable is called a free variable of that function.
• In short, free variables are variables that are used locally, but defined in an enclosing scope.
• In our case, x is a parameter of pow_later and y is a local variable of pow_later. But within lazy_pow, x and y are free variables.

import math
def pow_later(x):
    y = 2
    def lazy_pow():
        print('calculate pow({}, {})...'.format(x, y))
        return pow(x, y)
    return lazy_pow

2. What is Closure

• Generally speaking, a closure is a structure (code blocks, function object, callable object, etc.) storing a function together with an environment. The environment here means information about free variables that function bounded, especially values or storage locations of free variables.
• For example, a closure is created, returned and assigned to my_pow after following function call. Note that the closure for lazy_pow extends the scope of lazy_pow function to include the binding for the free variables: x and y.

>>> my_pow = pow_later(3)

py中常常会使用到__code__. 比如闭包中的 x.__code__.co_freevars。

In	Out
my_pow	<function __main__.pow_later.<locals>.lazy_pow()>	\
my_pow()	calculate pow(3, 2)...
my_pow.__code__.co_consts	(None, 'calculate pow({}, {})...') #函数内部的常量
my_pow.__code__.co_freevars	('x', 'y') # 自由变量，在本代码段中被引用，在外层代码段中被赋值的变量
pow_later.__code__.co_cellvars	('x', 'y') # 内层代码所约束的变量，在本代码段中被赋值，且被内层代码段引用的变量
dir(my_pow)	omit
my_pow.__closure__	(<cell at 0x0000011A55182648: int object at 0x00007FFDDA1E9380>, <cell at 0x0000011A55182918: int object at 0x00007FFDDA1E9360>) # Note that my_pow has an attribute named __closure__ and it's a tuple with two elements.
dir(my_pow.__closure__[0])	omit
my_pow.__closure__[0].cell_contents	3
my_pow.__closure__[1].cell_contents	2
import inspect inspect.getclosurevars(my_pow)	ClosureVars(nonlocals={'x': 3, 'y': 2}, globals={}, builtins={'print': <built-in function print>, 'format': <built-in function format>, 'pow': <built-in function pow>}, unbound=set())

3. Nonlocal declaration

• A Nested Function is a function defined inside another function. Noted that the nested functions can access the variables of the enclosing scope. However, at least in python, they are only readonly. However, one can use the nonlocal keyword explicitly with these variables in order to modify them.
• 在python的函数中和全局同名的变量，如果你有修改变量的值就会变成局部变量，在修改之前对该变量的引用自然就会出现没定义这样的错误了，如果确定要引用全局变量，并且要对它修改，必须加上global关键字。

import math
def pow_later(x):
    y = 2
    def lazy_pow():
        print('calculate pow({}, {})...'.format(x, y))
        result = pow(x, y)
        y = y + 1      # increase y
        return result
    return lazy_pow

>>> my_pow = pow_later(3)

In	Out
my_pow	<function __main__.pow_later.<locals>.lazy_pow()>
my_pow()	UnboundLocalError: local variable 'y' referenced before assignment The problem happens in this line: y = y + 1.

import math
def pow_later(x):
    y = 2
    def lazy_pow():
        nonlocal y    # nonlocal declaration
        print('calculate pow({}, {})...'.format(x, y))
        result = pow(x, y)
        y = y + 1
        return result
    return lazy_pow

>>> my_pow = pow_later(3)

In	Out
my_pow	<function __main__.pow_later.<locals>.lazy_pow()>
my_pow.__code__.co_consts	(None, 'calculate pow({}, {})...', 1) # 函数内部的常量
my_pow.__code__.co_freevars	('x', 'y') # 自由变量，在本代码段中被引用，在外层代码段中被赋值的变量
pow_later.__code__.co_cellvars	('x', 'y') # 内层代码所约束的变量，在本代码段中被赋值，且被内层代码段引用的变量
my_pow.__closure__	(<cell at 0x0000011A55182408: int object at 0x00007FFDDA1E9380>, <cell at 0x0000011A55182708: int object at 0x00007FFDDA1E9360>)

>>> my_pow()
calculate pow(3, 2)...
9
>>> my_pow.__closure__[0].cell_contents
3
>>> my_pow.__closure__[1].cell_contents
3
>>> my_pow()
calculate pow(3, 3)...
27
>>> inspect.getclosurevars(my_pow)
ClosureVars(nonlocals={'x': 3, 'y': 4}, globals={}, builtins={'print': <built-in function print>, 'format': <built-in function format>, 'pow': <built-in function pow>}, unbound=set())
>>> my_pow()
calculate pow(3, 4)...
81

def print_msg(number):
    def printer():
        number=3
        print(number)
    printer()
    print(number)
\
>>> print_msg(9)
3
9

def print_msg(number):
    def printer():
        nonlocal number
        number=3
        print(number)
    printer()
    print(number)
\
>>> print_msg(9)
3
3

什么时候闭包？

当嵌套函数引用其封闭范围内的值时，在Python中有使用了一个闭包。 在Python中创建闭包必须满足的标准将在以下几点 -

• 必须有一个嵌套函数(函数内部的函数)。
• 嵌套函数必须引用封闭函数中定义的值。
• 闭包函数必须返回嵌套函数。

闭包可以避免使用全局值并提供某种形式的数据隐藏。它还可以提供面向对象的解决问题的解决方案。

Decorators

Macro宏 for C++

Decorator for python

• 不用装饰器

def make_pretty(func):
    def inner():
        print("I got decorated")
        func()
    return inner
\
def ordinary():
    print("I am ordinary")

In	Out
ordinary	<function __main__.ordinary()>
ordinary()	I am ordinary
pretty = make_pretty(ordinary)
pretty	<function __main__.make_pretty.<locals>.inner()>
pretty()	I got decorated I am ordinary

• 用装饰器

def make_pretty2(func):
    def inner():
        print("I got decorated2")
        func()
    return inner
\
@make_pretty2
def ordinary2():
    print("I am ordinary2")

In	Out
ordinary2	<function __main__.make_pretty2.<locals>.inner()>
ordinary2()	I got decorated2 I am ordinary2

• Chaining Decorators in Python

在Python中链接装饰器：多个装饰器可以在Python中链接。 这就是说，一个函数可以用不同(或相同)装饰器多次装饰。只需将装饰器放置在所需函数之上。

def star(func):
    def inner(*args, **kwargs):
        print("*" * 30)
        func(*args, **kwargs)
        print("*" * 30)
    return inner
\
def percent(func):
    def inner(*args, **kwargs):
        print("%" * 30)
        func(*args, **kwargs)
        print("%" * 30)
    return inner
\   
@star
@percent
def printer(msg):
    print(msg)
printer("Hello")

Output:
******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Hello
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************
The above syntax of,

@star
@percent
def printer(msg):
    print(msg)

is equivalent to

def printer(msg):
    print(msg)
printer = star(percent(printer))

• @property Decorator @property decorator is a built-in decorator in Python for the property() function.

prop=property(getter, setter, deleter, docstring)

Summary

• 一切皆对象(可以将一个函数赋值给一个变量)
• 在函数中定义函数(嵌套函数)
• 从函数中返回函数(闭包)
• 将函数作为参数传给另一个函数(装饰器)

使用场景

• 授权(Authorization)
• 日志(Logging)

References

• Codevoila - Python Tutorial: Python Function And Python Closure
• learning Python with DataCamp's free Intro to Python tutorial - Closures
• Python装饰器
• Python 函数装饰器
• Wiki PythonDecorators
• TutorialsTeacher - @property Decorator
• What does the “at” (@) symbol do in Python?
• Python闭包
• 解决Python报错：local variable 'xxx' referenced before assignment
• Python @property装饰器详解