Decorators and Closures
A decorator is a function that takes a function and returns a replacement. That is the whole idea. The @ syntax is sugar, and closures are the mechanism that makes the replacement able to remember anything.
The @ is only sugar
def shout(function):
def wrapper(name):
return function(name).upper()
return wrapper
@shout
def greet(name):
return f"hello {name}"
print(greet("aisha")) # HELLO AISHA
Open this example in the editor and change it. Nothing you type is saved.
This is the same thing, written out:
def shout(function):
def wrapper(name):
return function(name).upper()
return wrapper
def greet(name):
return f"hello {name}"
greet = shout(greet) # exactly what @shout did
print(greet("aisha")) # HELLO AISHA
Open this example in the editor and change it. Nothing you type is saved.
@shout above a def means "after defining this, pass it to shout and rebind the name to whatever comes back". Nothing more.
Because the wrapper replaces the function, it must accept whatever the original accepted. *args, **kwargs is how a decorator stays agnostic:
def shout(function):
def wrapper(*args, **kwargs):
return function(*args, **kwargs).upper()
return wrapper
@shout
def greet(name, greeting="hello"):
return f"{greeting} {name}"
print(greet("aisha")) # HELLO AISHA
print(greet("sam", greeting="hi")) # HI SAM
Open this example in the editor and change it. Nothing you type is saved.
Keep the identity of what you wrapped
def plain(function):
def wrapper(*args, **kwargs):
return function(*args, **kwargs)
return wrapper
@plain
def area(width, height):
"""Return the area of a rectangle."""
return width * height
print(area.__name__) # 'wrapper' - the name is gone
print(area.__doc__) # None - and so is the docstring
Open this example in the editor and change it. Nothing you type is saved.
functools.wraps copies the name, docstring, module, annotations and a reference to the original:
from functools import wraps
def better(function):
@wraps(function)
def wrapper(*args, **kwargs):
return function(*args, **kwargs)
return wrapper
@better
def area(width, height):
"""Return the area of a rectangle."""
return width * height
print(area.__name__) # 'area'
print(area.__doc__) # Return the area of a rectangle.
print(area.__wrapped__) # the original function, for testing
Open this example in the editor and change it. Nothing you type is saved.
Always use @wraps. Without it, help(), debuggers, tracebacks, documentation tools and anything that introspects your code all see wrapper, and a codebase with forty decorated functions looks like forty copies of the same one.
Decorators that take arguments
A decorator with arguments is a function returning a decorator - one extra layer:
from functools import wraps
def repeat(times): # takes the argument
def decorate(function): # takes the function
@wraps(function)
def wrapper(*args, **kwargs): # takes the call
result = None
for _ in range(times):
result = function(*args, **kwargs)
return result
return wrapper
return decorate
@repeat(times=3)
def log(message):
print(message)
log("hello")
# hello
# hello
# hello
Open this example in the editor and change it. Nothing you type is saved.
@repeat(times=3) is evaluated first, producing decorate, which is then applied to log. Three layers, each with one job: arguments, function, call.
A decorator that works both with and without arguments needs a check, and is usually not worth the complexity:
from functools import wraps
def optional(function=None, *, loud=False):
def decorate(fn):
@wraps(fn)
def wrapper(*args, **kwargs):
result = fn(*args, **kwargs)
return result.upper() if loud else result
return wrapper
return decorate(function) if function else decorate
@optional
def quiet(name):
return f"hello {name}"
@optional(loud=True)
def loud(name):
return f"hello {name}"
print(quiet("aisha"), "|", loud("aisha")) # hello aisha | HELLO AISHA
Open this example in the editor and change it. Nothing you type is saved.
A cache, which is where this pays off
from functools import lru_cache, cache
@cache
def fib(n):
return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(50)) # 12586269025, instantly
print(fib.cache_info()) # CacheInfo(hits=48, misses=51, ...)
Open this example in the editor and change it. Nothing you type is saved.
Without the cache this is exponential: fib(30) would be over a million calls, and fib(50) would take longer than you are willing to wait. With it, each n is computed once.
@cache is unbounded; @lru_cache(maxsize=128) evicts the least recently used. Three rules for using either safely:
- The function must be pure. Same arguments, same answer, no side effects. Caching a function that reads a file or the clock caches the first answer forever.
- The arguments must be hashable. A list argument raises
TypeError; pass a tuple. - The cache holds references. Caching a method keeps every
selfalive for the process's lifetime, which is a memory leak with a respectable name. Usefunctools.cached_propertyfor per-instance caching instead.
from functools import cached_property
class Report:
def __init__(self, rows):
self.rows = rows
@cached_property
def total(self):
print("computing")
return sum(self.rows)
r = Report([1, 2, 3])
print(r.total) # 'computing' then 6
print(r.total) # 6 - no recomputation
Open this example in the editor and change it. Nothing you type is saved.
Closures are what make it work
The wrapper refers to function, which was a parameter of the enclosing call. That binding outlives the call because the wrapper still refers to it.
def multiplier(factor):
def multiply(n):
return n * factor # `factor` is captured
return multiply
double = multiplier(2)
triple = multiplier(3)
print(double(5), triple(5)) # 10 15
print(double.__closure__[0].cell_contents) # 2 - visible, if you look
Open this example in the editor and change it. Nothing you type is saved.
Each call to multiplier creates a new factor, which is why double and triple do not interfere.
To change captured state, you need nonlocal:
def make_counter():
count = 0
def increment():
nonlocal count # without this: UnboundLocalError
count += 1
return count
return increment
a = make_counter()
b = make_counter()
print(a(), a(), a()) # 1 2 3
print(b()) # 1 - its own count
Open this example in the editor and change it. Nothing you type is saved.
A worked example
import time
from functools import wraps
def instrument(name=None, *, threshold_ms=0.0):
"""Times a call, counts calls, and records the slowest - all in
state captured by the closure, so each decorated function has its
own and nothing is stored globally."""
def decorate(function):
label = name or function.__name__
stats = {"calls": 0, "total_ms": 0.0, "slowest_ms": 0.0, "errors": 0}
@wraps(function)
def wrapper(*args, **kwargs):
start = time.perf_counter()
try:
return function(*args, **kwargs)
except Exception:
stats["errors"] += 1
raise
finally:
elapsed = (time.perf_counter() - start) * 1000
stats["calls"] += 1
stats["total_ms"] += elapsed
stats["slowest_ms"] = max(stats["slowest_ms"], elapsed)
if elapsed > threshold_ms:
print(f" [{label}] {elapsed:.2f}ms")
wrapper.stats = stats # attached, so the caller can read it
wrapper.reset = lambda: stats.update(calls=0, total_ms=0.0,
slowest_ms=0.0, errors=0)
return wrapper
return decorate
@instrument("sum of squares", threshold_ms=0.5)
def work(n):
return sum(i * i for i in range(n))
@instrument()
def risky(n):
if n < 0:
raise ValueError("negative")
return n
print(work(50000))
print(work(5))
print(work.__name__) # work - thanks to @wraps
print(work.stats)
for value in (1, -1, 2):
try:
risky(value)
except ValueError:
pass
print(risky.stats) # calls=3, errors=1
print(work.stats is risky.stats) # False - separate closures
work.reset()
print(work.stats["calls"]) # 0
# Stacking decorators: the one nearest the def is applied first.
def tagged(tag):
def decorate(function):
@wraps(function)
def wrapper(*args, **kwargs):
return f"<{tag}>{function(*args, **kwargs)}</{tag}>"
return wrapper
return decorate
@tagged("b")
@tagged("i")
def text():
return "hello"
print(text()) # <b><i>hello</i></b>
Open this example in the editor and change it. Nothing you type is saved.
Two things are worth taking from that. The stats dict lives in the closure, so each decorated function has its own without any global registry - and attaching it to the wrapper as an attribute gives the caller a way in. And the finally records the timing even when the call raised, which a try/except without finally would not.
The stacking order is the one people get backwards: @tagged("b") is written first and applied last, so it ends up outermost. Read decorators bottom-up to know what wraps what.
When it goes wrong
| Symptom | Cause |
|---|---|
Every function reports the name wrapper |
Missing @wraps |
TypeError: wrapper() takes 1 positional argument |
The wrapper does not accept *args, **kwargs |
UnboundLocalError on captured state |
Assigning without nonlocal |
| A cached function returns stale data | It is not pure; do not cache it |
TypeError: unhashable type: 'list' from a cache |
lru_cache keys on the arguments |
| Memory grows with every object | lru_cache on a method pins self forever |
| The decorator ran once, not per call | The work was outside the inner wrapper |
| Decorators applied in the wrong order | The one nearest the def is applied first |
A check you can run
from functools import wraps
def noisy(function):
print("decorating", function.__name__)
@wraps(function)
def wrapper(*args, **kwargs):
print("calling", function.__name__)
return function(*args, **kwargs)
return wrapper
@noisy
def f():
return 1
print("--- defined ---")
f()
f()
Open this example in the editor and change it. Nothing you type is saved.
"decorating f" prints before the --- defined --- line, and only once; "calling f" prints twice, after. That tells you exactly where each piece of a decorator runs: the outer body at import time, the wrapper body at call time. Anything expensive put in the wrong one is either done once when it should be done per call, or done per call when once would do.