Functions, Arguments and Return Values
Python's argument handling is unusually expressive: positional, keyword, defaults, variadic, keyword-only, positional-only. Each exists for a reason, and one of them - the mutable default - is the most famous gotcha in the language.
The basics, and the implicit return
def area(width, height):
"""Return the area of a rectangle."""
return width * height
print(area(3, 4)) # 12
print(area(height=4, width=3)) # 12 - keywords, any order
print(area.__doc__) # Return the area of a rectangle.
def shout(message):
print(message.upper())
# no return statement
result = shout("hello") # HELLO
print(result) # None
Open this example in the editor and change it. Nothing you type is saved.
A function with no return statement returns None. So does a bare return. That is why print(xs.sort()) shows None - sort mutates and returns nothing, by deliberate convention.
The mutable default argument
def add_item(item, basket=[]): # evaluated ONCE, at definition
basket.append(item)
return basket
print(add_item("apple")) # ['apple']
print(add_item("pear")) # ['apple', 'pear'] - the same list!
print(add_item("plum")) # ['apple', 'pear', 'plum']
Open this example in the editor and change it. Nothing you type is saved.
The default is evaluated once, when the def statement runs, and the resulting list is stored on the function object. Every call that does not pass a basket gets that same list.
def add_item(item, basket=[]):
basket.append(item)
return basket
add_item("apple")
add_item("pear")
add_item("plum")
print(add_item.__defaults__) # (['apple', 'pear', 'plum'],)
Open this example in the editor and change it. Nothing you type is saved.
The fix is None, and building the real default inside:
def add_item(item, basket=None):
if basket is None:
basket = []
basket.append(item)
return basket
print(add_item("apple")) # ['apple']
print(add_item("pear")) # ['pear'] - a fresh list each time
Open this example in the editor and change it. Nothing you type is saved.
This applies to every mutable default: lists, dicts, sets, and anything whose constructor you call at definition time. def f(when=datetime.now()) freezes the time the module was imported, which is the same bug wearing a different hat.
*args and **kwargs
def report(label, *amounts, scale=1, **options):
total = sum(amounts) * scale
line = f"{label}: {total}"
if options:
line += " (" + ", ".join(f"{k}={v}" for k, v in options.items()) + ")"
return line
print(report("total", 1, 2, 3)) # total: 6
print(report("total", 1, 2, 3, scale=10)) # total: 60
print(report("total", 1, 2, currency="GBP", vat=0.2)) # total: 3 (currency=GBP, vat=0.2)
Open this example in the editor and change it. Nothing you type is saved.
*amounts collects extra positional arguments into a tuple. **options collects extra keyword arguments into a dict. Anything after *amounts in the signature is keyword-only - which is why scale must be passed by name.
The same symbols work in the other direction, unpacking:
def report(label, *amounts, scale=1, **options):
total = sum(amounts) * scale
line = f"{label}: {total}"
if options:
line += " (" + ", ".join(f"{k}={v}" for k, v in options.items()) + ")"
return line
numbers = [1, 2, 3]
settings = {"scale": 2, "currency": "GBP"}
print(report("unpacked", *numbers, **settings))
# unpacked: 12 (currency=GBP)
Open this example in the editor and change it. Nothing you type is saved.
Keyword-only and positional-only
def connect(host, port, /, timeout=30, *, retries=3):
return f"{host}:{port} timeout={timeout} retries={retries}"
print(connect("db.example", 5432)) # fine
print(connect("db.example", 5432, 60)) # timeout positionally
print(connect("db.example", 5432, retries=5)) # retries by name
# Each of these is a TypeError - uncomment to see it:
# connect(host="db.example", port=5432) # host and port are positional-only
# connect("db.example", 5432, 60, 5) # retries is keyword-only
Open this example in the editor and change it. Nothing you type is saved.
Everything before / is positional-only. Everything after * is keyword-only. The part between can be either.
Why bother? Positional-only means you can rename a parameter later without breaking anyone - nobody could have been passing it by name. Keyword-only means a call site cannot say f(data, True, False, True), which is unreadable and reorderable by accident. The rule of thumb: make boolean and configuration parameters keyword-only, always.
Scope, and closures
counter = 0
def increment_broken():
# counter += 1 # UnboundLocalError: assigning makes it local
pass
def increment():
global counter # explicit, and usually a smell
counter += 1
increment()
print(counter) # 1
Open this example in the editor and change it. Nothing you type is saved.
Assigning to a name anywhere in a function makes it local for the whole function, including before the assignment. That is why reading it first raises UnboundLocalError rather than finding the global.
nonlocal does the same for an enclosing function's scope, and is how a closure mutates captured state:
def make_counter(start=0):
count = start
def increment(step=1):
nonlocal count
count += step
return count
def current():
return count
return increment, current
bump, read = make_counter(10)
print(bump(), bump(5), read()) # 11 16 16
other_bump, other_read = make_counter(10)
print(other_read()) # 10 - a separate count
Open this example in the editor and change it. Nothing you type is saved.
Two counters do not share state: each call to make_counter made a new count.
The late-binding trap is the one to watch for:
# All three functions capture the VARIABLE i, not its value.
funcs = [lambda: i for i in range(3)]
print([f() for f in funcs]) # [2, 2, 2]
# A default argument is evaluated at definition, which binds the value.
funcs = [lambda i=i: i for i in range(3)]
print([f() for f in funcs]) # [0, 1, 2]
Open this example in the editor and change it. Nothing you type is saved.
This bites most often when building callbacks in a loop - a list of button handlers that all do the same thing because they all closed over the same loop variable.
A worked example
import time
from functools import wraps
def retry(attempts=3, delay=0.01, exceptions=(ValueError,)):
"""A decorator factory: the arguments are keyword-friendly and the
exception tuple is a default that is immutable, so it is safe."""
def decorate(function):
@wraps(function)
def wrapper(*args, **kwargs):
last = None
for attempt in range(1, attempts + 1):
try:
return function(*args, **kwargs)
except exceptions as error:
last = error
if attempt < attempts:
time.sleep(delay * attempt)
raise last
return wrapper
return decorate
calls = {"count": 0}
@retry(attempts=4)
def flaky(threshold, *, label="job"):
calls["count"] += 1
if calls["count"] < threshold:
raise ValueError(f"{label} failed on attempt {calls['count']}")
return f"{label} succeeded on attempt {calls['count']}"
print(flaky(3)) # job succeeded on attempt 3
print(flaky.__name__) # flaky - thanks to @wraps
# A function that takes anything and reports its own call, which is the
# shape every logging wrapper and test double has.
def describe_call(*args, **kwargs):
parts = [repr(a) for a in args]
parts += [f"{k}={v!r}" for k, v in kwargs.items()]
return "called with (" + ", ".join(parts) + ")"
print(describe_call(1, "two", key=3))
# called with (1, 'two', key=3)
# Forwarding: *args and **kwargs pass everything through untouched,
# which is how a wrapper stays agnostic about what it wraps.
def timed(function):
@wraps(function)
def wrapper(*args, **kwargs):
start = time.perf_counter()
try:
return function(*args, **kwargs)
finally:
elapsed = time.perf_counter() - start
print(f"{function.__name__} took {elapsed * 1000:.1f}ms")
return wrapper
@timed
def work(n, *, factor=2):
return sum(i * factor for i in range(n))
print(work(10000, factor=3))
# And the mutable-default bug, with the fix beside it.
def append_wrong(item, into=[]):
into.append(item)
return into
def append_right(item, into=None):
into = [] if into is None else into
into.append(item)
return into
# Both arguments are evaluated before print runs, and both calls
# returned THE SAME list - so it is printed twice, fully grown.
print(append_wrong(1), append_wrong(2)) # [1, 2] [1, 2]
print(append_right(1), append_right(2)) # [1] [2]
Open this example in the editor and change it. Nothing you type is saved.
The retry decorator shows the whole argument system in one place: a factory taking keyword arguments with safe defaults, a wrapper that forwards everything with *args, **kwargs, and @wraps to keep the wrapped function's name and docstring.
Type hints on functions
from collections.abc import Callable, Sequence
def summarise(
values: Sequence[float],
*,
key: Callable[[float], float] = abs,
limit: int | None = None,
) -> dict[str, float]:
chosen = sorted(values, key=key, reverse=True)
if limit is not None:
chosen = chosen[:limit]
return {"count": len(chosen), "total": sum(chosen), "first": chosen[0]}
print(summarise([3.0, -7.5, 1.2], limit=2))
# {'count': 2, 'total': -4.5, 'first': -7.5}
# first is -7.5, not 3.0: the sort is by abs(), so -7.5 ranks highest.
Open this example in the editor and change it. Nothing you type is saved.
Nothing checks these at run time. They are for the reader, for your editor's autocomplete, and for a type checker you run separately. Prefer Sequence and Callable from collections.abc in parameter positions, because they accept more than list does.
When it goes wrong
| Symptom | Cause |
|---|---|
| A default list accumulated between calls | Mutable default; use None |
UnboundLocalError reading a global |
Assigning anywhere makes the name local |
| Every callback in a loop does the same thing | Late binding; use a default argument |
| A decorated function lost its name | Missing @wraps |
TypeError: got multiple values for argument |
The same parameter given positionally and by name |
| A boolean argument's meaning is unreadable | Make it keyword-only with * |
f(*args) passed a string character by character |
A string is a sequence; wrap it in a list |
| A timestamp default never changes | datetime.now() evaluated at definition time |
A check you can run
def f(x, acc=[]):
acc.append(x)
return acc
print(f(1))
print(f(2))
print(f(3))
print(f.__defaults__)
Open this example in the editor and change it. Nothing you type is saved.
The fourth line prints ([1, 2, 3],) - the list is stored on the function object, visible from outside, shared by every caller. Once you have seen it sitting there in __defaults__, the rule stops being a thing you memorised and becomes a thing you understand.