As of Python 2.1, the lambda keyword become a whole lot more useful. Instead of having to pass in each variable value, Python is able to reference variables from the containing containing scope.
For instance, let’s say we have a function called shout(f), which calls its parameter f and prints the result:
>>> def shout(f): ... print f() ...
In the olden days, if you wanted to shout the numbers 10 through 12, you could pass a lambda function to shout, like this:
>>> for n in range(3): ... shout(lambda n=n: n+10) ... 10 11 12
While that syntax still works, with Python 2.1 and later, we are able to omit the n=n and write:
>>> for n in range(3): ... shout(lambda: n+10) ... 10 11 12
However, the two forms of lambda are not quite equivalent. Omitting the n=n causes the lambda to refer to the variable in the stack frame of the function that called the lambda. Consider what happens when we change the value of n after we make the lambda function, but before we call it:
>>> l = [] >>> for n in range(3): ... l.append(lambda: n+10) ... >>> for f in l: ... shout(f) ... 12 12 12 >>>
Putting in the n=n causes the value of n to be copied at the time the lambda function is created, which is exactly what we want in this case:
>>> l = [] >>> for n in range(3): ... l.append(lambda n=n: n+10) ... >>> for f in l: ... shout(f) ... 10 11 12 >>>
I found this little wrinkle while working on a multi-threaded application. One thread was producing lambda functions in a loop, and another thread was calling them – some before the loop was finished, and some after.
Comments
...and there is a big debate raging right now as to how generator expressions should behave... whether they should capture their variables or not. Most of the use cases suggest that capturing is more useful, but the semantics are a little unclear and Guido is tending strongly toward non-capturing.
This is confusing if one doesn't understand how closures work, but it is the right behavior: in one case the lambda is instantiated without any local binding for n (which means it gets looked up in the creator's frame), and in the other with a local binding. Capturing the binding is a feature.
To me your example looks contrived and like it's deliberately asking for trouble. There's a 0-argument lambda, for starters; since n is unbound in the body of the lambda, it captures state, for no apparent good reason; and you're instatiating this state-capturing lambda many times within the same execution frame. You're asking for trouble, and thus trouble finds you.
I think one needs to use anonymous funtions according to their common idioms. 0-argument lambdas in functional languages are used to delay the evaluation of an expensive operation; there aren't many interesting higher-order functions over 0-argument functions in most languages, and I don't think shout() counts as one. In idiomatic usage, anonymous functions either don't have any free identifiers, so they don't capture any state; or when they do, it's done deliberately, but they are created inside a factory function that immediately returns the lambda to the caller (which ensures that each lambda is created under a different binding).
The rules of thumb might be: 1. prefer stateless lambdas like "lambda n: n+10"; 2. don't instantiate the same lambda more than once in a scope (since it's going to give you the exact same closure).
em,
You're right; the example is contrived. The original program runs to about 3000 lines, and I thought, perhaps, that a shorter example would be more informative.
I'm hurt, though, that you didn't consider how useful shout() might be in the context of certain business applications. I am currently documenting the shout module, and will be submitting it to python-dev for inclusion into Python 2.4's standard library.
Thanks for your input,
The example is not contrived. I discovered the same "wrinkle" in an almost identical situation, with a thread creating callback lambdas and another thread calling them when data came in.