Tuesday 9 August 2011

Python Lambda Function 2.

When I first started learning Python, one of the most confusing concepts to get my head around was the lambda statement. I’m sure other new programmers get confused by it as well and some of you are probably wondering what I’m talking about. So, in the spirit of education, let’s have a pop quiz:
Q. What is a lambda?
A. the 11th letter of the Greek alphabet
B. the craniometric point at the junction of the sagittal and lamboid sutures of the skull
C. the driver in an Arm Slave mecha that allows it to change the user’s thoughts into reality
D. the name of a series of Japanese rocket
E. anonymous (unbound) functions
If you guessed, F – all of the above, you got it right! Of course, in the context of this article, “E” is really the right answer. The Python lambda statement is an anonymous or unbound function and a pretty limited function at that. Let’s take a look at a few typical examples and see if we can find a use case for it.
The typical examples that one normally sees for teaching the lambda is some kind of boring doubling function. Just to be contrary, our simple example will show how to find the square root. First we’ll show a normal function and then the lambda equivalent:
import math
 
#----------------------------------------------------------------------
def sqroot(x):
    """
    Finds the square root of the number passed in
    """
    return math.sqrt(x)
 
square_rt = lambda x: math.sqrt(x)
If you try each of these functions, you’ll end up with a float. Here are a couple examples:

>>> sqroot(49)
7.0
>>> square_rt(64)
8.0
Pretty slick, right? But where would we actually use a lambda in real life? Maybe a calculator program? Well, that would work, but it’s a pretty limited application for a builtin of Python! One of the major pieces of Python that lambda examples are applied to regularly are Tkinter callbacks. We’ll take a look at that, but we’ll also take that information and try it with wxPython to see if works there just as well.

Tkinter + lambda

We’ll start with Tkinter since it’s included with the standard Python package. Here’s a really simple script with three buttons, two of which are bound to their event handler using a lambda:
import Tkinter as tk
 
########################################################################
class App:
    """"""
 
    #----------------------------------------------------------------------
    def __init__(self, parent):
        """Constructor"""
        frame = tk.Frame(parent)
        frame.pack()
 
        btn22 = tk.Button(frame, text="22", command=lambda: self.printNum(22))
        btn22.pack(side=tk.LEFT)
        btn44 = tk.Button(frame, text="44", command=lambda: self.printNum(44))
        btn44.pack(side=tk.LEFT)
 
        quitBtn = tk.Button(frame, text="QUIT", fg="red", command=frame.quit)
        quitBtn.pack(side=tk.LEFT)
 
 
    #----------------------------------------------------------------------
    def printNum(self, num):
        """"""
        print "You pressed the %s button" % num
 
if __name__ == "__main__":
    root = tk.Tk()
    app = App(root)
    root.mainloop()
Notice the btn22 and btn44 variables. This is where the action is. We create a tk.Buttoninstance here and bind to our printNum method in one fell swoop. The lambda is assigned to the button’s command parameter. What this means is that we’re creating a one-off function for the command, much like in the quit button where we call the frame’s quit method. The difference here is that this particular lambda is a method that calls another method and passes the latter an integer. In the printNum method, we print to stdout which button was pressed by using the information that was passed to it from the lambda function. Did you follow all that? If so, we can continue…if not, re-read this paragraph as many times as necessary until the information sinks in or you go crazy, whichever comes first.

wxPython + lambda

Our wxPython example is pretty similar to the Tkinter one, only a little more verbose:
import wx
 
########################################################################
class DemoFrame(wx.Frame):
    """
    Frame that holds all other widgets
    """
 
    #----------------------------------------------------------------------
    def __init__(self):
        """Constructor"""
        wx.Frame.__init__(self, None, wx.ID_ANY,
                          "wx lambda tutorial",
                          size=(600,400)
                          )
        panel = wx.Panel(self)
 
        button8 = wx.Button(panel, label="8")
        button8.Bind(wx.EVT_BUTTON, lambda evt, name=button8.GetLabel(): self.onButton(evt, name))
        button10 = wx.Button(panel, label="10")
        button10.Bind(wx.EVT_BUTTON, lambda evt, name=button10.GetLabel(): self.onButton(evt, name))
 
        sizer = wx.BoxSizer(wx.HORIZONTAL)
        sizer.Add(button8, 0, wx.ALL, 5)
        sizer.Add(button10, 0, wx.ALL, 5)
        panel.SetSizer(sizer)
 
    #----------------------------------------------------------------------
    def onButton(self, event, buttonLabel):
        """"""
        print "You pressed the %s button!" % buttonLabel
 
# Run the program
if __name__ == "__main__":
    app = wx.PySimpleApp()
    frame = DemoFrame().Show()
    app.MainLoop()
In this case, we create a two-parameter anonymous function with the lambda statement. The first parameter is evt and the second is the button’s label. These are passed on to the onButtonevent handler, which is called when the user clicks on one of the two buttons. This example also prints the button’s label to stdout.

Wrapping Up

The lambda statement is used in all kinds of other projects as well. If you Google a Python project name and lambda, you can find lots of live code out there. For example, if you search for “django lambda”, you’ll find out that django has a modelformset factory that utilizes lambdas. The Elixir plugin for SqlAlchemy also uses lambdas. Keep your eyes open and you’ll be surprised how many times you’ll stumble across this handy little function maker.

Python Lambda Function.

Python: Lambda Functions

Note: Lines beginning with ">>>" and "..." indicate input to Python (these are the default prompts of the interactive interpreter). Everything else is output from Python.
Python supports the creation of anonymous functions (i.e. functions that are not bound to a name) at runtime, using a construct called "lambda". This is not exactly the same as lambda in functional programming languages, but it is a very powerful concept that's well integrated into Python and is often used in conjunction with typical functional concepts like filter(), map() and reduce().
This piece of code shows the difference between a normal function definition ("f") and a lambda function ("g"):

>>> def f (x): return x**2
... 
>>> print f(8)
64
>>> 
>>> g = lambda x: x**2
>>> 
>>> print g(8)
64
As you can see, f() and g() do exactly the same and can be used in the same ways. Note that the lambda definition does not include a "return" statement -- it always contains an expression which is returned. Also note that you can put a lambda definition anywhere a function is expected, and you don't have to assign it to a variable at all.
The following code fragments demonstrate the use of lambda functions. Note that you should have Python 2.2 or newer, in order to have support for nested scopes (in older versions you have to pass "n" through a default argument to make this example work).

>>> def make_incrementor (n): return lambda x: x + n
>>> 
>>> f = make_incrementor(2)
>>> g = make_incrementor(6)
>>> 
>>> print f(42), g(42)
44 48
>>> 
>>> print make_incrementor(22)(33)
55
The above code defines a function "make_inrementor" that creates an anonymous function on the fly and returns it. The returned function increments its argument by the value that was specified when it was created.
You can now create multiple different incrementor functions and assign them to variables, then use them independent from each other. As the last statement demonstrates, you don't even have to assign the function anywhere -- you can just use it instantly and forget it when it's not needed anymore.

The following takes this a step further.

>>> foo = [2, 18, 9, 22, 17, 24, 8, 12, 27]
>>> 
>>> print filter(lambda x: x % 3 == 0, foo)
[18, 9, 24, 12, 27]
>>> 
>>> print map(lambda x: x * 2 + 10, foo)
[14, 46, 28, 54, 44, 58, 26, 34, 64]
>>> 
>>> print reduce(lambda x, y: x + y, foo)
139
First we define a simple list of integer values, then we use the standard functionsfilter(), map() and reduce() to do various things with that list. All of the three functions expect two arguments: A function and a list.
Of course, we could define a separate function somewhere else and then use that function's name as an argument to filter() etc., and in fact that's probably a good idea if we're going to use that function several times, or if the function is too complex for writing in a single line. However, if we need it only once and it's quite simple (i.e. it contains just one expression, like in the above examples), it's more convenient to use a lambda construct to generate a (temporary) anonymous function and pass it tofilter() immediately. This creates very compact, yet readable code.
In the first example, filter() calls our lambda function for each element of the list, and returns a new list which contains only those elements for which the function returned "true". In this case, we get a list of all elements that are multiples of 3. The expression x % 3 == 0 computes the remainder of x divided by 3 and compares the result with 0 (which is true if x is evenly divisible by 3).
In the second example, map() is used to convert our list. The given function is called for every element in the original list, and a new list is created which contains the return values from our lambda function. In this case, it computes 2 * x + 10 for every element.
Finally, reduce() is somewhat special. The "worker function" for this one must accept two arguments (we've called them x and y here), not just one. The function is called with the first two elements from the list, then with the result of that call and the third element, and so on, until all of the list elements have been handled. This means that our function is called n-1 times if the list contains n elements. The return value of the last call is the result of the reduce() construct. In the above example, it simply adds the arguments, so we get the sum of all elements.

The following example is one way to compute prime numbers in Python (not the most efficient one, though):

>>> nums = range(2, 50) 
>>> for i in range(2, 8): 
...     nums = filter(lambda x: x == i or x % i, nums)
... 
>>> print nums
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47]
How does it work? First, we put all numbers from 2 to 49 into a list called "nums". Then we have a "for" loop that iterates over all possible divisors, i.e. the value of "i" goes from 2 to 7. Naturally, all numbers that are multiples of those divisors cannot be prime numbers, so we use a filter function to remove them from the list. (This algorithm is called "the sieve of Eratosthenes".)
In the above case, the filter function simply says: "Leave the element in the list if it is equal to i, or if it leaves a non-zero remainder when divided by i. Otherwise remove it from the list." After the filtering loop finishes, only prime numbers are left, of course. I am not aware of a language in which you can do the same thing with built-in features as compact and as readable as in Python (except for functional programming languages).
Note: The range function simply returns a list containing the numbers from x to y-1. For example, range(5, 10) returns the list [5, 6, 7, 8, 9].

In the following example, a sentence is split up into a list of words, then a list is created that contains the length of each word.

>>> sentence = 'It is raining cats and dogs'
>>> words = sentence.split()
>>> print words
['It', 'is', 'raining', 'cats', 'and', 'dogs']
>>> 
>>> lengths = map(lambda word: len(word), words)
>>> print lengths
[2, 2, 7, 4, 3, 4]
I think it doesn't need any further explanation, the code is practically self-documenting.
Of course, it could all be written in one single statement. Admittedly, this is somewhat less readable (not much, though).

>>> print map(lambda w: len(w), 'It is raining cats and dogs'.split())
[2, 2, 7, 4, 3, 4]

Here's an example from the UNIX scripting world: We want to find all mount points in our file system. To do that, we execute the external "mount" command and parse the output.

>>> import commands
>>> 
>>> mount = commands.getoutput('mount -v')
>>> lines = mount.split('\n')
>>> points = map(lambda line: line.split()[2], lines)
>>> 
>>> print points
['/', '/var', '/usr', '/usr/local', '/tmp', '/proc']
The getoutput function from the commands module (which is part of the Python standard library) runs the given command and returns its output as a single string. Therefore, we split it up into separate lines first. Finally we use "map" with a lambda function that splits each line (on whitespace, which is the default) and returns just the third element of the result, which is the mountpoint.
Again, we could write all of that in one single statement, which increases compactness but reduces readability:

print map(lambda x: x.split()[2], commands.getoutput('mount -v').split('\n'))
['/', '/var', '/usr', '/usr/local', '/tmp', '/proc']
When writing "real-world" scripts, it is recommended to split up complex statements so that it is easier to see what it does. Also, it is easier to make changes.
However, the task of splitting up the output of a command into a list of lines is very common. You need it all the time when parsing the output of external commands. Therefore, it is common practice to include the split operation on the getoutput line, but do the rest separately. This is a good trade-off between compactness and readability:

>>> lines = commands.getoutput('mount -v').split('\n')
>>> 
>>> points = map(lambda line: line.split()[2], lines)
>>> print points
['/', '/var', '/usr', '/usr/local', '/tmp', '/proc']
An even better idea is probably to write a small function for that task, which encapsulates the job of running the command and splitting the output.
On a related note, you can also use so-called list comprehesions to construct lists from other lists. Sometimes this is preferable because of efficiency or readability. The previous example could very well be rewritten using a list comprehension:

>>> lines = commands.getoutput('mount -v').split('\n')
>>> 
>>> points = [line.split()[2] for line in lines]
>>> print points
['/', '/var', '/usr', '/usr/local', '/tmp', '/proc']
In many cases, you can use list comprehensions instead of map() or filter(). It depends on the situation which one should be preferred.

Operator Overloading in Python.

Built in data structures are nice, but have you ever needed or wanted to use something that isn't built in? Have you ever wanted to use built in operators on your data structures? Operator overloading allows you to do so.

In Python the following are operators:


+	   -	   *	   **	  /	   //	  %
<<	  >>	  &	   |	   ^	   ~
<	   >	   <=	  >=	  ==	  !=	  <>


Simple examples of a few:
2 + 1
2*1
2.0-1.0
2/3+1/4

Hey wait a minute what was that last one? 2/3+1/4, according to Python, this is 0! That's not what we wanted now was it? We would like to work with rational numbers in python, so we'll begin by creating a new class called "Rational"
class Rational(object):
	def __init__(self, numerator=0, denomiator=1):	
		self.numer = numerator
		self.denom = denomiator
		self.fixSign()	 #will explain later
		self.reduce()	 #will explain later


Here I have defined the class and put in an initializing function, what __init__ does is creates an object with the initial parameters if no parameters are supplied for the object. If parameters are supplied, the object will be created with the supplied parameters.

Next we will want to overload some operators so we can use our new class.
	def __add__(self,rhs):  #self and rhs are Rational objects
		n1 = self.numer	#numerator of the self object
		d1 = self.denom	#denominator of the self object
		n2 = rhs.numer	 #numerator of the rhs object
		d2 rhs.denom	   #denominator of the rhs object
		r = Rational(d2*n1+d1*n2,d1*d2)	#creates a new object with d2*n1+d1*n2 as the numerator, and d1*d2 as the denominator
		return r
	def __sub__(self,rhs):
		n1 = self.numer	#numerator of the self object
		d1 = self.denom	#denominator of the self object
		n2 = rhs.numer	 #numerator of the rhs object
		d2 = rhs.denom	   #denominator of the rhs object
		r = Rational(d2*n1-d1*n2,d1*d2)	#creates a new object with d2*n1-d1*n2 as the numerator, and d1*d2 as the denominator
		return r


__add__ and __sub__ are the Python names used for the + and - operators. Now that we have these operators overloaded, we can use + and - on Rational objects just as we would on integers or floats.

Before we get to using our operators, let's first create a few more functions that make life easier.


	def gcd(self,a,b):
		if b == 0:
			return a
		return self.gcd(b,a%b)
	def fixSigns(self):	#takes - sign out of the denominator
		if self.denom <0:	#check if denominator is negative
			self.numer = -self.numer	#switch sign of numerator
			self.denom = -self.denom	#switch sign of denominator
	def reduce(self):	#reduce the fraction
		d = 1
		if self.denom != 0 and self.numer !=0:	  #make sure the fraction != 0
			d = self.gcd(self.numer, self.denom)	#finds the greatest common denominator in order to reduce 
		if d > 1:		#reduce away!
			self.numer /= d
			self.denom /= d
	def __str__(self):			#rules for printing our objects
		s = "%d /% d" % (self.numer,self.denom)
		return s


gcd finds the greatest common denominator of the numerator and denominator
fixSigns is used to remove negatives from the denominator
reduce reduces the fraction by using the greatest common denominator function
__str__ gives a rule on how we want our object printed



Complete class:
class Rational(object):
	def __init__(self, numerator=0, demoniator=1):
		self.numer = numerator
		self.denom = demoniator
		self.fixSign()
		self.reduce()
	def __add__(self,rhs):  #self and rhs are Rational objects
		n1 = self.numer	#numerator of the self object
		d1 = self.denom	#denominator of the self object
		n2 = rhs.numer	 #numerator of the rhs object
		d2 rhs.denom	   #denominator of the rhs object
		r = Rational(d2*n1+d1*n2,d1*d2)	#creates a new object with d2*n1+d1*n2 as the numerator, and d1*d2 as the denominator
		return r
	def __sub__(self,rhs):
		n1 = self.numer	#numerator of the self object
		d1 = self.denom	#denominator of the self object
		n2 = rhs.numer	 #numerator of the rhs object
		d2 = rhs.denom	   #denominator of the rhs object
		r = Rational(d2*n1-d1*n2,d1*d2)	#creates a new object with d2*n1-d1*n2 as the numerator, and d1*d2 as the denominator
		return r
	def gcd(self,a,b):
		if b == 0:
			return a
		return self.gcd(b,a%b)
	def fixSigns(self):	#takes - sign out of the denominator
		if self.denom <0:	#check if denominator is negative
			self.numer = -self.numer	#switch sign of numerator
			self.denom = -self.denom	#switch sign of denominator
	def reduce(self):	#reduce the fraction
		d = 1
		if self.denom != 0 and self.numer !=0:	  #make sure the fraction != 0
			d = self.gcd(self.numer, self.denom)	#finds the greatest common denominator in order to reduce 
		if d > 1:		#reduce away!
			self.numer /= d
			self.denom /= d
	def __str__(self):			#rules for printing our objects
		s = "%d /% d" % (self.numer,self.denom)
		return s


Now that we have all this done, we can start using our class and operators. Let's test!

from rational import *
def main():
	r1 = Rational(4,9)
	r2 = Rational(2,7)
	r3 = Rational(1,3)
	r4 = Rational(4,6)


	#use our __str__ function
	print "r1: ", r1
	print "r2: ", r2
	print "r3: ", r3
	print "r4: ", r4

	#use our operators

	print "r2-r1: ", (r2-r1)
	print "r1+r3: ", (r1+r3)

	r5 = r3+r4
	print r3, " + ", r2, " = ", r3

	a,b,c,d = 1,4,3,6
	print "%d/%d + %d/%d = %s" %(a,b,c,d,Rational(a,b) + Rational(c,d))
	print "%d/%d - %d/%d = %s" %(a,b,c,d,Rational(a,b) - Rational(c,d))

if __name__ == "__main__":
	main()


Python output:
r1:  4 / 9
r2:  2 / 7
r3:  1 / 3
r4:  2 / 3
r2-r1:  -10 / 63
r1+r3:  7 / 9
1 / 3  +  2 / 7  =  1 / 3
1/4 + 3/6 = 3 / 4
1/4 - 3/6 = -1 / 4


Now that we've overloaded the + and - operators, we can easily continue to overload more as needed. Perhaps you'd like to overload == (__eq__) or != (__ne__), you can do so by the same method.


Below is some Python code that shows how to do this:
You can do this as long as you overload the operator. To overload the operator, you need to implement the overload function(s) within your class object. To overload the equality == and non-equality != operators, you use __eq__ and __ne__.
Python Operator Overload
The one problem with overloading an operator is that it can sometimes become confusing when something is or isn’t overloaded. But I can’t offer and suggestions that would make it easier. I think that is a risk you take when doing something this.