default argument in Python and Lisp

The goal of this post is to go through:

  • how to pass default argument in Emacs Lisp (and Common Lisp)
  • why the rule “do not use mutable objects as default arguments” in Python
  • difference in how Python and Lisp interpreters treat default arguments

Every Lisp code snippet on this post is for Emacs Lisp, unless stated otherwise.

1. the Python mutable default argument gotcha

Here we define a Python function foo that takes two arguments aa and bb and they are optional and we have specified defaults arguments (the lists [0,0] and [0,0]), and so we call them without arguments twice:

def foo(aa = [0,0], bb = [0,0]):
    print aa, bb
    aa[0] = 7
    bb = [7,7]

foo()   # prints [0, 0] [0, 0] 
foo()   # prints [7, 0] [0, 0]

Does that output make sense? This is the gotcha.

Before we dive into what’s going on, since I want to compare this with Lisp, let’s see how one specify default arguments to optional parameters in Lisp functions.

2. how to give default arguments in Lisp

Here is how in Emacs Lisp.

(require 'cl-lib) ; for cl-defun macro

(cl-defun my-hello (aa bb &optional (cc 100) (dd (+ 100 100)))
  (list aa bb cc dd))

(my-hello 3 5)      ; ⇒ (3 5 100 200)
(my-hello 3 5 7)    ; ⇒ (3 5 7 200)
(my-hello 3 5 7 9)  ; ⇒ (3 5 7 9)

In other words, we use the cl-defun macro to define function my-hello which takes two mandatory arguments and two optional arguments, and we specify the default for cc to be 100, and the default for dd to be the sum of 100 and 100. (In Common Lisp, the usual defun macro can be used in this way.)

3. comparing Lisp and Python

We will define function bar and then call it twice. The function bar takes one optional argument and when we write a definition of this function, we will write an expression (for default argument) that calls another function cow. We will see when the function cow is called.

Python code:

def cow():
    print 'moo'
    return 'cow'

print 'defining bar'

def bar(cc = cow()):
    print 'bar'

print 'calling bar'

bar()
bar()

Lisp code:

(defun cow ()
  (print "moo")
  "cow")

(print "defining bar")

(cl-defun bar (&optional
               (cc (cow)))
  (print "bar"))

(print "calling bar")

(bar)
(bar)

Python output:

defining bar
moo
calling bar
bar
bar

Emacs Lisp output:

"defining bar"
"calling bar"
"moo"
"bar"
"moo"
"bar"

In case of Python, the expression cow() was calculated when the function bar was defined, not when bar was called. In case of Lisp, it’s the other way around, i.e., the expression (cow) was evaluated every time bar was called. The Lisp behavior is the expected behavior and the Python behavior is the surprising one and it does surprise many Python beginners. As far as I know, this is a Python-only gotcha. (On the other hand, the rule “don’t modify literal data” that Lisp users follow is related to a Lisp-only gotcha that is somewhat similar, but that’s another story)

So, this is why some Python users say you should not use mutable objects as default arguments (unless you know what you are doing).

4. the None trick and the nil trick

So in Python, some people use the None trick: specifying None as default arguments and then calculating the intended defaults when the function is called. Example:

def hello(aa, bb, cc = None, dd = None):
    if cc is None:
        cc = 100
    if dd is None:
        dd = 100 + 100
    print aa, bb, cc, dd

hello(3, 5)       # prints 3 5 100 200
hello(3, 5, 7)    # prints 3 5 7 200
hello(3, 5, 7, 9) # prints 3 5 7 9

hello(3, 5, None) # prints 3 5 100 200
hello(3, 5, None, 9) # 3 5 100 9

That example is a bit silly in that it is using the None trick for numbers which are immutable, nonetheless you get to see how the None trick is used. The statement hello(3, 5, None) printing 3 5 100 200 rather than 3 5 None 200 may or may not be what you want depending on situation.

Similarly in Emacs Lisp, the nil trick:

(defun my-hello (aa bb &optional cc dd)
  (if (eq cc nil)
      (setq cc 100))
  (if (eq dd nil)
      (setq dd (+ 100 100)))
  (list aa bb cc dd))

(my-hello 3 5)      ; ⇒ (3 5 100 200)
(my-hello 3 5 7)    ; ⇒ (3 5 7 200)
(my-hello 3 5 7 9)  ; ⇒ (3 5 7 9)

(my-hello 3 5 nil)   ; ⇒ (3 5 100 200)
(my-hello 3 5 nil 9) ; ⇒ (3 5 100 9)

Why would someone want to use the nil trick when Lisp has no default argument gotcha? Maybe if you don’t want to depend on the cl-lib library. It’s good to be aware of the nil trick anyway because you might need to read code that use the trick. As with Python, the expression (my-hello 3 5 nil) returning (3 5 100 200) rather than (3 5 nil 200) may or may not be what you want.

This expression

(if (eq cc nil)
    (setq cc 100))

can be alternatively written as

(if (null cc)
    (setq cc 100))

and because nil is the only false value in Emacs Lisp (and also in Common Lisp), that in turn can be alternatively written as

(unless cc
  (setq cc 100))

which then in turn can be written as

(setq cc (or cc 100))

and in fact, the last expression is probably what you would encounter most often in uses of the nil trick. (Equivalence of these four expressions apply to Common Lisp as well.)

On the other hand, None is not the only false value in Python, so you might want to think twice before you decide to use

if not cc:
    cc = 100

or this

cc = cc or 100

when you are using the None trick.

5. further reading

6. optional reading

6.1. more on the Python gotcha

Let’s recall the Python code we started with.

def foo(aa = [0,0], bb = [0,0]):
    print aa, bb
    aa[0] = 7
    bb = [7,7]

foo()   # prints [0, 0] [0, 0] 
foo()   # prints [7, 0] [0, 0]

Here’s Lisp code for comparison in case you are curious. (you may see something surprising if you write [0 0] in place of (vector 0 0) in the following code, but as I said earlier, that’ll be another story about literal data in Lisp.)

(cl-defun foo (&optional
               (aa (vector 0 0))
               (bb (vector 0 0)))
  (print (vector aa bb))
  (setf (elt aa 0) 7)
  (setf bb (vector 7 7)))

(foo)   ;; prints [[0 0] [0 0]]
(foo)   ;; prints [[0 0] [0 0]]

Let’s get back to Python. What happened when we called foo two times in Python code? When the Python interpreter was running the definition of foo, it calculated the first expression [0,0] (resulting in a Python list object), and it calculated the second expression [0,0] resulting in another list object. These two list objects are not the same object. In what sense? In the sense that your twin brother is not you. Let’s give these two list objects some nicknames. Let’s call the first list object Alice and the second object Bob.

When we called foo the first time, the name aa got assigned to Alice, and bb to Bob, and then aa and bb were printed, and then the statement aa[0] = 7 changed Alice’s state (because aa is Alice), and then the statement bb = [7,7] reassigned the name bb to another list object. Then the names aa and bb got expired as we exit the function body.

When we called foo the second time, the name aa got assigned to Alice, and bb to Bob as usual, and then aa and bb were printed, but this time Alice was in a different state.

6.2. the gotcha will get you

You might think that as long as you don’t do modify aa for example, you can get away with using mutable defaults in Python. Maybe, but as your code grow, you may end up modifying the object that the name aa refers to, through other names. For example, take a look at the class method example in this article.

6.3. it will get you again

Even with immutable objects, you need to be aware of the gotcha.

import random

def fizz(x = random.random()):
    print x

fizz()
fizz()
fizz()
0.451109142587
0.451109142587
0.451109142587
(require 'cl-lib)

(cl-defun my-fizz (&optional (x (cl-random 1.0)))
  (print x))

(my-fizz)
(my-fizz)
(my-fizz)
0.8515609502792358

0.14910626411437988

0.5142828226089478
This entry was posted in Emacs, Lisp, Python and tagged , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s