On Python

Culture and Care

Open any Python interpreter and type: import this. No other language I know has woven its culture into the language itself so cleverly or so openly.

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

This text is worth savoring carefully. You must be wondering: what exactly is this? If you know anything about Python’s package mechanism, you should know to hunt for the source file this.py under Python’s installation path. Open it, and you’ll find a familiar-looking but garbled block of text, followed by some code:

d = {}
for c in (65, 97):
    for i in range(26):
        d[chr(i+c)] = chr((i+13) % 26 + c)

print "".join([d.get(c, c) for c in s])

Seeing this, you can’t help but smile knowingly. This is the silliness and the charm of programmers — a kind of humanistic care concealed beneath geek flair (is the author paying homage to the Caesar cipher?).

I love Python not only for how code is written, but also for the pleasure of reading Python code. The truth is, we spend most of our working hours reading other people’s code. Reading Python code is always more comfortable than reading most other languages. Beyond unified formatting (I’m glad to see Go making a similar effort with gofmt), Python’s culture guarantees a certain baseline of readability.

Python’s humanistic care also shows up in the ubiquitous dir and help. In a Python shell (I recommend installing ipython), these two weapons plus a bit of necessary practice are enough for you to quickly master a new library. What could be friendlier to a beginner than that?

Use Cases

Python can be applied in many settings:

• Small tools and scripts
• Text processing
• Image processing
• Crawlers
• Servers
• Websites
• Numerical computation
• Prototyping
• …

In essentially every one of these domains there’s an excellent Python library for you to put to work.

Language Capabilities

Python’s language capabilities are solid and conventional, slightly less expressive than Ruby’s. But it still covers everything from object orientation and functional programming (limited support) to metaprogramming (limited support), spanning the mainstream ideas and methods. Since yesterday’s article already offered several Python examples demonstrating this, I won’t repeat them here. As a successful language, Python does not rest on its laurels — it keeps evolving in useful ways. Consider the with keyword added in 2.5, which simplifies the try..finally structure and makes code cleaner and more beautiful (this has always been the direction of Python’s efforts):

with open('hello.txt') as f:
    f.write('Hello world!\n')

This is equivalent to the older way of writing it:

f = open('hello.txt')
try:
    f.write('Hello world!\n')
finally:
    f.close()

More importantly, just implement a few magic functions and this language capability becomes yours to wield:

class MyOpen:
    def __init__(self, filename, mode = 'r'):
        self.filename = filename
        self.mode = mode
    def __enter__(self):
        self.f = open(self.filename, self.mode)
        return self.f
    def __exit__(self, **unused):
        self.f.close()

with MyOpen('hello.txt') as f:
    f.write('hello world!\n')

Interesting Libraries and Tools

Competition among modern programming languages is competition over language capabilities, but it is also competition over libraries and tools. For Python, the standard library gives you all sorts of fundamental capabilities, and the abundant third-party libraries in the community push those capabilities to new heights. Take the famous “kennethreitz/requests” library on GitHub: it makes writing an HTTP client as simple and intuitive as writing prose:

In [5]: import requests

In [6]: r = requests.get('https://api.github.com', auth=('user', 'pass'))

In [7]: r.status_code
Out[7]: 200

In [9]: r.json()
Out[9]:
{...}

In [10]: r.text
Out[10]: u'{"..."}'

With libraries like this plus the GitHub API, you can build a full-text code search tool in ten or twenty lines of code. That’s the productivity of the web era.

Take scrapy (a crawler framework) as another example: you just define the rules for crawling and how the harvested data should be stored, and it will concurrently crawl and format the data for you. Best of all — and most principled — it offers a wonderful shell in which you can interactively iterate and trial-and-error until the crawling rules are correctly defined.

Concurrency Support

Python gained generators in 2.3. Generators let you suspend the current point of execution, turning synchronous code into asynchronous code and giving a sequentially executed program the ability to execute concurrently. Take, for instance, a Fibonacci sequence:

def fabonacci():
    a, b = 1, 2
    while True:
        a, b = b, a+b
        yield b

Through generators, code suddenly gains lazy evaluation: data is computed only when you need it. Perhaps you can’t see any trace of concurrency here — but imagine a world full of generators, each with its own execution stack. If you wrote a scheduler that swapped generators in and out, wouldn’t that be a coroutine? Of course, Python has its own coroutine library, gevent, built on the highly efficient libev. For instance, using gevent to implement the actor model (the cornerstone of Erlang):

import gevent
from gevent.queue import Queue

class Actor(gevent.Greenlet):

    def __init__(self):
        self.inbox = Queue()
        Greenlet.__init__(self)

    def receive(self, message):
        raise NotImplemented()

    def _run(self):
        self.running = True

        while self.running:
            message = self.inbox.get()
            self.receive(message)

Simple and clear.

Python’s Flaws

All right — every language has its dark side. Python (CPython) has the notorious GIL (not unique to Python, of course; Ruby has MRI too): the Global Interpreter Lock. Any code execution must first acquire this global lock, which is released when there’s an I/O operation. With this global lock in place, Python’s multithreading is effectively an illusion: no matter how many threads you have, only one core can be used. You can run a multithreading experiment:

import threading

def dead_loop():
    while True:
        pass

# new dead loop thread
t = threading.Thread(target=dead_loop)
t.start()

# dead loop on main thread
dead_loop()

t.join()

And a multiprocessing test:

import multiprocessing

def dead_loop():
    while True:
        pass

# new dead loop process
p = multiprocessing.Process(target=dead_loop)
p.start()

# dead loop on main process
dead_loop()

p.join()

Look at the difference in CPU utilization between the two. Multithreading really doesn’t pull its weight!

Of course, there’s no need to be too pessimistic. Multithreading can’t exploit multiple cores, but multiprocessing can — and although multiprocessing carries more overhead, it can still make up for some of the shortcomings the GIL imposes.