A particularly cheap way of breaking cycles are simple boolean flags:

class Foo(object):
   listen = True
   def some_method(self):
       self.listen = False
       try:
           # do stuff that triggers some_event
       finally: 
            self.listen = True
 
    def on_some_event(self):
        if not self.listen:
            return
        # handle event normally

Boolean flags are great. They’re simple, they’ll make your code harder and half of the time, you’ll forget to set them to True after you’re done. Or you forget the try...finally... block. Or you forget to check them in the event handler. What could possibly go wrong.

Fortunately, some GUI toolkits provide ways to temporarily disable event handling for specific events, like GObject’s handler_block_by_func. This approach has two problems:

• You have to know the object (or objects) that emits the event.
• It only works for GObject events (signals).

Since I do have classes with their own event handling mechanism, in order to be independent of GObject and since it’s not really difficult to implement, I wanted a cross-event-framework way of temporarily blocking event handling. Or, maybe I just wanted to write another decorator/context manager very badly. Maybe a little bit of both.

class Foo(object):
    def some_method(self):
        with self.on_some_event.suspend():
            # do stuff that triggers some_event
 
    @suspendable
    def on_some_event(self):
        # handle event normally

The event handlers are wrapped inside another method which swallows the event when the context manager is in suspended mode. The code remains blissfully ignorant of threading issues for now and also breaks down if the event handlers have meaningful result values. I’ve never encountered this so far, but adding default return values to the decorator is a simple extension. The methods are still proper bound methods and docstrings etc. are all conserved.

Without further ado, the code:

class Suspender(object):
    def __init__(self):
        self.is_suspended = 0
 
    @contextmanager
    def __call__(self):
        self.is_suspended += 1
        try:
            yield
        finally:
            self.is_suspended -= 1
 
    @classmethod
    def suspendable(cls, meth):
        suspend_manager = cls()
 
        wrapper = suspend_manager.add_suspendable(meth)
        wrapper.suspend = suspend_manager
        wrapper.add_suspendable = suspend_manager.add_suspendable
        return wrapper
 
 
    def add_suspendable(self, meth):
        @wraps(meth)
        def suspended_wrapper(*args):
            if self.is_suspended == 0:
                meth(*args)
 
        return suspended_wrapper
 
suspendable = Suspender.suspendable

It’s possible to have several event handlers block by a single context manager using the add_suspendable decorator added to suspendable methods:

@suspendable
def on_some_event(self): ...
 
@on_some_event.add_suspendable
def on_misc_event(self): ...

Calling the suspend() context manager for on_some_event will also block on_misc_event.

The context manager does not prevent the event from being propagated, so it’s not a speed optimization; due to the added boolean check event + call indirection, handling actually becomes slightly slower. It likely won’t make a difference. If it does, event cycles are the least of your problems.

Apart from having mastered a small coding challenge, as it turned out to be, this also gives you greater control over the scrolling itself, like making sure that certain elements are visible, viewport panning etc.

Anyway, especially when using PyGTK, it’s a bit unclear on how to proceed. From the documentation, it somehow gets clear that it has to do with the signal set_scroll_adjustment_signal:

This signal is emitted when a widget of this class is added to a scrolling aware parent, gtk_widget_set_scroll_adjustments() handles the emission. Implementation of this signal is optional.

This is not a signal name, but a signal ID¹ that has to be set in GtkWidgetClass².

Some more documentation reading reveals that you can set this signal by using the set_set_scroll_adjustments_signal method on a widget:

class ScrollableWidget(gtk.DrawingArea):
    __gsignals__ = {
        "set-scroll-adjustments": {
            gobject.SIGNAL_RUN_LAST,
            gobject.TYPE_NONE, (gtk.Adjustment, gtk.Adjustment)),
    }
 
    def __init__(self):
        gtk.DrawingArea.__init__(self)
        self.set_set_scroll_adjustments_signal("set-scroll-adjustments")

It doesn’t really matter how you call the signal as long as it takes two arguments (the horizontal and vertical adjustment). This will make the method set_scroll_adjustments (which you can’t override from within Python) return True when it is called and signal that the widget supports scrolling.

This, however, is only half the way, because the scrollable widget still needs the adjustments handed in via said methods. It’s of course possible to connect to the signal explicitly, but there’s an even more direct way by using action signals.

Action signals are the C programmer’s idea of “generic methods”. In order to create such a signal, it has to have the flag gobject.SIGNAL_ACTION and they are directly connected to a function which is then called on each signal emission. While in C, you have to provide a function pointer, in Python you can just implement functions with a compounded magic name³ and have it called automatically. I haven’t found any documentation on that in the PyGObject or PyGTK docs, only some examples in the web:

class ScrollableWidget(gtk.DrawingArea):
    __gsignals__ = {
        "set-scroll-adjustments": {
            gobject.SIGNAL_RUN_LAST | gobject.SIGNAL_ACTION, 
            gobject.TYPE_NONE, (gtk.Adjustment, gtk.Adjustment)),
    }
 
    def __init__(self):
        gtk.DrawingArea.__init__(self)
        self.set_set_scroll_adjustments_signal("set-scroll-adjustments")
 
    def do_set_scroll_adjustments(self, h_adjustment, v_adjustment):
         # do some useful stuff here, like saving them
         ...

The method being called on emission has to start with do_, following by the signal names with hyphens replaced by underscores.

The adjustment objects can then be configured to one’s own liking to have scroll bars show up or not. However, to know when the user did some scrolling, it’s necessary to listen on some signals:

    def do_set_scroll_adjustments(self, h_adjustment, v_adjustment):
        h_adjustment.connect("value-changed", self._scroll_value_changed)
        self._hadj = h_adjustment
        v_adjustment.connect("value-changed", self._scroll_value_changed)
        self._vadj = v_adjustment

To make the scroll bar show up, modify upper, lower and page_size on the adjustments.

self._hadj.lower = 0
self._hadj.upper = 50
self._hadj.page_size = 10

This tells the scrollbar that the size of the underlying picture (upper - lower) is 50, while the visible size (page_size) is 10.

The page size obviously depends on the current size of the widget, which can be retrieved from the underlying gtk.gdk.Window:

width, height = self.window.get_size()

The current position of the scroll bar is controlled by the property value of the adjustment object and should be in the range [lower .. upper - page_size]. Whenever the property is changed, the value-changed signal is emitted, which we’ve connected to previously, and the widget can be repainted.

If you’re curious, you can also see the whole gloriousness in actual working code.

1. which you usually don’t seen when coding with PyGTK
2. ditto
3. a technique which I thoroughly dislike and should be converted to be used with decorators

Surprisingly, all those toolkits were written in Python and made our C++-affine lecturer wonder if he probably should look into this language some more, apart from suggesting that we „maybe […] should get a life.”

So what does my toolkit do?

• Determinization of NFSAs to DFSAs
• Creation of DFAs from simple regular expressions
• Application of FSTs
• dot graph output for *FSAs

All in all, not very impressive, and not very hard. Still, if you like PyParsing and generator-expression-prone Python code, you might want to have look at the TinyFST code.

1
2
3
4
5
6
7

def uncurry(f):
    return lambda t: f(*t)
 
def longest_common_prefix(Sa, Sb):
    return len(list(
        takewhile(uncurry(eq),
                  izip(Sa, Sb))))

1. I bet the tenses are all wrong again.

For those who are wondering what kind of code name Gamla Stan is: STA releases are named after Stockholms subway stations.

Align your trees while the release is still hot!

I simply cannot confirm often enough how true that is.

Nachdem das Release nur ein paar Monate verspätet war, machen wir uns auch gerade daran, die technische Basis mit ein paar gezielten geradezurücken. Die Hauptattraktionen für das nächste Release (Codename „Solna”, geplant für Mitte März) sollen Diff-Views für verschiedene Versionen eines Korpus und eine verbesserte & erweiterte Suche per TSQL (TigerSearch, für einzelne Baumdatenbanken) und AQL (Alignment Query Language, für parallele Baumdatenbanken).

Helfer werden natürlich immer gebraucht – bei entsprechenden Programmierkenntnissen oder Interesse reicht eine Email; Arbeit gibt es genug!.

• unoptimized: 61.01s
• optimized: 39.68s

That still seemed a bit too slow to me, and I decided to try out some parallelization: since the ID reference remapping and graph store code in the parser is decoupled from parsing and the creation of the node data structures, I tried to put that into a separate thread. Both threads are mostly CPU-bound, so I quickly ran into the good old GIL problem. In short: in a Python process, only one thread my execute Python byte code at a time. This is not a problem if you are doing a lot of I/O or lurking around in message loops, but it’s deadly for CPU-bound tasks. It made the parser considerably slower, even when I used the shelve module instead of storing the graphs to memory (which involves a lot more disk I/O).

In times where multi-core CPUs are becoming a commodity, more fine-grained locking would be desirable in CPython, or, even better, more advanced types of parallelization support.

There are other approaches to parallel computing in Python readily available (based on forks, some with shared memory, some with pipes [↪]), but they seem too heavyweight to be dragged in for this small problem: a non-trivial dependency just to shave off some seconds from the parsing process is not a good tradeoff.

Nach einem Jahr verschärftem Toolmonkeyism auf der Arbeit bin ich zu der Überzeugung gelangt, dass auch beim Python-Programmieren ein wenig IDE-Unterstützung nicht schlecht wäre. Eclipse (besonders das erstaunlich stabile 3.3) wächst einem dann doch ans Herz. Nicht dass ich für Python unbedingt Ctrl-1 bräuchte (z.b. den infernalischen Java-Typinferenz-Hack, Variablen ohne Typ zu deklarieren und sich per Tastendruck den Typ hinschreiben zu lassen – geht leider nicht in new-style For-Schleifen!), aber Refactoring-Support wäre schon was feines.

Refactoring in Python

Natürlich ist Refactoring in Python beliebig schwierig und der Bicycle Repair Main verhaut den Code teilweise schon ziemlich. Aber „Extract Method”, „Inline Variable”, „Extract Local Variable” und ähnlich einfache Sachen sollten durchaus zu machen sein, solange man keinen Unfug mit locals(), globals(), __dict__ oder (get|set|has)attr(...) macht. Und wenn man auf dieser Ebene programmiert, sollte man wissen, dass man sich mit automatischen Refactorings vielleicht nicht unbedingt nur Freude ins Haus holt.

Python-IDEs

In Zuge des TreeAligner-Workshops habe ich tatsächlich wieder mal ernstzunehmend Python programmiert. Um die ganze Sache etwas lehrreicher zu gestalten (PyParsing ist zwar nett, aber nicht mehr neu!) habe ich mich entschlossen, den Emacs zu verlassen und einige Python-IDEs auszuprobieren. Da ich eric schon zur Genüge kenne und auch nicht wirklich benutzbar finde, habe ich micht auf Pydev 1.3.8 und WingIDE 3.0.0b1 beschränkt.

Das schlechte zuerst: Pydev

Pydev ist wirklich erschreckend schlecht. Zwar hat man die üblichen Eclipse-Vorteile (run configurations etc.), aber dafür hat der Python-Editor keine Indentation-Engine: pro TAB-Taste wird einfach ein Level tiefer eingerückt, automatische Indentierung scheint nicht zu gehen. Das ist besonders nervig wenn man sich der datenzentrierten Programmierung verschrieben hat und andauernd große Dictionaries oder Listen als Literale im Code erstellt. Allein dieses Manko lässt Pydev schon aus dem Wettbewerb fallen.

Die automatische Code-Vervollständigung erschließt sich mir nicht wirklich, zumindest wird wohl die aktuelle Datei gescannt (das kann Emacs auch…). Die wenigen Refactorings, die Pydev unterstützt, scheinen aber zu funktionieren, wenigstens ein kleines Plus; insgesamt aber nicht überzeugend.

Etwas besser: WingIDE 3.0

WingIDE hat eine überraschend gute Code-Vervollständigung (alle Imports werden gescannt), auch Funktionsaufrufe auf Objekten werden vervollständigt, soweit WingIDE den Typ inferieren kann, ohne das Programm auszuführen. Der Editor ist gelungen und die Emacs-Emulation überzeugend. Die Indentation-Engine ist sehr gut und produziert bei tief verschachtelten Datenstrukturen subjektiv schönere Ergebnisse als der Emacs.

Der Debugger hat alles, was man erwartet, obwohl er manchmal etwas schneller reagieren könnte. Auch nervt, dass das Starten eines Debug-Laufes und das Weiterausführen nach einem Breakpoint ein Befehl sind. Alle anderen Zusammenstöße, die ich mit dem Debugger hatte, rechne ich meinem extravaganten Programmierstil zu.

WingIDE hat keinerlei Unterstützung für Refactorings, es sei denn, man versteht under „Rename Variable” eine Search/Replace-Operation. Für ein neues major release einer professionellen kommerziellen IDE im Jahr 2007 ein sehr schwaches Zeichen. Außerdem nerven einige kleine Bugs, die aber bei einer Beta-Version entschuldbar sind – von der Reaktion auf mein Feedback werde ich wohl eine Kaufentscheidung abhängig machen.

Einige Sache würde ich mich nur wünschen (z.b. ein ähnliches Feature wie Quick Outline in Eclipse). Darüber hinaus habe ich mir die Skriptbarkeit der IDE noch nicht angeschaut, und wie stark man sie tatsächlich erweitern kann. Unterstützung für py.test wäre schon etwas, das man implementieren könnte.