I See Dead Code

Metro 2033 is a game that gets its atmosphere right. While the narrative is entirely forgettable and somewhat trope-infested, the design and the mechanics managed to make the player feel ill at ease all the time. With almost no HUD, a feeble headlamp and weapons that need to be manually pumped/recharged or offer the choice between ‘somewhat more useless ammo’ or ‘expensive but little more powerful ammo’, I never felt like I was on top of things.

Among the most disorienting experiences are the sequences when you forced to use a gas mask, unless you want your character to slowly suffocate within 45s. Use of the gas mask drowns out environmental sounds, which are usually a cue for enemies, especially non-human ones. Instead, you are forced to listen to the sound of your own breath, getting rustier the more the filter in your mask runs down. And while there are some filters scattered throughout most levels, there is never an abundance of it.

Simply knowing that filters and therefore time are extremely limited already makes these sequences quite stressful. The gas mask impairs the field of view quite a lot, but the worse your filters get, the more water condenses on the inside of your mask. You could of course change the filters early and improve your view, but then you’d shorten the time you have left.

If you happen to be on the outside, you are probably completely lost by now. Surrounded by pits with radioactive water and the odd flying demon casting its shadow over you, how do you find back on track? Unlike Fallout 3, whose PIP-Boy 3000 interface is a pretty standard (if awkward) RPG interface disguised as an in-world artifact, in Metro 2033, in-world artifacts are your interface. Want to know your objectives and the direction? Put your weapon away and take a lot at your notepad—and do not forget to use your lighter if it is dark. There is no map, there is just a compass. Lost in the dark? Use your lighter to look at the notepad.

In some of the more open levels, this all comes together and adds up to a quite impressive experience, especially in the open-air chapters Dead City 1 & 2, Outpost, Alley and the multi-route underground level Frontline. But under all their clever disguise and disorienting design, the levels are entirely linear, and there is very little interaction with the world per se. Plucking guitar strings here and rescuing some soldiers there, finding a hidden stash or looking out some window collect points towards the ‘good’ ending, but nothing you do has an influence on the game world—and even if it did, you could not go back to check, because there is no way back.

The point were this realization is most terrible is when you enter Polis, the destination that you originally set out to reach and that shall hold salvation for you and your tribe. You are driven through a large underground Metro station and all those people are walking around, so many people to talk to, so many things to discover, but you cannot move. All you can do is move your head during those precious seconds where the game seems to promise to open up, only for a fence to close behind you. Then there’s a little room, a short scripted sequence with only the slightest hint of why things would not go the way you want them to, and then off you go, the next dark corridor, the next poisonous stretch of destroyed city awaits.

Addendum

When I wrote the original post, I remembered Alley, the walk down a derelict part of the city towards the library as one of the more tense parts of the game. Since Alley itself is reasonably short, I selected it as a benchmark level, without giving much thought to the enemies. As I found out after countless runs with different hardware and different settings, the whole level is just smoke and mirrors, exquisitely crafted to freak you out. As it turns out, just running through the level does not get you killed((probably not true at hardcore difficulties)). The Nosalises turn up at always the same positions, attack a couple of times and disappear again, not even bothering to attack the other characters in the level. This makes it possible to finish the whole level in less than 60s, as opposed to the multiple attempts that I needed when I first played the level.

After a bit more than 2½ years, I upgraded the my desktop system.

Back in 2009, I built my machine around the MSI P55-GD65, a Core i7 860, 4 GiB (later 8 GiB) Kingston HyperX T1 DDR3-1600 (clocked at 1333 MHz) and a Sapphire HD 5870. All in all, I could not have been happier with the system all the time I was using it. It was rock-solid under Windows 7 64 bit. I’ve never had any BSODs (although I needed to run off of beta graphics drivers in the beginning) and thanks to my case, the whole setup was reasonably cool, at the expense of loudness.

While I did have a Linux running on it for some, I never really cared and did not end up using it for anything productive; the machine was (and is) my dedicated gaming system. For a blissful time in 2009/2010, I was able to run most games at full settings (Batman: Arkham Asylum, Metro 2033 and the like). Even after more than two years, the Windows installation has not deteriorated into uselessness, although I diligently upgraded drivers as often as AMD brought out new ones (much to my delight when seeing improved Unigine Heaven scores).

Earlier this year, it became increasingly apparent that especially the graphics card was not appropriate for new games any more. Especially Batman: Arkham City and Skyrim brought the system to its knees. I had planned to do a rebuild after all, having jumped one generation (Sandy Bridge and the Radeon HD 6000 series). After my good impressions with 5870, which still is a phenomenal card with very low power consumption, I decided to get a HD 7950. While not the dedicated single GPU flagship (unlike the 5870), it impressed me with its good performance/power consumption ratio.

In February, not late after the 7000 series was launched, I bought the ASUS HD 7950 DirectCU II TOP. As the somewhat bulky name already suggests, the card is outrageous in its space demands, since it takes 3 slots and is almost 30 cm long—2 cm more and the card would not fit anymore.

Just this upgrade alone already boosted the performance of the whole system, almost doubling performance in synthetic benchmarks (Unigine Heaven, Furmark), and brought impressive improvements in real-world loads. The card also supports new AA modes and, due to the design, stays very cool. I have yet to manage to bring the GPU core temperate above 70℃.

When planning the upgrade, I had given up on exchanging everything at the same time relatively early. Sandy Brigde was already on the way out, and the new Ivy Bridge boards (with USB 3.0) where not for a couple more months. It was also not entirely clear whether all Sandy Bridge boards would be able to host Ivy Bridge CPUs, so I decided to bide my time and wait for the general availability.

Last Monday, I eventually settled on the new parts:

• Gigabyte GA-Z77X-UD5H
• Core i7 3770K
• 16 GiB Corsair Vengeance DDR3-1600

Originally, a RAM upgrade was not part of the plan, because the Kingston memory would still have run at the highest possible (non-OC) clock. However, I decided that 16 GiB are cooler than 8 GiB of memory, especially given the fact that I’d like to run a Linux VM on the machine. The RAM decision was the most difficult one, between compatibility lists from Gigabyte and the different RAM makers; and while I originally had planned to stay with Kingston, I eventually switched. All in all, I paid pretty much the same for the amount of money for mainboard + CPU + RAM as I had in 2009.

The new hardware arrived yesterday and I started the upgrade. Searching for how to prepare a Windows system for a mainboard exchange had come up with mixed recommendations, alternating between Sysprep, removing all drivers and booting up in safe mode or doing nothing except for making sure the latest drivers are installed. Doing nothing sounded like it would require little effort, so I downloaded the lastest drivers from the Gigabyte page and proceeded.

The physical part of the upgrade was quick and painless. The documentation for the board was good, taking out the old motherboard, putting in the new one and plugging all cables back in cannot have taken more then 60 min. I was pleasantly surprised by several factors:

• The mainboard has two BIOS chips, making it harder to screw up with a bad flash
• while the MSI board had three on-board chassis fan connectors, the Gigabyte has 4, so I was able to connect all chassis fans to the system board instead of using the power supply.

The first poweron was already a success, and the BIOS menu is well-done. The 3D mode (click on certain parts of a rendering of the mainboard to change the related settings) is a nice touch, but ultimately just a gimmick.

The first Windows boot succeeded, but then came to a quick end when I realized that none of the input devices worked. My suspicion that I should have used the USB 1.1/2.0 ports instead of the USB 3.0 ports was correct (and even recommended in the Gigabyte FAQ for Windows 7). Audio worked out of the box, but the ethernet interfaces, the USB 3.0 and the chipset needed new drivers, since they are not supported by Windows 7 natively.

I also updated the BIOS using the flash utility embedded in the BIOS, which reads a BIOS image from USB, which was a refreshing change from the MSI way. The new BIOS added a couple of fixes, but nothing spectacular.

When all of the hardware was working, I ran a couple of benchmarks and stability tests:

• FurMark
• IntelBurnTest
• SpikeFun
• Unigine Heaven
• Blender 2.61 Cycles Benchmark

None of the benchmarks destabilized the system or brought the CPU temperate into uncomfortable ranges, although IntelBurnTest (which is a wrapper around LinPack) produces impressive temperate graphs.

Unfortunately, I didn’t run all of the benchmarks on the old system before upgrading, but the Blender test differences are informative (using the official Blender 2.63a 64bit build from blender.org):

Core i7 860 (Lynnfield)

5:51 min

Core i7 3770K (Ivy Bridge)

4:17 min

Not a new definition of fast, but good enough. Weirdly enough, the performance of the default 64bit Blender builds is very low, especially when compared to the performance on other platforms¹. When I switched to an optimized build from graphicall.org, the time dropped to 2:37 min with the 3770K. Unfortunately, I have no comparison figures for the 860.

The scores in Unigine Heaven did not budge the slightest bit (~880 at 1920×1200, 8xEQAA, normal tesselation, 16xAF), more or less as expected.

I did two full Memtest86+ runs, one with the RAM clocked at 1333 MHz, and one with 1600 MHz. The differences between the two mem clocks are a bit larger than I expected, the Blender benchmark takes 10s less (2:27 min instead of 2:37 min). At 1333 MHz, SpikeBench reports an average memory bandwith of 17.03 GiB/s for the large_max_bandwidth 64bit, vs. 20.45 GiB/s at 1600 MHz².

Overall, I am pleased with the upgrade, especially the features of the mainboard are a step up from the MSI board, which was already marketed towards enthusiasts.

I am still undecided whether an HDD upgrade, or even a switch to SSDs makes sense. The 1 TB harddrive is far from full, mostly because I radically deleted most games I wasn’t playing at some point last year. Prices for SSDs with more than 256G of space are still prohibitive, so I might opt for an SSD/HDD combo to improve my Windows startup experience later this year.

EDIT: Updated with benchmark numbers for 1600 MHz memory clock, minor fixes and Blender footnote.

1. According to this bug, the official Blender builds are done with the Microsoft compilers, which are worse than GCC at optimizing the cycles rendering code [↩]
2. And I try not to think about how fast it might be if I went and bought even faster RAM [↩]

Q: “How many constituent trees fit on a single A0 poster in style?”
A: “886”
[Read more →]

A recurring problem in (not only) GUI programming are event cycles, i.e. the receiving of events oneself has triggered. These can quickly lead to event cycles, where change triggers change triggers change, until something gives out—usually the stack.

A particularly cheap way of breaking cycles are simple boolean flags:

class Foo(object):
   listen = True
   def some_method(self):
       self.listen = False
       try:
           # do stuff that triggers some_event
       finally: 
            self.listen = True
 
    def on_some_event(self):
        if not self.listen:
            return
        # handle event normally

Boolean flags are great. They’re simple, they’ll make your code harder and half of the time, you’ll forget to set them to True after you’re done. Or you forget the try...finally... block. Or you forget to check them in the event handler. What could possibly go wrong.

Fortunately, some GUI toolkits provide ways to temporarily disable event handling for specific events, like GObject’s handler_block_by_func. This approach has two problems:

• You have to know the object (or objects) that emits the event.
• It only works for GObject events (signals).

Since I do have classes with their own event handling mechanism, in order to be independent of GObject and since it’s not really difficult to implement, I wanted a cross-event-framework way of temporarily blocking event handling. Or, maybe I just wanted to write another decorator/context manager very badly. Maybe a little bit of both.

class Foo(object):
    def some_method(self):
        with self.on_some_event.suspend():
            # do stuff that triggers some_event
 
    @suspendable
    def on_some_event(self):
        # handle event normally

The event handlers are wrapped inside another method which swallows the event when the context manager is in suspended mode. The code remains blissfully ignorant of threading issues for now and also breaks down if the event handlers have meaningful result values. I’ve never encountered this so far, but adding default return values to the decorator is a simple extension. The methods are still proper bound methods and docstrings etc. are all conserved.

Without further ado, the code:

class Suspender(object):
    def __init__(self):
        self.is_suspended = 0
 
    @contextmanager
    def __call__(self):
        self.is_suspended += 1
        try:
            yield
        finally:
            self.is_suspended -= 1
 
    @classmethod
    def suspendable(cls, meth):
        suspend_manager = cls()
 
        wrapper = suspend_manager.add_suspendable(meth)
        wrapper.suspend = suspend_manager
        wrapper.add_suspendable = suspend_manager.add_suspendable
        return wrapper
 
 
    def add_suspendable(self, meth):
        @wraps(meth)
        def suspended_wrapper(*args):
            if self.is_suspended == 0:
                meth(*args)
 
        return suspended_wrapper
 
suspendable = Suspender.suspendable

It’s possible to have several event handlers block by a single context manager using the add_suspendable decorator added to suspendable methods:

@suspendable
def on_some_event(self): ...
 
@on_some_event.add_suspendable
def on_misc_event(self): ...

Calling the suspend() context manager for on_some_event will also block on_misc_event.

The context manager does not prevent the event from being propagated, so it’s not a speed optimization; due to the added boolean check event + call indirection, handling actually becomes slightly slower. It likely won’t make a difference. If it does, event cycles are the least of your problems.

Over the last months, I gave two talks, one at TaCoS in Heidelberg on Text+Berg digital, the other one in our local Python user group, swiss.py, on Python metaclasses.

The slides can be found on the Talks page.

Beginnings

After the bleak, joyless work of releasing software, plugging holes of preventing users from clicking buttons they should not have clicked in the first place, writing unctuous documentation and release notes and wrapping code into neat little installers with tiny bows and bells that gently tingle when touched, there are few things as profoundly satisfying as taking code that looked like line noise in the first place, applying some non-trivial transformation to it and still have it look like line noise, only now it’s twice as fast, or half as long or of some other inaccessible quality that can, on some technical level, be referred to as “cool”.

Quite some time ago, I decided that this time, “cool” will mean “runs on graphics hardware, possibly faster”. For this end, I took my trusty T61 to work, downloaded the latest CUDA toolkit, found out that nearly all examples crashed when compiled, installed the latest beta drivers from NVIDIA, noticed that everything worked now and started playing around with the examples.

After some time marveling at ball pit simulations and similarly telling examples, I printed out the CUDA programming guide and spent a considerable amount of time marveling at how well the NVidia green looked when printed with our institute’s color laser printer, before reading its more important parts.
[Read more →]

My Master thesis “Integration of Light-weight Semantics into a Syntax Query Formalism” is available from the SALSA project pages.

Abstract

In the Computational Linguistics community, much work is put into the creation of large, high-quality linguistic resources, often with complex annotation. In order to make these resources accessible to nontechnical audiences, formalisms for searching and filtering are needed.

The TIGER query language can, by describing partial structures, be used to search treebanks with syntactic annotation. Recently, augmented treebanks have been published, including the SALSA corpus which features frame semantic annotation on top of syntactic structure. Query languages, however, need to keep up with newly introduced annotation, allowing it to be searchable and easy to access.

We design an extension for the TIGER language which allows searching for frame structures along with syntactic annotation. To achieve this, the TIGER object model is expanded to include frame semantics, while remaining fully backwards-compatible.

Finally, these extensions have been added to our own implementation of TIGER, which includes novel indexing features not found in the original work of Lezius (2002a).

[Read more →]

It is possible to write custom GTK widgets that have “native” scrolling support, as opposed to just shoving them into a GtkViewPort and forgetting about them.

Apart from having mastered a small coding challenge, as it turned out to be, this also gives you greater control over the scrolling itself, like making sure that certain elements are visible, viewport panning etc.

Anyway, especially when using PyGTK, it’s a bit unclear on how to proceed. From the documentation, it somehow gets clear that it has to do with the signal set_scroll_adjustment_signal:

This signal is emitted when a widget of this class is added to a scrolling aware parent, gtk_widget_set_scroll_adjustments() handles the emission. Implementation of this signal is optional.

This is not a signal name, but a signal ID¹ that has to be set in GtkWidgetClass².

Some more documentation reading reveals that you can set this signal by using the set_set_scroll_adjustments_signal method on a widget:

class ScrollableWidget(gtk.DrawingArea):
    __gsignals__ = {
        "set-scroll-adjustments": {
            gobject.SIGNAL_RUN_LAST,
            gobject.TYPE_NONE, (gtk.Adjustment, gtk.Adjustment)),
    }
 
    def __init__(self):
        gtk.DrawingArea.__init__(self)
        self.set_set_scroll_adjustments_signal("set-scroll-adjustments")

It doesn’t really matter how you call the signal as long as it takes two arguments (the horizontal and vertical adjustment). This will make the method set_scroll_adjustments (which you can’t override from within Python) return True when it is called and signal that the widget supports scrolling.

This, however, is only half the way, because the scrollable widget still needs the adjustments handed in via said methods. It’s of course possible to connect to the signal explicitly, but there’s an even more direct way by using action signals.

Action signals are the C programmer’s idea of “generic methods”. In order to create such a signal, it has to have the flag gobject.SIGNAL_ACTION and they are directly connected to a function which is then called on each signal emission. While in C, you have to provide a function pointer, in Python you can just implement functions with a compounded magic name³ and have it called automatically. I haven’t found any documentation on that in the PyGObject or PyGTK docs, only some examples in the web:

class ScrollableWidget(gtk.DrawingArea):
    __gsignals__ = {
        "set-scroll-adjustments": {
            gobject.SIGNAL_RUN_LAST | gobject.SIGNAL_ACTION, 
            gobject.TYPE_NONE, (gtk.Adjustment, gtk.Adjustment)),
    }
 
    def __init__(self):
        gtk.DrawingArea.__init__(self)
        self.set_set_scroll_adjustments_signal("set-scroll-adjustments")
 
    def do_set_scroll_adjustments(self, h_adjustment, v_adjustment):
         # do some useful stuff here, like saving them
         ...

The method being called on emission has to start with do_, following by the signal names with hyphens replaced by underscores.

The adjustment objects can then be configured to one’s own liking to have scroll bars show up or not. However, to know when the user did some scrolling, it’s necessary to listen on some signals:

    def do_set_scroll_adjustments(self, h_adjustment, v_adjustment):
        h_adjustment.connect("value-changed", self._scroll_value_changed)
        self._hadj = h_adjustment
        v_adjustment.connect("value-changed", self._scroll_value_changed)
        self._vadj = v_adjustment

To make the scroll bar show up, modify upper, lower and page_size on the adjustments.

self._hadj.lower = 0
self._hadj.upper = 50
self._hadj.page_size = 10

This tells the scrollbar that the size of the underlying picture (upper - lower) is 50, while the visible size (page_size) is 10.

The page size obviously depends on the current size of the widget, which can be retrieved from the underlying gtk.gdk.Window:

width, height = self.window.get_size()

The current position of the scroll bar is controlled by the property value of the adjustment object and should be in the range [lower .. upper - page_size]. Whenever the property is changed, the value-changed signal is emitted, which we’ve connected to previously, and the widget can be repainted.

If you’re curious, you can also see the whole gloriousness in actual working code.

1. which you usually don’t seen when coding with PyGTK [↩]
2. ditto [↩]
3. a technique which I thoroughly dislike and should be converted to be used with decorators [↩]

Even before I started working at UZH, I got my new hardware. Back in February, when I visited Zürich to look for rooms, I was offered to choose a new notebook for myself. Since I already have a large notebook (at least that’s what I consider my 14.1″ T61) and toyed around with the idea of getting a desktop again (after 4 years of exclusive notebook use!), I chose an X301. I included some hardware upgrades that weren’t included in the basic offer:

• +2 GB RAM (4 GB overall)
• 3G card
• USB Port Replicator
• DisplayPort->DVI converter

Being in Switzerland, I had the choice between CH and US keyboard layout. Since CH is physically the same as the German layout (105 keys), I took it and haven’t really noticed that the keys have different symbols (as far as most of the special characters are concered) printed on that what appears on the screen when I hit them. I should really get Das Keyboard after all.

The X301 is not quite the workhorse the T61 is, which is especially noticeable from graphics speed, although that might be due to an Intel driver being in several different transitions right now. The SSD compensates for that, booting and starting up is a breeze. Fortunately, I don’t have to do much booting these days, because unlike other major graphics hardware creators, Intel’s developers are able to support powersave modes on Linux hardware, both Suspend-to-RAM and Suspend-to-disk.

Everything else works more or less, even the DisplayPort, dutyfully serving my 24″ screen at the university. The pièce de résistance is definitely the 3G card. I had to install it myself, which was quite a bit more involved than what I would have liked, but it worked on the second try. I got myself a prepaid data contract which allows me to surf the net for 3 CHF/h, which is a good complement to free wireless access at most public hotspots in Switzerland, which is due to SWITCH. The wireless became especially hand today, when I was shopping for a new router without having a clue about what’s good. Standing in the Media Markt (yes, this very pillar of the German retail market also metastasised into Switzerland) network hardware aisles browsing the web with a notebook probably didn’t look as superior as doing the same thing with a smartphone, but it helped me finding a router.

What really sets the X301 apart is the weight. I’m quite used to it by now, since I’ve had it for over a month, but it’s usually the first response I get when I hand it over to somebody else. It is, however, very noticeable in direct comparison, and the T61 feels clunky and hugely oversized when I carry it around or use it on my lap. The battery lifetime is okay, usually I get 3h under Linux. There’s probably some more running time to be gotten, but I’m more or less done tweaking the system for now and using it for actual work; or for blogging, emailing and chatting on my nice, new, comfy chair.