Wishful Coding

It’s distro hopping time again. This time I decided to Pick Fedora with Xfce as the desktop environment and i3 as the window manager.

I previously used Crunchbang and Arch before that. On Arch I built my own setup from scratch, but kept running into missing features and breakage. On Crunchbang I used the default setup, but being a Debian derivative, it has very old software.

What I hope to find in Fedora’s Xfce spin is a light distro that is up-to-date and provides a nice out-of-the-box experience. I only want to replace xfwm4 with i3.

Most of the stuff in this post is based on this guide adapted for Fedora and my personal preferences.

The first step is easy:

yum install i3

Next, you need to stop the Xfce window manager and start i3 instead.

In the Xfce menu, go to “Session and Startup” and then to “Application Autostart”. Add an entry for i3 there.

Stopping xfwm4 is done in the “Session” tab. Simply set its “Restart Style” to “Never”.

You might also want to stop xfdesktop and xfce4-panel. xfdesktop provides a window with icons, which is awkward when it gets tiled by i3. xfce4-panel provides the menu bar, which I kept around until I was sure I could do everything I needed using i3.

Then I made some modifications to the i3 config file.

# use the Xfce terminal instead of urxvt
bindsym $mod+Return exec xfce4-terminal
# mimic Crunchbang, I remapped tabbed layout to $mod+t
bindsym $mod+w exec firefox
# make sure xfce notifications display properly
for_window [class="Xfce4-notifyd"] floating enable;border none; focus mode_toggle
# uncomment the i3 dmenu wrapper
# this gives you easy access to Xfce settings
# and hides obscure utilities
bindsym $mod+d exec --no-startup-id i3-dmenu-desktop
# remap the exit command to properly exit Xfce
bindsym $mod+Shift+e exec xfce4-session-logout
# use conky for displaying some stats
bar {
    status_command conky
}

My .conkyrc is very simple

out_to_console yes
out_to_x no
TEXT
RAM ${membar} | \
CPU ${cpubar} | \
BAT ${battery_bar BAT1} | \
$acpitemp C | \
${time %a %d %b %R}

The only thing that remains is installing patented software for proper font rendering and playing multimedia. Fedora is made by RedHat which is based in the US. It turns out you can’t have subpixel smoothing or play MP3 files because of that.

If you’re wondering why this is not a problem for Arch or Ubuntu, it is because Canonical is based on the Isle of Man and Arch isn’t based anywhere at all.

The solution to all of this is to take a look at RPM Fusion. A third-party repo which has all the stuff you need.

Alternatively there is also a repo for the Freetype infinality patches, but so far I have not managed to make my fonts look the way I like.

Just before I went to EuroClojure I read about Gershwin, a concatenative programming language based on Clojure.

With a lot of time to think and code on the train between Amsterdam and Berlin and back, I thought: Why make this a language at all?

Using the threading macro you can make a basic stack program as follows.

(def add [stack]
  (conj (drop 2 stack)
        (apply + (take 2))))

(-> () ; the stack
  (conj 1)
  (conj 2)
  add)
;; => (3)

After that, things got interesting. What if I used a queue instead of a stack? What if I used any number of queues and stacks?

I switched to using thrush, wrote a function to generate words from Clojure functions, and did some magic to apply words to multiple stacks.

(defn nth-stack [n f]
  #(apply update-in % [n] f %&))

(thrush [() ()]  ; data stack and return stack
  (lit 2) dup c+ ; add numbers
  (lit 4) c=     ; verify result
  >r)            ; put bool on return stack
;; => [() (true)]

What if I used clojure.core.reducers/fold instead of reduce in thrush? Could I use dataflow programming and continuations to parallelize programs?

I failed at parallelization, but I surely achieved dataflow programming.

The idea is that you have a contiunation-passing-style function for popping of the stack. That function either calls the continuation with the head and tail of the stack, or returns a negative stack which will call the continuation when an item is conj‘d.

(deftype NegStack [popfn stack]
  clojure.lang.ISeq
  ...
  (cons [_ o] (popfn o stack))
  ...)

(defn mpeek [stack cont]
  (if (seq stack)
    (cont (first stack) (next stack))
    (NegStack. cont stack)))

(defn add [stack]
  (mpeek stack (fn [car cdr]
    (mpeek cdr (fn [cdar cddr]
      (conj cddr (+ car cdar)))))))

(thrush () (lit 2) add (lit 2))
;; => (4)

Yes, you can have prefix, infix, postfix and otherfix all you want. But not on a fork-join pool.

I had it all figured out, including concatenating negative stacks. It’s just not conceptually sound. Imagine this program

(thrush () (lit 2) (lit 2) mul (lit 5) (lit 6) add add)
;; => (15)

Using my fork-join-based pthrush thish might be split up as

(mconcat
  (thrush () (lit 2) (lit 2))
  (thrush () mul (lit 5) (lit 6) add add))

The first block produces (2 2), the execution of the second block is more tricky.

1. mul produces a negative stack
2. 5 and 6 are conj‘d onto this stack, resulting in (30)
3. The add’s consume 30 and produce a negative stack
4. This negative stack is concatenated to (2 2), resulting in (34)

The problem is that mul got applied to (5 6) instead of (2 2) because of how the program was split.

It might still be possible to paralellize by analyzing the stack effect of words, but dataflow is definitely not the way to go.

All my experiments so far can be found on Github at pepijndevos/staack

I’ve been working with a number of HTTP API’s using Python recently, and have accumulated some helper functions that I hope you don’t need.

PHP

It might be possible to write a good REST API in PHP, but it’s certainly not easy… or common.

Array parameters

Lets say your brilliant API design requires an object to be passed as a parameter. How do you go about this? Hint: It’s not JSON.

Either you call serialise on the object and send it, or you use this other PHPism that lets you say ?param[field]=val.

In the first case, use the phpserialise package, in the second case, use this handy pre-processing function:

def php_flatten(data):
    res = {}

    def inner_flatten(path, value):
        if isinstance(value, dict):
            for k, v in value.items():
                newpath = "%s[%s]" % (path, k)
                inner_flatten(newpath, v)
        elif isinstance(value, list):
            for k, v in enumerate(value):
                newpath = "%s[%s]" % (path, k)
                inner_flatten(newpath, v)
        else:
            res[path] = value

    for k, v in data.items():
        inner_flatten(k, v)

    return res

XML

I heard Java people like XML, so your API should probably offer it as an alternative to JSON, serialised PHP and INI files.

I was surprised there is no such thing as xml.dumps in Python, so I wrote it. There is no 1-on-1 mapping between dicts and XML, but this a a way to go from Python data to XML.

import lxml.etree as ET

def maybesub(el, tag):
    """
    Create a subelement if a tag was given.
    Otherwise return the current node.
    Used for the root node.
    """
    if tag:
        return ET.SubElement(el, tag)
    else:
        return el

def walk(el, tag, data):
    """
    Recursively add child nodes named tag to el based on data.
    """
    if isinstance(data, dict):
        sub = maybesub(el, tag)
        for key, value in sorted(data.items()):
            walk(sub, key, value)
    elif isinstance(data, list):
        for value in data:
            walk(el, tag, value)
    else:
        sub = maybesub(el, tag)
        sub.text = str(data)

def dumps(data, root="response"):
    root = ET.Element(root)
    walk(root, None, data)
    return ET.tostring(root, pretty_print=True)

INI

That was no joke. Take a look at configparser if you ever encounter this.

The internet is s-l-o-w

So take this handy Redis-based memoization function with you, and twiddle with the TTL.

import redis
from functools import wraps
import pickle
import logging

pool = redis.ConnectionPool()

def memoize(ttl=3600):
    def decorator(f):
        @wraps(f)
        def wrapper(*args, **kwargs):
            try:
                r = redis.StrictRedis(connection_pool=pool)
                # Compute the function signature
                sig = (f.__module__, f.__name__, args, kwargs)
                pargs = pickle.dumps(sig) 
                # Try to get the result
                pres = r.get(pargs)
                if pres:
                    return pickle.loads(pres)
                else:
                    # Or compute and store
                    res = f(*args, **kwargs)
                    r.setex(pargs, ttl, pickle.dumps(res))
                    return res
            except redis.RedisError:
                # Show must go on!
                logging.exception("redis oopsed")
                return f(*args, **kwargs)

        return wrapper
    return decorator