Wishful Coding

Team Relay Chat: File Sharing and Registration

2013-05-08T00:00:00-07:00

Almost a month ago I uploaded the first version of Team Relay Chat, now I have a couple of users and some nice new features to show you.

Registration

Previously it required several not-so-trivial steps to add a user to your IRC server. Now you can just enable the registration module. Your employees will be able to just pick a user name and password and be good to go.

File Sharing

I realised file sharing was important right from the beginning. It has been possible to share files from the web interface for quite some time now, but I added two important features.

You can now share files from your desktop client too. Just send them to Hubot via DCC, and Hubot will take care of uploading them.

The web client now displays a preview of uploaded images inline. No more no less.

Documentation

I’m working hard to make TRC as easy to use as possible. Both by just making things simpler and by writing documentation and even doing some videos.

All of this can be found at teamrelaychat.nl/wiki

Log Viewer

ZNC comes with a lovely logging module that stores log files of all your chats in a folder deep inside the server.

It is now possible to browse all these files via the ZNC web interface. This interface will grow even better in the next few weeks.

Firewall friendly

Nowadays almost everyone is behind a NAT or firewall, and I’ve been working to make sure you can use Team Relay Chat even then.

Alternative IRC ports

You can now configure ZNC to listen on any port you want, even lower port numbers like 25(SMPT) or 80(HTTP) are supported. So even if port 6697 is blocked, you can still chat.

Passive DCC SEND

File sharing from the desktop uses DCC. Normally this means the sender(you) opens a port from which the receiver(hubot) downloads the file. This means you have to forward external ports to your machine.

However, Hubot also support passive DCC. In this case Hubot opens a port and you simply upload the file. No configuration needed.

Conclusion

You should try the demo server(with SSL) and read more.

Hosted IRC for Teams

2013-04-23T00:00:00-07:00

It has started. A few weeks ago I was just playing around with Pallet, now I have been on Hacker News and have a couple of users.

My workflow is pretty desktop oriented, so when I was required to use a web application to keep in touch with teammates and Hacker School friends, I thought I could make a perfectly fine group communication tool based on IRC with plenty of desktop clients available.

The idea is pretty simple: Give every team a VPS with an IRC server, but the devil is in the details. A vanilla IRC server is a bit awkward as a collaboration tool. You might as well telnet to each other directly.

An important piece to the puzzle is ZNC. ZNC is a powerful bouncer with a ton of plugins. It stays connected to the server on your behalf 24/7, replaying all the messages you missed while disconnected. So that is user accounts and persistent chat right there.

Another important part is Hubot. Hubot is an IRC bot that can execute commands and notify you about things like Git commits, build failures and more.

There is also a web interface, because… you know, some people do live on the web.

Finally, I have some grand plans for file sharing, but that is for another time.

The result speaks for itself, check it out: teamrelaychat.nl

I realise this is not for everyone, but if you’re anything like me, chances are it fits your workflow better than these web based services.

clojure.core.logic magic square

2013-03-31T00:00:00-07:00

Last night my brother told me about this puzzle where you have a square of 3x3 that you have to fill with 1-9 in a way that all columns and rows sum up to 15.(this is not a real magic square, but anyway)

After discussig our strategies for a bit, we went on to think about generalising the problem to bigger squares.

The next morning we tested our theory in excel and produced a 4x4 square. The method we used was to fill in numbers at random, then switching them around horizontally until all collumns equalled 34 and finally switching them around vertically to make the rows work out.

Once we started thinking about doing the 5x5 grid, I suggested I might do it faster using core.logic. I was wrong.

user=> (time (test/magic 3))
(1 5 9)
(6 7 2)
(8 3 4)
"Elapsed time: 451.880413 msecs"
nil
user=> (time (test/magic 4))
(1 2 15 16)
(7 8 9 10)
(12 11 6 5)
(14 13 4 3)
"Elapsed time: 6386.272944 msecs"
nil
user=> (time (test/magic 5))
OutOfMemoryError GC overhead limit exceeded

The code simply generates N² logic variables in the 1-9 domain that are distinct. These are then sliced up in rows and collumns and constrained to sum up to the “magic number”.

(ns test
  (:refer-clojure :exclude [==])
  (:use clojure.core.logic
        clojure.pprint)
  (:require [clojure.core.logic.fd :as fd]))

(defn grid [n] (repeatedly (* n n) lvar))

(def rows partition)

(defn cols [n grid] (apply map list (rows n grid)))

(defn sum [ls res]
  (conde
    ((== ls []) (== res 0))
    ((== ls [res]))
    ((fresh [h t inter]
       (conso h t ls)
       (fd/+ h inter res)
       (sum t inter)))))

(defn magic [n]
  (let [g (grid n)
        nums (range 1 (inc (* n n)))
        ndom (apply fd/domain nums)
        lsum (/ (apply + nums) n)
        lines (concat (rows n g) (cols n g))]
    (->> (run 1 [q]
           (everyg #(fd/dom % ndom) g)
           (fd/distinct g)
           (everyg #(sum % lsum) lines)
           (== q g))
      first
      (rows n)
      (map println)
      dorun)))

I suspect I can still beat my brother to 6x6 by implementing one of the techniques outlined on wikipedia in a good old imperative style, especially since he’s not even trying.

VMfest Base Image

2013-03-20T00:00:00-07:00

I’m playing with Pallet to setup an IRC server and bouncer. Pallet uses vmfest to deploy to VirtualBox.

While vmfest comes with some prepared images, I found that my laptop would not run them, so I made my own, with a lot of help from Antoni Batchelli.

Create an image in VirtualBox with a NAT and host-only network.
Install your favourite distro.
Install an SSH server.
Setup passwordless sudo

Make sure /etc/network/interfaces contains both eth0 and eth1.

  auto eth0
  iface eth0 inet dhcp
  auto eth1
  iface eth1 inet dhcp

Update and upgrade all packages.
Install the guest additions
On some Debian based distros, remove persistent-net.rules.

This should give you a working machine, but we’re not done yet. You still need to make the hard disk multi-attachable.

If you where to use this image in Pallet, you could only use it once. Multiattach means that every time a machine is made, a new copy-on-write image is created so the original stays intact.

To do this, delete the VM, but not the vdi file, and run the following command:

VBoxManage modifyhd the/disk.vdi --type multiattach

Finally, you need to create a meta file with the same name as the disk image, but with the .meta extension. As an example:

 {:os-type-id "Ubuntu_64",
  :sudo-password "vmfest",
  :no-sudo false,
  :username "vmfest",
  :os-family :ubuntu,
  :os-version "12.04",
  :os-64-bit true,
  :password "vmfest",
  :description "Ubuntu 12.04 (64bit)"
  :packager :apt}

The Prolog Cook

2013-01-06T00:00:00-08:00

brain dump alert

I had read this article on core.logic a few days back, and while making myself a dinner, I came up with this analogy.

At first, I was confused about disjunction and conjunction, so let’s start with the Prolog cook.

The prolog cook cooks depth-first. The cook has several recipes and steps to produce them. The first recipe on the list is an egg with ham and cheese.

The cook bakes the egg on top of the ham, adds some salt, finds out there is no cheese, throws everything away and moves on to the next recipe.

The next recipe is is a simple pasta. The cook checks if there is pasta, herbs, tomato, and finally checks to see how much water is in the tap.

Next up is the miniKanren cook. This cook has fair disjunction, but unfair conjunction.

So the cook looks at her recipes, picks the egg recipe, checks if there is an egg. Then, picks another recipe, like a pasta, checks if there is any pasta.

The cook keeps going over the recipes, discarding ones that can’t be made. But at some point this cook will also check how much water is in the tap. And keep checking until there is no more water.

Finally, the fair cook, with fair conjunction.

This cook goes over his recipes the same way the miniKanren cook does, but upon checking for water, this cook opens the tap and continues cooking while keeping an eye on the tap.

Maybe the recipe can be adjusted not to check for all the water. Even the fair cook is wasting a lot of water while doing other stuff, but at least she does other stuff in the meantime.

There have been very successful Prolog cooks, but not all recipes can be adjusted to work for the Prolog cook.

Philips 162 Post-Mortem

2012-12-13T00:00:00-08:00

You read that right, I used my phone untill it died. This is not a review or even a preview, but a postview.

I wish there was a gadget website of which the authors are not too happy about ditching their almost-new phone.

– some tweep lost in history

The upside of a post-mortem about a phone is that I used it for years, so I know all the details and annoyances. The downside is that Philips no longer makes the phone, but no matter, I wouldn’t recommend you buy it anyway.

It’s a practical phone. It lies confortably in your hand, it’s super sturdy, even after years, it lasts a week on a charge, and it even has a few games on it.

It doesn’t have a camera, wifi, bluetooth or pretty mutch anything else, but it’s great for calling and texting. The T9 feature works good, but always uses the interface language and does not allow adding words.

My number one annoyiance with the phone is that in the phonebook the screen displays only 8 of the 10 digits of a phone number in a hard to follow sliding motion.

Speaking of the phone book, I recently discovered that copying contacts to the SIM card is hard.

The menu of the phone is easy to use, but there is hardly any reason to. On a daily basis, I tend to use the directional pad to get to the most frequently used functions. This does mean it is fairly easy to do stuff accidentally if you forget to lock it.

One thing to keep in mind is that it doesn’t warn you when the inbox is full. It just doesn’t receive anymore text messages. Once in a while, when people stop replying to my messages, I need to check if it’s me or my phone.

Overall, this phone served me well. R.I.P. phone…

Pure CSS Gauge

2012-11-29T00:00:00-08:00

I got nerdsniped

still looking for pure css gauges, ideas?
— Almog Melamed (@radagaisus) November 28, 2012

So I had to make this gauge using CSS animation.

I used two nested divs with round borders. The outer one does the quadrants with a top/left/bottom/right border. The inner one has only a top border, and slides over the bottom one.

Note that halfway through, the inner border changes from sliding out, to sliding in. Otherwise you’d get in trouble during the last quadrant.

A good way to see how it works is removing the negative margin, or change the inner border colour.

Incremental E-paper update

2012-11-23T00:00:00-08:00

This is what I’ve been up to since last post. I almost completely rewrote the Arduino code to support different waveforms and both full updates and incremental ones.

I admit the old update looked smoother, but it faded away over time, defeating on of the main features of E-paper, that it is bi-stable.

I do think the incremental updates look cool, kind of like an evolving Pokémon.

Get the source code

I am using a modified version of shiftOut that can send any number of bits. This is needed to send 1 bit for the background and common, and 16 bits for a character.

The EIO pin is no longer used, but just tied low. This causes the first chip to be selected when it wakes up. The enable pins are chained so that when the first chip is full, it pulls the second chip low.

In short, this is how I update the screen:

Step one is to convert all 8 bit characters to 16 bit segment data using a big table.

Next, I create a struct with the changes. This struct contains 6 arrays. One with the added segments, one with the removed segments, one with all changed segments and 3 negatives.

In case this is a complete update, this struct is very easy to make. In this case the negative of the aded segments equals the deleted segments.

In the case of a partial update, this struct is generated using a lot of binary logic, based on an array of the current segments.

Now that we have a struct of all the changes, we actually need to write this to the shift register and toggle the latch to send it to the display. The writing itself is fairly trivial, but the waveform is not.

I will cover the simple waveform in this post, and leave the 757 for you to figure out. The principle is the same.

First we wakeup the display. After every update, we shut it down again.

Now, you need to look at the datasheet to see the waveform for segments to white and segments to black. Unchanged segments need to be the same as the common bit, so no current flows through them.

Note that for white-on-black the added/deleted bits are the other way around, but the unchanged segments aren’t.

Finally, for every section of the waveform I take the corresponding array out of my changes, send it, and wait for a bit.

Improved library for Sparkfun E-paper display

2012-11-12T00:00:00-08:00

I got my E-paper display from Sparkfun, but I was disappointed by the quality of the library.

My version has more and nicer looking(IMO) characters, uses black text on white background, uses sleep mode, and most importantly, no ghosting.

What remains to be done is implementing temperature awareness. The colder the display, the longer it takes to update. The Atmega chip has a sensor built in.

Original library
My updates

Thanks to the people at Hack42 for helping me out with my bootloader and understanding the E-paper datasheet.

One little byte of information that I found particularly interesting is that the E-paper controller is basically a shift register with an extra latch. Maybe the code can be simplified using shiftOut?

From the library

// Second ePrint removes the un-used segments - I know, this is weird…

I thought the same, until I understood the COM(mon) bit. This indicates that all specified segments should be pulled either high or low. But not both at the same time.

This means that the first pass, you put +35v on all the segments you want to use, and the second pass -35v on the ones you don’t want to use. Order doesn’t matter, but gives a different intermediate state.

Sparkfun 1.8v USB to serial

2012-10-26T00:00:00-07:00

I am trying to hack my Kindle 4 so that I can plug in a keyboard and use it as a Raspberry Pi with battery and E-paper screen.

To support a keyboard, the Kindle needs to use its USB port in host mode. This is called USB On The Go, and someone modified the Kindle kernel to support this.

So far, it does not work on my Kindle, but since the Kindle does not display any kernel messages while booting, I had no idea what went wrong.

Thus, I had to open up the Kindle to attach a serial console. My brother is writing a guide about this at http://jurriaandevos.nl

The next hurdle was that the Sparkfun USB-serial breakout comes with a solder jumper for 5v and 3.3v, but the Kindle uses 1.8v.

Therefore, I devised this small stripboard that lets you select any of these three voltages with 2 jumpers.

With the top jumper removed, you can switch the lower jumper to either jumper Vcc or the 3.3v reference to the VCIO pin.

With the top jumper connected and the bottom one in the lowest position, you engage a voltage divider between 3.3v and gnd, resulting in half that voltage(1.65v) on VCIO.

Before this works, you should desolder the built-in jumper, and remove the LEDs.

The LEDs are powered from Vcc, so they leak current into the circuit. I measured 2.7v on VCIO with the LEDs still on.

The voltage divider was made using two 10k resistors, but anything between 1k and 10k should be fine, I’ve read.

I’m looking forward to blog about my Kindle 4 keyboard soon.

Kindle 4 as a paper terminal

2012-10-08T00:00:00-07:00

After reading about the Kindleberry Pi, I really wanted to experiment with e-paper as a computer monitor.

Sadly, I own a Kindle 4, which has no keyboard and consequently a lot less hacks. Still, I’m typing this post from my Kindle, so I will document how I got here.

Initially, I could not find a jailbreak for the Kindle 4, but at some point, I found this wiki page that documents everything you need to get started.

I used the universal method, which involves downloading data.tar.gz, entering and leaving diagnostic mode and rebooting a few times.

The jailbreak only installs a developer key, that allows you to install software. The piece of software we need is usbnetwork, which allows you to SSH over USB.

Download usbnetwork from here and follow the instructions.

Before you enable usbnetwork by renaming the auto file, go into /usbnet/etc/config and set USE_VOLUMD="true"

If all went well, the next time you connect your Kindle, it will present a network instead of mass storage.

You should now be able to ssh root@192.168.2.2. Linux users should run sudo ifconfig usb0 192.168.2.1 netmask 255.255.255.0 first, to bring up the network.

The terminal emulator used in the Kindleberry hack does not work on the Kindle 4. It expects a keyboard, I’ve been told.

However, vdp wrote a Java kindlet terminal emulator called KindleTERM that seems to run on Kindle 4.

KindleTERM was writen 2 years ago, and while it pretty much worked out of the box, it did not make an SSH connection to my PC.

I finally got something working using the remote keyboard feature, sshing to localhost and using dbclient to ssh into my Mac and start a tmux session.

Hawhill made a new version that just telnets to localhost, but he removed the remote keyboard. That’s useless.

Then hippy dave came around and added back the remote keyboard. Yay!

Then I came around and added a config file to specify which host/port/username/password/command to use.

Long story short, follow his instructions, but upload my version of KindleTerm to your Kindle. You should be using SCP or SFTP, since you can no longer use mass storage.

By default, my version still telnets to localhost, but it tries to read from /developer/KindleTermPV/work/kindleterm.properties. Mine looks like this:

host=192.168.2.1
login=pepijn
password=secret
cmd=tmux -S /tmp/kindle

After rebooting, you should see KindleTermPV on your home screen. When you start it, it should telnet to your PC.

If you see a blank screen, press back+keyboard or relaunch the app a few times.

If you are on Mac, you can run telnetd with sudo launchctl load -w /System/Library/LaunchDaemons/telnet.plist

You might want to export PS1=">" to save some precious screen space.

You should also export TERM=ansi if you experience any formatting problems. The VT320 implementation is a bit buggy.

Linux Time Machine Backup with netatalk 3.0

2012-09-14T00:00:00-07:00

There are quite a few lengthy guides on how to back up your Mac to a Linux box with Time Machine and netatalk 2.x.

You don’t need any of that. With netatalk 3.0, it is a single line in your config file, time machine = yes, and you don’t even need to set TMShowUnsupportedNetworkVolumes.

On Arch Linux, all you need to do is

pacman -S netatalk
Append dbus avahi-daemon netatalk to your DAEMONS in /etc/rc.conf.

Add your backup folder to /etc/afp.conf like so:

[ShareName]
path = /path/to/backup/drive
time machine = yes

Now start the daemons and you’re good to go. Even Bonjour is taken care of automatically.

On Ubuntu, netatalk is still at version 2, so you’ll need to compile it from source. Make sure you install Avahi.

This is How I Want to Write Markdown

2012-09-03T00:00:00-07:00

Most websites us a split-pane approach to Markdown editing. You enter your text in one field, and the resulting HTML appears in another.

I think this is silly UI, and would like to see websites adopt the iA Writer approach to Markdown editing.

Sadly, I don’t know of any library or parser that lets you do this, so I started writing one.

This thing is currently easier to break than to use, partially because my approach is broken, partially because the content editable API is broken, and partially because it’s just a one-day hack.

Attending Hacker School

2012-08-22T00:00:00-07:00

As a male without $6000 in savings.

Hacker School is awesome, so if you enjoy coding, you should apply, but what really prompted me to write this post is that someone on Reddit or HN said giving grants only to woman is unfair to men who need it.

While I do think it’d be good to have more gender balance in programming, giving grants based on need only to woman seemed unfair to me too. After three months of Hacker School, there are 2 things that altered my opinion on the matter.

First-hand stories of all the crazy little subtle male culture bias. There is no way I can second-hand this to male readers, but let me say there is a lot more to it than just getting the job and getting promotion.

“Affirmative action.” Discrimination is a big part of American culture. If you think giving grants to woman is discriminative to man, you would be completely right.

But it’s okay, because woman are a minority. Affirmative action is a very normal and accepted thing over here, and it’s just that, discrimination in favor of the minority.

I still managed to make it

New York is both more expensive and cheaper than you can possibly imagine.

If I would have rented a decent room and ate out like everyone else, I would not have been able to do this from my savings. But the thing is, you don’t have to do all that.

The first thing to do is to contact all your relatives in and around New York. I’m basically staying here for free in exchange for some help with AERO on my free days. I do have a long commute, but I have a Kindle and a Gameboy with Pokemon.

My friend Six has taken an alternative route, and is just crashing on couches and roofs, with Couchsurfing as a backup. With a lot alumni living in the city, and a dangling fire escape here and there¹, this is perfectly doable. However, it might be less pleasant in the Fall batch.

A good lunch can easily cost you $10, but it doesn’t have to. You can buy bread, there is a fridge, microwave and hot plate², use them. Me and Six had a lot of cheap and delicious meals that way.

I hope that helps.

WARING ERROR WARNING WARNING don’t do that, unless you want to get in trouble. Alumni roofs are okay though.
↩
Well, there was, until something started to smell. I’m convinced that’s the trash/fridge though, not my hot plate ;)
↩

The Multiple Index Problem

2012-08-12T00:00:00-07:00

I would like to name this problem in functional programming. There has been some talk aout how Clojure solves the Expression Problem, but th Multiple Index Problem has deserved so little attention it doesn’t even have a name.

A simple example of multiple indices can be seen in relational databases, where you can querry a table like SELECT * FROM foo WHERE id=1 but you can add an index to the table and querry on other fields like SELECT * FROM foo WHERE name="Pepijn".

If you would translate this to hash maps, in imperative languages, you could do the following

>>> val1 = []
>>> val2 = []
>>> index1 = {1:val1, 5:val2}
>>> index2 = {"pepijn":val1, "ben":val2}
>>> index2["pepijn"].append("foo")
>>> index1[1]
['foo']

If you want to do this in a functional language, you’re out of luck. In the most basic case, you’d have to add some indirection.

user=> (def index [[], []])
#'user/index
user=> (def index1 {1 0, 5 1})
#'user/index1
user=> (def index2 {"pepijn" 0, "ben" 1})
#'user/index2
user=> (def new-index (update-in index [(get index2 "pepijn")] conj "foo"))
#'user/new-index
user=> (get new-index (get index1 1))
["foo"]

Next up, I’ll show 2 cases where this is a real pain, rather than just anoying.

Dijkstras Algorithm

Dijkstras algorithm is used to find the shortest path in a graph. It is fairly easy to implement using a vector, but also very slow.

Every itteration you need to find the closest unexplored node. With he flat list, you need to walk them all.

The way to solve this is to use a Fibonacci Heap, which has constant time and logarithmic time opperations for getting and decrementing the lowest key.

However, it has no methods for lookingg up other keys, other than… walking the tree? The way this is done in imparative languages is to just have pointers to the nodes, but in a functional language, youre stuck walking the tree.

This defeats the whole purpose of using a Fibonacci Heap. Is it at all possible to implement Dijkstra in a pure functional way in an optiomal time complexity?

I ended up using a mutable heap and just not decrementing keys. Yuk!

You can see the code me and Mary wrote at Hacker School at her github(fibbonaccy and flat array), and mine(Using PriorityBlockingQueue)

Collision Detection

I struggled with the same problem in my Clojure game engine, before I realised this was a repeating pattern.

So I started out modeling my world as a list of objects, but quickly found that the bat needed to know where the ball was, so I turned the world into a hash map of IDs to objects, and all was good.

Until I needed to do collision detection on more than a dozen objects. If you just compare all objects to al others to see if they collide, you’re doing O(n²), which is no good.

Lukily there are datastructures that do spatial indexing. Indexing… see my problem?

I wrote a Quadtree in Clojure, but as soon as I stuck all my objects in the tree, there was no way to look them up by ID.

I think I ended up doing some black magic to sorted maps to have lookup by ID and okayish collision detection.

That is all I have to say about the problem. I don’t have a soltuion or a satisfying workaround. You can try adding intermediate indices, or introducing mutability, but it’s painfull.