Wishful Coding

Inspired by a great video by Veritasium about analog computers and Andrew Carol’s Lego Antikythera mechanism I decided to try to build Sir William Thomson’s tide predicting machine out of Lego. Before I get into the details, here is a video of it running.

It uses 96 gears and a Lego Robot Inventor to drive the device and measure the hight of the output with an ultrasonic sensor. The motor angle and distance are then logged to the computer to plot the tide over time. This is super exciting! Compare real tide data form Hawaii to the data generated by my machine. Looks similar right!! Of course not all components are aligned perfectly, but there is definitely a striking similarity.

With all the pretty pictures out of the way, time to get into the making of.

As Veritasium explains in his video, predicting the tides was alway an important problem, but only after we had been blessed with the Fourier transform, did it become possible to decompose tidal data in frequency components, and then add these components back up to predict the tide. Except this was before digital computers existed, so Sir William Thomson invented an analog computer with gears, ropes, and pulleys to create the different frequency components and add them together.

So the idea is simple. Create gear trains that run at the correct frequencies, and then attach pulleys to them. One small problem, Lego gears only come in a small set of fixed sizes, so you can’t just make a 468/424 gear ratio by meshing a 468 and a 468 tooth gear, you have to build it up from the fixed gear ratios that you have. This turns out to be a bit of a challenge. To solved it I wrote a Julia script to find the optimal gear ratios.

To save me half of the problem, of decomposing tidal data into frequency components, I found that the Wikipedia page Theory of Tides contains a handy table with the frequencies of the components, and their magnitude at various locations. I only took the 5 biggest ones.

First I defined all the frequencies and normalized them and scaled them down. This is important because all these gears will have a ton of friction and backlash, so by introducing reduction both of these problems can be reduced a lot.

using Optim
M2 = 12.4206012
S2 = 12
N2 = 12.65834751
K1 = 23.93447213
O1 = 25.81933871

periods = 1 ./[M2, S2, N2, K1, O1] .* 12 ./ 24

Then I computed all the possible gear ratios you can make out of Lego gears. But I later found that not all combinations are easy to build, in fact some are downright impossible (12:16 for example). So I narrowed down the list to just 4 combinations: 8:24, 12:20, 12:28, 16:20. I found that these 4 ratios required almost the same amount of gears as using all combinations, but gives a result that is much easier to build. These can all be assembled on a normal grid of technic beams.

gears = [8, 12, 16, 20, 24, #=36, 40=#]
gearratios = Dict{Rational{Int64}, Vector{Tuple{Int64, Int64}}}()
for g1 = gears
    for g2 = gears
        if g1 < g2
            push!(get!(gearratios, g1//g2, []), (g1,g2))
        end
    end
end
delete!(gearratios, 12//16) # impossible
delete!(gearratios, 36//40) # huge
delete!(gearratios, 20//36) # wide gap, large
delete!(gearratios, 16//40) # wide gap, large
delete!(gearratios, 20//24) # wide gap
delete!(gearratios, 8//40)  # large
delete!(gearratios, 16//24) # large

grs = []
for gr = values(gearratios)
    if length(gr) == 1
        push!(grs, gr[1])
    else
        push!(grs, first(filter(x-> 8 ∉ x, gr)))
    end
end

Then I ran a simulated annealing algorithm to find the optimal gear ratios. I formulated the problem as a big product of all the gears, where I can optimize the exponents to vary the number of gears. Negative exponents just flip the gears around. I added a small factor for the required number of gears so it’d hopefully find an optimum that also doesn’t require 500 gears like my first attempts.

A final little trick I used to reduce the number of gear is that the neighbor function tries a random permutation that doesn’t exceed the hard limit on the total number of gears I set. Playing around with this a bit I got much better convergence to much more reasonable results. Note that the final result uses less than this limit! It just stops the optimizer from generating too much BS proposals.

x0 = zeros(length(gearratios))
lim = 20

function neighbor(x::AbstractArray, x_proposal::AbstractArray)
    while true
        for (idx, xval) = enumerate(x)
            val = rand((1, 0, -1))
            x_proposal[idx] = xval+val
        end
        sum(abs.(x_proposal)) < lim && break
    end
end

temperature(t) = 1/log(t/1000)

ngs = ones((length(periods),length(gearratios)))*lim
for (idx, ratio) = enumerate(periods)
    cost(x) = abs(prod(keys(gearratios).^x)/ratio-1)+sum(abs.(x))/1e4
    res = optimize(cost, x0, SimulatedAnnealing(;neighbor, temperature), Optim.Options(iterations = 10000000))
    mg = Optim.minimizer(res)
    mm = Optim.minimum(res)
    ng = sum(abs.(Optim.minimizer(res)))
    ngs[idx,:] = mg
    println()
    println(ratio)
    println(mm)
    println(mg)
    println(ng)
end

And then all that is left to do is compute the bill of materials and verify the error margin looks fine.

num = sum(abs.(ngs); dims=1)
thebom = Dict(zip(gears, zeros(Int64, length(gears))))
for (num, (g1, g2)) = zip(num, grs)
    thebom[g1] += num
    thebom[g2] += num
end
thebom

display([permutedims(grs); ngs])
display(thebom)
display(sum(values(thebom)))
res = map(x->float(prod(keys(gearratios).^x)), eachrow(ngs))
display(res./periods)
display(res.-periods)

Pretty good!

6×4 Matrix{Any}:
   (12, 20)   (12, 24)   (16, 20)    (8, 24)
  1.0        1.0        9.0         0.0
  0.0        3.0        0.0         1.0
 -1.0        0.0        2.0         3.0
 11.0        0.0        2.0        -2.0
  9.0        0.0        2.0        -1.0
Dict{Int64, Int64} with 5 entries:
  16 => 15
  20 => 37
  12 => 26
  24 => 11
  8  => 7
96
5-element Vector{Float64}:
 1.0002389240748448
 1.0
 1.0001657291851853
 1.0003226141581112
 0.9991658743311826
5-element Vector{Float64}:
  9.618055961925498e-6
  0.0
  6.546240931305791e-6
  6.739529419302198e-6
 -1.6153118369648806e-5

Now one last simple thing I need to calculate is the length of the levers to match the amplitude of the frequency components. Looking at the table on Wikipedia I picked Hawaii because the differences in magnitude are relatively small. That means I will not have to make a lever of 20 studs and one of 0.2 studs.

Normalizing the amplitudes to 2 studs for the biggest component, resulted in a smallest component of a bit shy of 0.5 studs. I rounded these distances to 0.5 studs, which resulted in levers of 0.5, 1, 1.5, and 2 studs. Very reasonable! I experimented with variable length levers, but they were too flimsy.

5-element Vector{Float64}:
 2.0
 0.7999999999999999
 0.3826086956521739
 1.452173913043478
 0.7999999999999999

Next, I took my bill of materials to Bricklink, robbed some poor seller of literally all of their gears (I had to supplement some types from my existing collection because they didn’t have enough!), and two weeks later I could start building.

Packing all these gears as tightly as possible was a real challenge. The diagonal ones can just zig-zag around,but the orthogonal ones require a sort of snake pattern that’s extremely tight. Each axle has 4 positions and is surrounded by 5 other axles, meaning the smaller gears have to double up. But after two days of trying, I was successful. The first one took forever, an then the rest was done in a matter of hours.

After that I stacked them on top of each other, and hooked them up with a long driveshaft, attached levers and pulleys, and the rest is history.

If you’re looking at the video and wondering which gear train corresponds with which frequency components, but don’t want to go figure it out from pictures of gears, here is an annotated picture. Green are the semi-diurnal ones, red the diurnal ones, which are twice as slow.

I’m currently developing Mosaic, a schematic editor built on web technology. In the process of adding online collaboration, I wrote a library to help me manage and synchronize state. The library is called Hipflask.

In my previous post I wrote about different strategies for dealing with conflicts when multiple users are editing the same object. In this post I want to focus more on the library I wrote to manage state.

Hipflask is basically a ClojureScript/Reagent atom backed by PouchDB.

Mosaic is written in Reagent, a ClojureScript React wrapper. The fundamental building block of Reagent is the Ratom (Reagent atom), a data structure that allows atomic updates by applying functions to its state. It then monitors state updates, and redraws components that depend on the Ratom. It’s quite elegant.

The semantics of a Clojure atom (after which ClojureScript and Reagent atoms are modelled) is that there is a compare-and-set! function that atomically updates the state if the expected old state matches the current state. swap! is a wrapper over that which repeatedly applies a function until it succeeds.

The neat thing is that CouchDB update semantics are exactly like compare-and-set!. You update a document, and if the revision matches it succeeds, if there is a conflict the update is rejected. You can layer swap! behavior on top of that which applies a function to a document repeatedly until there is no conflict.

So what you get if you combine PouchDB with a Ratom is that you can apply functions to your state, which will resolve database conflicts and rerender your app. The PouchDB atom even listens for database changes, so that remote changes will update the cache and components in your app that depend on it.

Basically it transparently applies functions to the database and updates your UI on database changes.

One thing that sets Hipflask apart from my previous CouchDB atom is that it manages a prefix of keys. This plays well with sharding, so you can have a PouchDB atom containing all foo:* documents. Worth noting is that it’s only atomic per document, and assumes the first argument to your update function is a key or collection of keys that will be updated.

You can use Hipflask in two main ways. The first way is offline first. The atom is backed by a local PouchDB which can by replicated to a central database. This way the app remains fully functional offline but replication conflicts are not handled automatically. You can also make it talk directly to a central CouchDB, in which case it does not work offline but conflicts in the central database are resolved automatically.

As a demo of the library, I made a Global Cookie Clicker app. It’s an offline-first collaborative cookie clicker. You can click cookies offline, and they are then synchronized to a central CouchDB. The entire code is less than 70 lines.

It basically defines a PouchDB atom

(def db (hf/pouchdb "cookies"))
(def rginger (r/atom {}))
(def ginger (hf/pouch-atom db "gingerbread" rginger))

And then when you click the button it calls this function that increments your cookie document.

(defn make-cookie [a n _]
  (swap! a update-in [(str (.-group a) hf/sep me) :cookies] #(+ % n)))

This library has been very useful in building Mosaic, and I hope it is useful to others as well.

It’s important to stay hydrated, but hard to know how much you are actually drinking. Some number of liters or cups does not easily translate to absentmindedly sipping from an oddly sized mug. There are health apps that let you manually track how much you eat and drink, but that is such a hassle. When I’m in the zone, it’s already hard to refill my cup, let alone enter it into some app.

So I made a smart cup holder that weighs my cup and keeps track of how much I drink.

This setup consists of the following parts:

• Lego cup holder (building instructions)
• load cell (TAL220)
• load cell amplifier (HX711)
• Arduino-compatible board (Teensy 3)

The basic idea is quite simple. Every time the cup is placed down, its weight is recorded. If the weight is less than the last recorded weight, the difference is assumed to be consumed by me, and added to the total. The Arduino code is indeed quite simple.

The only noteworthy part of the code was calibrating the load cell. I did this simply by taking a measuring cup and pouring 100ml of water at a time into a cup. Repeat a few times with a few different cups and determine the scale factor.

The cup holder had to go through several design iterations. The first ones had either too much friction, inconsistent weight measurement, or just weren’t structurally stable enough. The load cell is not a Lego part, so the tricky part was making the cup rest only on the load cell. I used a Technic Flex-System Hose inserted into Technic Plates, which just about align with the load cell screw holes.

Fun fact: a 1 x 2 Grille is slightly thicker than a 1 x 2 Tile. Maybe I’m imagining things, but it seems to be the difference between a tight fit or not. With normal tiles I had to insert some pieces of paper to keep it from flexing.

With the mechanical parts out of the way, I went on to displaying the information. Maybe a 7-segment display would have been the obvious choice, but I only had one digit. I do have a full colour LCD panel, but that seems overkill and distracting.

If it’s going to sit on my desk and I’m going to power it from my computer’s USB ports, I might as well display the information on my computer. A little icon should be less obtrusive than a blinky LED thing. I may have underestimated how easy it is to get USB serial data in my taskbar, but in the end it worked out.

I forked a Gnome extension that displays the status of wireless earbuds, which it reads from the log file of some daemon. All I had to do was change the icons a bit and make it read form the TTY… right? Well turns out reading a TTY from Gnome JavaScript is a bit painful because there don’t seem to be any functions to configure it. It would also randomly close and reset for some reason. In the end I shelled out to stty to set up the TTY to not block and then just get the whole contents of it. In the process I learned that a JS extension can definitely hang and crash your entire Gnome shell. Great design that.

So that’s it, right? Throw the code on Github, job done. Well, except I decided that in the name of reproducibility, I should make building instructions for the Lego cup holder. I used to sell Lego Mindstorms building instructions, so I’ve done it before… many years ago. So I installed LeoCAD and started making the model.

At first it went pretty well, but then I ran into two problems. First of all, I spilt the model into a base and a cupholder submodel. But when generating building instructions, LeoCAD just inserted the submodel as a part. The second challenge was making the flexible hoses.

I tried to open the model with LPub3D, which did render the submodel instructions, but did not show the flexible hose correctly because LeoCAD uses a non-standard format for those. Then I installed Bricklink Studio under Wine, and redid the flexible hose there. For some reason Bricklink Studio did not render the parts list in the building instructions, so I ended up going back to LPub3D to render the final building instructions. Phew!

Update: I have now hooked up my smart cupholder to InfluxDB using the tail input using the following Telegaf configuration.

[[inputs.tail]]
  files = ["/dev/ttyACM0"]

  # this avoid seeks on the tty
  from_beginning = true
  pipe = true

  # parse csv
  data_format = "csv"
  csv_column_names = ["cup present", "current weight", "last weight", "weight difference", "total consumed"]
  csv_delimiter = ";"