Wishful Coding

I’m a jack of many trades master of some, who has worked his way down the stack from frontend to chip design. I’m in my early thirties, so officially old in terms of software (jk but startup culture can be kinda ageist I’ve heard), so I feel somewhat qualified to give a “30000 foot view” of the different disciplines in software development.

I’ll briefly describe my (in some cases limited) experience with each, and then break them down in the good, the bad, and the ugly. The exciting stuff, the frustrating stuff, and the bread-and-butter seen-one-seen-all stuff.

This post was inspired by someone on Twitter getting into “software”, meaning frontend JS, which made me feel like I maybe have some useful perspective to share on why you might want to do frontend, or what else you could be doing.

Frontend

I started my career building Wordpress websites, back when jQuery was cool. Later I had a gig working in Angular.js and did some React hobby stuff. I’m currently building Mosaic in Reagent (a ClojureScript React wrapper).

Frontend problems are all about design and state. You make a user interface and then hook up the buttons and fields to state, and communicate that state to the backend.

The good part in frontend is in my opinion that you can very quickly get a tangible result. If you think software is changing pixels on the screen, this is close to that ideal. It’s also a part of software very close to the user, and it obviously has a big design aspect. It’s very easy to get started with, there are lots of resources, and it’s easy to land a job.

The bad parts of frontend basically all stem from the fact that everything has to run in a browser. It’s maybe not as bad as in the IE6 days, but you still have to fight browser compatibility bugs and limit yourself to the lowest common denominator in features. It also means JavaScript, the language that was famously invented in 10 days, and will plague us for the rest of eternity.

People have been building things that improve on JS and CSS (SCSS, Sass, TypeScript, CoffeeScript), and then use transpilers and bundlers to convert this into the lowest common denominator stuff suitable for browsers. This, combined with its immense ubiquity, has lead to a sprawling ecosystem of boilerplate spewing tools that changes continuously. The tool I used for the Angular gig doesn’t even exist anymore. To me the churn and boilerplate is what I despise most about the frontend ecosystem.

But the ugly part is that most of the work is pretty similar from the tech side. Every Wordpress theme is technically almost identical. These days JS apps are a bit more involved, but still, one CRM dashboard isn’t going to be that different from the next CRM dashboard. The appeal is probably more in the design aspect, working with customers.

Backend

As said, I started doing Wordpress. After that I worked at a company on modernizing their Perl backend, writing Python APIs and workers. I also did some hobby backend stuff in Clojure.

“Backend” is very broadly everything web related that’s on the server. I’d say the main thing is implementing HTTP endpoints that implement business logic and communicated with a database. It doesn’t have the visual aspect, but not being constrained to the browser grants a lot of freedom. You can use literally any programming language out there, and most of them have good web frameworks.

So sorta the good thing is that there are not a lot of bad things. There are bad parts but you can mostly avoid them.

If I had to pick a bad thing, I’d say deployment. Back in the day you’d just throw some PHP in your shared host LAMP stack and that’s it. Now everything has to be web scale and cloud native and what not. That brings with it headaches like eventually consistent databases that throw away your data, and apparently writing lots and lots of YAML to configure your fleet of containers, and then getting a €50k bill because you clicked the wrong button on AWS.

The ugly part is just never ending CRUD (create, read, update, delete) apps. Sure, there are interesting problems in backend but the bread and butter stuff is HTTP endpoints that create, read, update, and delete data from a database. Better get cozy with SQL. (or an ORM if you want to invite the scorn of the SQL experts)

Systems programming

At my current job I’m working on a SPICE simulator and a C++ RPC thing. I’ve also written some toy compilers and databases and stuff.

Ok this is an extremely nebulous definition, but like… everything that runs on a normal operating system and is not web related. It’s all the tools and services used by the backend people and more. Roughly speaking, if its leading characteristic is performance, it’s probably systems stuff.

The good is that this domain has lots of hard and interesting problems, with lots of room for implementing clever algorithms and data structures, and optimizing stuff for maximum performance. Like backend, you have a choice in what tools to use, but there is a strong bias towards the more low-level and high-performance ones. Expect lots of C/C++. And Rust if you hang out with the cool kids.

The bad is no doubt build systems. Being more low level exposes you more to operating system differences, so in a way it’s a bit similar to the frontend browser compatibility stuff, but much more unchanging. The build systems you’ll run into are pretty much Make, autoconf, and CMake. Always have been, always wil be, sorta. You’ll spend endless time making your software build on Linux, OSX, and in particular Windows, and then compiling all your dependencies on all these platforms.

The ugly is that there is never as much of the hard and interesting problem as you hoped, and lots of stuff around it. A database has a high performance storage part, but also a network API and config files. A compiler has interesting compiler bits, but also needs a library of bog standard functions. A simulator has an interesting mathematics bit, but also needs a library of models to work with. It’s no surprise that hobbyists tend to build game engines and then never build the game.

Desktop software

Had a job building a teleoperation platform in Qt. Also some hobby apps.

Briefly put, desktop software is like a mishmash of frontend and systems programming. The objectives are visual like frontend, but the environment is the OS, like systems programming. You can spend hours optimizing pixels, and days getting dependencies to build. Most UI toolkits are written in C/C++, with wrappers available for other languages. You can write GUI apps in Python for example, but then you have to ship Python to your users, which is a PITA. Also, the desktop is kinda getting taken over by the web. Lots of serious tasks can these days be done in web apps or Electron apps, and React-style development is usually more pleasant than imperative UI toolkits.

I guess game development also fits in here, which I don’t really have experience with. It’s also graphics + performance + low level, but has quite a different dynamic to it from what I’ve heard. It looks glamorous from the outside, but I would recommend to read some first hand accounts from game devs, because it can be gruesome and toxic.

Scientific computing

I have a degree in electrical engineering and IC design. Lots and lots of small experiments.

This one sits more between backend and systems programming, in that there are a diverse set of hard problems to solve, but you tend to do them in more high-level languages like Python, Julia, Matlab, or R. It also has a big data visualization aspect, but there isn’t as much design involved because it’s mostly tucked away in neat libraries. Usually it’s not so much a software job, but a science job that requires software as a means to an ends. Lots of this stuff is written by physicists who don’t necessarily have a software background.

I guess machine learning is also scientific computing? I know literally nothing about it, but people seem excited about it. Here too it’s probably the case that there is a small core of really interesting science, but also a lot of feeding data to an API (PyTorch etc.) and visualizing the results.

So the good is that you’re probably doing science and getting cool results. The bad is that you’re probably dealing with code written by scientists. The ugly is that it’s a lot of just cleaning up data and plotting it.

Embedded

Did some gigs involving microcontrollers, an internship in FPGA tools, and a metric ton of hobby projects involving Arduino’s and FPGAs.

I’d define embedded as software that runs without an operating system. I’ll also throw PLCs (industry automation) in this category, because they seem to kinda have the same properties and I don’t know a huge deal about them. The main objectives in embedded are resource usage and latency. (if you would not have these requirements, you could just run Linux and do whatever)

Certainly at the hobby level, embedded has the same immediate tangible results as frontend. You write code and the LED blinks or the robot moves, very rewarding. You’re also in complete control over your environment, you can build your own castle from scratch by poking registers, and justify the ugliest handwritten assembly because of latency and low power or whatever.

The bad is the tools. Outside of the happy lalaland of Arduino and embedded Rust, there are cases where you get some GCC fork in a hacked up Eclipse IDE and good luck. In FPGAs open source tools are even less common. For hobby stuff you can certainly get an ECP5 and use Yosys and Nextpnr or whatever, but chances are if you get a job as an FPGA developer you’ll be using proprietary tools by the FPGA vendor.

The set of languages available for embedded stuff are also pretty much C and more C for microcontrollers and Verilog or VHDL for FPGAs. There are of course cool languages like Rust and Amaranth HDL, but the industry hasn’t really picked up on that yet. No experience with PLCs but from what I’ve heard they also have a pretty limited set of languages and vendor tools.

The ugly is probably that in an industrial setting, you’re probably not building glamorous robots but maybe just some glue logic that reads out a sensor over SPI and if it goes above a certain value sends a message on the CAN bus.

Conclusion

Every branch of software development has rewarding parts, frustrating parts, and some amount of mundane drudgery. Some may suit you better than others. If you find a thing with only good parts, do share ;)

One thing that stands out that if you do it as a hobby you can avoid some of the bad and ugly parts. You can just pick up the hot new tools and do only the interesting bits. If you’re doing it purely as a hobby it’s also worth mentioning retrocomputing and programming as art. You can certainly have a ton of fun hacking Game Boy’s or marking art with code, but these are not typical career paths.

Inspired by a great video by Veritasium about analog computers and Andrew Carol’s Lego Antikythera mechanism I decided to try to build Sir William Thomson’s tide predicting machine out of Lego. Before I get into the details, here is a video of it running.

It uses 96 gears and a Lego Robot Inventor to drive the device and measure the hight of the output with an ultrasonic sensor. The motor angle and distance are then logged to the computer to plot the tide over time. This is super exciting! Compare real tide data form Hawaii to the data generated by my machine. Looks similar right!! Of course not all components are aligned perfectly, but there is definitely a striking similarity.

With all the pretty pictures out of the way, time to get into the making of.

As Veritasium explains in his video, predicting the tides was alway an important problem, but only after we had been blessed with the Fourier transform, did it become possible to decompose tidal data in frequency components, and then add these components back up to predict the tide. Except this was before digital computers existed, so Sir William Thomson invented an analog computer with gears, ropes, and pulleys to create the different frequency components and add them together.

So the idea is simple. Create gear trains that run at the correct frequencies, and then attach pulleys to them. One small problem, Lego gears only come in a small set of fixed sizes, so you can’t just make a 468/424 gear ratio by meshing a 468 and a 468 tooth gear, you have to build it up from the fixed gear ratios that you have. This turns out to be a bit of a challenge. To solved it I wrote a Julia script to find the optimal gear ratios.

To save me half of the problem, of decomposing tidal data into frequency components, I found that the Wikipedia page Theory of Tides contains a handy table with the frequencies of the components, and their magnitude at various locations. I only took the 5 biggest ones.

First I defined all the frequencies and normalized them and scaled them down. This is important because all these gears will have a ton of friction and backlash, so by introducing reduction both of these problems can be reduced a lot.

using Optim
M2 = 12.4206012
S2 = 12
N2 = 12.65834751
K1 = 23.93447213
O1 = 25.81933871

periods = 1 ./[M2, S2, N2, K1, O1] .* 12 ./ 24

Then I computed all the possible gear ratios you can make out of Lego gears. But I later found that not all combinations are easy to build, in fact some are downright impossible (12:16 for example). So I narrowed down the list to just 4 combinations: 8:24, 12:20, 12:28, 16:20. I found that these 4 ratios required almost the same amount of gears as using all combinations, but gives a result that is much easier to build. These can all be assembled on a normal grid of technic beams.

gears = [8, 12, 16, 20, 24, #=36, 40=#]
gearratios = Dict{Rational{Int64}, Vector{Tuple{Int64, Int64}}}()
for g1 = gears
    for g2 = gears
        if g1 < g2
            push!(get!(gearratios, g1//g2, []), (g1,g2))
        end
    end
end
delete!(gearratios, 12//16) # impossible
delete!(gearratios, 36//40) # huge
delete!(gearratios, 20//36) # wide gap, large
delete!(gearratios, 16//40) # wide gap, large
delete!(gearratios, 20//24) # wide gap
delete!(gearratios, 8//40)  # large
delete!(gearratios, 16//24) # large

grs = []
for gr = values(gearratios)
    if length(gr) == 1
        push!(grs, gr[1])
    else
        push!(grs, first(filter(x-> 8 ∉ x, gr)))
    end
end

Then I ran a simulated annealing algorithm to find the optimal gear ratios. I formulated the problem as a big product of all the gears, where I can optimize the exponents to vary the number of gears. Negative exponents just flip the gears around. I added a small factor for the required number of gears so it’d hopefully find an optimum that also doesn’t require 500 gears like my first attempts.

A final little trick I used to reduce the number of gear is that the neighbor function tries a random permutation that doesn’t exceed the hard limit on the total number of gears I set. Playing around with this a bit I got much better convergence to much more reasonable results. Note that the final result uses less than this limit! It just stops the optimizer from generating too much BS proposals.

x0 = zeros(length(gearratios))
lim = 20

function neighbor(x::AbstractArray, x_proposal::AbstractArray)
    while true
        for (idx, xval) = enumerate(x)
            val = rand((1, 0, -1))
            x_proposal[idx] = xval+val
        end
        sum(abs.(x_proposal)) < lim && break
    end
end

temperature(t) = 1/log(t/1000)

ngs = ones((length(periods),length(gearratios)))*lim
for (idx, ratio) = enumerate(periods)
    cost(x) = abs(prod(keys(gearratios).^x)/ratio-1)+sum(abs.(x))/1e4
    res = optimize(cost, x0, SimulatedAnnealing(;neighbor, temperature), Optim.Options(iterations = 10000000))
    mg = Optim.minimizer(res)
    mm = Optim.minimum(res)
    ng = sum(abs.(Optim.minimizer(res)))
    ngs[idx,:] = mg
    println()
    println(ratio)
    println(mm)
    println(mg)
    println(ng)
end

And then all that is left to do is compute the bill of materials and verify the error margin looks fine.

num = sum(abs.(ngs); dims=1)
thebom = Dict(zip(gears, zeros(Int64, length(gears))))
for (num, (g1, g2)) = zip(num, grs)
    thebom[g1] += num
    thebom[g2] += num
end
thebom

display([permutedims(grs); ngs])
display(thebom)
display(sum(values(thebom)))
res = map(x->float(prod(keys(gearratios).^x)), eachrow(ngs))
display(res./periods)
display(res.-periods)

Pretty good!

6×4 Matrix{Any}:
   (12, 20)   (12, 24)   (16, 20)    (8, 24)
  1.0        1.0        9.0         0.0
  0.0        3.0        0.0         1.0
 -1.0        0.0        2.0         3.0
 11.0        0.0        2.0        -2.0
  9.0        0.0        2.0        -1.0
Dict{Int64, Int64} with 5 entries:
  16 => 15
  20 => 37
  12 => 26
  24 => 11
  8  => 7
96
5-element Vector{Float64}:
 1.0002389240748448
 1.0
 1.0001657291851853
 1.0003226141581112
 0.9991658743311826
5-element Vector{Float64}:
  9.618055961925498e-6
  0.0
  6.546240931305791e-6
  6.739529419302198e-6
 -1.6153118369648806e-5

Now one last simple thing I need to calculate is the length of the levers to match the amplitude of the frequency components. Looking at the table on Wikipedia I picked Hawaii because the differences in magnitude are relatively small. That means I will not have to make a lever of 20 studs and one of 0.2 studs.

Normalizing the amplitudes to 2 studs for the biggest component, resulted in a smallest component of a bit shy of 0.5 studs. I rounded these distances to 0.5 studs, which resulted in levers of 0.5, 1, 1.5, and 2 studs. Very reasonable! I experimented with variable length levers, but they were too flimsy.

5-element Vector{Float64}:
 2.0
 0.7999999999999999
 0.3826086956521739
 1.452173913043478
 0.7999999999999999

Next, I took my bill of materials to Bricklink, robbed some poor seller of literally all of their gears (I had to supplement some types from my existing collection because they didn’t have enough!), and two weeks later I could start building.

Packing all these gears as tightly as possible was a real challenge. The diagonal ones can just zig-zag around,but the orthogonal ones require a sort of snake pattern that’s extremely tight. Each axle has 4 positions and is surrounded by 5 other axles, meaning the smaller gears have to double up. But after two days of trying, I was successful. The first one took forever, an then the rest was done in a matter of hours.

After that I stacked them on top of each other, and hooked them up with a long driveshaft, attached levers and pulleys, and the rest is history.

If you’re looking at the video and wondering which gear train corresponds with which frequency components, but don’t want to go figure it out from pictures of gears, here is an annotated picture. Green are the semi-diurnal ones, red the diurnal ones, which are twice as slow.

I’m currently developing Mosaic, a schematic editor built on web technology. In the process of adding online collaboration, I wrote a library to help me manage and synchronize state. The library is called Hipflask.

In my previous post I wrote about different strategies for dealing with conflicts when multiple users are editing the same object. In this post I want to focus more on the library I wrote to manage state.

Hipflask is basically a ClojureScript/Reagent atom backed by PouchDB.

Mosaic is written in Reagent, a ClojureScript React wrapper. The fundamental building block of Reagent is the Ratom (Reagent atom), a data structure that allows atomic updates by applying functions to its state. It then monitors state updates, and redraws components that depend on the Ratom. It’s quite elegant.

The semantics of a Clojure atom (after which ClojureScript and Reagent atoms are modelled) is that there is a compare-and-set! function that atomically updates the state if the expected old state matches the current state. swap! is a wrapper over that which repeatedly applies a function until it succeeds.

The neat thing is that CouchDB update semantics are exactly like compare-and-set!. You update a document, and if the revision matches it succeeds, if there is a conflict the update is rejected. You can layer swap! behavior on top of that which applies a function to a document repeatedly until there is no conflict.

So what you get if you combine PouchDB with a Ratom is that you can apply functions to your state, which will resolve database conflicts and rerender your app. The PouchDB atom even listens for database changes, so that remote changes will update the cache and components in your app that depend on it.

Basically it transparently applies functions to the database and updates your UI on database changes.

One thing that sets Hipflask apart from my previous CouchDB atom is that it manages a prefix of keys. This plays well with sharding, so you can have a PouchDB atom containing all foo:* documents. Worth noting is that it’s only atomic per document, and assumes the first argument to your update function is a key or collection of keys that will be updated.

You can use Hipflask in two main ways. The first way is offline first. The atom is backed by a local PouchDB which can by replicated to a central database. This way the app remains fully functional offline but replication conflicts are not handled automatically. You can also make it talk directly to a central CouchDB, in which case it does not work offline but conflicts in the central database are resolved automatically.

As a demo of the library, I made a Global Cookie Clicker app. It’s an offline-first collaborative cookie clicker. You can click cookies offline, and they are then synchronized to a central CouchDB. The entire code is less than 70 lines.

It basically defines a PouchDB atom

(def db (hf/pouchdb "cookies"))
(def rginger (r/atom {}))
(def ginger (hf/pouch-atom db "gingerbread" rginger))

And then when you click the button it calls this function that increments your cookie document.

(defn make-cookie [a n _]
  (swap! a update-in [(str (.-group a) hf/sep me) :cookies] #(+ % n)))

This library has been very useful in building Mosaic, and I hope it is useful to others as well.