Wishful Coding

Imagine you are the maintainer of an open source FPGA toolchain such as Apicula and you want to make sure your code works. The industry standard for making sure software works is to set up continuous integration that tests your code for every change you make.

But what do you test? We are reverse engineering the inner workings of complicated devices, and if you’re slightly wrong you can generate a good looking bitstream that just doesn’t work. So of course during development you test your bitstreams on real hardware to make sure they work. And on top of that we do set up CI, but it can only test that we can generate a bitstream.

This is not idle thought, just yesterday we had one FPGA silently fail due to improvements in another. It is very hard to catch these regressions reliably without constantly manually testing every single devices, which is completely infeasible.

For a long time I’ve dreamed of setting up a raspberry pi with a bunch of FPGAs and somehow running automated tests on it. But it always felt like a huge ordeal to really put everything together. Running and maintaining a raspberry pi with a bunch of FPGAs, writing useful self-tests that automatically verify correct behaviour, triggering a test run, downloading or generating bitstreams, programming the FPGAs, obtaining and verifying the results, and communicating that back to Github.

But this week I looked at the problem again and realized all the pieces had falling into place to make this not only feasible but almost trivial.

1. I am already running a Raspberry Pi with Home Assistant on my very own Mini-ITX motherboard: Sentinel Core
2. For running LLMs on my pi, I built custom Docker addons for Home Assistant
3. To build those Docker containers, I’m using a Github self-hosted runner on my VPS
4. We have femto RISC-V UART examples now based on Bruno Levy’s FPGA tutorials

So the plan is simple, plug some FPGAs into Home Assistant, run a self-hosted runner as an addon, and add a CI task that uploads an example and verifies its UART output against a reference.

For the addon I just forked an existing addon and added USB access and openFPGALoader: home-assistant-github-runner-add-on

For the CI side, it basically just adds a self-hosted step after our current bitstream generation that fetches the bitstream artifacts, uploads them, and diffs the UART output: Add hardware-in-the-loop CI test infrastructure

I’m excited to bring this new level of reliability to Apicula, and curious if other projects could do something similar.

It seems that people on my part of Twitter, in particular the rationalists, are very into Newcomb’s problem, probably because their guru wrote about it, so they all believe you should pick one box.

So I thought it would be a fun idea to take two boxes and some money to Treeweek 2 and try to actually play Newcombs Problem.

Rules of Byzantine Mini Newcomb’s problem

This is a thought experiment made real, as a thought experiment about what it means to really play this game. What’s up with the name? Mini: I am not made of money so we are playing at reduced stakes and on a donation basis. Byzantine: How do you tell Omega from a charlatan? How reliable are my predictions really? Are you really precommitted? If I know that you know that I know that you know that…

The Rules

• Think carefully about the nature of the game and your strategy, this is the most important step!!
• Give me your (nick)name, twitter handle, and a minimum donation of €10
• I give this information to my reliable predictor and prepare the boxes accordingly.
• You will be offered two boxes, a transparent one containing €10 and an opaque one that contains either €100 (if I predict that you will only choose the opaque box), or €0 (if I predict that you will take both boxes).
• You take any money from your chosen box(es) and reflect on what happened. Any money in any remaining boxes goes back to the bank.

Byrules

To keep it fun for everyone:

• Don’t donate more than you’re willing to lose.
• Officially you can only play once, but if you ask nicely maybe you can go again.
• Anyone can choose to join the “ethical review board” and I’ll explain my whole spiel but then you can’t play or leak info.
• At the end of the week all will be revealed, and if you wrote down your strategy we can compare notes.

My reasoning behind all of this

First of all, why “mini byzantine”? I would argue it is the only version that can be played.

First of all. There is no scenario where I trust someone who claims to be an alien in posession of millions of earthly USD and giving it to me. In the real world you’re always dealing with the question of who is this guy, why is he giving me money, and how can he possibly know my choice?

There are only two types of scenarios where you are offered large sums of money: scams and games of chance like a lottery or casino. And both of them make money rather than lose it, by asking for a fee.

You could run this game like a scam or game of chance where there is just a vanishingly small chance there is anything in the opaque box, but that’s not really in the spirit of the problem is it?

Although you could argue that a predictor that reliably predicts two boxes could be argued to be a logical if not ethical solution to the problem. “this is obviously a scam so I should take two boxes” as a sort of self fulfilling prophecy.

So lets impose that this is a zero sum game where I’m neither scamming you out of your money or just giving away money for free, which is the only fair and reasonable way to play.

The introduction of a donation actually meaningfully alters the game under this constraint.

Your donation can be seen as the expected value you place in this game. If you donate 10 euro you obviously place low odds on there being anything in the opaque box, and should logically take two boxes to not lose money. While if you donate 100 euro you obviously expect there to be money in the opaque box that you intend to take. These are the two stable equilibrium.

If you donate 110 you expect to outsmart the predictor which is irrational to singal, and if you donate something inbetween you’re not sure what to expect. So the question is, if your expected value/donation is 50 euro, what should the prediction be?

A reasonable choice in this case depends on the past donations. If people keep donating 50 euro I can’t just keep paying out 100, so we need to look at past donations and payouts compute our account balance and historical expected value. So in order for a logical prediction to be one box, it needs to be higher than or equal to the historical average payout, and the current account balance needs to actually be enough to fill the boxes. That can be easily implemented in an excel spreadsheet, which I did.

I’m really curious if we’ll just get stuck on the 10 euro equilibrium or a trickle of higher donations and mispredictions will lower the expected value low enough for a 100 euro payout to happen. Or will there be people with sufficient trust and money who have come to the same conclusion as me.

My predictor

I put the following equation in a spreadsheet that checks if your donation is more than the average payout, and the current account balance allows a payout.

=IF(AND(C2>AVERAGE($E1:E$2), SUM($C$2:C2)-SUM($E1:E$2)>=110), "one box", "two box")

Results

Based on the conversations I’ve had and the number of actual games played (0), we can conclude:

The only winning move is not to play.

I have recently become a dad, which is sometimes amazing, and sometimes suffering, such as when the baby is in a fussy mood and refusing to sleep. Some friends were very enthusiastic about a $1.5k smart crib that makes white noise and rocks your baby to sleep.

You know what also makes noise, rocks your baby to sleep, and doesn’t cost $1.5k? That’s right, this Lego model. This is already by far my most useful Lego creation, and we’re using it regularly to great effect.

At the core of this creation are two bogies, driven by Lego motors, strapped to the wheel of a stroller with rubber bands. Each bogie is driven by two motors for a total of 4 driven wheels and 4 motors for maximum traction and power.

You could honestly get away with two motors and maybe gear them down 12:20. This design has excess torque and velocity and is limited by traction. The traction is provided by the rubber bands, and kept straight with coaster wheels.

This particular design uses parts from the Lego Mindstorms Robot Inventor kit, which has been discontinued. It should however be possible to construct a similar model from Powered Up motors using the Technic Hub.

ps: bugasnoo is obviously from buggy and snooze and not from any similar sounding trademarks ;)
pps: I have considered turning this into a product but I’d need a cofounder who’s more into Industrial Design Engineering and business.