Imagine you are the maintainer of an open source FPGA toolchain such as Apicula and you want to make sure your code works. The industry standard for making sure software works is to set up continuous integration that tests your code for every change you make.
But what do you test? We are reverse engineering the inner workings of complicated devices, and if you’re slightly wrong you can generate a good looking bitstream that just doesn’t work. So of course during development you test your bitstreams on real hardware to make sure they work. And on top of that we do set up CI, but it can only test that we can generate a bitstream.
This is not idle thought, just yesterday we had one FPGA silently fail due to improvements in another. It is very hard to catch these regressions reliably without constantly manually testing every single devices, which is completely infeasible.
For a long time I’ve dreamed of setting up a raspberry pi with a bunch of FPGAs and somehow running automated tests on it. But it always felt like a huge ordeal to really put everything together. Running and maintaining a raspberry pi with a bunch of FPGAs, writing useful self-tests that automatically verify correct behaviour, triggering a test run, downloading or generating bitstreams, programming the FPGAs, obtaining and verifying the results, and communicating that back to Github.
But this week I looked at the problem again and realized all the pieces had falling into place to make this not only feasible but almost trivial.
- I am already running a Raspberry Pi with Home Assistant on my very own Mini-ITX motherboard: Sentinel Core
- For running LLMs on my pi, I built custom Docker addons for Home Assistant
- To build those Docker containers, I’m using a Github self-hosted runner on my VPS
- We have femto RISC-V UART examples now based on Bruno Levy’s FPGA tutorials
So the plan is simple, plug some FPGAs into Home Assistant, run a self-hosted runner as an addon, and add a CI task that uploads an example and verifies its UART output against a reference.
For the addon I just forked an existing addon and added USB access and openFPGALoader: home-assistant-github-runner-add-on
For the CI side, it basically just adds a self-hosted step after our current bitstream generation that fetches the bitstream artifacts, uploads them, and diffs the UART output: Add hardware-in-the-loop CI test infrastructure
I’m excited to bring this new level of reliability to Apicula, and curious if other projects could do something similar.
