Wishful Coding

Didn't you ever wish your computer understood you?

AVR pseudo-DMA

I’ve been thinking about making my own 8-bit retro hand-held console, not unlike a GameBoy Color. 8-bit microcontrollers and buttons are easy to come by, memory, sound and a display not so much.

The Atmega2560 used in the Arduino Mega has support for external memory, which is nice. This allows up to 64kb of RAM, and even more if you implement bank switching.

The display is an unsolved problem for me so far. Sure, there are dozens of display shields and breakouts, but pushing pixels over a serial bus with an AVR is just too slow. If you just make the Arduino sit in a busy loop sending bytes over SPI at top speed, it takes hundreds of milliseconds to update the whole screen.

The proper way to do it seems to be to have a framebuffer that you DMA to the screen controller, while the CPU is busy making the next frame. Except the only chips I found that support DMA can hardly be considered retro, and often not even 8-bit. We’re talking 32-bit, 100Mhz chips like AVR XMega, dsPIC or ARM Cortex-M3 here. While the GameBoy does all of that at 4Mhz. I’m not sure what kind of black magic that is.

In this moment of despair, I came up with a way to do something like DMA on the Arduino Mega. I figured that with a couple of diodes I could transfer data from a RAM chip to the screen.

Normally you’d connect both the RAM and the screen to the SPI lines and give them a chip-select line each. What I did is disconnect the MISO of the screen and connect the MISO of the RAM to the MOSI of the screen. This way, if you lower both CS lines and send a zero, a byte is transferred from the RAM to the screen.

Pseudo-DMA Schematic

The hack that makes this work combines code from Adafruit_ILI9341 and Adafruit_FRAM_SPI, which results in the following glorious disaster

void framDMA() {
  // set up the display
  spi_begin();
  tft.setAddrWindow(0, 0, ILI9341_TFTWIDTH-1, ILI9341_TFTHEIGHT-1);
  // set up the RAM
  digitalWrite(fram._cs, LOW);
  fram.SPItransfer(OPCODE_READ);
  fram.SPItransfer(0);
  fram.SPItransfer(0);

  digitalWrite(tft._dc, HIGH);
  digitalWrite(tft._cs, LOW);
  // at this point BOTH the display and FRAM are active
  // Send zero's
  for(int i=0; i<ILI9341_TFTWIDTH; i++) {
    for(int j=0; j<ILI9341_TFTHEIGHT; j++) {
      SPDR = 0;
      while(!(SPSR & _BV(SPIF)));
      SPDR = 0;
      while(!(SPSR & _BV(SPIF)));
    }
  }
  // release the cs lines
  digitalWrite(tft._cs, HIGH);
  digitalWrite(fram._cs, HIGH);
  spi_end();
}

I think it’s a nice hack, and if you rewrite the loop as an ISR it could definitely free up CPU cycles. But the actual updating of the screen is still too slow for any sort of annimations.

Pepijn de Vos

Breaking the Respondus Lockdown Browser

At the University of Twente there was a pilot program for digital testing using Pearson MyLabsPlus in the Respondus Lockdown Browser. This browser is supposed to stop students from using their laptops for other things besides the test.

It does this by minimizing all applications, disabling a bunch of things and maximizing itself. It also forces you to shut down software such as Skype and Steam.

After the test, a friend mentioned he had received a notification from WhatsApp web during the test. Surely enough, I could easily reproduce this with a few lines of JS using the notification API.

This got me thinking if you could show other things on top of Respondus Lockdown Browser. I literally copied some code for making a transparent always-on-top window in Java, added a Thread.sleep (so it would open after I started Respondus) and I was in business.

A little more copy-pasting and a dozen lines of Swing and I had a transparent WolframAplha client sitting on top of my lockdown browser.

WolframAlpha client on top of Respondus

I’ve become convinced Respondus is selling snake oil, pure snake oil. To drive the point home, another friend demonstrated you could run Respndus Lockdown Browser in a virtual machine.

Respondus in a VM

Time-line of events:

  • 10 June 2016
    • 13:45 - 15:45 Math exam
    • Intensive hacking
    • 23:08 University of Twente and Respondus notified
  • 12 June 2016, UT says they will bring it up with respondus

Respondus never even responded.

Below is the result of my copy-pasting. I didn’t even bother to rename anything.

import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import static java.awt.GraphicsDevice.WindowTranslucency.*;
import java.awt.image.BufferedImage;
import javax.imageio.ImageIO;
import com.wolfram.alpha.WAEngine;
import com.wolfram.alpha.WAException;
import com.wolfram.alpha.WAImage;
import com.wolfram.alpha.WAPod;
import com.wolfram.alpha.WAQuery;
import com.wolfram.alpha.WAQueryResult;
import com.wolfram.alpha.WASubpod;

public class TranslucentWindowDemo extends JFrame {
    private static String appid = "wolfram app id";
    private WAEngine engine = new WAEngine();
    private JLabel picLabel = new JLabel();

    public void requestWA(String input) {
        WAQuery query = engine.createQuery();
        query.setInput(input);
        try {
            // For educational purposes, print out the URL we are about to send:
            System.out.println("Query URL:");
            System.out.println(engine.toURL(query));
            System.out.println("");
            
            // This sends the URL to the Wolfram|Alpha server, gets the XML result
            // and parses it into an object hierarchy held by the WAQueryResult object.
            WAQueryResult queryResult = engine.performQuery(query);
            
            if (queryResult.isError()) {
                System.out.println("Query error");
                System.out.println("  error code: " + queryResult.getErrorCode());
                System.out.println("  error message: " + queryResult.getErrorMessage());
            } else if (!queryResult.isSuccess()) {
                System.out.println("Query was not understood; no results available.");
            } else {
                // Got a result.
                System.out.println("Successful query. Pods follow:\n");
                for (WAPod pod : queryResult.getPods()) {
                    if (!pod.isError() && !pod.getTitle().equals("Input") && !pod.getTitle().equals("Input interpretation")) {
                        System.out.println(pod.getTitle());
                        System.out.println("------------");
                        for (WASubpod subpod : pod.getSubpods()) {
                            for (Object element : subpod.getContents()) {
                                if (element instanceof WAImage) {
                                    WAImage waimg = (WAImage)element;
                                    waimg.acquireImage();
                                    System.out.println(waimg.getFile());
                                    System.out.println("");
                                    BufferedImage myPicture = ImageIO.read(waimg.getFile());
                                    picLabel.setIcon(new ImageIcon(myPicture));
                                    SwingUtilities.invokeLater(new Runnable() {
                                      public void run() {
                                        revalidate();
                                        repaint();
                                    }});
                                    return;
                                }
                            }
                        }
                        System.out.println("");
                    }
                }
                // We ignored many other types of Wolfram|Alpha output, such as warnings, assumptions, etc.
                // These can be obtained by methods of WAQueryResult or objects deeper in the hierarchy.
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
      
    public TranslucentWindowDemo() {
        super("TranslucentWindow");
        setUndecorated(true);
        getRootPane().putClientProperty("Window.shadow", Boolean.FALSE);
        getContentPane().setBackground(Color.WHITE);

        setLayout(new FlowLayout());

        engine.setAppID(appid);
        engine.addFormat("image");
        //requestWA("integrate x^2");

        addMouseListener(new MouseAdapter() {
            @Override
            public void mouseEntered(MouseEvent me) {
                setOpacity(0.55f);
            }
            @Override
            public void mouseExited(MouseEvent me) {
                setOpacity(0.1f);
            }
        });

        setSize(300,200);
        setLocationRelativeTo(null);
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);

        //Add a sample button.
        final JButton button = new JButton("Solve");
        final JTextField textfield = new JTextField(20);
        ActionListener submitListener = new ActionListener() {
          public void actionPerformed(ActionEvent ae){
            requestWA(textfield.getText());
          }
        };
        button.addActionListener(submitListener);
        textfield.addActionListener(submitListener);
        //add(button);
        add(textfield);
        add(picLabel);
    }

    public static void main(String[] args) {
        try {
        Thread.sleep(10000);
        } catch (InterruptedException e){}
        // Determine if the GraphicsDevice supports translucency.
        GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
        GraphicsDevice gd = ge.getDefaultScreenDevice();
	final Rectangle rect = gd.getDefaultConfiguration().getBounds();

        //If translucent windows aren't supported, exit.
        if (!gd.isWindowTranslucencySupported(TRANSLUCENT)) {
            System.err.println(
                "Translucency is not supported");
                System.exit(0);
        }
        
        //JFrame.setDefaultLookAndFeelDecorated(true);

        // Create the GUI on the event-dispatching thread
        SwingUtilities.invokeLater(new Runnable() {
            @Override
            public void run() {
                TranslucentWindowDemo tw = new TranslucentWindowDemo();

                // Set the window to 55% opaque (45% translucent).
                tw.setOpacity(0.1f);
                tw.setAlwaysOnTop(true);

                // Display the window.
                //tw.pack();
		tw.setLocation((int)rect.getMaxX()-400,(int)rect.getMaxY()-300);
                tw.setVisible(true);
            }
        });
    }
}

I’m not sure it’s wise to release a “weaponized” version of my “exploit”, but I think anyone who can figure out how to compile and run this code and its dependencies knows enough to do the copy-pasting.

Pepijn de Vos

The Church of Science

The church of science

This, ladies and gentleman, is the University of Twente. To the left you see a church of god, submerged in water, that I see as a representation of the opinion of science towards the religions of god. To the right you see a bell tower that I affectionately call the church of science, representing the beliefs of scientists.

This desire to drown a religion should sound familiar to you. It is a common trait to most religions to make a claim to the truth and declare all other religions false.

To call science a religion might raise some eyebrows, so let me explain. There are three concepts that I will talk about:

The first is god. God by itself is not true or false. It is simply the concept of an omnipotent being. This concept is internally consistent and valid as a concept.

The second is math. Like god, math is simply a concept. It is a set of rules and symbols that are internally consistent.

The final one is observations. It is not until you apply these concepts to observations that they become meaningful. It is when you say there are two birds and god created them, that you apply these concepts to observations.

All of math, god and observations are true in themselves. It is when you say that your observations are explained by your concepts that they become models that can be correct and accurate or not.

You could say that a bird is so perfect it could only have been designed by an omnipotent being. You could say that because the field of a point charge decays with one over the radius squared exactly to one part in a billion, the electric field must be explained by math.

But they are still just concepts applied to observations. Models that help us think about our universe. But the map is not the territory, which for me means that a model should not be judged by its accuracy, but by how useful it is to think about a problem.

Ancient Greek medicine, which centered about balancing the four bodily humours, often lead to treatments that would do the right thing for what we currently consider the wrong reason. It must have been a very useful model that saved countless lives up until after the middle ages despite its inaccuracy.

Maxwell’s equations might describe the whole domain of electromagnetism very accurately, but using it to design an electric circuit is impractical. The equations of Ohm and Kirchhoff will serve you better.

One day science might make a map that is the territory (they seem to be trying very hard at least), but it will not help you think about ethical and moral dilemma or the meaning of life. The stories told in your spiritual book of choice are much more practical for those kinds of problems.

Pepijn de Vos