Wishful Coding

Didn't you ever wish your
computer understood you?


I came across a random tweet, found a Wikipedia page and bumped into some smart people and long story short, apparently compression is equivalent to general intelligence.

So, how do you build ChatGPT with data compression? What if you compress a large corpus of text to build up the encoding table, then you compress your prompt and append some random data and decompress the random data, and hope it decompresses to something sensible.

It feels vaguely similar to diffusion, but what do I know. Look, this is just a dumb idea, let’s just see what happens ok? Well, here is my progress so far. It’s kind of whack but it’s hilarious to me that it produces something resembling words.

import nltk
import lzma
import random

my_filters = [
    {"id": lzma.FILTER_LZMA2, "preset": 9 | lzma.PRESET_EXTREME},
lzc = lzma.LZMACompressor(lzma.FORMAT_RAW, filters=my_filters)

corp = nltk.corpus.reuters.raw().encode()
out1 = lzc.compress(corp)

corp = ' '.join(nltk.corpus.brown.words()).encode()
out2 = lzc.compress(corp)

corp = nltk.corpus.gutenberg.raw().encode()
out3 = lzc.compress(corp)

out_end = lzc.flush()

lzd = lzma.LZMADecompressor(lzma.FORMAT_RAW, filters=my_filters)

# mess around to avoid LZMAError: Corrupt input data
# insert prompt????

Here are a few runs. Note how the start is always , and tri, usually completing it into some word. Are we doing some primitive accidental “prompting” or just flushing the buffer? Either way, not bad for mere seconds of “training”!

$ python train.py 
, and triof billioerse,
ht and see th,
Thy smile, in to be happy,
Over tout as aThy smile;t as aThyrged in
ent, foldehe snoion since how long,
my roomr? Is ic books 

$ python train.py 
, and triompact, sca,
Take deepcky fouy vitaliz  bodiehow there i,
Nor drummiwisibly wile of the-ations, dutway?
Yet ld woman'okesmanall whoy slow bekesmanalle me 

$ python train.py 
, and tri billions of the boftier, faie no acqutory's dazzd haOr that thpages:
(Sometimeseathe ihern, Sounte, fld Turkey n one,
Worlseathe Border Minstrelsy,ine, New-Ene Queen.

'Thelicate l

$ python train.py 
, and tri, sleepinlke babes  bent,
Forhis fair n!
By thea mystic strangehe gifts ofhe body aering
 t: I haue a lugs whipageantuperb-fnz

$ python train.py 
, and triions of b--n the gra, with
e open the countless buAnd bid theng;
Billi  toward you.i ally undya Songs

To f Death, istas of was mar to be UY 9,30
Pepijn de Vos