SDRs and Compression Algorithms

by Drannex42 (t. Macleod Sawyer) :: Jan 16, 2022

Human brains are equivalent to a zip archive/tarball, they use Sparse Distributed Representations (SDR) to represent values across neurons and minimize data using similar processes such as overlap (how things are different) and union (key table duplication, or the representation of multiple ideas that are similar with one linked value) properties.

The concept is simple, you can have an example sentence (or string as they are called in programming) that says "I am a woman, a woman is me, I am existing and not existing for I am both on your screen and not on your screen."

How does a nominal compression algorithm work? Well, you take the words and statements that have been repeated and assign a number such as:

# Statement Times Said
1 "I am" 3 times
2 "a woman" 2 times
3 "existing" 2 times
4 "on your screen" 2 times
5 "not" 2 times

Now we have a key table of values (key: "value")! We can now rewrite the original statement like this:

 "1 2, 2 is me, 1 3 5 3 for 1 both 4 5 4"

Let's assume that every letter and space is a size of 1 byte, that means the original statement has a size of 112bytes while the second version only has 38! That means you have saved 69% of your drive space just by doing a simple find and replace!

Now you could do that all the way down to part of words, statements, and more and further improve this rather simple compression algorithm.

The above uses the principle of a 'Union' which is found in what neuroscience calls SDR's or Sparse Distributed Representations. These can be attributed to images, colors, experiences, smells, and more. You have a memory that consists of key value tables to detect and apply replacements within that memory to ensure that you aren't having to remember things twice such as the color red a thousand times, instead you just have '1: RED!" saved and see a lot of 1's linking to that color.

Then you have the principle of 'Overlap' which is how things are different. These things relate to Union in similar ways, for example: You see red, but its a different new lighter shade of red, you don't remember ever seeing this red so it becomes "3: 1+lighter" because you know its red, not pink (that is coded as '2: Pink'), but while it it lighter it is not light enough to be pink.

This is roughly how compression algorithms and SDR's work in a nutshell. We humans have this strange tendency to recreate or mirror our own lives unknowingly in the things that we make whether that be emotions, dreams, or even how our own natural brain chemistry works! We do it over and over and over again. These are the reasons why in the future we might be able to create a new form of living being, or even one where we can upload and replicate our minds (that's called the field of mindcloning - a real thing!) and be able to quickly travel the stars without having to worry about how long it will take or if our 'body' will live long enough to get to another planet! We would just be able to upload and download into a new body in hours instead of centuries of travel.

We take a lot of inspiration from nature, but we can experiment and tinker a lot faster than it can naturally. We've gone from the game of Pong to this entire cyber connected and graphically intensive world in less than fifty years, 99.9% of the population never having or seeing electricity to nearly everyone having it in less than a hundred years!

What will be possible in a hundred years? a thousand years? I'm looking forward to being alive to find out!


This was first posted on my tumblr.