QUANTIFYING IMAGES
All data processed by machines is stored as binary code—0s and 1s, organized into bytes. The same applies to digital images in our galleries; they are essentially arrays of numbers representing pixels. While we see these images as cherished memories, for a machine, they are merely thousands of pixels to be processed.
In this experiment, I encoded an image from my digital gallery by extracting its RGB values. I used p5.js to create the code for the image encoding and fed the resulting RGB values to ChatGPT to see whether it could visualize the original image just by using the given values. The objective is to quantify something that cannot be quantified (memories). For me, the image represents memories of my childhood; for machines, they are just numerical data represented in RGB formats.
I also did additional experiments, in which I matched the numbers with words extracted from the values of ASCII characters. The outputs are more poetic, consisting not just of numbers but words that creates a poem.
EXTRACTING RGB VALUES
Using p5js sketch code, I extracted the RGB pixel values of an image that marks my first ballet performance when I was a kid. The sketch code allows me to download the text file as well and give me the total characters count from the file.
I was surprised to find that even a compressed 90x70-pixel image—similar in size to a favicon—produced over 50,000 characters.
MANIPULATING RGB VALUES
The collected RGB values are then manipulated with ASCII characters to create poems from numerical values. When any pixels from the RGB values match with the text given, it will replace the text with the numbers. The diagram below shows an example of how the RGB values are match with given text.
By inputting my own descriptions of the image as an input text, the text will be converted into ASCII value sums and tries to pair with the RGB value pixel. This text will then be or not be integrated into the RGB values data, depending on whether it matches with the RGB values.
As the given texts matches with the RGB values of an image inputted, this will create an array of pixel values filled with texts as well, creating poems of digital memory with numbers.
FEEDING TO CHATGPT
These manipulated numbers and texts are then fed to ChatGPT to see how it reconstructs the image based on its "understanding.” We can see that it's not picture perfect as the original image, however, the machines are able to create this image with just the values given.
This reflects how images perceived by machines are just numerical data that can be processed. No matter how we think of this image as a precious memory, its understanding are just merely number of values to be analyzed.
Hover over the image
