That's one creative mofo.
The amplifying changes isn't new, it's used in computer graphics all the time, but using just an rgb images for input and measuring such subtle changes is amazing.
The sound stuff is amazingly creative. Crazy to turn visual input into audio so accurately.