Poised at the top of a mild slope, a lone, intrepid sledder pauses to adjust his hat before pushing off into the unknown. It’s an easy ride at first, a few jolts here and there but some pleasant, wintry scenery overall. A light breeze tosses his scarf behind him in a flash of red as his velocity increases. Poor sucker. Before long he’s hanging on for dear life on the ride of his life, whizzing over increasingly large jumps and crashing down onto steeper and steeper slopes, zipping around backwards and upside down with only the occasional sighting of another rider on a parallel track to provide any comfort. There’s something about that other rider. Is it adrenaline-fueled infatuation? Is it love at first sight? He doesn’t know, but if he makes it, he’s tracking down that green-scarfed fellow daredevil, if nothing else just to ask, “Did you hear the first movement of Beethoven’s Fifth coming from somewhere, or was that just me?”
It’s not that unlikely this year. There’s Beethoven in the air, or at least in every concert hall, in this the 250th anniversary season of Beethoven’s birth. But let’s back up for a second. The video above, Line Riders — Beethoven’s Fifth, was built via the simple, goofy Line Rider computer game where players build tracks for little animated stick figures to sled down. Somewhere along the way, synchronizing the sled tracks to music became popular, leading to wildly intricate tracks set to all sorts of soundtracks, including famous works in the classical music canon like Beethoven’s Fifth, In the Hall of the Mountain King, “Dance of the Sugarplum Fairy” and more.
These videos and classical music visualizers have attracted hundreds of millions of views. DoodleChaos runs the YouTube channel with the most-watched Line Rider videos, with many reaching into the millions, while Californian resident and former computer programmer Stephen Malinowski’s visualizers alone have attracted more than 170 million views alone.
Why the fascination? Scientists have long documented that humans process visual information faster and more clearly than aural information, and the same part of your brain that processes musical rhythm also controls your sense of timed movements — motions like dancing, walking, bicycling, and even organizing a calendar. Our brains appreciate synchronization between how we move and what we hear, and there’s an innate pleasure in watching animations synchronized with rhythms. It simply feels right. Also, the sledding Line Riders figures are funny in a perverse “wow that would definitely kill someone” kind of way.
Malinowski studied music theory and composition in college and has been working on synchronizing graphic scores to music since the 1970s. He’s been at it full time since retiring from computer programming in 2010 and has developed visuals for the Alexander Quartet, Björk, numerous orchestras, and educational institutions. “The way I look at it, I’m doing something which will eventually be an art form,” Malinowski said. “What I’m doing now is going in that direction. I feel like I’m laying down some tools.” In honor of Beethoven’s 250th, has created visualizers for the entirety of the composer’s string quartets — more than eight hours of music — in collaboration with the Alexander Quartet.
Watching Malinowski’s Beethoven visualizations is enchanting. Little shapes and squiggles track the pitch and duration of the different instruments while implying additional information in the way they flit around the screen. “Every expressive or structural thing in music that I’m aware of I’m trying to visualize,” Malinowski said. “I started working on phrasing a couple years ago, moving pitches as groups.” He’s an able score reader himself, but he recognizes that such documents are impenetrable for people who don’t read music and wanted to build something to help listeners visually process the information. Viewing his work really feels like watching a Beethoven score in real time, especially given that the instruments are stacked vertically, which suggests a typical score. Focusing on the cello squiggles draws the ear to the lower parts of the score, while glancing the viola line helps your ears identify and bring out that particular instrument’s sound.
Humanity has a long history of attempting to draw scientific lines between color and sound, but you don’t have to be synesthetic to appreciate a good Mickey Mouse moment, or a moment when music synchs perfectly to something happening on screen. Disney coined the term after the very first Mickey Mouse cartoon in 1928, which had plenty of such moments. The 1933 King Kong film continued the trend’s climb, and by the time of 1940’s Fantasia, the popularity of “Mickey Mousing” a soundtrack was well established.
Malinowki started his work in the ’70s, and the Windows Media Player became available in 1991, which later added a visualizer that roughly synched shapes and lines to music, though this was largely dependent on volume alone. Apple’s iTunes followed suit, though in the more recent shift from owned music to streaming services Spotify doesn’t seem to have any interest in developing a visualizer.
DoodleChaos also builds Rube Goldberg machines and domino tracks to music. (They didn’t respond when I reached out to pester them with questions like “how many hours does it take to get the dominoes and marbles to fall just right” or “why do you do this?”) Malinowski says he started working with computer visualizers to “scratch an itch.” “My tool kit is built up of renderers,” he said. “I take the note in the score and do something with the display to mimic what the note is doing. Half of what I do is building more of those. I’m at the point now where I have pretty much all of the renderers that I need, it’s just a question of figuring out how to use them in different pieces.” Aside from the Beethoven videos, his channel — which has more than 186,000 subscribers — also features a number of full-orchestral works and opera arias. Each is unique, though they bear a distinctive style.
Malinowski’s projects have been featured in live concert halls as well over the years, including a TEDx Talk in Zurich in 2012. He cofounded the company Music:Eyes to bulk out educational uses for his visualizers and is headed towards a major launch in the not too distant future. Like musicians, he says he’s getting better with practice. “I go back and look at my old videos, and as I get better at what I’m doing, what I’m doing in my old videos,” he said. “Every piece, I’m learning hundreds of little things that add up, how all these things combine into something. I think visual meaning arises from the context.”
