Dynamic Pictures

Bret Victor / March 3, 2011

I believe that dynamic pictures will someday be the primary medium for visual art and visual explanations. The medium is currently in its infancy -- working in the medium is more about technological fumbling than artistic expression. The revolution cannot happen until there is a means for artists to draw dynamic pictures.

This essay describes what dynamic pictures are, what dynamic drawing is, why the subject is so important, and some initial project ideas.

If any of this inspires you to play around with these concepts, I'd love to see what you come up with.

What is a dynamic picture?

Dynamic means that the picture changes when you change some input. A dynamic picture looks different in different situations.

Here are examples of different types of pictures:

This is a static image. Exactly the same, every time you see it.

This is a static animation. Exactly the same, every time you see it.

These are some instances of a single dynamic image. I didn't draw these images individually -- I made a "master" image, and then varied parameters. (Click to vary some randomly.)

This is an interactive picture. Try dragging or flicking. A dynamic picture can be interactive, but it doesn't necessarily have to be.

Most pictures that you see today, in books and on websites, are static pictures -- one-offs, unable to adapt to their surroundings, unable to reflect new ideas or information, unable to respond to the viewer's needs and interests.

Here are a few examples of the sorts of dynamic pictures that you see today:

In general, today's dynamic pictures tend to be confined to niche applications, exhibit a relatively low level of artistic sophistication, and emphasize technology over artistic expression (more focus on the medium than the message). These are symptoms of an art form in its infancy.

What is dynamic drawing?

Drawing means that the artist creates the picture by directly manipulating the picture itself, instead of working with some indirect representation that doesn't resemble the art-in-progress.

Next to each picture below, I've placed a screenshot of the canvas that the artist typically works in:

Artwork

Canvas

Static image.

Static animation.

Dynamic image.

Interactive image.

Do you see the things that are not like the other things? In the first two examples, the artist works directly with the picture on the canvas. These pictures can be drawn, but they're not dynamic. In the last two examples, the canvas consists of pages of abstract text, and the artist has to maintain an intricate mental mapping between text and picture. These pictures are dynamic, but can't be drawn.

With today's tools, dynamic design requires creating pictures by writing text. It is only because we are so accustomed to this situation that we don't recognize how bizarre, even barbaric, it is.

Below are some common environments for creating dynamic pictures. I've highlighted the part of the canvas where the artist designs the dynamic behavior.

Flash.

Processing.

Nodebox.

Context-Free.

Quartz Composer.

Scratch.

In many cases, the output picture is juxtaposed with the code. In some cases, the canvas is (so-called) "visual", with code in boxes instead of sentences. But in no case does the artist create dynamic behavior by manipulating the picture itself. This is not drawing.

Why do I care?

Magic Ink is my treatise on why dynamic pictures matter. The general motivations for dynamic drawing tools are laid out in Substroke and a chapter of Magic Ink.

Here are some more personal reasons why I'm so passionate about all this.

Information, Explanations

I'm passionate about enabling people to understand, and visual explanations are crucial for understanding many concepts. But because our tools are so weak, we usually resort to describing when we should be depicting. And those few visual explanations are almost all static. One-off pictures, tediously drawn for a specific situation. One-size-fits-all pictures, identical for every reader.

Dynamic pictures are ideal for visual explanations, because the parameters can represent information to be conveyed. As the information changes, so does the picture. For example, we could draw a picture that explains the carbon emissions of a country. Which country? Any country -- just plug in its data. The static artist is stuck drawing one-off pictures for each country, instead of a single generalized picture.

Dynamic pictures needn't be one-size-fits-all either. A dynamic picture can adapt to the specific reader and that reader's context. It can teach the reader what he actually wants to know, not merely what the artist guessed that everyone wanted to know.

Artist Independence

A "user interface" is simply one type of dynamic picture. I spent a few years hanging around various UI design groups at Apple, and I met brilliant designers, and these brilliant designers could not make real things. They could only suggest. They would draw mockups in Photoshop, maybe animate them in Keynote, maybe add simple interactivity in Director or Quartz Composer. But the designers could not produce anything that they could ship as-is. Instead, they were dependent on engineers to translate their ideas into lines of text. Even at Apple, a designer aristocracy like no other, there was always a subtle undercurrent of helplessness, and the timidity and hesitation that come from not being self-reliant.

It's fashionable to rationalize this helplessness with talk of "complementary skillsets" and other such bullshit. But the truth is: An author can write a book. A musician can compose a song, an animator can compose a short, a painter can compose a painting. But most dynamic artists cannot realize their own creations, and this breaks my heart.

Adopting, Adapting

Suppose you want to take some elements from my picture, and build on them in your own work. With a static picture, it's easy -- you open up my source file (in Illustrator, say), and copy-and-paste objects into yours. Even if you don't have the source file, you can at least cut out elements from the flattened output file or a screenshot.

With a dynamic picture today, appropriation is a nightmare. The picture is described by pages of code. You have to read through all the code, figure out what each part is doing, figure out how to disentangle the code you want from its surroundings without breaking dependencies, and figure out how to incorporate this foreign code into yours, which might be using different conventions, libraries, or even languages.

A drawing tool for dynamic pictures should make sharing elements as easy as copy-and-paste. I believe that the evolution of the medium requires such ease of sharing. Artists must be able to build on each other's work.

What to do?

I don't know yet. That's why this is a research project.

I've explored two approaches before. In the Designing a Design Tool section of Magic Ink, I describe an artist drawing static snapshots corresponding to sets of example parameters, with the tool inferring the mapping between parameters and picture elements. Substroke was based around parameterized transformations, where the artist's drawing history could be parameterized at each stage.

I think there is promise in both of these approaches. But these were both very ambitious attempts to design a general-purpose dynamic drawing tool ex nihilo. I've come to believe, or at least suspect, that such ambitious goals are better approached incrementally.

I'm interested in exploring a series of small, self-contained projects for doing specific dynamic and meta-drawing tasks. With an emphasis on making tiny tools, each immediately useful. Hopefully, the insights gained from this work will lead to a general-purpose tool or conceptual framework down the road.

I have a bunch of ideas for such little tools. I will outline them in a separate essay.

Stay tuned. Or better, try making some yourself.


Update from 2024

My later work in this direction includes Stop Drawing Dead Fish and Drawing Dynamic Visualizations.

See also Toby Schachman's Apparatus and Cuttle, the work of Rubaiat Habib Kazi, Ravi Chugh's Sketch-n-Sketch, and associated references.