🎛 How Apple Designs a Virtual Knob

💡 Author's note: This unpublished post was originally written in May 2012, and refreshed for publication in November 2022.

Just want to see the code? Check out github.com/jherrm/knobs

When Apple introduced Garageband for the iPad back in March 2011, I just had to try it out. It wasn’t just the ability to create music that drew me in, it was the realization that this was a flagship app that uses the strengths of a multitouch interface to create experiences that weren’t possible with a keyboard and mouse. My favorite example is how you can hit a note on the sound sampler keyboard then drag your fingers left and right to bend the pitch.

Some time after a new type of interface is introduced, a handful of concepts tend to “win” and become a standard way of interacting with the interface. Keyboard shortcuts and right click contextual menus aren’t tied to a specific OS, they’re part of the standard toolset you get by using a keyboard and mouse. The iPhone popularized pinch-to-zoom, swipe, long tap, and many other useful multitouch interactions. With Garageband, Apple explored uncharted waters in user interaction and I’m going to talk about one simple control that has a few surprises in both use and implementation: the virtual knob.

Apple’s Three Ways to Turn a Virtual Knob

After using the knobs in Garageband for a while, I noticed that they didn’t always react the way I thought they would. Most of the time the little indicator dot on the knob would follow my finger as I spun the knob around in a circle. Other times the knob wouldn’t follow my finger at all and seemed to go in random directions. I eventually figured out that I had stumbled on three different ways to turn a virtual knob. I was so impressed by this added functionality that I had to try and reproduce it myself, which led to me writing Knob.js, a javascript implementation of multitouch knobs that attempt to replicate the knobs found in Garageband. Knob.js powers all of the examples used in this post.

Spin

The natural interaction with a knob is to spin, so being able to touch the knob and drag it in a circle is key to a successful virtual knob. You can start the spin at any point on the knob, and if your finger is hiding the indicator you can drag away from the knob as you spin, so you can see what’s going on. Dragging further away from the center of the knob also lets you be more precise in setting the value.

Vertical Slide

Just as pinch-to-zoom actually wraps zoom, pan and rotate in one gesture[1], Apple’s knobs also do double duty as sliders. By sliding your finger in a vertical line, the knob will lock into vertical slider mode. This makes it super easy to make the knob spin several full revolutions without giving your finger a workout. Just like a real life physical knob, the direction the knob spins is dependent on which side of the knob you turn. If you start sliding down on the right side of the knob, it will spin clockwise, and if you start sliding down on the left side of the knob it turns counter-clockwise. (spin disabled)

Another bonus of using the vertical slide gesture with multitouch is you can easily change multiple knobs with one gesture on one hand. Using one finger per knob, drag straight up or down as if they were sliders on a audio mixing board.

Horizontal Slide

Like the vertical slide, if you start sliding in a horizontal line the knob will lock into horizontal slider mode. Start sliding at the top and moving right causes the knob to spin clockwise, and starting at the bottom and going right will spin the knob counter-clockwise. (spin and vertical slide disabled)

All Together Now

The demos above restrict the functionality to show each type of interaction one at a time. Try all three interactions together, as well as mouse/trackpad scrolling support on this button:

How Were They Built?

When I set out to replicate the knobs in Garageband, the first thing I did was open up the Garageband.ipa file and extract the assets. What I found was that in order to recreate the classic knobs from various instruments and amps, the developers used several techniques to bring the knobs to life in an efficient and realistic way.

Knobs with static backgrounds

Below we have a knob made with two images: a non-moving knob “background” and an indicator. The indicator is simply repositioned to create the turning effect. In Garageband, the generic and clavinet knobs all use this technique.

Here’s another knob with a non-moving background, but instead of positioning the indicator to simulate turning, it rotates the indicator instead. Another neat thing about this knob is it, along with the knobs shown at the beginning of this post, are created with CSS rather than images. You can also render a knob with SVG and canvas. The pad and panner knobs in Garageband uses the rotating indicator method, though panner looks like a simple repositioned indicator knob.

A third variation of a knob with a non-moving background is found by combining the other two indicator transformations. An indicator that’s both positioned and rotated offers a nice way to emulate real knobs. The rhodes electric piano knobs in Garageband show off this style.

Knobs with repeating backgrounds

Apple uses repeating backgrounds for knobs that change more than just their indicator as they’re spun around. To make one of these knobs just create the knob image without the indicator, rotate the image a few degrees and save a new image until the pattern is repeated. Combine all of the generated images side by side into one wide grid. Thanks to Google Trimble 3D Warehouse, I was able to find some 3D knobs and recreate the process using blender and photoshop:

Here’s one that I made that has a 13 image repeated progression:

Of course you can also combine a repeating background with the indicator styles. The hammond knobs on the organs in Garageband are an example of a repeating background and a positioned indicator. The knobs on the generic synthesizers show off a repeated background and an indicator that’s rotated and positioned.

Fully Rendered 3D Knobs

Some knobs just can’t be reproduced with static images or repeating backgrounds. Instead of embedding an OpenGL 3D model of the knob, Apple chose to use a similar solution to the repeating background knobs. They basically created a 3D model of then knob then took a picture of it as they spun it around 3 degrees at a time. Since a circle has 360 degrees that means they’re creating 120 different snapshots of the knob! They line them up in order and save it off as a single image that looks like this:

Out of the 25 knobs in Garageband that are able to be turned in all directions, 18 were created using this method.

Thanks to WebGL we could even do one better and use an actual 3D knob! Unfortunately that will have to wait for another day!

---

Garageband Secrets

While going through the assets extracted from the Garageband, I ended up finding some knobs that didn’t make the cut for inclusion in the final release. I’m happy to breathe some life into these knobs for the first time outside of Apple labs.

Test Knob

Before Apple settled on one image for every 3 degrees for 3D knobs, they tested out one image for every 5 degrees - 72 sprites in all. I can only assume they thought the jump from frame to frame was too jarring and was worth the increase in file size.

Knob Rejects

There were also three knobs that were fully finished but not included. Two were simple color/material changes to existing knobs, but the round arrow knob is new. I especially enjoy the specular highlights that appear on the rounded part of the arrow as you spin it.

Conclusion

Apple’s attention to detail is what has propelled it to be the most valuable company on earth. Whether it’s the click of a physical button or the math behind inertial scrolling, Apple employees work really hard to make products that are deceptively simple and just feel right. The virtual knobs found in Garageband are no exception and I hope others enjoyed learning about them as much as I have.