Alex Moerlein, May 14, 2004

Edward Sensorhands

I. Goals

Edward Sensorhands is a continuation and expansion upon the midterm project we created using flex resistors. As our midterm project showed, however, flex sensors don’t work very well when blown over by a fan. We decided instead to attach them to handyman’s gloves in order to get better, more controlled results. The presentation by the gentleman from MalletKat gave us the idea to use the gloves, not as much to create a piece of completely original music, but to modify loops and clips, basically acting as a sort of DJ. While the user is limited by what kind of clips he has to work with, the amount of unique sounds and effects he can produce is unparalleled.

II. Description

The gloves use bend sensors on all the fingers of the left hand, and force sensing resistors (FSRs) on the tip of each right-hand finger. We chose the hands as the centers of gesture control because they are the most articulate part of the body, and the gloves provided a convenient mounting platform. The gesture controlling the bend sensors is the same as one uses to make a fist, and for the FSRs, it is simply tapping one’s finger on a tabletop. In the original plan we were going to use an infrared distance sensor as well, which would be controlled by raising and lowering one’s forearm.

III. Chronology and Project Description

We split the project up in order to best emphasize our individual skills, but it ended up generating a fair amount of crossover. Fernando set on designing the loops using various Reason patches, Jeff programmed the MAX patch and wired stuff, and I handled the construction end of things. Our first step was to make sure that our sensors would work as they were intended, and were not overly sensitive or finicky. Jeff’s electronics expertise came in very handy at this point, as he was able to use resistors to condition the bend sensors’ signals. Otherwise, all our sensors were very easy to use and wire, though the FSRs were somewhat of an unknown.

After we knew the gloves could theoretically work, each section was able to proceed pretty much independently. The gloves we used were chosen for their close fit and reasonable sturdiness. A loose-fitting glove would cause trouble with the sensors attached to it, as the bend sensors might not bend as much, and looseness at the tips might mis-trigger the FSRs. The first problem we noted in attaching the bend sensors was that when the user flexed his hands, the gloves stretched, meaning that we could not simply glue the sensor to the gloves. I headed down to the Goodwill and purchased a nice silk tie, then chopped it up to form the slide tubes for the sensors (see Figure 1). After testing it out, the sensors still did not slide easily enough for our uses, so we glued a cut up Ziploc bag around each sensor, which did the trick. After that it was short work to glue all the pieces in place. We were careful to glue the sensors so that they would not get caught inside the silk holders, but could still get significant bend during use. The input wires were soldered to the ends, which were in turn glued to the gloves.


Figure 1: Bend Sensors

 

The FSRs were a bit trickier, as they don’t react well to sharp bends, yet had to be attached to the underside of the user’s fingers. Running them across the tops of the fingers was a preliminary idea, but the necessary bend to get the sensor under pad of the finger was unacceptable. A slight bend over the length of the FSR, however, is perfectly fine, so after some experimentation we worked out a decent solution. The user does not get perfect range of motion, but assuming he would play the gloves with fingers crooked no more than if playing a keyboard, the range is plenty. The sensors are glued at the very tip, and then again just before where the leads are attached, allowing them to bend in a sensible, non-destructive manner (see Figure 2). Initially we had trouble getting the FSRs to trigger well, because the extra fabric in the tip of the gloves pads the sensors fairly well. After some position adjustment, we found a spot with reasonable response, though the user needs to play with the gloves for a bit to find that "sweet spot" where the FSR triggers reliably.

Figure 2: Force Sensing Resistors

While all this was going on, we were simultaneously working on the MAX and Reason patches. Fernando brought his extensive techno background to bear on creating some catchy beats, using the various patches available. Initially, the song Fernando created was exceedingly complex, so we pared it down to its elements and designed a more modular set of loops. A lot of what we created was handled by Fernando, and is beyond the scope of the course, so I can’t honestly say that I understood all that he did. However, the effects we chose to modify are quite straightforward. We chose ones which would show off the abilities of our system as well as possible. They are as follows:

Finger

Effect

Thumb

Mod wheel in Maelstrom controlling background strings

Index

Resonance effect on distortion in melody

Middle

Low-pass filter frequency on bass

Ring

Flanger on drums

Pinky

Delay on drums

Auditory effects are rather difficult to describe in writing, but I will attempt a rough description without the reader having heard them. The mod wheel is hooked up to a variety of effects, including filter frequency and a couple LFOs, and creates a stuttering, high pitched effect. The resonance effect creates a feedback-like noise in the melody. The low-pass filter on the bass simply takes out the higher frequencies. The flanger on the drums creates a "jet engine" type noise, and the delay causes the drum loops to repeat more or less quickly. The FSRs are hooked into the Mute control for each loop, and triggering them causes the individual track to turn on or off.

The major challenge behind our MAX patch was getting the program to send out note-ons from the FSRs when we needed them. Todd Ryan had a similar problem when designing his midterm project, in that MAX simply doesn’t like to just send out one bang every time it’s triggered. There was extensive trouble with double bangs, and no response, and programs only working after you fiddle around with sending out bangs and resetting everything for a while (that’s still the case). At any rate, Jeff came up with a system which, after the initial weirdness at start-up, works like a charm. It uses a series of gates and bangs to finally produce a note-on when we need it. A problem we ran into when using the FSRs was that there was a significant delay between hitting the FSR and getting it to effect the track. The latency issues probably came about due to the complexity of the Reason program and number of parameters it was controlling.

The bend sensor program is cake compared to the one for the FSRs. It consists of a control-in, a few equations tailored to each sensor to condition the reading between 0 and 127, followed by a control out.


Figure 3: Our Workstation

IV. Closing Remarks

This project went quite smoothly given our lack of experience with much of the materials we were using, most notably the sensors and Doepfer box. The few hitches we ran into were straightforward and did not result in major project changes. The distance sensor we had initially planned to include was scrapped after it ceased to function for reasons unknown. Given the chance to completely redo the gloves, I can’t say I would want to change much. The FSRs were a little twitchy, so I might shop around for something a little smaller, that doesn’t restrict the user’s motion quite as much, and hopefully gets slightly better readings.

We have a modular, easy-to-use system with a short learning curve. After about twenty minutes of experimentation, any user can get the gloves to do what they want. The mechanical and software parts are robust, providing the user with no worries that the slightest tap will cause a malfunction. All in all, Edward Sensorhands is a functional, fun electronic musical instrument. •

Jeff Hamalainen's report
Fernando Kriete's report