Electronic Musical Instrument Design

lecture notes • Spring, 2019

April 4

lets you control multiple modules with one MIDI channel, send multiple controller messages from one control.
Insert modules within combi. Can have mixer inside combi or mix externally.

Open "Programmer", select the module you want to assign a controller too, insert source (mod wheel, breath control, expression, etc.) and destination and scaling in table. A source can be assigned to one or more parameters, with scaling and polarity, in each module. (Mod wheel gets through as mod wheel regardless of whether it's assigned.)
Rotaries are controllers 71-74.
Buttons, binary, controllers 75-78

Main panel
Modulation routings

pre-defined routings, but also virtual patches in the Mod routings section below.

analog: simple waveform.
Wavetable: sample, just a few cycles, move through it using position.
Phase modulation: combine two waves in series, and second one modulates first.
FM pair: two sine waves, relative pitch creates sidebands. FM control determines level of sidebands.
Multi: multiple osc same wave, detunable.
Noise: white, colored, band-limited

Filter types: LP (Moog-style), Comb (many teeth), Formant (x-y, harmonic-related peaks, vowel sounds), state variable (notch or peak sweepable between high-pass and low-pass)

LFOs have delays, assignable in matrix.

Shaper: distortion
Chorus, delays

Global section: affects everything. Envelope assigns to filter, or somewhere else. LFO 2 is free-floating, must be assigned.

Mar 28

Mixing and Processing in Reason
Assign each module a MIDI channel in Reason Hardware Interface.
Controller numbers assigned to each parameter are in the MIDI Implementation Chart PDF.
Processing modules: reverb, complex reverb, flanger, delay, scream, etc.
Use Max to assign different incoming data to different channels to control efx, mix, aux sends, panning, etc.

Meta-events: events about events. How do you implement them in Max?
Use Objects: metro, tempo, sliders, dials, lists of numbers, random, number boxes with limits and offsets
  Tempo, accelerate & ritard
  Range, also restrict to key, scale, chord
Tendency masks: "clouds" of random numbers controlling function to keep it within certain limits, and changing the limits over time
  Crescendo and diminuendo, using velocity (control over individual notes, but only one velocity byte per event) vs. volume (continuous control of level, affects all notes on that MIDI channel)
  Harmony: distance between notes. Does it follow key, scale, or chord?
  Counterpoint: how musical voices move against each other, parallel, opposing, combination
  Orchestration: use multiple Reason modules on multiple MIDI channels to layer different sounds or create an orchestra.
Density: growing/shrinking chords or clusters using module's polyphony setting

Mar 26

Keep software files off the desktop. Create folders for your team’s work on Google Docs.
Task assignments within teams.
Getting started on first projects. Start working on Max and Reason patches.

After each lab session:
Put tools away.
Put common parts away.
Put power tool batteries in their chargers.
UNPLUG soldering equipment.
Don't leave any files on the Mac Desktops.

Mar 7

Designing an instrument:
What gestures will control it?
What sensors do you need for those gestures?
What will need to be done with the sensors to make them "playable", e.g., mounted on wood, put in tubes, duct-taped to your head?
How do the sensors translate into switch contacts, or 10KΩ resistors, or +5V DC sources, to use with the Arduino? Will there need to be supplementary electronics?
What processes will be applied to the MIDI data coming from the Arduino in Max?
What will the instrument sound like, and how will the sound be controlled by the gestures?

Assigning team tasks:
Specifying and obtaining sensors.
Specifying materials to make sensors playable, i.e., the human interface.
Wiring the sensors making the wiring robust.
Adding circuits to process sensor data.
Writing the Max patch to process the data.
Designing the Reason patch.
Document wiring and circuits!

First teams:
Meet and exchange info: email, phone, experience
Groups meet, start on proposal for first project. Assign tasks, set a meeting time before next class. Designate one person to write it up and bring it Tuesday.

Project is due Tuesday April 10.

Mar 5

More about soldering:
• Using heat sinks
• Unsoldering and removing components: use sucker, brush, and wick as apporpriate. If practical, pull gently on the component as you remove the solder to get it out of the board.

Reason NN19 module: samples, with many of the same filtering and envelope parameters as Subtractor.
Building sample map, assigning controllers.
Aligning pitches in sample

map—setting root note. Keyboard tracking (no pitch change).
AIFF audio format. Samples can be mono or stereo.
Recording, importing and converting sounds, SFX libraries.
Self-contain setting

Using mixers. Assign a MIDI channel, look at controller chart. Two different mixers. We don’t use master mixer.

Reason NN19 in manual 985-1005
Read: in course pack: "The griDJ MIDI grid system" and Watch vide

Feb 28

Lab safety and rules: Goggles when using any power tools. No power tools when you’re alone.
Don’t use tools you don’t know how to use, especially soldering irons, jigsaw.
Unplug soldering irons after use. Unplug glue gun after use.
Don’t overuse hot glue. Heat destroys components.
No spray painting in the building: take objects outside and use cardboard backing so paint doesn’t get on pavement or grass.

Wiring and soldering:
• Safe use of the soldering iron
• Assembling components on a breadboard and soldering them
• Good soldering practice
Graphic instructions on how to solder here

3D printer: design in Onshape (online). Save (right-click) file as STL. Put on SD card, bring over to printer

Feb 14


Powerpoint from class is here

Switches (binary/digital):
Toggle, push-button, momentary, lever, magnetic
Rotary encoder: has two different switch contacts, “0” and “1”. Speed of encoder determined by how fast switch contacts are made; direction determined by order of contacts.

Continuous (analog)
Continuous sensors can act as resistors or current generators. Some, like rotary or slide potentiometers, can be connected directly to the Arduino. Others need buffering, amplification, and/or filtering to provide usable signal.
• Force-sensing resistor=pressure. Button-type or strip.
• “Softpot”, flat variable resistor, ribbon or circular
• Joystick=two-dimensional variable resistor
• Flex sensor=bending angle
• Infrared distance sensor=from about 1.5-10 inches. Available in various ranges. Get too close and the field inverts.
• Photocell: presence or absence of light
• Accelerometer: 1-axis, 2-axis, 3-axis. More sensitive ones can measure tilt (responding to gravity)
• Piezo electric foil=striking or bending force. High voltage.
• Capacitive
• Color and gesture: https://www.sparkfun.com/products/12787
• Myelogram https://www.sparkfun.com/products/13027

Feb 7

Reason NN19 module: samples, with many of the same filtering and envelope parameters as Subtractor.
Can drag samples into keymap, load samples from disk, or record directly (30-second limit, automatically trims silence at beginning)

Intro to Max:
Check AudioMIDI Setup to see that MPD or keyboard is recognized.
Launch Max. Before opening a patch, check Max's MIDI Setup: “in a” is MPD or keyboard, “out a” is "fromMax1" (goes to Reason). Disable all other inputs and outputs
Launch Reason. Set Preferencex>Sync Bus a to "fromMax1"
Edit and locked mode. command-E toggles, or click on lock icon.
object box = function
start typing, list of objects appears. To list them all, Max Help: objects A-Z or by function
Number box = for monitoring.
Can choose to display MIDI note numbers OR names (get info)

Patch cables connect everything.

Notein, 4 arguments: port (a-z), note#, velocity, channel
noteout, ditto
Makenote: port, note#, vel, length (in milliseconds). Has to then go to a noteout. test for not-zero [ != 0 ]
cntlin: port, controller value, controller #, channel (if no channel, will accept all. If wrong channel, won't respond)
cntlout: same
Sliders (0-127), can change minimum and range (Get Info).
Kslider, shows keyboard.
Pipe=delay: number of arguments is number of messages passed, with last one being delay time in ms. "0 0 0 time" will delay a MIDI message (zeroes are placeholders).
Random n=puts out random value 1-n
Metro=clock, number of milliseconds. Start and stop with toggle at input.

Feb 5

Akai MPD26 and Reason
Using MPD with Reason--you can now control anything on the Subtractor front panel with the proper controller numbers. Controller assignments for the Reason modules are in the "Reason controller charts" folder in EMID Resources

Multiple modules in Reason: addressable by MIDI channel, and by different pads and controls on MPD

Vyzex MPD editor instructions
Launch application
Click OK at MIDI Ports Setup
Wait! Image of the device will appear on screen.
Click on a pad or control to change it.
MIDI CH CC=current global channel (usually 1) You can set a pad or control to any channel, but use bank A (e.g., 1A)
Pads can be set to momentary or toggle (on/off). Pressure (aftertouch) can be on or off (don’t use Poly Pressure-PPr)
When you have set a note number, move the cursor off the note number—or else playing another pad will impose that pad’s note number on the current pad! (It's a feature: it lets you program the thing using a keyboard.)
Sliders can be set to controller number or Pressure.
Knobs similar, also option for NRPN—don’t use.
Click on name to name your preset. You can have up to 30 presets.
Finally: File>Save Set As….to save entire bank of presets. When you come back to the lab, File>Open to get it back.

Jan 29

Virtual MIDI connections (inside operating system, using software synths), no channel limit, no speed limit.

What are synth parameters? Subtractor:
Key number
Pitch, detune
Filter freq and resonance
Envelope, filter envelope
LFO rate and depth

Mapping keyboard physical gestures to musical ones:
Mod wheel to pitch vibrato, volume vibrato, timbral change, envelope
Velocity to amplitude
velocity upside down to volume
velocity to envelope attack time
velocity to filter envelope (brightness)

Using knobs: Control mappings (numbers and functions) for keyboards are on the desktop.

Saving the whole rack (.reason) in Reason: create your own folder inside theEMID Resources folder.

Jan 24

Basic things to think about when designing an instrument:
Depth & Virtuosity: As you learn it, you get better and can do more with it.
Something interesting to look at? Audience needs to pay attention, be able to connect what they see visually with what they hear.

Three types of musical interfaces:
Familiar: keyboard, guitar, drums, malletboard, violin, woodwind, brass. Advantage, people already know how to use it. Doesn't require new skills, practicing, rethinking how you make music.
Most commonly in electronic world: keyboard. Used with Moog synths, Switched-on Bach. Became glorified organs with thousands of stops, people play all of them the same way.
But, can extend technique to play keyboard in new way that has different expressive parameters (aftertouch, wheels, pedals).
Adaptive: or extended, like keyboard with pedals; stringless guitar; wind controllers with more buttons and levers than a conventional wind instrument; violins with sensors on the bow, etc.
Unfamiliar: like Theremin: hard for guitar or string players used to articulating with right hand. Using different parts of the body, or in different ways. Finger position or movement on surface or in free space. Relative positions of fingers -- spread or angle. Bend of joints: wrists, elbows, knees. Pressure on surface. Requires practice and mastery! Think of music in different ways than simple button-pushes/discrete events.

MIDI command structure
Real-time control language.
Some commands three-bytes, some two, some one, some longer.
Best expressed in hexadecimal notation: 0-255 decimal ($)=00-FF Hex
Numbers below 128$ (80H) are data bytes. Numbers 128$-255$ are command bytes.
Channels: second half of command byte, 0-F=read as 1-16. Different insruments respond to different channels. In Reason, each module is on its own MIDI channel.
Note on (9n) + note number + velocity, off (velocity). Decimal ($): 144-159
Note off (8n)+ note # + velocity. Duration is time between on and off. 128-143$
9n with velocity zero is equivalent to note-off
Controller (Bn), controller number, value. Some continuous (wheel, slider, breath, foot control), some switched (sustain pedal). 127 of them, not all defined. Used for any kind of continuous command. 176-191$
Pitchbend: (En) + LSB + MSB. Like controller but its own command, double precision. 224-239$
Mono Aftertouch or Channel Pressure: (Dn) + value. 208-223$
Poly aftertouch (An) + note # + value. 160-175$
Program change, (Cn) 0-127. Reason doesn't respond to it. 192-207$

MIDI Hardware
Uses 5-pin DIN connectors. 5v DC, 10-bit words with 8 data bits and 2 framing bits. Data rate is 31,250 bits/second=3125 bytes/second=about 1000 commands/second. MIDI choke is a problem when too much data is flowing.
Solution 1: Multiport MIDI interaces, with multiple MIDI cables
Solution 2: MIDI over USB/Firewire/Thunderbolt. However, these are not specified in the MIDI spec. But Apple and Microsoft have included MIDI over USB in their operating systems. If a device follows those guidelines it is "class compliant". If it does not, then the manufacturer has to provide driver software to install on the host computer.

Jan 22

An electronic musical instrument uses gestures to control electronic circuits.
Can be simple (direct) like theremin or complex (through microprocessor) like Wriggle Screamer (video)
Computers provide ultimate flexibility: they can produce any sound, and interpret any gesture however we like, once we get the gestural information into the computer.
The physical aspects of an instrument no longer have to have any relationship to the sounds it makes.

Two metaphors/paradigms for musical instrument:

1) Instrument metaphor:
Causes a sound to be made
Control of pitch(es)
Expressivity takes many forms:
Pitch articulation (more than just turning on a note)
Amplitude articulation (initial and subsequent)
Timbre articulation

2) Controller/Mixer metaphor:
Sequence or pattern generated automatically or by a single gesture.
Gestures control the parameters of the sequence as it plays.
Selection, adjustment, nudging, changing volume/balance/timbre on the fly

Basic sound parameters in a synthesizer:
Pitch (inc. glide, bend)
timbre: waveform, filter freq, filter resonance
Envelopes of all of the above (ADSR)
Vibrato (LFO) + vibrato envelopes affecting pitch, volume, timbre

Jan 17

1) What's music? Varese: organized sound. Working definition: Sound that is created deliberately, with intent, and has interest as sound.
2) Elements of music: melody, rhythm/tempo, harmony, timbre/orchestration.
Elements of music performance: prescription, improvisation, interaction (and grey areas between)
3) What's a gesture controller? Something that responds to a physical action by one (or more) human beings. Examples: Squeeze, blow, pluck, bow, hit with hand or foot or stick, press with fingers.

How we do it: Links in the chain:
• Electronic sensors to detect gestures: touch, pressure/force, movement, acceleration, distance, displacement
• Framework to hold the sensors and make them playable.
• Device to turn data from sensors into MIDI: Arduinos (formerly Doepfer boards), Bela
• Software to interpret and process the MIDI data: Max
• Synthesis software to turn the processed MIDI into sound: Reason, Max MSP
• Audio system to produce the sound

What we will do in the class:
• Study existing electronic instruments, see them demonstrated on video and live
• Conceptualize physical gestures as they can be used to make music
• Learn MIDI, what the commands mean, how they can be used to control music
• Learn Reason, a software synthesis system, and how to set its parameters
• Translate gestures into electronic form, using sensors, and then translate them into MIDI using Arduinos
• processing real-time MIDI data with Max
• optional: using Max/MSP to generate audio, Bela for self-contained audio
• Building new controllers and systems
• Work in teams of students with complementary skillsets
• at the end, do a public demonstration

Resources: Course pack (buy at Gnomon Copy); reference books, manuals, and catalogs in lab, on computers, and on-line.

Lab: Electronic parts and tools, sensors, software, mechanical parts, hand and power tools for woodworking and assembly. 3D printer, X-Carve 3D computerized router/carver.

Laser cutter, more tools in Bray Labs and in Makerspace in Robinson. Available to all with training.