Do we build a 2d system, and then later convert to 3d   or   directly build a 3d system?

MotionComposer website.  System must be robust , and easy to use. We do not want the user to have to take a background picture.

Some of the music environments work fine with 2d camera using flow or activity measurement (no background necessary).  Others, not so well.

With 2d, how much will we be able to accomplish with filters and algorithms?  The patch called "Drums", for example, needs to locate where the body is in order to put instruments around it --

Also,

How much latency will we have with 3d?    Can we get an asus with different bus ?

One of the most effective ways to map music to movement is through activity -- the larger and faster the movement, the louder (and possibly somewhat higher pitched) is the sound. If you stop, the music stops. This method is both highly intuitive and simple to implement. Since it is based on a modulated wave file, variation can be built directly into the file -- but only up to a point! too much variation denies the user her role. Also, avoid silences since they contradict the mapping and frustrate the user. By applying the OSC data to external software (i.e. Contribution Method B), many interesting (more sophisticated) modulations are possible.
Credits:  Music: Hannah Groninger Engineering: click here.

When using continuous controllers one runs the risk of losing palpability. The fact that body and sound are both changing does not by itself insure coherence. After all, constant change is pretty normal in both movement and sound. To make the connections coherent one needs discrete events. If they are tightly synchronized the connection is unmistakable. In sound this means a short attack, short decay or sound particles (see Particles below).
Credits:  Dancer: Anja Wirthmann, Engineering: click here.

You will notice when I open and close my eyes in the video that two different sounds are heard. This reflects the bilateral nature of many movements -- the eyes and mouth are open or closed, the head turns right or left, a palm faces up or down. This simple technique helps to build a stronger interactive relationship.
Credits:  Music: Beethoven, or thereabouts Engineering: click here.

Discreet physical gestures are used in this example to turn continuous sounds on and off. In this way, tracks can be layered -- brought in and out in a pre-determined sequence. You may wonder if rhythmic music can work. The answer is yes. we are building an automatic beat synchronizer into the device. (See Rhythms below).
Credits:  Music: Marc Sauter
Engineering: click here.

Tags refers to a little trick in which a small sound or "tag" is added at the start or end of a longer sound. Tags give the user a stronger sense of control. Adding a start-tag is easy since you can build it directly into the wave file. Ending tags are a bit trickier: Without knowing when the user is going to stop, how can you build the tag into the long sound? The answer is you play the tag at the same moment the longer sound is stopped and it will sound like the longer sound is finding its end. Two kinds of end-tags -- short decay and long decay -- are demonstrated in the video.
Credits:  Music: Marc Sauter
Engineering: click here.

Notice that the female dancer's eyes modulate the sound that is triggered by her mouth, i.e. the two controllers are functionally linked.
Credits:  Music: Dan Hosken Dancer: Mimi Joeng
Engineering: click here.

Field1 = height
Field2 = width
Field3 = centerX axis
Field4 = expansion
Field5 = activity
Credits:  Music: Marc Sauter
Engineering: click here.

Two height fields to the left and right of the dancer are used to trigger and bend sounds. The sounds were designed to be bent in this way. A Chinese bo was recorded and then stretched. Since this instrument generates a large number of overtones the effect of pitch bending is amplified. Such linear, position-based controllers can also be used to move forwards and backwards through melodies and scales (there is a brief example in Video 12, part 'e' ).
Credits:  Music: Pablo Palacio
Engineering: click here.

Stretching the arms overhead is an easily tracked gesture, but puts us in a position in which additional movement is difficult. Thus, when we use this mapping to freeze the music and we tell everyone dancing to freeze as well. (see Video 12, part 'c' ). Making music "easy to play" is not our goal, indeed more the opposite. We want people to bend, reach, twist and turn beyond their normal lazy ranges.
Credits:  Music: Hannah Groninger Engineering: click here.

Rhythm presents a challenge for interactive music. For one thing, non-musicians are not generally very good at playing music, duh. What, if anything, can they add to a piece of music that will not ruin it? In the jazz club when someone in the audience shouts something, it can make the shouter, as well as the people around him, feel like they are part of the music. Thus, one way is to let people contribute accents on top of a fixed music. Another way is by adding and removing rhythmic layers. In the video I am manually synchronizing (or attempting to synchronize) five tracks. One can also use software to open and close gates (i.e. with multiple tracks running unheard in the background). A fourth possibility involves giving the user the ability to shift non-rhythmic aspects of the music, such as key or instrumentation. Finally we have the option of simply trashing the beat. It may be temporary, like a stumble, or leave some background elements undisturbed. To summarize the options we have:
1 triggered accents
2 manual track synchronization
3 automated track synchronization*
4 changes in the non-rhythmical aspects of the music
5 trashing the beat
* -If you give us 5 tracks of exactly the same length, we will do the rest.
Credits:  Music: Burnt Friedman
Engineering: click here.

One way to combine the coherence of discrete changes with the expressive subtlety of continuous controllers is through the use of sound particles. As the name implies, particles are short sounds. Examples in nature include the sound of water drops, gravel or the crackle of twigs or dried leaves underfoot. Many particles can combine to build textures and shapes.
Credits:  Music and engineering: Andreas Bergsland and here.

We have just begun to work with people with disabilities and will continue to learn from them and the music and dance therapists that work with them.