Milestone I draft

As milestone one is approaching, I have a draft of its description.

- finish research on the existing systems

During my research I found many projects of various people doing various real-time performance / improvisation related tasks. However, I did not find a ready to use system, that would behave in a way I want it to behave - listen to the live input and respond in a meaningful musical way.
Existing systems may be listed here (...)

- collect information on the gestures to be analyzed by the system

I plan to extract musical information from the audio input using five analysis techniques:

• onset detection (rhythm)
• pitch tracking (pitch)
• amplitude (loudness)
• beat tracking (meter) 
• FFT (timbre).

At this stage last two are optional, their usability will be evaluated on the implementation stage.

Musical qualities I want to extract from these elements consist of:

• rhythmic speed
• rhythm regularity (this is where beat tracking can help)
• pitch range and character (spread pitches vs. linear structure)
• loudness (RMS)

and optional parameters:

• discrete pitch and rhythm values (may be usable to model computer response)
• pitch class set (mode/harmonic structure)
• beat detection (it would be great to have some kind of "synchronized" response from time to time).

It's still unclear to what extent those parameters would model system's response, which will only transform sounds from live input. This may be too difficult technically for me if I'll try to use very specific parts from the recorded audio for system's playback (like single pitches extracted from the recording).

At this stage I need to start thinking about possible responses/transformations of the audio material already, along with coding the gesture recognition.

Another problem is phrase recognition, which will be manual at the beginning (keyboard/pedal input). However I plan to use mixture of pitch and rhythm values to make system split the gestures and recognize phrases automatically.

- start gesture recognition implementation

Implementation hasn't been started yet.