Thursday, October 04, 2007

Midi Speech Controller Application

I am currently working on the first version of my Midi Speech Controller Application. I am doing the final programming and testing now while some of my (professional) colleagues are helping me with the UI design (thanks guys!). The application is written in C# using the .NET 3.0 framework. It uses WPF (User Interface) and the Speech API. While waiting on the UI design, I'm also looking into building an installer using the Windows Installer Xml (WIX). Unfortunately I am unable to find a redistributable for the Microsoft Speech technologies. I may end up advising the users of my application to install the Speech SDK…

The application runs in two modes: Run-mode and Edit-mode. In Edit-mode the user can define a Preset with one or more Patches. A Patch is collection of Midi Commands that are executed in order when the Patch is activated. A patch can be activated by speaking its text phrase out loud (into a microphone that is connected to the PC) –or- if the speech engine is not installed, by selecting the patch from a list and activating it manually through a button click.

In Run-mode the application can be made to listen by clicking on the Start button. All text phrases for all Patches for the current selected Preset are registered with the speech recognition engine. When the user speaks out a phrase for a Patch, the speech is recognized and displayed to the user, the Patch is located and its Midi Commands are executed.

Preset
The Preset represents one set of Patches that can be active at one time. There can be many Presets, but only one is active. Presets (with their Patches and Midi Commands) can be save to- and loaded from disk. A Preset also contains two options for audio feedback. The options for successful or unsuccessful speech recognition are: None, Beep and Speak. So when a text phrase for a Patch is successfully recognized the computer can say the Patch name back to you – or just beep – or do nothing. When the Speak option is selected for the unsuccessful recognition option, "Not Recognized" is spoken.

Patch
The Patch represents a set of Midi Commands that will all be executed when the Patch is activated. The Patch has a (human readable) name and a (pronounceable) text phrase associated with it. Both can be set to any text, although it is recommended to keep the text phrase as short as possible.

Midi Command
A Midi Command contains the definition of a valid Midi message that is sent to a specific Midi Port (and channel) on the PC. A Program Change, a Control Change or a System Exclusive message can be defined. The data for the System Exclusive message has to be hand-typed into a text field (for now), including the Start and End markers (F0 and F7 resp.).

The application is targeted at users that literally have their hands full or just need a simple solution to output a large number of Midi messages at the click of a button (when the speech engine is not installed). People that do live performances can benefit from this application. Imagine speaking the title of the song you're about to play and all you equipment being initialized for that song.

I came about the idea for this application when I was playing my (electronic) V-Drums and wanted to change kits. With no hands and feet to spare, speech seemed the only option.

1 comments:

runagate said...

That's really wonderful, Marc. I'm not sure how I missed this particular blog but I want you to know someone is interested ;)