Meeting with Ciara O'Toole

Similar Applications Ciara mentioned:

1. http://www.uiowa.edu/~acadtech/phonetics/# - Phonetics flash animation-The Sound of Spoken Language
2. Speechviewer3 with english voice rec - no longer made
3. Visispeech
   

Phonetics:The Sounds of Spoken Language

http://www.uiowa.edu/~acadtech/phonetics/#

VISISPEECH

An Autonomous Speech Rehabilitation System for Hearing Impaired People

Final Report

http://www.xanthi.ilsp.gr/olp/pdf/HARP_final_report.pdf

NOTES TAKEN WHILE READING:

• aim: to develop a visual speech training aid for hearing-impaired speakers
• suitable for use in schools
• provide advice on pronounciation errors thereby providing rehab assistance
• guides user through a set of lessons which gradually improve pronounciation (in my case vocab)
• provides a flexible, accessible and highly motivating teaching tool (I could add extensible)
• used to complement and extend services already offered (primary cirriculum for example)
• speech analysis provided
• prototype coursework provided integrating excercises in pitch, loudness, intonation, vowel & consonants
• successful attempts to exploit recent advances in multimedia suer interface such as interactive computer graphics & speech processing... the project completed a detailed investigation of how such multimedia schemes can be exploited for pronounciation teaching for hearing impaired people
• stimulating and motivational material
• bilingual-whole interface designed to be
  

EXTRA: INDIANA SPEECH TRAINING - ISTRA
The path of speech technologies in computer assisted language learning: from ...

By V. Melissa Holland, F. Peter Fisher

• stuff on HMM and common errors with training the corpus

NOTES (from meeting):it would be a good idea to integrate it with the primary school cirriculum-can download the primary school books online so can check them-try to as extension work from vocab learnt in school class room-as an optional extension to do extra work at home if they wanted.

Check school ciriculum for first and second class…APPLY IT!
And test to see if it worked/was helpful; in schools?

There are 2 types of learning : expressive and comprehension/understanding
I’m aiming at expressive so first and second class level
Before that they just learn the actual words, but im interested in their use of it

Books that may be of use:

Cead focal, the first hundred words

Buntus foclora: a children’s irish picture dictionary

Phonemes that are unique to Irish – get native speakers to pronounce

For an articulation game (not language), the apps in existence already measure pitch & loudness

GOOD WAY OF LEARNING WOULD BE TO:

To have a range of acceptability for each word rather than a straight-cut right or no & if there was a way of testing what they did incorrectly

English App Learning Games:

• http://www.manythings.org/

USER-INPUT-Audio- EXTRACTION, STORAGE & REPLAY

Extracting Audio Files from user input

• It is not possible to record and store microphone input locally.
• Therefore it has to be sent to a server (Flash Media Server for it to be compatible).
• It is also difficult to extract audio files only from a .flv file

http://www.gotoandlearnforum.com/viewtopic.php?f=29&t=21974&p=115759

"it's enough if I can record it, replay it and overwrite it again if the user records another sound clip.

So far I gather this is impossible in Flash without the use of a mediaserver. Then again if I'm right the only way a media server can help me is by actually streaming the microphone input to the mediaserver, actually storing the sound as an mp3 file or similar, and then serving the mp3 file back to the app. It seems a bit overkill for just replaying short soundclips while the app is up, soundclips which are going to be re-recorded over and over while the user is using the app and trashed when he stops using the app.

I'm afraid not. I spend some time looking around and there seems to be no way to do this purely with Flash alone. As far as I can tell the only way to record audio is by streaming the microphone input to a media server such as adobe's own media server or an open source alternative like Red5 and then play back the mp3 stored by the server.

Quite an ugly sollution so we ended up building a simple java applet to take care of the temporary recording. ( HYPERLINK "http://www.jsresources.org/examples/audio_playing_recording.html" http://www.jsresources.org/examples/aud ... rding.html )"

http://www.jsresources.org/examples/aud ... rding.html

"Could use java applet (but would have privacy issues)

Purpose. Plays a single audio file. Capable of playing some compressed audio formats (A-law, μ-law, maybe ogg vorbis, mp3, GSM06.10). Allows control over buffering and which mixer to use. Usage:

java AudioPlayer -l

java AudioPlayer [ -M mixername ] [ -e buffersize ] [ -i buffersize ] audiofile"

PROBLEM:

Have to record as video file for the nelly moser converter!!

http://blog.andrewpaulsimmons.com/labels/Flash%20Media%20Server.html

• "My team is currently developing a series of interactive speech recognition application. One of the applications requires us to create a web front end that allows us to record audio from a user microphone and return it to the server... We decided to use Flash and quickly found that we could not extract the audio from our recorded FLV files... But we simply could not get the audio out of the files that were being streamed to our Flash Media Server. "
• "We discovered that all files converted from another format to FLV store audio in an embedded MP3. Unfortunately, all FLV files recorded from the user’s microphone in by the Flash Player use the Nellymoser audio format. Nellymoser is a highly proprietary mono audio format designed solely for streaming speech. When we looked for a program to decompress this converter we found that Nellymoser offered a converter for $7,500."
• "one other converter that would do our decoding, the Total Video Converter. for only $50"
• "You can convert the flv to many different audio or video formats... To convert an FLV that contains video or video and audio, but not audio only, you may use the GUI which is self explanatory. Audio only clips can not be converted with the GUI at this time. (The application simply locks up when we try to convert Nellymoseraudio only FLV files)."

Therefore to record and extract audio from a user's mic it is necessary to record video as well &

this can be done using the following code:

var bandwidth:int = 0; // Specifies the maximum amount of bandwidth that the current outgoing video feed can use, in bytes per second. The default value is 16384.
var quality:int = 50; // this value is 0-100 with 1 being the lowest quality.
var camera:Camera = Camera.getCamera();
camera.setQuality(bandwidth, quality);
camera.setMode(320,240,15,true); // setMode(videoWidth, videoHeight, video fps, favor area)

// Now attach the webcam stream to a video object.
var video:Video = new Video();
video.attachCamera(camera);
addChild(video);

Depending on the project, you can change bandwidth, quality, and frame-rate settings to find the best combination.

http://blog.728media.com/2009/02/24/actionscript-3-webcam-configure/

Sending to Server:

Make network connection with local host...

var nc:NetConnection = new NetConnection();
nc.connect("rtmp://YOUR_SERVER_URI/vod/");

var ns:NetStream = new NetStream(nc);
ns.play("NAME_OF_STREAM");
// ns.play("mp4:NAME_OF_STREAM.mp4");