A few months ago, Microsoft made headlines by announcing at Code Conference that Skype would soon be able to translate language between speakers in real time—that product, which Microsoft calls simply Skype Translator, if successful will be the first real time language translator—a primitive version of the Universal Translator of Star Trek fame.
The demo showed two people talking in real time using Skype—one in English, the other German. The words by the speakers were displayed as translated text on each other's screen and were also played aloud by a voice generator that allowed the listener, to listen in their own language. Now new details of how Microsoft has achieved this feat are coming out, and some of them are a little surprising, Phys.org reported.
Representatives with Microsoft have revealed that the main "trick" in real-time language translation is recognizing the difference between spoken and written languages. When people talk, they note, they use all manner of cues to convey meaning—they hem and hah, and use words like um to give them more time to respond. They also pronounce words differently, using tone changes to convey added information, such as whether what they are saying is a statement, or a question. To build their translator, Microsoft started by feeding text from books and articles into a software engine designed to tear sentences apart looking for meaning in smaller phrases—and then mapping them to similar phrases in the target language. That approach led to some advancements in the neural network (learning part of the software) but more work still needed to be done. It turns out, Microsoft engineers looked to social media sites because of the unique language structure used by the people who use them to convey meaning in a way that is more like conversation than typical written documents.
Because the different social media sites tend to have their own language characteristics, Microsoft started by writing software that could normalize posts from the various sites, arriving at a common format. Adding the normalized data to the original software engine led to a dramatic improvement in the sound of conversational language, and it's what users will hear once Skype Translator is finally released as a beta app for Windows 8 by the end of this year.