Computer Learns Sign Language By Watching TV

Posted By Ben Goulding, 9 July, 2009 | permalink

British scientists have created a computer that is able to learn sign language by watching television shows that are both subtitled signed.

Researchers at the University of Oxford, came up with an algorithm to recognize gestures made by people. The team exposed the computer to around 10 hours of TV footage that was both signed and subtitled, tasking the software to learn 210 nouns and adjectives that appeared during the footage, of which it correctly learned 136 words.

The software uses a person’s arms to estimate the rough location of fast-moving hands and identifies flesh-colored pixels in those areas to understand the precise hand shape, however difficulty often arises when some words have different meanings depending on the context. E.g. cutting a tree has a different sign to cutting a rose.

Although this is a clever piece of technology, it still has a long way to go. Rather than working round the problem, maybe the researchers should concentrate their efforts more on making deaf people hear again. What do you think?

Related posts:

  1. Gross Mechanical Tumor is a Computer Peripheral
  2. Extreme Computer Junkie
  3. Road Sign Misspells Three Out Of Four Words
  4. Geek’s Remains Stored In Computer Casket
  5. “Thinking Cap” Closer To Reality

Bookmark and Share
Posted in: Sci/Tech | Television
GEAT ARTICLES ON OTHER BLOGS...

Comments
Gareth Jones March 24, 2010

“Although this is a clever piece of technology, it still has a long way to go. Rather than working round the problem, maybe the researchers should concentrate their efforts more on making deaf people hear again. What do you think?”

No all people who sign are deaf, many people who sign have speaking difficulties. Even so developing this technology will be much simpler and quicker than working out a cure for all forms of deafness. A stop gap solution is better than waiting an age for a complete solution.

One useful application on this technology would allow non-signers to understand sign language, it would, therefore be easier for signers to communicate with non-signers. I know a profoundly deaf man who can get by with lip reading in order to understand non-signers, however he is still left with writing in order to communicate with them.

Sorry, the comment form is closed at this time.