Researchers Use Video Game to Crack the 'Language Code'

May 18, 2011 -

Carnegie Mellon University's Lori Holt and Sung-Joo Lim and Stockholm University's Francisco Lacerda are using video game training with a mock "alien" language to replicate the challenges of learning languages as an infant. The research found that listeners were to quick recognize word-like units. The study was funded by grants from the National Science Foundation, National Institutes of Health and Riksbankens Jubileumsfond.

To uncover how spoken sounds are decoded by the brain, the research team designed a video game narrated in deliberately distorted speech. The soundtrack (unintelligible babble in any language) was the only source of instruction for the 77 adult players in the study. After only two hours of play, the participants could reliably extract world-length sound categories from continuous alien sounds and apply that learning to advance through the game.

"Traditionally, when we study adult learning in the lab, it's nothing like how infants learn language," said Holt, professor of psychology at CMU and a specialist in auditory cognitive neuroscience. "This video game models for adults the challenge language learning poses to infants. This presents the opportunity to study learning in ways that are just not feasible with infants."

Lacerda, professor of phonetics and an expert in language acquisition, agrees that using video games is a promising new way to explore language learning.

"This is a wonderful opportunity to approximate the task facing infants by creating a setting where adults are forced to infer what the meaning of different sound elements might be, and to do it in a functional way."

The research has the potential to help researchers better understand and effectively treat a number of conditions including dyslexia and improving second language learning.

Lim, a graduate student in psychology at CMU and lead author of the study, has used the game to help adults learn English.

"Native speakers of Japanese can use this type of training to learn English consonants they have difficulty distinguishing," she said

Holt, director of CMU's Speech Perception and Learning Laboratory, is interested in taking the study further to determine how the video game and its alien soundtrack engage different areas of the brain to produce rapid learning. The next step is to investigate this by observing players with functional magnetic resonance imaging (fMRI) to view real-time brain reactions to the video game.

Researchers will present their findings at the Acoustical Society of America's annual meeting May 23-27 in Seattle.

For more information on research at Carnegie Mellon, visit www.cmu.edu/research/brain.


Comments

Re: Researchers Use Video Game to Crack the 'Language Code'

I love this stuff.  I majored in CS and minored in linguistics; Watson's performance on Jeopardy was one of the most fascinating things I've ever seen.

Re: Researchers Use Video Game to Crack the 'Language Code'

That is just too cool 

Re: Researchers Use Video Game to Crack the 'Language Code'

This is really cool. Like really cool. I want to try doing that, play an unfamiliar game in an unfamiliar language, too bad imports won't work on my consoles.

-Austin from Oregon

Feel free to check out my blog.

 
Forgot your password?
Username :
Password :

Poll

Which Feminist Frequency video are you looking forward to most?:

Shout box

You're not permitted to post shouts.
prh99Devolvers response to Nintendo's creator program https://mobile.twitter.com/devolverdigital/status/56082061398444851301/31/2015 - 3:24am
Andrew EisenI only know of him being annoyed that IGDA mistakenly hosted on its anti-harassment tools page a blocklist he was on. What else was he critical of?01/31/2015 - 2:19am
Goth_Skunk@WonderKarp: Given Rosario's outspoken criticisms of the conduct of the IGDA, I would not be one bit surprised if he's being strong-armed into silence in closed-door meetings. He's not towing the party line, after all.01/30/2015 - 11:38pm
MechaTama31EZK: Thank you! (And I was able to see that you did it, only because you did it. How appropriate! :)01/30/2015 - 11:37pm
Matthew Wilson@Monte Nintendo is a Japanese company though. consoles are dead in japan. ps sony is a international company, so they do not much care about Japanese sales much.01/30/2015 - 10:11pm
MonteOnly one generation ago nintnedo had the best selling console. They know their games are still popular and can move their hardware. They just need to focus on better gaming hardware and advertising (their youtube policy not helping)01/30/2015 - 9:58pm
MonteI'd expect Nintendo to keep making consoles, much like the gamecube i doubt nintendo will see the Wii U as a sign of the end for consoles. There are multiple reason why the Wii U failed a big one being marketting.01/30/2015 - 9:55pm
Wonderkarpwhen asked on Twitter, the Chairman said He couldnt comment at this time https://twitter.com/siloraptor/status/561284497275367424. still looking for facts01/30/2015 - 9:41pm
Wonderkarphttp://pressfarttocontinue.com/2015/01/30/igda-puerto-rico-closed/ this is what I have so far01/30/2015 - 9:27pm
Wonderkarpso allegedly, IGDA has closed their Puerto Rico branch. This comes after the Chairman of the Puerto Rico branch was blocklisted by Randi Harper. I'm trying to dig up more info on it. all I have is a link with some info, but its not cited.01/30/2015 - 9:27pm
Andrew EisenSo, kinda like the Wii U just with a GamePad that's portable?01/30/2015 - 9:13pm
Matthew Wilson@AE I hope they do a Ipad/iphone thing. make a home console/handheld where games are playable on both. that is what I would want from them.01/30/2015 - 9:00pm
Andrew EisenThat would be a shame but as long as I can play comfortably on my TV I'll probably be fine with whatever the future brings.01/30/2015 - 8:56pm
Matthew Wilson@AE I would get used to it. I do not think nintendo is going to do another home console , but I suspect a hybrid one. 1. they are doing worse than the gamecube. 2. the home console market is dead in japan. even the ps4 isnt selling there.01/30/2015 - 8:54pm
Andrew EisenAside from that (and the aiming isn't too smooth but it's turn based so it's not a big deal (maybe better with Circle Pad Pro or New 3DS?)) my only other quibble is I'd much prefer playing it on a console. But I say that about nearly every handheld game.01/30/2015 - 8:39pm
Andrew EisenFrom what I've seen of Valkyria Chronicles, this is significantly slower. Mechanically, it's fine and plays into the game well. It just takes too long.01/30/2015 - 8:37pm
Matthew Wilsonyup like VC it moves every enemy each turn one by one.01/30/2015 - 8:33pm
Andrew EisenI played most of the Codename Steam demo. Pretty neat so far but the enemy turn (it's turn based) takes way too long.01/30/2015 - 8:31pm
Goth_SkunkJust finished the first episode of Life is Strange. The writing is hit-or-miss, the lip-syncing is awful (but I can't fault an indie dev for that), but the chronokinesis mechanic is dynamite. Love the concept. Can't wait to play more.01/30/2015 - 8:23pm
Andrew EisenNintendo invites disaster!01/30/2015 - 7:34pm
 

Be Heard - Contact Your Politician