Researchers Use Video Game to Crack the 'Language Code'

May 18, 2011 -

Carnegie Mellon University's Lori Holt and Sung-Joo Lim and Stockholm University's Francisco Lacerda are using video game training with a mock "alien" language to replicate the challenges of learning languages as an infant. The research found that listeners were to quick recognize word-like units. The study was funded by grants from the National Science Foundation, National Institutes of Health and Riksbankens Jubileumsfond.

To uncover how spoken sounds are decoded by the brain, the research team designed a video game narrated in deliberately distorted speech. The soundtrack (unintelligible babble in any language) was the only source of instruction for the 77 adult players in the study. After only two hours of play, the participants could reliably extract world-length sound categories from continuous alien sounds and apply that learning to advance through the game.

"Traditionally, when we study adult learning in the lab, it's nothing like how infants learn language," said Holt, professor of psychology at CMU and a specialist in auditory cognitive neuroscience. "This video game models for adults the challenge language learning poses to infants. This presents the opportunity to study learning in ways that are just not feasible with infants."

Lacerda, professor of phonetics and an expert in language acquisition, agrees that using video games is a promising new way to explore language learning.

"This is a wonderful opportunity to approximate the task facing infants by creating a setting where adults are forced to infer what the meaning of different sound elements might be, and to do it in a functional way."

The research has the potential to help researchers better understand and effectively treat a number of conditions including dyslexia and improving second language learning.

Lim, a graduate student in psychology at CMU and lead author of the study, has used the game to help adults learn English.

"Native speakers of Japanese can use this type of training to learn English consonants they have difficulty distinguishing," she said

Holt, director of CMU's Speech Perception and Learning Laboratory, is interested in taking the study further to determine how the video game and its alien soundtrack engage different areas of the brain to produce rapid learning. The next step is to investigate this by observing players with functional magnetic resonance imaging (fMRI) to view real-time brain reactions to the video game.

Researchers will present their findings at the Acoustical Society of America's annual meeting May 23-27 in Seattle.

For more information on research at Carnegie Mellon, visit www.cmu.edu/research/brain.


Comments

Re: Researchers Use Video Game to Crack the 'Language Code'

I love this stuff.  I majored in CS and minored in linguistics; Watson's performance on Jeopardy was one of the most fascinating things I've ever seen.

Re: Researchers Use Video Game to Crack the 'Language Code'

That is just too cool 

Re: Researchers Use Video Game to Crack the 'Language Code'

This is really cool. Like really cool. I want to try doing that, play an unfamiliar game in an unfamiliar language, too bad imports won't work on my consoles.

-Austin from Oregon

Feel free to check out my blog.

Forgot your password?
Username :
Password :

Poll

How do you feel about microtransactions in $60 video games?:

Shout box

You're not permitted to post shouts.
RedMageOh hey, a news station is trying to scapegoat gaming. That takes me back. https://twitter.com/GamingAnarchist/status/63754361236507443208/29/2015 - 1:40pm
Big PermA link to TB twitter with Matt Lees in the replies - https://twitter.com/Totalbiscuit/status/52528604482949939208/29/2015 - 12:16pm
Brad GlasgowWhy would he say the company lies about getting abuse? Oh, because people don't abuse GG?08/29/2015 - 10:38am
Big PermBrad - Matt Lees was also quick to say the company "Gamers Gate" lies about getting abusive messages thinking they were an official GG channel08/29/2015 - 9:11am
Goth_SkunkMGSV: The Phantom Integrity - A Rant by RazörFist. (NSFW on account of language). RazörFist discusses the latest batch of unethical journalist conduct, with a caveat. - http://ow.ly/RwXYT08/29/2015 - 7:10am
Goth_Skunk@Brad: I can.08/29/2015 - 6:13am
Goth_SkunkI assume "Stacy" is a pseudonym. After reading what she went through, I would not be one bit surprised if it is.08/29/2015 - 6:13am
Goth_SkunkA Year of #GamerGate: From Neutral To Anti To Neutral To Pro by "Stacy" - http://ow.ly/RwVeT08/29/2015 - 6:12am
Brad GlasgowI can't believe Matt Lees deleted his positive review of Ethan Carter because Chmielarz is sympathetic to GG.08/29/2015 - 5:30am
Goth_SkunkA GameDev's Year With #GG: The Good, The Bad, and The Ugly by Adrian Chmielarz - http://ow.ly/RwSCd08/29/2015 - 5:18am
Goth_SkunkDespite not being a fan of fighting games I had to check out that R Mika trailer. Loved it. Still won't buy the game though, on account of Isuckatstreetfighteritis.08/29/2015 - 2:42am
MechaCrashI use a Dynex DX-840 headset, but it's discontinued. :( I wanted a mono headset so I could keep the other ear free for my speakers, but it has the bonus of being very light and comfortable, so you don't notice it.08/29/2015 - 12:41am
Big PermSora - I was just having a slow day at work earlier. Now I'm home with vidya!08/28/2015 - 7:54pm
ZippyDSMleeSora-Chan: Blender is easy compared to 3Dmax :P08/28/2015 - 6:51pm
Sora-Chantime to take up a hobby? maybe messing around in GIMP to make wallpapers? use qCAD to design somethin? open Blender and stare at it for a couple hours trying to figure what does what?08/28/2015 - 6:41pm
Big PermAlso, yes. I've been spamming the shoutbox. I don't have much going on today, don't judge me08/28/2015 - 3:25pm
Big PermThanks, but yeah. Not sure I wanna drop that kind of cash :P I don't even mind the sound quality of my krakens, it just hurts to wear em after a couple hours.08/28/2015 - 3:25pm
Sora-Chan@Big Perm: I'm a bit of a fan of the Omega Recon3D headset from SoundBlaster. Though it is a bit expensive.08/28/2015 - 2:36pm
Big PermI actually need to look into a new headset. I have those green razer krakens and I would not suggest them. Though maybe they're better for people without glasses08/28/2015 - 11:32am
Big PermI've never heard if Apple was dominant in Japan.08/28/2015 - 11:25am

Be Heard - Contact Your Politician