Learning French with Sophie

This project proposed an approach to language learning for preschool aged children using social robots as conversation partners within a shared play context for children and their families. It addresses an underserved age for language learning, where early learning can greatly impact later educational success, but that cannot benefit from text-based interventions.
With the goal of establishing a shared physical context between multiple participants without
absorbing all of the children’s focus onto digital content, a hybrid physical and digital interface
was iteratively designed and play-tested. This interface took the form of a “café table” on which
the child and robot could share food. A robot was programmed to introduce itself and name
foods in French, eat some foods and express dislike towards others, respond with distress to a
new object, show its focus of attention through gaze, and in one experimental condition,
express feedback about its comprehension when spoken to in French or English.
This project was about the following ideas:

1. How might one establish a shared context for play that is digital, and therefore can be adapted easily, but also maintains focus on social cues and not only a screen?
2. What is a play pattern that supports conversation anchored in a specific set of objects and actions but is still open-ended, and how might the technology’s form prompt this pattern?

This project was supported by an NSF Cyberlearning grant.

We ran a controlled study with 16 participants between the ages of 3 and 6, their parents, and in some cases their older or younger siblings. The study was designed to learn if feedback from the robot could influence the child’s use of language towards it.
In the experimental condition, it was presented as a completely French- speaking robot that non-verbally expressed interest whenever the child spoke to it – confused interest if the child spoke English, as though it was listening hard but did not understand, and pleased interest if the child spoke French to it. In the control condition, it engaged in the same shared activity but did not give the child feedback when it was spoken to.
The primary research question was about children’s use of language during the interaction with the robot in response to its behaviors. Would they treat the robot as a conversational partner? Would they try to speak to it in French, or use other strategies to communicate? Would they be more likely to speak to it in the “feedback” condition? A secondary question was whether children would learn language from the robot by listening to and/or using its utterances.
I was also interested in observing parents’ participation throughout the interaction. Would they speak to the robot directly, modeling treating it as a social agent? Would they “play pretend” as though they were playing with dolls with their child? In what ways would they scaffold the child’s social behaviors and speech? Would the “play pattern” of eating together support parent’s participation?
The study found that some children as young as 3 years old would treat a social robot as an
agent capable of understanding them and of perceiving a shared physical context, and would
spontaneously modify their use of language and gesture in order to communicate with it –
particularly when the robot communicated confusion. The study also found that parents tended
to frame their scaffolding of the children’s behavior with the robot in a social context, and
without prompting aligned their guidance and reinforcement with language learning goals.
After one exposure to the robot and new French vocabulary, children did not retain the robot’s
utterances, but engaged in communicative and social behaviors and language mimicry
throughout the interaction. The system appeared to support multi-user social participation,
including both caretakers and siblings of the participants.