Robot Vocal Expressivity

Prior research with preschool children has established that book reading, especially when children are encouraged to actively process the story materials through dialogic reading, is an effective method for expanding young children’s vocabulary.
A growing body of research also suggests that social robots have potential as learning companions and tutors for young children’s early language education. Social robots are new technologies that combine the adaptability, customizability, and scalability of technology with the embodied, situated world in which we operate.
In this project, we asked whether a social robot can effectively engage preschoolers in dialogic reading. Given that past work has shown that children can and do learn new words from social robots, we investigate what factors modulate their learning. In particular, we looked at whether the verbal expressiveness of the robot impacted children’s learning and engagement during a dialogic reading activity.
This project was funded by an NSF Cyberlearning grant.

We invited 45 preschoolers with an average age of 5 years who were either English Language Learners (ELL), bilingual, or native English speakers to engage in a dialogic reading task with a social robot. The robot narrated a story from a picture book, using dialogic reading techniques and including a set of target vocabulary words in the narration. Children were post-tested on the vocabulary words and were also asked to retell the story to a puppet. A subset of children performed a second story retelling 4-6 weeks later.
For approximately half the children, the robot’s dialogic reading was expressive in the sense that the robot’s voice included a wide range of intonation and emotion (Expressive). For the remaining children, the robot read and conversed with a flat voice, which sounded similar to a classic text-to-speech engine and had little dynamic range (Flat). The robot’s movement was kept constant across conditions.
We performed a verification study using Amazon Mechanical Turk to confirm that the Expressive robot was viewed as significantly more expressive, more emotional, and less passive than the Flat robot.

Overall, we saw that children learned from the robot, emulated the robot’s story during the story retelling, and treated the robot as a social being. Children did not report any differences between the robot conditions (i.e., they did not dislike or devalue the flat robot vs. the expressive one).
Nevertheless, the robot’s behavior still had an effect on children’s story processing. First, children showed more engagement in the Expressive condition. Second, in the Expressive condition, children who responded to the dialogic questions were also more likely to correctly identify more of the target vocabulary words, whereas this correlation did not emerge for children in the Flat condition.

Children in the Expressive condition emulated the robot’s story more in their story retells, using similar phrases as the robot, both during the initial session and during the follow-up. In the follow-up, children from the Expressive condition retained the story better than children in the Flat condition.
These results suggest that while children engaged and learned with both robots, they engaged more deeply with the expressive robot than with the flat robot.

The following people collaborated with us on this project:
Paul Harris – Graduate School of Education, Harvard University
Samuel Ronfard – Graduate School of Education, Harvard University
David DeSteno – Dept. of Psychology, Northeastern University