Scholar Yang Wu on “Emotion as Information: Inferring the Causes of Others’ Expressions of Emotion”

By Tayna Gonzalez-Rivera, Teacher

Adults infer and learn so much from others’ emotional expressions. If we see someone looking at a screen with a terrified look, we can infer that this person might be watching a scary movie or show. If we see a picture of people gathered around a TV watching excitedly or nervously, we can infer they could be watching a sports event. Are infants and children capable of these kinds of inferences? Can infants and children use observed emotional expressions as ways to deduce information about the world? These are the questions postdoctoral scholar Yang Wu posed in her talk at this spring’s staff development day. Wu works alongside Professor Michael Frank in Stanford’s Language and Cognition Lab and Assistant Professor Hyowon Gweon at Stanford’s Social Cognition Lab, and she is interested in how we observe emotional cues to gain information about the world, and specifically how and to what extent children and infants can do this as well.

Wu spoke about her work, which focuses on external causes of people’s emotional responses, citing these external causes as “reliable predictors of particular emotional responses.” To illustrate this, Wu showed pictures of moldy food, a broken bridge, fireworks, a peaceful landscape and a cute baby. These pictures corresponded with feelings of disgust, fear, excitement, awe and affection. With this principle in mind, Wu’s first study investigated whether children map emotional expressions onto their external causes. To do so, Wu selected simple and easily identifiable external causes that would generate distinct, positive emotional vocalizations. She presented these images and their corresponding vocalizations. The image categories were: silly faces (which corresponded to laughs), cute babies (which corresponded to “awws”), slices of cake (which corresponded to “mmms”), light-up toys (which corresponded to “ooos”) and crying babies (which corresponded to “oooh”). The study was conducted with 2- to 4-year-olds who were presented with two images from the categories above, e.g., a silly face and a light-up toy, at random. The researchers introduced the children to a doll named Sally, and explained to them that Sally makes a sound when she’s looking at a picture. They then asked the children to identify what image Sally was looking at, based on the sound she emitted. For example, if Sally says “mmm,” is she looking at a cute baby or a piece of cake? The study found that 2- to 4-year-olds all performed accurately in pairing emotional vocalizations to their external causes, and 4-year-olds reached adult-level performance in all categories.

This encouraged Wu and her researchers to look at even younger children. This study aimed to determine if 12- to 17-month-old infants could match emotional vocalizations to their external causes, and the task was entirely nonverbal. Each child was shown two pictures on a screen while a vocalization matching one of the images was played for four seconds, followed by three seconds of silence. Each sound was repeated twice. Wu was interested in knowing if the infants would shift their gaze to the picture that matched the vocalization. The study showed that this was the case, and concluded that infants between 12 and 17 months old can distinguish at least five kinds of positive emotional vocalizations and can connect them to their probable causes.

Wu’s next study delved even deeper. She asked the question: “Can infants use this fine-grained understanding of emotions to guide their exploration of the world?” Before the session, a researcher placed an item inside a box: an orange or a toy puppy. During the session, a child did not know what was inside the box but watched the researcher look in the box and make a vocalization, such as “aww.” The box was then handed to the infant. Wu predicted that if the infant found an orange, he or she might keep searching, since the “aww” sound indicated something adorable. By comparison, if the child found a toy puppy in the box, he or she would probably not keep searching. The study found that, indeed, infants searched longer when what they found failed to match the vocalization than they did when the item matched. Through these studies, Wu established that 12- to 17-month-old infants can not only distinguish five positive emotional vocalizations and connect them to their probable cause, they can also use this fine-grained understanding to guide their exploration of hidden causes of emotional expressions, and that these abilities are early-emerging and sophisticated.

Wu then talked about a study she’s conducting at Bing, exploring the role of surprise in learning. Wu wanted to answer whether or not children can use others’ expressions of surprise to gather information. In this study, a researcher presented 3- and 4-year-old children with a novel toy. The researcher acted as though she did not know how the toy worked. The researcher then played with the toy in front of the child and discovered a function that the child then tried out. This established common ground between them: They both knew something about the toy and its function. Then a second researcher, who never played with the toy, came in the room. In one condition, the first researcher continued to play with the toy behind an occluder and expressed a big surprise; the child could see her expression but not the toy. The researcher then gave the child the toy to play with while she took up some paperwork with the second researcher in the corner of the room. In the second condition, everything was identical except that the second researcher, rather than the first researcher, played with the toy behind the occluder and expressed surprise.

Wu predicted that children would be more persistent in searching for a novel function of the toy in the first condition than in the second condition. Wu stated that in the first condition, the researcher had common ground with the child, and her surprise could indicate the discovery of a novel function. In the second condition, by contrast, the second researcher did not have common ground with the child, and her surprise might either indicate the discovery of a novel function or the discovery of the same function that the child already knew about. The study did, in fact, show that children were more persistent in searching for a novel function in the first condition. Wu highlighted how this study shows that preschoolers consider not only pedagogical and verbal cues, but also others’ expressions, and they use them to guide their learning of the world.

The findings of Wu’s studies show us that children, starting at a very young age, have a sophisticated ability to read others’ emotional expressions, and that they gather information through this process. To finalize her talk, Wu reminded us that we are constantly and consistently emoting and expressing around children, so we have the responsibility not only to mind what we say and how we act, but also how we emote in front of children.