An interactive soft tangible interface helping kids with autism recognize, learn, and express their emotions
Oct 2017 - Ongoing
Interaction & visual design
The ability to express and manage one’s emotions is fundamental to children’s social and educational development, but this is a significant challenge for children on the autism spectrum. Autistic children often struggle with social interaction, verbal communication, and emotional and cognitive overload, making it difficult for them to express their emotions to others. Without an understanding of their emotions, caretakers and teachers cannot adequately support their needs or guide them towards problem-solving strategies.
How can we design an experience that helps kids on the autism spectrum communicate their emotions?
We designed a playful and comforting experience that helps children recognize, learn, and express their emotions. Knowing that children find comfort, familiarity, and enjoyment in plush objects, we designed our interface as a hybrid between a pillow and a stuffed animal. The interface is comprised of a pillow-like base and six cute plush "bubbles," which represent emotions.
Emoji-like faces on the bubbles create mental mapping between facial expressions and six core human emotions: sadness, happiness, disgust, anger, fear, and surprise.
When squeezed, the bubbles light up and gently vibrate to acknowledge the interaction. We limited the design to two modalities of sensory feedback in order to create a delightful but not overwhelming experience.
Users can express not only what they are feeling, but how much of a particular emotion they are feeling—as they squeeze a bubble with increasing pressure, the bubble and pillow's light becomes more and more intense.
Emotibubbles captures the fact that emotional states are complex, and it's normal for people to feel a mix of emotions. When multiple bubbles are squeezed, a color combination is displayed in the base.
After pivoting from another concept (due to technical reasons), we designed and developed Emotibubbles in 7 weeks.
Initially inspired by the popular Pixar movie Inside Out, we began by exploring other tools that support children's emotional intelligence development. We learned that children with autism and other developmental challenges often need support in this area.
To learn more about social emotional learning and autism, we read academic papers, pored through parenting blogs, and talked to parents & teachers.
"Social and emotional learning (SEL) is the process through which children and adults acquire and effectively apply the knowledge, attitudes, and skills necessary to understand and manage emotions, set and achieve positive goals, feel and show empathy for others, establish and maintain positive relationships, and make responsible decisions." — CASEL
"People on the autism spectrum often experience states of emotional or cognitive overload that pose challenges to their interests in learning and communicating." — Rosalind Picard, "Future Affective Technology for Autism and Emotion Communication" (2009)
"For many children on the autism spectrum, reading facial expressions is a daily struggle. Is my teacher happy with me or irritated? Is my sister worried or is she sad? Many children with autism can have a difficult time determining what other people are thinking and feeling, and because of this, struggle to partake in what neurotypical people may deem normal social interactions." — Autism Parenting Magazine (2015)
We learned a lot from speaking to Maureen Morgan, Director of the Wellesley College Child Study Center & former public school teacher who has 20 years of experience working with children who have autism and other special needs:
Many children with autism are sensory seekers who find various sensory inputs such as texture, resistance, and vibration to be comforting.
In classrooms today, teachers use tools such as PECs, stuffed animals, journaling activities, picture cards, and digital toy-like interfaces to help young kids with social emotional learning.
For teachers, having students who are able to manage and regulate their emotions is extremely important. Without this fundamental building block, teachers must use limited class time working with kids on their behavior instead of teaching them material.
Our goal was to design something that felt familiar and comforting to young kids. Thus, we began exploring the idea of a pillow-like plush toy, which eventually became Emotibubbles.
We learned how to sew, work with a microcontroller, build circuits, solder wires, and more, all for the first time. Dozens of wires, many needle-pricked fingers, and 1 fried Lilypad Vibe Board later, we finally got a working prototype!
Complete list of all technologies and materials used:
At the end of the semester, we had the opportunity to present our project to over 50 Wellesley students and faculty members.
After the presentation, there was a demo session where people got to interact with Emotibubbles and learn more about our design & development process from our research poster (see larger version):
We are partnering with the Wellesley College Child Study Center to conduct user testing with preschool children. In the future, we plan to explore different resistances and vibration patterns that correspond to the emotions (for example: anger may have high resistance and a slow vibration pattern when squeezed). To better inform our next steps, we are actively seeking advice from autism and child development research groups.