top of page

Curlybot+

a robotic toy recording and replaying sound and motion to foster multimodal play 

 

Project team: Quincy Kuang, Lingdong Huang, Lucy Li

Based on the original Curlybot made by Phil Frei

*Note: Curlybot+ is an ongoing work, AI component have not been implemented yet.

Introduction

Physical play is integral to childhood development. Often children pair sound, dialogue, and onomatopoeia with their actions and interactions. Through these imaginative experiences, simple toys become portals to fantastical worlds, enriching cognitive and social growth. However, screen-based video games and entertainment media often fail to preserve the tangible aspects of physical play. This paper introduces Curlybot+, a robot designed to record and replay the auditory and kinetic elements of play. By aligning with children's natural play patterns, Curlybot+ supports imaginative engagement through an interactive modality of sound and movement playback, fostering deeper creative expression. We aim for Curlybot+ to enhance physical play by making tangible toys dynamic and interactive, providing an experience that outperforms digital media. This effort seeks to reintroduce a more embodied form of play, encouraging physical situated engagement in an increasingly mediated digital world.

Evolution of Curlybot+

CurlyBot TangibleAI.png

Toys in an increasingly digital world

In recent years, tangibility has steadily declined with the rise of digital entertainment platforms. Screen-based media, with simulated worlds, captivating narratives, and interactive experiences, have become the primary source of entertainment for children. However, several recent studies have revealed that increased screen-time significantly impacts children’s mental imagery performance \cite{suggate_screentime_2020,muppalla_effects_2023}. Similarly, early exposure to video-streaming platforms, social media or adrenaline driven video games, severely impacts children’s identity formation \cite{holloway2024} and emotion regulation. On the other hand, while physical toys are generally more beneficial for children's development, they often lack the engaging and dynamic experiences that digital platforms provide. Overall, the call for a rethinking of children’s play in a digital world steadily rises \cite{livingstone_beyond_2022}. 

One interesting approach are computational toys that situate themselves between screen-based devices and passive physical playthings. Previous HCI works of \cite{Merrill,Raffletopobo,Schweikardt} extensively demonstrate computational toy’s potential to stimulate imagination whilst preserving an adequate exposure to digital technologies. The educational opportunities of computational toys include but are not limited to objects-to-think-with \cite{papert_mindstorms_1993}, making knowledge approachable in new ways and support multiple playing styles.

In this paper we present Curlybot+, a robot toy that captures and plays back sound and motion to foster physical play. The Curlybot+ is a further development of Curlybot introduced by Frei in 1997 \cite{frei_curlybot_2000}. We were given permission by the author to build on the technical concept of Curlybot - an autonomous two-wheeled vehicle with embedded electronics that can record how it has been moved and then play back that motion accurately. However, instead of applying it to educate young children about advanced mathematical concepts away from a traditional computer, Curlybot+ aims to explore multimodal interaction to support tangible play to address contemporary educational challenges resulting from an increased screen time as discussed before. 

To tackle this, we expanded the Curlybot+ in two ways.First, we implemented the ability to record and replay sound alongside movement, linking the two through a variety of interactive experiences. Because of this, children can capture and directly manipulate ambient sounds, music or even their own voices and onomatopoeias to engage in multisensory play. Secondly, we designed the Curlybot+ to have swappable and modular shells. Pretend play \cite{lillard_why_2017} toys such as doll houses or doctor’s kits are well known for their benefit in supporting fantasy play. Similarly, the playful shells of Curlybot+ allows it to visually take on the form of animals or functionally become a musical instrument. 

Technology approach

We selected the ESP32-S3 module as the main controller. Clocked at 240MHz, it also offers extensive PSRAM and flash options, both of which are needed for fast storage and retrieval of audio and motion data on the board. We use the built-in USB functionality for firmware updates and debugging. We utilized a pair of HEDS5540 motors with Integrated rotary encoders to record the turning of the wheels while the user is holding and moving the curlybot. Rotary encoders may be read by observing the two out-of-phase signals sent through their output lines, and the direction of the rotation is thus inferred by checking the state of one of the lines (A) when the other (B) is rising (or falling). We record such data at a high sample rate, the small interval of which shall be thus referred to as “ticks”. At each tick, we expend two bits of internal SRAM of the microcontroller to record the raw state of both lines. During the playback phase, these two-bit-states are read back, and compared with the immediately preceding sample, and the motor, (or rather, the motor driver), is given a corresponding two-bit signal: LOW-HIGH (turning in one direction) for an A-LOW on a B rising edge; HIGH-LOW (turning in the other direction for a A-HIGH on an B rising edge; all other conditions lead to a LOW-LOW (not turning). We use the H-bridge-based, TB67H451AFNG DC brushed motor driver to actuate the motors. Each motor is provided with a separate 9V battery–this ensures enough power and simplifies power management circuits. 

Prototyping process:

Final Prototype

This is the initial prototype of Curlybot+, more development will come.

Find me on:

  • 77-behance-512
  • YouTube - Black Circle
  • LinkedIn - Black Circle
bottom of page