Music Box Toy that Transforms Drawings into Sound

music box toy that uses crayon drawings as inputback of toy that shows where to input paper

A Music making toy that takes marker drawings and turns them into sound so that even young kids aged 6-7 can learn the beginnings of music composition.

Professor

Kimberley Snyder

Team

Individual Project

Duration

2 months

Tools

3D printing, lasercut wood, Fusion360, RPi Zero, OpenCV, Keyshot, Figma

Research

Only 20% of high schoolers are involved in music education. If music is a core cultural practice that many enjoy, then why is participation so low ?

Despite growing up learning how to play classical music, I always struggled to figure out how to play modern pop songs, jazz music, or any other modern genre of music. Why is it so hard for me to play my favorite songs on the piano? Why didn’t my peers, who love listening to music, participate in music education? This question led me to explore music education as a topic.

Traditional music education focuses on performance, meaning only the dedicated few can participate.

Music education traditionally emphasized performance, excluding those who couldn’t keep up. In contrast, composition doesn’t rely on physical ability, making it more inclusive for a wider audience of people.

Keeping this in mind, a comparative analysis of competing products shows a product opportunity space in the physical music composition space.

I took note of non-western modes of music such as the japanese mode which has a Pentatonic scale (5 notes per octave) compared to the western 7 note per octave.

Of particular interest to me was the Shakuhachi notation.

The use of Western Classical Notation today is a highly contentious and controversial topic, with opponents declaring that “Notation Must Die”, or that the written score is an elitist “cult”

There is a surge in interest in ‘vintage’ and creation toys

Focused on toys for the project, I explored the concept of educational toys and interviewed the owners of Rhen’s Nest, a local shop specializing in educational and puzzle toys for kids.

Ideation to Prototyping

I focused on non-screen interactions familiar to children

The move away from screens gave me space to explore unique designs. After considering the day-today activities of a child, I settled on drawing as a interaction.

The Music box, a simple and familiar visual story.

Utilizes OpenCV Image Recognition on a Raspberry Pi Zero to enable limitless ways to draw composition.

I detect colors on the paper, line by line, translating each to a different note using MIDI. A VST plugin at the other end takes the notes in and converts this into sound. Find the source code here

Unlimited ways of guided or unguided learning through different printed pages. The versatility allows for first-class support of non-western world music.

Redesign

Rethinking familiarity in visual stories for children through calling back to the classic machined wooden toys.

The form felt unresolved, so I went back to the drawing board. I wanted to call back to the classic machined wooden toys, rather then the lasercut method I originally used.

Reflection

This project speaks to me personally, growing up I had learned to play classical music. However, in my later years I came to the realization that this knowledge was often difficult to apply into modern music. I was fascinated with modern UX developments in music production, and also post-colonial critiques on western music theory. This project let me experiment with applying my own philosophy of design, alongside learning OpenCV, woodworking, and lasercutting.

Simon Zhang is available in the following addresses:

linkedin