Shad Gross

  1. home
    1. overview
  2. research
    1. materiality
    2. publications
  3. design
    1. tangibles
    2. system design
  4. bio
    1. background
    2. education
    3. honors/awards
    4. interests
  5. cv
    1. pdf
    2. read online

3. design/melody bot


  • concepting
  • developing – processing, java
  • deployment

promotional material

a. concept

I, as part of a team with Dr. Shaowen Bardzell and Dr. Jeffrey Bardzell, created this design as part of the Intel Science and Technology Center (ISTC) for Social Computing with the goal of encouraging discussions surrounding the ideas of self-tracking, body image, and surveillance. Specifically, this design focuses on how a sedentary lifestyle can result in weight gain.

The Melody Bot takes video captured by a custom web camera and uses it to generate custom music videos. Similar to the CC, the web camera takes on a playful form – a three inch tall, stylized and cartoonish human form with the actual camera embedded in it’s television-shaped head. Clear epoxy resin makes the functional circuits of the web camera’s PCB visible, using it as an aesthetic element, rather than something to be covered. Through capturing time spent at the computer, a sedentary activity, the goal of this design is to cause cognitive dissonance between the often-upbeat nature of music videos and the video collected.

melody bot profile melody bot head on

b. prototype design

Polyester resin was used, as opposed to 3D printing, to obtain greater clarity and smoothness. This required the creation of a two-part mold for the resin, created by first modeling the character in the polymer clay and then using that model to create molds for the resin.

The application that captures video was written in Processing. Video clips are captured through the Melody Bot camera, while the amount of change at the pixel level is used to create a measurement of the amount of motion the video contains. When the user uploads a song, the waveform of the song is scanned for peaks, and the distances between these peaks used to determine patterns in the music – specifically determining a basic interval that is repeated throughout the song. The recorded video clips are then cut into smaller sub clips that are the length of the interval and are assembled together based on a comparison of the amount of movement in the sub clip and the number of peaks in each interval of the song.

While initially, the plan had been to simply have the videos cut and assembled with no further modification. However, due to an accidental error in the order of video compression, artifacts were introduced at the points where the video was cut. Ultimately, these errors were left in as an aesthetic element.

Melody Bot Use Sketches