Technical details
With this idea we were able to transcribe human movement to MIDI (digital music protocol) form. By using MotionScore live music performances and dance videos, robot ensembles becomes possible. Performances where dances can be “remixed” with the music live may be possible.MotionScore converts human movement into a musical score. Currently computer generated characters and robotic movements are controlled by motion data, like video data, at 60fps. However human movements such as dancing moves to a tempo or a beat.
Think about dancing as an example; people move to the beat or rhythm. You count off 1…2…3…4 before you move, never would you hear a person say “raise your leg during the 3rd and 4th second!”
MotionScore focused on this aspect of human nature. Until now motion data took commands such as “from 3 seconds to 3.5 seconds move the right hand forward” and transcribed it into commands such as “from the 2nd beat to 3rd beat move your right hand forward.” This technology allows tempo to be used to control commands.
Movies
Exhibition content
1.Virtual DancerUsing Motion Score you can sync it with an uploaded dance video and the virtual dancer with dance along with it. It also reacts to users tapping the screen and will change its dance moves in real time!
2.ODDLETTER
ODDLETTER, an app which uses MotionScore technology. Upload a photo of yourself and a 2d version of you will start dancing within the app. The app is available on the app store now! If you can’t make it to the exhibit be sure to try the app!
