3D Motion Tracked Interactivity on HTML5 Video of an iPhone

Just a short snippet of some further experiments I’m doing with interactive 3D motion tracking. This is a continuation of this experiment.

I shot some footage of me holding an iPhone, tracked the screen and converted the tracking data. I then added interactivity to the buttons (in Adobe Edge Animate) so that you can press the actual buttons on the video and those numbers appear on the phone (all tracked in 3D space).

You’ll notice that buttons presses (and the beep sound) are a bit laggy – I haven’t quite worked out how to get this properly sync’d yet. If anyone knows then feel free to leave a comment – I’d love to know!

And in true reality-feedback style, this does actually run on an iPhone 4 (remember them?), albeit a bit stuttery.

Update: The button/audio lag is now fixed – I realised that I had made the schoolby error of using the ‘mousedown’ event for the buttons instead of ”touchstart’ – I had also missed off the view-port meta tags which can also avoid the 300ms lag in some browsers – thanks to @VirgilRocks for the nudge!

Chris Gannon Edge Animate Templates




5 thoughts on “3D Motion Tracked Interactivity on HTML5 Video of an iPhone

    1. I realised I had made the schoolboy error of using ‘mousedown’ instead of ‘touchstart’ – all fixed now (I’ve also added in the view-port meta tag just to be sure). Cheers!

  1. Hi Chis
    Really cool experiment! Just a question. Can’t seem to find any guide on how you hook up the touch/scrub event to the draggable – you know, so you can scrub forth and back in the video. Are you using some kinde of currentFrame to do that or?

    Thanks for a really great blog.

  2. Thats amazing! Can you please tell me what code you used for clicking the numbers?

    When I make a:


    It seem to replace the existing text rather than add more text to it.

    Though it is properly af newb question, I look forward hearing from you

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s