I read an interesting article the other day about IKEA sticking with Flash to deliver its ideas and there’s a quote in it from their marketing manager Nicolas Johnsson that says ‘Mobile-friendly tech [HTML5] does not support streaming video, animation of hotspot positions on top of video or sophisticated animation’ and it got me thinking about the features and functionality of the components of HTML5 when compared to Flash especially when it comes to interactive video. We can do all sorts of clever stuff with Flash video, from overlaying interactive elements and embedding hidden links to animating the entire video itself in 3D space.
It’s all very cool and something we have grown to expect – the immersive and beautifully crafted experiences that companies like IKEA are creating show off Flash to its full potential (and often take it further) – however having spent the past few weeks delving into HTML5 video I’m pretty disappointed not only with what’s on offer but the reliability and stability of it; having to encode multiple versions for different browsers is a pain and then finding that certain functionality you wrote that works in one browser just ever-so-slightly doesn’t work as expected in another.
So, with the words of Nicolas Johnsson ringing in my ears (specifically the animation of hotspot positions) I’ve worked hard to show that it can actually be done albeit without the stability and browser penetration that the Flash player offers.
The process to get here has been pretty convoluted, involving several different bits of software and libraries (Flash, After Effects, Adobe Edge, Greensock JS, my own bespoke JS video timeline library – I had to get my hands dirty on this one!) and yes, it had to involve Flash in order to gather and generate the tracking data. So whilst this demo may well be a world first (I think it might be, I haven’t found it done anywhere else online yet but do ping me if I’m wrong) it still feels like it couldn’t have been possible without a technology that it seeks to replace.
Some things to note: I have tested it on Safari 5.1.7, Chrome 19, IE 9 and Firefox 13 and it seems to work well. It doesn’t work on an iPhone because the video is loaded into and played in the iOS native video player (which operates outside of the realms of HTML control). I’m pleased to report it does work on the iPad – it has been a little unpredictable although I have now managed to reduce that unpredictability down to almost zero now. There is a lot more work needed to simplify and streamline the workflow – if you happen to root through the code you’ll find it’s not very neat right now and lots of the code methods could do with optimising but it’s my first stab at it! Also the video is from the Adobe blog Creating Interactive Video with After Effects and Flash .
You can click/tap several times on the motion tracked label and see some data about the rider – the last bit of data is a link to a YouTube video to demonstrate that links can also be placed in these overlaid elements. What’s also cool is that the tracked data is synchronised with the video timeline using Greensock’s TimelineMax so when a user scrubs back and forth using the video controls (which aren’t enabled here but you can right click the video to enable them) the overlaid elements will also scrub with it.
I hope this experiment demonstrates the possibilities that HTML5 video can offer – I’m pretty excited by these results and the possibilities it presents.
And please do leave your thoughts and experiences of it in the comments – I’m really interested to hear how this is working in your browser!
Click here to launch the Interactive Motion Track demo
Click here for a tutorial on how to get Greensock’s Tween library working in Adobe Edge.
Update July 2014: If you liked this post check out my more recent 3D motion tracking stuff.
CHRIS GANNON 
