Sing It KittyThree
Sing It Kitty is a mobile web-app that turns a selfie into an all-singing, all-dancing music video.See the project
Three continues their mission to encourage the UK to share silly stuff online, with Sing It Kitty. We worked with Wieden + Kennedy to create a mobile-first experience, which allowed users to take selfies and photos of their mates, and have then mapped in 3D, in real-time, into a music video featuring a singing kitten.
We began by protoyping a core approach in Adobe After Effects, taking care to simulate only technique we knew would be transferable into a programmatic environment.
We experimented with different object marker methods before settling on hypoallergenic adhesive dots, which were easy to add and remove.
We produced a number of test shots to determine what would work well and what was best avoided, which really helped when it came to supervise on set.
Once the shoot had wrapped, we developed a workflow for tracking the resulting footage in SSONTech Syntheyes, before exporting animated point clouds for editing and refinement in Maxon Cinema 4D.
We used these point clouds to reconstruct a generic 3D face mesh, which we projectively textured with a variety of placeholder images to ensure a good average fit. We then exported the results as a series of 16 bit UV maps compatible with our server side rendering system.
Jaw and eye movements were implemented using 'Moving Least Squares' warping functionality, driven by an animated spline we created in After Effects before exporting via a JSX script.
We had a separate day's shoot to capture assets for the singing mouth, which we then pulled apart in post production into a number of different separate elements. These were then recombined in various distorted and warped combinations to generate a total of 9 variations of mouth scale and lip fullness.
To integrate the singing mouth seamlessly with the user's uploaded face, we utilised an advanced blending algorithm known as 'Mean Value Coordinate Image Cloning'. To blend the whole face with the entire scene, we performed white point normalisation, contrast stretching and hue alignment on the uploaded image.
To drive all this programmatic compositing we utilised SkyBiometry facial analysis API, making the process of uploading a face completely automatic - there was no need for the user to indicate eye, mouth or jawline positions.
Finally we worked with Impossible Software to specify a robust and scalable server solution that would cope with the large spikes in traffic caused by an extensive supporting media campaign.
FWAMobile Of The Day