This is an exciting product. Exciting for many reasons but mostly because this is the first step. When Microsoft’s Kinect was still named project Natal, I immediately thought MoCap. I thought to myself, “Screw gaming, this thing is WAY bigger than just gaming.” Especially when we all saw the flurry of hacks come out that made use of the Kinect in a bunch of fascinating ways, event if most of them were not really up my alley like robotics or 3D scanners. While they were definitely cool in their own right, most of these techniques were niche or incomplete, not ready to be applied. When I see the Kinect, I see lowering the bar for people to express themselves creatively. Traditional MoCap studios are naturally more expensive, require massive space and jumpsuits that need to be donned to be properly captured.
Capturing the motion is only one part. It’s what you need to do with it afterwards that’s important. This is where Reallusion steps in. Reallusion has been developing it’s series of software titled “iClone” that attempts to operate as a WYSIWYG (What You See Is What You Get) 3D modeling program. Recently with the new version, iClone 5, there is plugin support for Kinect or other natural motion interfaces because Reallusion worked with PrimeSense early on to get this to a functional state. As far as I can search, Reallusion is well ahead of the curve here and have positioned themselves as the only company that has official Kinect support in a 3D modeling suite tailored to ease 3D animation. For me, the super exciting part was the Kinect integration, but the awesome part is how much smart software is assisting the whole process.
Again, let me touch on how I introduced this topic. This is exciting because it’s the first step. iClone 5 combined with Kinect is still limited by what Kinect is capable of. Kinect’s IR camera, the part that is gathering the spacial data to render a 3D point cloud, is capable of resolving 640×480 lines of resolution but is in fact only operating at 320×240 lines of resolution. Perhaps this limitation is only applied to the Xbox 360, but I can’t find any data that this is the case for PC or not. Considering the fact that Microsoft released the SDK for PC, I would assume that the framework would be similar, especially since applications might want to traverse from developing on PC to running on the 360. Considering the Kinect is only running at half capacity and Microsoft has been able to eke out respectable performance from the limited specs (especially when demoing the Kinect Labs features on the 360), the future looks bright. However, because of the limitations currently found on the Kinect, you are limited to the grand movements that Kinect can reliably translate.
This is where the secret sauce of the iClone 5 comes in. Because you can record your movements, you can then finesse and guide the movements exactly as you wish in software. Check out the video below for a demonstration:
But it doesn’t end there, the iClone5 software suite assists in texturing 3D models by using a photo to 3D mapping technique by simply pointing a dotted line along a face and you can see it in real time apply that texture to a 3D model. A library of locations to put your models in and props that react realistically when physics demands them to are available as well. Even cooler is that you can create and import 3D models from Google’s awesome sketchup application to help people that have difficulty making 3D objects using traditional 3D modeling applications. There is also an extremely handy lip-sync tool that tries very hard to properly animate based on the sound file imported.
I’d like to emphasize what I saw at Reallusion’s NYCC Booth isn’t this amazingly easy tool that doesn’t require any real work on your part. iClone 5 with Kinect doesn’t do everything for you. It creates this broad picture of what the desired output will look like. You’ll need to tweak an elbow here or a shoulder shrug there. You’ll need to wrangle the nuance out of the lip-sync tool to properly reflect the desired emotion. iClone 5 makes it so that you don’t need to touch every single frame of animation to get your desired output. You’ll start with a rough carving and refine as much as you need to.
I’m excited. Reallusion is spearheading a new frontier in creative outlets by lowering the bar for the masses to quickly and effectively produce 3D animation. I’m excited that it works so well even though this is just the beginning. Ten years ago, I wouldn’t have imagined that we would have such powerful technologies available so cheaply. Because Kinect is capable of linearly evolving via better hardware updates, perhaps one day in the future we will have true 1:1 motion capturing. Reallusion is doing impressive software to facilitate those who wish to express themselves creatively but lack the capitol or technical savvy to accomplish it. I see a future where Reallusion will power a full length motion picture. The future is bright indeed.