How to Animate a Character in Augmented Reality

I’ve always wanted to have fun and try to make a character of myself for animation.  With Unity, there are a number of really fun opportunities.  You can take a digital model and turn him or her into a video game character, view in virtual reality, or create a character in augmented reality.

The goal here was try to make a character with a likeness to myself and animate him with augmented reality on top of a book.  Moving forward, I will eventually edit the character to have a life-like 3d scanned head in full color, but for now I made an abstracted version.

I modeled the character in a “T” position (with arms spread out) using polygon modeling in Maya.  I created a jacket, a shirt, and shoes, and applied different materials to each in the rendering editor.  Then, I exported the model as a .obj (with an associated .MTL file).   Once I had that, I put the .obj and .mtl file in a .ZIP file together.

Step 1: Model/ Download the Character

If you don’t have the facilities to create your own character from scratch and apply materials, you can use other software like Adobe Fuse.  Adobe Fuse is a great program for creating a custom character avatar simply.

Step 2: Rig Character using Mixamo

Mixamo is a web based software that makes rigging 3d models very simple.  This allows you to upload any “humanoid” character (with limbs, wrists, knees, head, etc) and be able to automatically rig that model for animations.  In Unity, this library of movements is helpful for creating specific movements of a game character.  In this case, I wanted to make the character do a simple walk, but Mixamo has a library of movements including dancing, jumping, backflips, even sitting down and working on a laptop.

For this process, it’s important that you model your character in a “T” pose – with arms spread out.

For the rigging process, import your model into Mixamo and then choose where the chin, wrists, knees, elbows, and groin are.  And that’s it!  You can download your characters movement into a .FBX file.

I uploaded the .FBX movement file to Sketchfab to illustrate and view the character.  Here is the result.

Step 3: View Your Animated Character in AR with Unity and Vuforia

Creating an Image Target

In order for your 3d model to register and have a reference plane, you need to find an image target to use.  Image targets can be nearly anything – a dollar bill, a QR code, a page from a book, or a printed out piece of paper.  The role of the image target is to have a flat surface in the real world that you want to map your 3d model onto.  This flat surface is uploaded to vuforia and imported into Unity as an Image target for reference.  That way, when the camera views and recognizes the image target, it knows how to display and project the 3d model into view.  Many people that experiment with or use Vuforia will use an image from online and then print that image to use as a real world reference plane.

For this instance, I took a book cover and then found the cover artwork on Google Images.  The image on the left is the image I uploaded.  The image on the right shows the registration markers that Vuforia is using.

The integration of Vuforia and Unity requires following some steps pretty carefully.  I recommend using this tutorial below, which goes through the steps clearly and in order.

The Result:

Here’s the result.

Here’s a breakdown of hardware and software used:

  • Autodesk Maya for modeling character (can use Fuse instead)
  • Mixamo (free)
  • Unity (free)
  • Vuforia (free)
  • Laptop
  • Logitech Webcam

2017-07-17T18:14:34+00:00 July 17th, 2017|AR, Augmented Reality, Maya, Unity, Vuforia|