Skin And Bones...

In order to further develop and test my Antron fleece style puppet shader, I used Maya to quickly model a generic hand puppet loosely based on the Glorified Sock Puppet pattern which I originally used to build No. 1.

The mesh is very straight forward and is bound to a minimal rig, the image below illustrates the construct of the model.

I made some adjustments minor adjustments to the length and the density to the fur feedback once it and the Antron shader were applied to the mesh in order to make it fit better and while there is still some room for improvement I feel that the effect, when rendered, is quite convincing.

While I rather like the aesthetic of the puppet, looking solid as though it is made from skin, muscle and bone, I would still prefer it to appear to appear like it is made from puppet making materials. To achieve this I need to think about the way that Antron hangs and reacts to motion and I will be looking at a number of techniques, including sculpting, blendshapes and cloth simulation, in order to determine the best solution for this problem before I commit to building my final puppet..

Have a Happy Christmas,

Mat :)

On Antron Antron Antron....

There is a material called Antron fleece which is used for puppet making, it is particularly recognisable in the construction of the Muppets, and I would like my CGI puppet to have a look similar to that Antron. Below you can see the texture that I am after in the puppet from Project Puppet.

Antron fleece is a rather complex material with a mixture of loose and tight fibres and a specularity that varies depending on the viewing angle. I have spent the morning trying to emulate the material in Maya and, while I have not quite got it nailed, I believe that I am definitely on the right tracks with my investigations.

I have applied my shader to an n-cloth plane which has been simulated to drape over a sphere in order to illustrate tone variation through folds and light angles.

The main texture of the material has been generated using an MIA Material X node into which I have plugged in a ramp to the Colour attribute and fractal noise nodes into the Reflection Colour and the Bump attributes.

In order to get the fuzzy look I have applied a Fur node to the plane which can be more clearly seen in the following close up render.

My fur node started life as the Duckling preset. The presets are a great starting block for any fur feedback in Maya and, providing you understand the nature of each of the samples, it is relatively simple to adjust your initial choice to suit the requirements of the job.

Because of the short length and light weight of the fuzz on Antron fleece it is not necessary for the fur to be dynamic. However, the processor requirements may still slow things down a bit so I will need to create an option on the user interface, when I get round to designing it, through which I can make it possible to switch the fur on and off depending on whether the model is being captured or rendered.

Meet No.2 .....

So, with the realisation that a sock puppet is not entirely suitable for this project I have moved my sights towards the construction of a foam puppet with the idea that the foam structure will provide a kind of exoskeleton for the controller components.

I grabbed the free pattern for a round head puppet from Pupppetvision, purchased a glue gun, watched the video below and set about the construction of my first foam puppet head.

I was really pleased at how easily the puppet head came together, the hot glue gun makes for very speedy sticking!! With the head constructed I made the mouth from felt covered mounting board and stuck it into place.

Although it wasn't really necessary I decided to cover the head with fleece, partly to make it look for aesthetically pleasing but mainly because I had never done it before. So, figuring that the stretch in the material would allow it, I cut some fleece from the same pattern that I used for the head and then sewed it together in much the same way that I glued the foam for the head. I left a small amount of the seam at the nape of the neck open so that I could pull the skin over the head like a helmet and then I sewed that up to ensure that it wouldn't slide off.

There's a lot of friction between the foam and the fleece which is ideal as it means that there is practically no slide, it also means that pulling the skin into place took a little manhandling but thankfully everything stayed together. The last thing to do was to glue the mouth seam to the foam and presto, covered foam puppet head... done!

As I hoped, I believe that the head will be ideal for the purpose of the controller and, having a physical object to work with I can already imagine possible design solutions for the control system and how I might be able to incorporate it into the puppet.

Obviously, being a first effort there are a few things that I would like to address before I can call it a final design.

• The head is too small - I'll try up-scaling the pattern by 50% next time.
• The skin is too tight in some areas - slightly modifying the pattern I should add more material where I want it.
• The sewing is a bit of a mess - learn to use the "Henson Stitch" and practice.
• A body or sleeve is required.

OK, I've got a to-do list I'd best get on and do it!

Little Baby Steps.....

I thought it would be a good idea to make a puppet being as I'm planning on making a puppet controller and so I did.... Meet No.1 (that's the puppet not the child)

Armed with a pattern from Project Puppet, some felt, some fleece, a spool of thread with a needle and some sharp scissors I am very proud that I was able to cobble together my first ever puppet regardless of the fact that it is completely unsuitable for this project.

Yep, that's right, this type of puppet, as endearing and full of character as he is, just doesn't lend himself to being stuffed with electronic do-dads and thingummy-jigs, and so my next foray is going to be a journey into the realm of foam puppets.

No.1 is essentially a sock puppet, he has no skeleton beyond that of the hand which is controlling him. I believe that a foam head puppet will be more suitable for my project because the foam structure will provide a semi-rigid enclosure for whatever inputs I end up using.

Making No.1 has by no means been a waste of time though, I can apply everything that I've learnt from making him towards the next puppet and just look at how much of a friend he has in Sooz.

No More Waldo.....

The other day I contacted Andrew Young, a puppet enthusiast and film maker, and he has informed me that at some point The Character Shop, a company that makes pretty awesome animatronic characters, trademarked the term Waldo despite it having been used for many years to describe any control system that was similar in nature to the one in Robert Heinlein's short story "Waldo".

Here's what The Character Shop says about their Waldo systems.....

"Waldo®:TCS' trademark for its brand of ergonomic-gonio-kineti-telemetric input devices for controlling its puppets and animatronics. Ergonomic because it is engineered to fit a puppeteer's or performer's body (and/or head and/or face) and comfortably allow a wide range of physical freedom. Gonio- and kineti-metric because it measures the angle and movement of the wearer's joints and limbs. And telemetric because the movement data is measured and sent via remote control. In simpler terms, an electro-mechanical rig you wear that makes a puppet (whether actually three dimensional or a CGI "electronic puppet") mimic your movements."

Who knew that I was making an "ergonomic-gonio-kineti-telemetric" input device?....

Here's some more from TCS' Waldo statement....

"That's right! Waldo®, Facial Waldo®, Body Waldo®, Warrior Waldo® , and any use of the term Waldo when referring to data-capture input devices are all trademarks of The Character Shop. You can make use of a Waldo®-like system, but you can't call it a Waldo®!

So call it something else, please! Who wants to spend money on lawyers?"

So, warning heeded.... Despite my project being small and it being very unlikely that TCS will ever hear about it I shall no longer consider my controller as a Waldo®.

I guess I better get used to ergonomic-gonio-kineti-telemetric input device......

Testing Other Sensors....

After the success of my initial proof of concept for this project in which I used a trio of potentiometers to adjust the rotational attributes of the test object, I started to think about other means by which I could record the motion from my puppet controller.

With some consideration towards the type of motion that would be required to replicate a Muppet style puppet I decided that the best options at this stage were to investigate flex sensors and gyros.

A flex sensor is basically a variable resistor that is adjusted through a bending motion, ideal for use in glove type control applications.... check out the Nintendo Power Glove from 1989, a neat idea but perhaps a bit ahead of it's time it used flex sensors in the fingers.

Gyro's and accelerometers are found in pretty much all smart phones and tablets. These chips enable the device to orient the screen when it's rotated and are used in a lot of games that rely on motion to control the action rather than a traditional control system... take Doodlejump for example.

The following video demonstrates my application of functionality from these two devices through Arduino in a Maya environment.

Applying the data from the gyro and flex sensor was possible without too much adjustment to my original Maya python code. Both of the sensors are great but, despite being calibrated, the gyro can get a bit lost if it's moved about to much which requires the device to be reset on a regular basis.

In light of this test I have ordered a new gyro, the MPU 9150, which has a built in magnetometer, this should combat the "getting lost" syndrome that I have found with the MPU 6050 as it will have a constant directional reference from magnetic north.

Proof of Concept..

My initial idea was to employ a game pad to drive the animation of a digital puppet in a similar way to the demonstration in the video below.



The problem with this solution is that the controller looks nothing like a Waldo controller and it would have to be dismantled to then be reconfigured in a manner that would suit my objective. While this could potentially work I would rather have a controller that was built for purpose and so I decided to use an Arduino micro-controller as the heart of this project instead of the game pad.

The process to get the analogue inputs speaking with Maya was far more straight forward than I could ever have imagined and was implemented in three stages.

1. Get my Arduino Uno to listen to analogue inputs and then send the data to my computer via the serial port.
2. Send the data from the serial port to the command line because Maya 2015 is unable to read the serial port.
3. Get Maya to read the data from the command line and apply it to the relevant attributes of the demonstration model.

With help from the book "Getting Started With Arduino" and with some tips from various forums, my favourite tip being to read each analogue input twice in order to reduce the likelihood of noisy data, I was able to cobble together the sketch that would read my inputs and send them to the serial port on my computer.

I was able to use a short script written by Eric Pavey which writes serial data to the command line, this took part of the second stage of the process.

The last pat of the puzzle was probably the hardest for me to solve because I needed to find a way to separate the incoming data from the command line and then turn the values into numbers to be used as individual attribute values in Maya. I was eventually able to do this with a little luck and by reading the Python help documentation in Maya.

I'm really pleased with this initial development as it demonstrates the viability of my project. The one drawback is that the Arduino only has 6 analogue inputs and so only has the potential to drive all 3 axis of only 2 joints in Maya which is not really enough for what I have in mind in this project.

There is hope though, a small amount of investigation has brought to light a project from Fluid Forms in which they used digital multiplexing to increase the number of analogue inputs to a potential of 96 which is plenty more than I would ever require for this project.

So I'm pretty excited that this is working and that I have a way to utilise as many inputs as I need. Next step is to grab some multiplex goodness and get that working!!

Where's Waldo?

These two clips illustrate two very different contemporary Waldo Control systems.

Channel 4''s Black Mirror programme has used a digital Waldo for one of it's episodes in which a CGI character from an educational TV show turns political satirist. The character demonstrates the current apex of digital Waldo technology with a control system that looks like a fighter plane cockpit and which requires 3 puppeteers to manipulate.



There is a very clear reason for the producers decision to use a digital puppet over traditional key-framed CGI in that the performers are able to act and react in real time to the other actors and the environment. This partly reflects my own rationale towards developing a puppet controller, I say partly because other motivation for this project includes the simplification of the initial computer animation process in the hope that it may engage people who might otherwise avoid it.

This second clip shows a Waldo controller that was developed by Spencer Riedel for a robotics project while studying at college. While it is plain to see that the functionality of the controller is far more limited than that of the Waldo used in Black Mirror the concept is the same, an analogue input is used to control a digital object.



Despite the limited functionality of Spencer's Waldo it illustrates a foundation which could be built upon to develop a puppet controller that would offer the user a more natural puppeteering experience. I don't believe that it is too ambitious to hope that my controller will lie somewhere between the complexity of the puppet used in Black Mirror and Spencer's home-brew creation.

Who Is Waldo Anyway?

Until I started to investigate these Waldo remote systems I figured that the name "Waldo" was an acronym for something too technical for the layman to understand. In actual fact it turns out that the name was given in homage to a short science fiction story by the author Robert A Heinlein.

The story of Waldo Farthingwaite-Jones describes a born weakling who uses his intellect and family money to develop a device, the "Synchronous Reduplicating Pantograph", that would enable it's user to control much larger and more powerful machines through the manipulation of a remote glove.

There is an obvious difference between Heinlein's idea and the focus of my investigations in that the remote controller from the story was used to manipulate robotic machinery while my own aim is to control digital objects. This manipulation of a digital environment is simply a natural evolution of Heinlein's original fiction which is based on the potential use of available technologies.

Prior to Henson's digital controller Waldo controllers have been used and still are used to manipulate robotic devices in scientific, entertainment and industrial environments. The following video illustrates the evolution of the control system, from rod to Waldo, for the Doozers, characters from the TV show Fraggle Rock.

I realise that my reference appears to continually point to the work of the Henson company but what other bench mark is there for the technological development of puppetry for entertainment? It will be very interesting to discover the work other pioneers in this field through my investigations towards the development of my own remote manipulator.

In The Beginning.....

What's all this about then?

Well, after going to the Children's Media Conference this year I have been inspired to work towards pitching an idea for commission at the conference in a couple of years time, I'm not entirely certain what that idea is yet but it will include puppets, narrative, education and entertainment...... a tall order but hey what is life without a challenge?

My intention is to align the project development with my masters qualification in Animation and Digital Effects which, as a part time course, will conclude in 2017... this seems a long way off but I am certain that it will fly by as I have so much to get on with before then like develop a new IP, create the tools and assets to realise the project and produce a short pilot episode on top of working as a senior lecturer and having a family. Hmmm, is that too ambitious?

So, the title of this blog eludes to the fact that the aforementioned puppets are not actual. That's right folks,as fun and endearing as real life puppets are I am a computer animator and so computer generated puppets are my goal.... actually that description somewhat undersells my plan as I really intend to create a digital puppet or puppets that can be manipulated in real time using a physical control system, oh yeah, I need to be able to record the manipulations as they happen and I'm going to be investigating the potential for including he Oculus Rift in this set up..... that ambition thing might need to be reigned in a bit there Mat!!

Well, that's about it, my intentions in a nutshell, but I really dislike a post without images or video so here's a clip of Jim Henson's "Waldo" digital puppet and controller to give you a better idea of the direction that I am aiming with this project....

As amazing as the Henson's "Waldo" controller is I am rather hoping that the advancements in technology over the past 25 years will be enable me to develop something a little less cumbersome.