Making rotoCap Data Useful

In order to be adaptable to each user's situation I have come up with a scheme to re-interpret data coming from rotoCap into a usable format. Motion data such as this is converted to keyable attributes which could be used to drive the user's own character rigs or anything possible via the connectionEditor. All this is done by essentially drawing the slider around the drone.

The image above shows the boundary box of the sphere's motion-extremes across the image surface. The dark triangles were some previous diagnostic graphics, ignore them. Scaling the green box redefines the parameter range of the drone (sphere) outputs by clamping the X and Y attributes. Additional number fields will be added to the drone's GUI section for North, South, East, West. Each field will define what the extreme value is for that direction or to ignore that parameter or clamp it to infinity. The user should be able to just plug in the extreme values from the control rig, blank fields are ignored:

• yourFaceRig_eyeControlSlider.minY = 0.0 = drone.South
• yourFaceRig_eyeControlSlider.maxY = 1.0 = drone.North
  

Example: The user would like the drone's Y-axis movement to control the blink slider for an eye rig. When the sphere rises, the eye gets wider; completely open equals an attribute slider value of 1.0. When the sphere lowers, the eye closes till 0.0 is reached. By translating and transforming the box, the drone's movement is filtered to the desired animation curve equivalent of amplitude and frequency in a visual and intuitive manner.

Simulation
User Defined Drone Attributes

• Target Name = eyeControlSlider
• North = 1.0
• South = 0.0
• West = SKIP //ignore
  
• East = SKIP //ignore

Output = 0.85

.85 would equal a slightly closed eye, or 85% open. The eye is parented to a null-dummy node which acts as a buffer between rotoCap and the user's rig just in case the user needs to remove portions or all of rotoCap from a scene which actually is the ultimate goal. rotoCap should work behind the curtains and leave as little impact to a scene beyond passing attributes in the form of a bare-bones node network.

rotoCap Progress (Updated)

The past weekend has yielded many successes. After finally fixing a bug that wouldn't advance the frame properly I've fully re-engineered the core tracking engine and even ran some benchmark tests which will be posted shortly (updated, see below). The movie above shows the raw output, single pass with "Simplify Curves" command in from the Graph Editor, and a final hand-tweaked version. The raw output creates lots of key frames that aren't all necessary so by keeping the important key frames, this makes it much easier for the final manual cleanup due to the reduced key set.

Test Results & Observations
Environment

• 50 JPG image sequence
• 1 Drone sampled (Pink)
• timerX MEL function for timing

Parameters

• Tolerance: (0.0 to 1.0) range of RGB value rotoCap accepts as close enough to desired color.
• Step: (0.0 to 1.0) distance to next sampled UV coordinate
• # of Keys: how many keyframes rotoCap created for a drone
• Time Elapsed: (seconds) starts when "Launch Drones" is pressed, ends after last drone is tested

Test 1: Initial Test

• Tolerance .2
• Step .05
• # of Keys 7
• Time Elapsed = 101.47

Observations:

Almost all of the keys occurred at the beginning while the target color was stationary. Tracking was not detecting motion over time. I suspected the step size was too large and rotoCap just couldn't sample enough UV's to find the drone.

Test 2: Smaller Step Size/Centering Off

• Tolerance .2
• Step .025
• # of Keys 16
• Time Elapsed = 1217.69

Observations:

Reducing the step turned out to fix the sparse key issue. rotoCap still seemed to favor the edge of the color mass instead of centered which makes sense since this was run with self-centering turned off.

Test 3: Centering On

• Tolerance .2
• Step .025
• # of Keys 35 (simplified: 17)
• Time Elapsed = 71.86

Observations:
The time difference is substantial due to the centering sub-process passing the centered UV coordinates to the next cycle of the colorSeek process. The key generation also went up significantly. Running a simplify curves on the XY curves in the Graph Editor essentially deletes the unnecessary stationary keyframes. I'm very happy how well the script is working now.