2/28/09
Alternate Search Modes
While working on rotoCap today I thought a nice feature would be allowing the user to manually launch rotoCap on a per-frame basis instead of a run of the whole image sequence. Additionally I am adding a flag to the keying sub-process whether or not to overwrite existing keyframes on a drone. If existing keyframes are found then the engine can use them as initial search coordinates, saving time by not having to start at the origin of a UV plane during the initial UV survey.
2/20/09
Homogenous RGB Mass Self-Centering
The colorSeek engine of rotoCap has been given some new methods for searching and evaluating masses of targeted RGB ranges. Instead of basing tracker position on a single pixel test, the new method treats a positive RGB test as the origin for a localized survey of the surrounding pixels. If it detects the pixels immediately surrounding it are within the RGB range it will expand the survey (patch) till it finds the edges of the color mass.
The engine then calculates the center of the mass by simply offsetting the target tracker half the size of the mass. This theoretically will eliminate any false positives due to the quality of the source image due to sensor noise and blurs due to fast movement. The centered coordinates are then passed on to the next frame's processing cycle to use as an origin for the new search.
The engine then calculates the center of the mass by simply offsetting the target tracker half the size of the mass. This theoretically will eliminate any false positives due to the quality of the source image due to sensor noise and blurs due to fast movement. The centered coordinates are then passed on to the next frame's processing cycle to use as an origin for the new search.
2/16/09
Break for Debugging
While not moving as quickly as I had hoped, the ideas I've been integrating have been encouraging. Though like any grand scheme, there comes a time to pause and make sure future development is on a stable foundation. For the past weekend I've been going back and fixing little bugs so I don't have to worry about them later. Though not as satisfying, it is essential to maintain a level of coding discipline.
1/19/09
Integrating Path Prediction
The core tracking engine has been successfully installed into the main program. It remains virtually unchanged since its creation but there are some adjustments to the search-style I'm working out currently. When the engine spots a target RBG value it uses that "hit" as the initial center for successive UV searches. Limiting the search area was the first attempt at speeding up processing and now predicting where the target is in the given area is the next step.
Noting Newton's First Law of Motion, a target will tend to continue along if not a straight line, a near approximation. The engine will make a list of coordinates to look at first in order of probability (the opposite direction being the last). This will require re-writing the engine but every engine needs an overhaul eventually. I believe the time I spend on this portion will be worth it in terms of process efficiency.
Noting Newton's First Law of Motion, a target will tend to continue along if not a straight line, a near approximation. The engine will make a list of coordinates to look at first in order of probability (the opposite direction being the last). This will require re-writing the engine but every engine needs an overhaul eventually. I believe the time I spend on this portion will be worth it in terms of process efficiency.
12/3/08
Drone Generation Subprocess "makeDrone"
I'm calling the visual representations of each color track drones. These, like worker bees and pollen, will track the RGB elements across the UV space. Currently the makeDrone subprocess creates:
- visible geometry
- RGB attributes from a colorEditor call
- unique object name
- colorSeek inclusion toggle (include in batch color tracking or not)
- RGB swatch via canvas command
- target-rig attribute to control (what does this drone control)
- "edit RGB" and delete options
- per-drone area constraints
- initial search pattern
- per-frame census (how many frames does drone find target RGB)
11/10/08
Over 100!
My method of file numbering is when I get a portion of the script running it gets saved as a separate file. This past weekend, the bugs squashed broke 100 (as of this writing at 103).
The GUI is being refined into a sleeker interface, but still with the consideration of an Adobe-like familiarity. Currently I'm working on a timeline-independent playback system for an animated texture. While scrubbing and stepping frames was relatively easy, the animation of the texture at a constant framerate is a little trickier. I've been using the timerX function to wait a certain amount of time then step the frame. It works but I'm trying to make it run consistently on different machines. The amount of times the function checks the elapsed time varies depending on the speed of the machine. I don't think this is a big issue, but I'm trying to make rotoCap as consistent and stable as I'd rather have all the processing power for the color-tracking engine than the GUI.
The GUI is being refined into a sleeker interface, but still with the consideration of an Adobe-like familiarity. Currently I'm working on a timeline-independent playback system for an animated texture. While scrubbing and stepping frames was relatively easy, the animation of the texture at a constant framerate is a little trickier. I've been using the timerX function to wait a certain amount of time then step the frame. It works but I'm trying to make it run consistently on different machines. The amount of times the function checks the elapsed time varies depending on the speed of the machine. I don't think this is a big issue, but I'm trying to make rotoCap as consistent and stable as I'd rather have all the processing power for the color-tracking engine than the GUI.
10/16/08
Simple Shape Tracking
Andy Wilson has developed an elegant control method based on the shape of the hand. By comparing the background shapes to newly created shapes (formed by the hand), he is able to track the movement allowing for mouse-less interactions. (Source: http://procrastineering.blogspot.com/2008/10/andy-wilson.html)
9/25/08
Video-Based vs Laser-Based Tracking of Dynamic Environments
In both fields of motion-capture and surface-capture (scanning) the need to discriminate unique points in space is paramount to the accuracy of the capture system. When I was first conceiving rotoCap I tried to figure out this problem by looking what I had to work with and not having ready access to the specialized cameras of a Vicon system or laser scanner I turned to consumer video cameras. I conceived I could lessen the amount of computer load by not having the program calculate trajectories but color. An infrared machine-vision camera is suited to higher resolutions and frame rates, how could the lowly camcorder compete? The answer is color.
Most machine vision cameras used in mocap and 3d scanning are grayscale in which their output is turned into a binary image. A binary image is simply a high-contrast image that is only black and white with no gradient. This makes the image easier to process for the software since it only deals with white dots on a black background, but the other calculations it uses to discriminate between two white dots which come near to each other can be complex and costly to the performance. By adding in the color value, the need to track each dot is made easier by giving each its own RGB value.
Why bring this up? The DARPA Grand Challenge is a great source for inspiration because of the real-world applications of scanning/mocap. The Stanford University team's Standley uses a hybrid system of laser scanning and a video overlay allowing it to use color to tell the difference between the road color and everything else. This is a great confirmation that we're both on the right track when it comes to using color over binary alone.
Most machine vision cameras used in mocap and 3d scanning are grayscale in which their output is turned into a binary image. A binary image is simply a high-contrast image that is only black and white with no gradient. This makes the image easier to process for the software since it only deals with white dots on a black background, but the other calculations it uses to discriminate between two white dots which come near to each other can be complex and costly to the performance. By adding in the color value, the need to track each dot is made easier by giving each its own RGB value.
Why bring this up? The DARPA Grand Challenge is a great source for inspiration because of the real-world applications of scanning/mocap. The Stanford University team's Standley uses a hybrid system of laser scanning and a video overlay allowing it to use color to tell the difference between the road color and everything else. This is a great confirmation that we're both on the right track when it comes to using color over binary alone.
9/14/08
Sequence Viewer and Shader Progress (UPDATE)
rotoCap is really coming along as more pieces of code unlock multiple areas of development. The past few weeks have seen a streamlining of file management, GUI revisions for better interactivity, and it's now ready to start processing images.
Since the core tracking engine relies on an image plane to calculate movement, rotoCap needs to:
Currently, the shader generator works great the first time the user loads up the image sequence into rotoCap but changing the image directory results in errors as the script tries to create new shaders connected to previous file nodes. The tricky part at this time is how to query if a shader exists since the shaderNode command does not have an -exists flag to test. To get around this I'm loading up all the shaders into an array and deleting the rotoCap-created shader network based on wildcard name-space search.
UPDATE
The shader handling is fixed now. The bug was due to old nodes conflicting with the newly generated nodes. Each time the user changes the image sequence directory a process is called to delete all nodes generated by rotoCap from previous executions. The nodes are filtered by a gmatch wildcard name-space execution, then deleted.
Since the core tracking engine relies on an image plane to calculate movement, rotoCap needs to:
- generate a shader with an animated texture
- disconnect the animated texture from timeline dependency
- connect animated texture to rotoCap control
- apply the shader to an image plane
- scale image plane the same aspect ratio as source image file
- create a camera to lock onto the image plane perpendicularly
- use the new camera's viewport as part of the rotoCap GUI
- create playback widget for different framerates
Currently, the shader generator works great the first time the user loads up the image sequence into rotoCap but changing the image directory results in errors as the script tries to create new shaders connected to previous file nodes. The tricky part at this time is how to query if a shader exists since the shaderNode command does not have an -exists flag to test. To get around this I'm loading up all the shaders into an array and deleting the rotoCap-created shader network based on wildcard name-space search.
UPDATE
The shader handling is fixed now. The bug was due to old nodes conflicting with the newly generated nodes. Each time the user changes the image sequence directory a process is called to delete all nodes generated by rotoCap from previous executions. The nodes are filtered by a gmatch wildcard name-space execution, then deleted.
Subscribe to:
Posts (Atom)