CameraPoseFinder.ProcessFrame Method

Kinect for Windows 1.8

Adds the specified camera frame to the camera pose finder database if the frame differs enough from poses that already exist in the database.

Syntax

public void ProcessFrame (
         FusionFloatImageFrame depthFloatFrame,
         FusionColorImageFrame colorFrame,
         Matrix4 worldToCameraTransform,
         float minimumDistanceThreshold,
         out bool addedPose,
         out bool trimmedHistory
)

Parameters

• depthFloatFrame
  Type: FusionFloatImageFrame
  The depth float frame to be processed. This frame must have valid camera parameters and have a minimum size of 80×60. Also, this frame must be the same size and have been captured at the same time as the colorFrame parameter.
• colorFrame
  Type: FusionColorImageFrame
  The color frame to be processed. This frame must have valid camera parameters and have a minimum size of 80×60. Also, this frame must be the same size and have been captured at the same time as the depthFloatFrame parameter.
• worldToCameraTransform
  Type: Matrix4
  The current camera pose. This is usually the camera pose result from the most recent call to the FusionDepthProcessor.AlignPointClouds or Reconstruction.AlignDepthFloatToReconstruction method.
• minimumDistanceThreshold
  Type: Single
  Threshold in the range [0.0f, 1.0f] that specifies how different the worldToCameraTransform pose must be from the poses that are already stored in the database. Input frames that have a minimum distance equal to or above this threshold when compared to existing poses will be added to the database. Set this value to 0.0f to always add the pose to the database when this function is called. Note that setting this value to 0.0f can lead to many duplicated poses unless your application implements its own test.
• addedPose
  Type: Boolean
  Returns true if the input frame was added to the camera pose finder database; otherwise, false.
• trimmedHistory
  Type: Boolean
  Returns true if the pose limit set by the CameraPoseFinderParameters.MaxPoseHistory property was exceeded when this input frame was stored. This indicates that the oldest pose in the camera pose finder database was overwritten by the latest pose.

Exceptions

Remarks

CameraPoseFinder accumulates whether the values at each sample location pixel in a saved pose frame are less than or greater than a threshold which is randomly chosen between minimum and maximum boundaries (for example, this range is [0, 255] for each color). Given enough samples, this accumulation represents a unique key frame signature because different poses have different values for surfaces that are closer or further away, or that contain different colors.

Unlike depth, the robustness of finding a valid camera pose in a color image can have issues with ambient illumination levels. For best matching results, both the Kinect camera and the environment should have exactly the same configuration as when the database key frame images were captured. If you used a fixed exposure and custom white balance to capture the poses in the database, you should use these same values when testing the database later.

You can improve matching accuracy by replacing the RGB values in the color image with three different channels of matching data scaled from 0–255. Some of the non-RGB information that your application could use in the color image to improve matching accuracy are the following:

• Compute hue and saturation instead of RGB to reduce problems caused by illumination variance.
• Convert RGB to LAB and use the AB channels.
• Use other values such as texture response or corner response.

Requirements

Namespace: Microsoft.Kinect.Toolkit.Fusion

Assembly: Microsoft.Kinect.Toolkit.Fusion (in microsoft.kinect.toolkit.fusion.dll)

See Also

Reference

CameraPoseFinder Class
CameraPoseFinder Members
Microsoft.Kinect.Toolkit.Fusion Namespace