Getting and Displaying Depth Data in C#

Article
09/12/2013

Kinect for Windows 1.5, 1.6, 1.7, 1.8

Overview

This How-To topic describes user tasks from the DepthBasics C# sample.

Code It

To get and display depth data, you must first initialize the sensor. Here are the tasks for initializing the sensor to generate depth data.

Once you have initialized the sensor, you are ready to get the next frame of data.

After getting the next frame of data, these tasks store the data.

Register an event that fires when data is ready

A Kinect streams data out continuously, one frame at a time, until you tell it to stop streaming. One method your application can use to get each frame of data, is to set the runtime to notify your application each time a frame is ready by raising an event.

  if (this.sensor != null)
  {
      this.sensor.DepthFrameReady += this.SensorDepthFrameReady;
  }

Do this by adding a handler on the DepthFrameReady event. This will cause the Kinect to raise the event each time a new frame of data is ready.

Implement an event handler

The system notifies your application that a depth frame is ready by firing the DepthFrameReady event. Use this event handler to copy the pixel data from the sensor into the depthPixels member. The object sender is the KinectSensor that fired the event.

  private void DepthImageReady(object sender, DepthImageFrameReadyEventArgs e)
  {
      using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
      {
          if (depthFrame != null)
          {
              depthFrame.CopyDepthImagePixelDataTo(this.depthPixels);
          }
          else
          {
              // depthFrame is null because the request did not arrive in time
          }
      }
  }

Use the OpenDepthImageFrame method to access the next frame of data that is returned from the sensor. Once you have access to the frame, use the CopyDepthImagePixelDataTo method to copy the data to your local memory. You will need to run this code for every depth frame that arrives from the sensor that you want to save, so be sure to only include in this handler processing that must be done every frame.

The else case may occur when your application does not respond quickly enough to the event notification. If that is the case, the depthFrame parameter will be null, which means you no longer have access to this frame of data as the sensor has now begun to generate the next frame of data and this frame is no longer valid.

For best performance, allocate the memory once outside of the handler that runs every frame.

Allocate storage for the data

        private DepthImagePixel[] depthPixels;
        private byte[] colorPixels;

        this.depthPixels = new DepthImagePixel[this.sensor.DepthStream.FramePixelDataLength];

        this.colorPixels = new byte[this.sensor.DepthStream.FramePixelDataLength * sizeof(int)];

First, declare a member variable to store the pixel data and then allocate the memory array. Each element will store the data from one pixel. Get the size of the array to allocate from the FramePixelDataLength property or the PixelDataLength property.

Again, for best performance, allocate memory once outside of the handler that runs every frame.

Get the data

    using (DepthImageFrame depthFrame = e.OpenDepthImageFrame())
    {
        if (depthFrame != null)
        {
            // Copy the pixel data from the image to a temporary array
            depthFrame.CopyDepthImagePixelDataTo(this.depthPixels);
            ...
        }
     }

Create a bitmap

Create a bitmap to store the new pixel data.

        private WriteableBitmap colorBitmap;

        this.colorBitmap = new WriteableBitmap(this.sensor.DepthStream.FrameWidth, this.sensor.DepthStream.FrameHeight, 
          96.0, 96.0, PixelFormats.Bgr32, null);

A WriteableBitmap is a Windows Presentation Foundation (WPF) construct, available in .NET Framework 4, that allows access to the bitmap bits. You could create a Bitmap every frame, but you will get much better performance creating a WriteableBitmap only when the pixel format changes, such as when you start the application.

Convert the depth data to rgb

    // Get the min and max reliable depth for the current frame
    int minDepth = depthFrame.MinDepth;
    int maxDepth = depthFrame.MaxDepth;
 
    // Convert the depth to RGB
    int colorPixelIndex = 0;
    for (int i = 0; i < this.depthPixels.Length; ++i)
    {
        // Get the depth for this pixel
        short depth = depthPixels[i].Depth;
 
        // To convert to a byte, we're discarding the most-significant
        // rather than least-significant bits.
        // We're preserving detail, although the intensity will "wrap."
        // Values outside the reliable depth range are mapped to 0 (black).
 
        // Note: Using conditionals in this loop could degrade performance.
        // Consider using a lookup table instead when writing production code.
        // See the KinectDepthViewer class used by the KinectExplorer sample
        // for a lookup table example.
        byte intensity = (byte)(depth >= minDepth && depth <= maxDepth ? depth : 0);
 
        // Write out blue byte
        this.colorPixels[colorPixelIndex++] = intensity;
 
        // Write out green byte
        this.colorPixels[colorPixelIndex++] = intensity;
 
        // Write out red byte                        
        this.colorPixels[colorPixelIndex++] = intensity;
                                                
        // We're outputting BGR, the last byte in the 32 bits is unused so skip it
        // If we were outputting BGRA, we would write alpha here.
        ++colorPixelIndex;
    }

Store the data in the bitmap

After you get the color data and save it locally, use the WriteableBitmap to display the mapped depth data.

    this.colorBitmap.WritePixels(
        new Int32Rect(0, 0, this.colorBitmap.PixelWidth, this.colorBitmap.PixelHeight),
        this.colorPixels,
        this.colorBitmap.PixelWidth * sizeof(int),
        0);

Use the WriteableBitmap.WritePixels method to save the pixel data. This code will run every frame to get each new frame, so try to do as little work inside this loop as you can to get the best performance.

Share via