Microsoft Kinect SDK中的Event Model

Microsoft Kinect SDK在初始化的时候，需要设置一个ManualReset的Event Handle丢进NuiImageStreamOpen里。

        hNextColor = CreateEvent(NULL, TRUE, FALSE, NULL);
    
        hr = m_pNuiSensor->NuiImageStreamOpen(
        NUI_IMAGE_TYPE_COLOR,
        NUI_IMAGE_RESOLUTION_640x480, 0, 2, 
        hNextColor, 
        &hColorStream);

但在处理的时候，没有任何地方调用ResetEvent来重置hNextColor。看微软提供的SkeletonViewer里面的有一段函数：第18行，调用一次WaitForMultipleObjects后，又在41/50/55行再次调用WaitForSingleObject来确认当前是否有Depth/Color/Skeleton帧。

DWORD WINAPI CSkeletalViewerApp::Nui_ProcessThread( )
{
    const int numEvents = 4;
    HANDLE hEvents[numEvents] = { m_hEvNuiProcessStop, m_hNextDepthFrameEvent, m_hNextColorFrameEvent, m_hNextSkeletonEvent };
    int    nEventIdx;
    DWORD  t;

    m_LastDepthFPStime = timeGetTime( );

    // Blank the skeleton display on startup
    m_LastSkeletonFoundTime = 0;

    // Main thread loop
    bool continueProcessing = true;
    while ( continueProcessing )
    {
        // Wait for any of the events to be signalled
        nEventIdx = WaitForMultipleObjects( numEvents, hEvents, FALSE, 100 );

        // Timed out, continue
        if ( nEventIdx == WAIT_TIMEOUT )
        {
            continue;
        }

        // stop event was signalled 
        if ( WAIT_OBJECT_0 == nEventIdx )
        {
            continueProcessing = false;
            break;
        }

        // Wait for each object individually with a 0 timeout to make sure to
        // process all signalled objects if multiple objects were signalled
        // this loop iteration

        // In situations where perfect correspondance between color/depth/skeleton
        // is essential, a priority queue should be used to service the item
        // which has been updated the longest ago

        if ( WAIT_OBJECT_0 == WaitForSingleObject( m_hNextDepthFrameEvent, 0 ) )
        {
            //only increment frame count if a frame was successfully drawn
            if ( Nui_GotDepthAlert() )
            {
                ++m_DepthFramesTotal;
            }
        }

        if ( WAIT_OBJECT_0 == WaitForSingleObject( m_hNextColorFrameEvent, 0 ) )
        {
            Nui_GotColorAlert();
        }

        if (  WAIT_OBJECT_0 == WaitForSingleObject( m_hNextSkeletonEvent, 0 ) )
        {
            Nui_GotSkeletonAlert( );
        }

若无ResetEvent的话，势必会造成下次调用WaitForMultipleObjects/WaitForSingleObject仍能成功，于是会导致重复帧。但实际情况上不会出现重复帧，于是猜测是在用户手动调用NuiImageStreamGetNextFrame更新FrameHandle时，这个函数在SDK内部将当前的 hNextColor重置。

模拟函数如下：

#include <iostream>
#include <Windows.h>
using namespace std;

HANDLE hNextRgb;
int timestamp = 0;

void FakeNuiSkeletonGetNextFrame()
{
    // HRESULT NuiImageStreamGetNextFrame(HANDLE hStream,DWORD dwMillisecondsToWait,const NUI_IMAGE_FRAME **ppcImageFrame)
    // update rgb stream handle : "HANDLE hStream" and reset next rgb event
    ResetEvent(hNextRgb);
}

DWORD WINAPI RgbCaptureThread(LPVOID lpParameter) 
{ 
    while(1)   
    { 
        Sleep(30); // assume rgb frame is update 30ms/frame
        timestamp++;
        cout << "+ ";
        SetEvent(hNextRgb);
    }
    return 0; 
} 

DWORD WINAPI MyProcssThread(LPVOID lpParameter) 
{ 
    while(1)   
    { 
        if ( WAIT_OBJECT_0 == WaitForSingleObject(hNextRgb,INFINITE) )
        {
            FakeNuiSkeletonGetNextFrame();
            cout<< timestamp <<endl;
            {
                // copy the rgb frame, run your own algorithm
                // if your process time is beyond 30ms, you will lose some rgb frames
                int ProcessTime = 30;  
                Sleep(ProcessTime);
            }            
        }
    }
    return 0; 
}

void simulateKinect()
{ 
    hNextRgb = CreateEvent(NULL,TRUE,FALSE,NULL); 

    HANDLE hThrd[2];
    hThrd[0] =  CreateThread(NULL,0,MyProcssThread,NULL,0,NULL); 
    // hThrd[1] is hidden in SDK, there should be a rgb/depth thread that continues update kinect rgb/depth camera
    hThrd[1] =  CreateThread(NULL,0,RgbCaptureThread,NULL,0,NULL); 

    WaitForMultipleObjects(2, hThrd, TRUE, INFINITE);
    CloseHandle(hThrd[0]); 
    CloseHandle(hThrd[1]);   
    CloseHandle(hNextRgb);
} 

int main(){
    simulateKinect();
    return 0;
}

可以看到，如果用户在MyProcessThread中处理一帧的时间过长，超过30ms，就会导致错过下一帧的更新。

eg：ProcessTime = 40时，每更新4个Frame，只处理3个Frame，程序输出如下：

为了证实想法的正确性，搜了一下MSDN，发现如下描述：

http://msdn.microsoft.com/en-us/library/hh973076.aspx

Event Model

The event model supports the ability to integrate retrieval of a skeleton frame into an application engine with more flexibility and more accuracy.

In this model, C++ code passes an event handle to NuiImageStreamOpen to open a color or depth stream. The event handle should be a manual-reset event handle, created by a call to the Windows CreateEvent API. When a new frame of color or depth data is ready, the event is signaled. Any thread waiting on the event handle wakes and gets the frame of color or depth data by calling NuiImageStreamGetNextFrame or skeleton data by calling NuiSkeletonGetNextFrame. During this time, the event is reset by the NUI Image Camera API.

 

后话：Microsoft Kinect SDK目前的内部同步（RGB/DEPTH/SKELETON）仍然做的不是特别精准，所以下一步将要考虑如何设置缓存，并根据timestamp手工同步这三组数据。

下一篇文章，先测试下NuiSetFrameEndEvent～