(翻译)Gestures and Tools for Kinect

• Sign In

 

MSDN Blogs > Eternal Coding > Gestures and Tools for Kinect

Gestures and Tools for Kinect

David Catuhe 

4 Jul 2011 9:58 AM 

• 27

You have certainly not missed (as a regular reader of this blog ) that the Kinect for Windows SDK is out!

For now, however, no gestures recognition services are available. So throughout this paper we will create our own library that will automatically detect simple movements such as swipe but also movements more complex such as drawing a circle with your hand.

The detection of such gestures enable Powerpoint control the Jedi way ! (similar to the Kinect Keyboard Simulator demo).

If you are not familiar with the Kinect for Windows SDK, you should read a previous post that addressed the topic:http://blogs.msdn.com/b/eternalcoding/archive/2011/06/13/unleash-the-power-of-kinect-for-windows-sdk.aspx

如何检测手势 ?

检测手势有很多解决方案，文章中将介绍两种:

• 算法搜索
• 基于模板搜索

Note that these two techniques have many variants and refinements.

You can find the code used in this article just here: http://kinecttoolbox.codeplex.com

GestureDetector class

To standardize the use of our gestures system, we will therefore propose an abstract class GestureDetector inherited by all gesture classes:

This class provides the Add method used to record the different positions of the skeleton’s joints. 

It also provides the abstract method LookForGesture implemented by the children. 

It stores a list of Entry in the property Entries whose role is to save the properties and timing of each recorded position.

描画已存储的位置点

The Entry class also stores an WPF ellipse that will be used to draw the stored position:

Via the TraceTo method of the GestureDetector class, we will indicate which canvas will be used to draw the stored positions. 

In the end, all the work is done in the Add method:

1. public virtual void Add(Vector position, SkeletonEngine engine)
2. {
3.     Entry newEntry = new Entry {Position = position.ToVector3(), Time = DateTime.Now};
4.     Entries.Add(newEntry);
5.  
6.     if (displayCanvas != null)
7.     {
8.         newEntry.DisplayEllipse = new Ellipse
9.         {
10.             Width = 4,
11.             Height = 4,
12.             HorizontalAlignment = HorizontalAlignment.Left,
13.             VerticalAlignment = VerticalAlignment.Top,
14.             StrokeThickness = 2.0,
15.             Stroke = new SolidColorBrush(displayColor),
16.             StrokeLineJoin = PenLineJoin.Round
17.         };
18.  
19.  
20.         float x, y;
21.  
22.         engine.SkeletonToDepthImage(position, out x, out y);
23.  
24.         x = (float)(x * displayCanvas.ActualWidth);
25.         y = (float)(y * displayCanvas.ActualHeight);
26.  
27.         Canvas.SetLeft(newEntry.DisplayEllipse, x - newEntry.DisplayEllipse.Width / 2);
28.         Canvas.SetTop(newEntry.DisplayEllipse, y - newEntry.DisplayEllipse.Height / 2);
29.  
30.         displayCanvas.Children.Add(newEntry.DisplayEllipse);
31.     }
32.  
33.     if (Entries.Count > WindowSize)
34.     {
35.         Entry entryToRemove = Entries[0];
36.         
37.         if (displayCanvas != null)
38.         {
39.             displayCanvas.Children.Remove(entryToRemove.DisplayEllipse);
40.         }
41.  
42.         Entries.Remove(entryToRemove);
43.     }
44.  
45.     LookForGesture();
46. }

注意SkeletonToDepthImage 方法是将三维坐标转成二维坐标，且在每个坐标轴上的值介于0和1之间。

除了保存节点位置之外，GestureDetector 类还可以将他们在屏幕上画出来以使开发和调试更加简化和直观:

As we can see above, the positions being analyzed are shown in red above the Kinect image. To activate this service, the developer just needs to put a canvas over the image that shows the stream of the Kinect camera and pass this canvas to the GestureDetector.TraceTo method:

1. <Viewbox Margin="5" Grid.RowSpan="5">
2.     <Grid Width="640" Height="480" ClipToBounds="True">
3.         <Image x:Name="kinectDisplay"></Image>
4.         <Canvas x:Name="kinectCanvas"></Canvas>
5.         <Canvas x:Name="gesturesCanvas"></Canvas>
6.         <Rectangle Stroke="Black" StrokeThickness="1"/>
7.     </Grid>
8. </Viewbox>

The Viewbox is used to keep the image and the canvas at the same size. The second canvas (kinectCanvas) is used to display the green skeleton (using a class available in the sample : SkeletonDisplayManager).

Event-based approach

GestureDetector class provides one last service to his children: the RaiseGestureDetected method that reports the detection of a new gesture via anOnGestureDetected event. The SupportedGesture argument of this event contains the following values:

• SwipeToLeft
• SwipeToRight
• Circle

Obviously the solution is extensible and I encourage you to add new gestures to the system. 

The RaiseGestureDetected method (with MinimalPeriodBetweenGestures property) also guarantee that a certain time will elapse between two gestures (in order to remove badly executed gestures). 

Now that our foundations are laid, we can develop our algorithms. 

Algorithmic search

The algorithmic search browses the list of positions and checks that predefined constraints are always valid. 

The SwipeGestureDetector class is responsible for this search:

以SwipeToRight挥到右边手势为例, 我们使用了以下限制条件:

• 每个新的位置需在上个位置的右边
• 每个位置在高度上不能超过第一个位置指定的长度(20cm)
• 首末位置点的间隔时间必须在250ms和1500ms之间
• 手势至少要有40cm长

The SwipeToLeft gesture is based on the same constraints except for the direction of the movement of course.

To effectively manage these two gestures, we use a generic algorithm that checks the four constraints mentioned above:

1. bool ScanPositions(Func<Vector3, Vector3, bool> heightFunction, Func<Vector3, Vector3, bool> directionFunction, Func<Vector3, Vector3, bool> lengthFunction, int minTime, int maxTime)
2. {
3.     int start = 0;
4.  
5.     for (int index = 1; index < Entries.Count - 1; index++)
6.     {
7.         if (!heightFunction(Entries[0].Position, Entries[index].Position) || !directionFunction(Entries[index].Position, Entries[index + 1].Position))
8.         {
9.             start = index;
10.         }
11.  
12.         if (lengthFunction(Entries[index].Position, Entries[start].Position))
13.         {
14.             double totalMilliseconds = (Entries[index].Time - Entries[start].Time).TotalMilliseconds;
15.             if (totalMilliseconds >= minTime && totalMilliseconds <= maxTime)
16.             {
17.                 return true;
18.             }
19.         }
20.     }
21.  
22.     return false;
23. }

为了使用以上方法，我们必须提供三个函数以及延迟时间来检测，

为了检测这两个手势，只要简单调用如下代码:

1. protected override void LookForGesture()
2. {
3.     // Swipe to right
4.     if (ScanPositions((p1, p2) => Math.Abs(p2.Y - p1.Y) < SwipeMaximalHeight, // Height
5.         (p1, p2) => p2.X - p1.X > -0.01f, // Progression to right
6.         (p1, p2) => Math.Abs(p2.X - p1.X) > SwipeMinimalLength, // Length
7.         SwipeMininalDuration, SwipeMaximalDuration)) // Duration
8.     {
9.         RaiseGestureDetected(SupportedGesture.SwipeToRight);
10.         return;
11.     }
12.  
13.     // Swipe to left
14.     if (ScanPositions((p1, p2) => Math.Abs(p2.Y - p1.Y) < SwipeMaximalHeight,  // Height
15.         (p1, p2) => p2.X - p1.X < 0.01f, // Progression to right
16.         (p1, p2) => Math.Abs(p2.X - p1.X) > SwipeMinimalLength, // Length
17.         SwipeMininalDuration, SwipeMaximalDuration))// Duration
18.     {
19.         RaiseGestureDetected(SupportedGesture.SwipeToLeft);
20.         return;
21.     }
22. }

骨骼检测的稳定性

为了确保检测的正确性，我们必须确认骨骼是静态的以免检测生成错误的手势(例如检测整个身体都在移动的情况)

为了做以上工作，我们使用BarycenterHelper 类：

1. public class BarycenterHelper
2. {
3.     readonly Dictionary<int, List<Vector3>> positions = new Dictionary<int, List<Vector3>>();
4.     readonly int windowSize;
5.  
6.     public float Threshold { get; set; }
7.  
8.     public BarycenterHelper(int windowSize = 20, float threshold = 0.05f)
9.     {
10.         this.windowSize = windowSize;
11.         Threshold = threshold;
12.     }
13.  
14.     public bool IsStable(int trackingID)
15.     {
16.         List<Vector3> currentPositions = positions[trackingID];
17.         if (currentPositions.Count != windowSize)
18.             return false;
19.  
20.         Vector3 current = currentPositions[currentPositions.Count - 1];
21.  
22.         for (int index = 0; index < currentPositions.Count - 2; index++)
23.         {
24.             Debug.WriteLine((currentPositions[index] - current).Length());
25.  
26.             if ((currentPositions[index] - current).Length() > Threshold)
27.                 return false;
28.         }
29.  
30.         return true;
31.     }
32.  
33.     public void Add(Vector3 position, int trackingID)
34.     {
35.         if (!positions.ContainsKey(trackingID))
36.             positions.Add(trackingID, new List<Vector3>());
37.  
38.         positions[trackingID].Add(position);
39.  
40.         if (positions[trackingID].Count > windowSize)
41.             positions[trackingID].RemoveAt(0);
42.     }
43. }

通过IsStable方法可以得知骨骼是正在移动着的还是静止的。

我们利用此信息，只有当骨骼不是移动着时才告知检测系统节点位置信息。

1. void ProcessFrame(ReplaySkeletonFrame frame)
2. {
3.     Dictionary<int, string> stabilities = new Dictionary<int, string>();
4.     foreach (var skeleton in frame.Skeletons)
5.     {
6.         if (skeleton.TrackingState != SkeletonTrackingState.Tracked)
7.             continue;
8.  
9.         barycenterHelper.Add(skeleton.Position.ToVector3(), skeleton.TrackingID);
10.  
11.         stabilities.Add(skeleton.TrackingID, barycenterHelper.IsStable(skeleton.TrackingID) ? "Stable" :"Unstable");
12.         if (!barycenterHelper.IsStable(skeleton.TrackingID))
13.             continue;
14.  
15.         foreach (Joint joint in skeleton.Joints)
16.         {
17.             if (joint.Position.W < 0.8f || joint.TrackingState != JointTrackingState.Tracked)
18.                 continue;
19.  
20.             if (joint.ID == JointID.HandRight)
21.             {
22.                 swipeGestureRecognizer.Add(joint.Position, kinectRuntime.SkeletonEngine);
23.                 circleGestureRecognizer.Add(joint.Position, kinectRuntime.SkeletonEngine);
24.             }
25.         }
26.  
27.         postureRecognizer.TrackPostures(skeleton);
28.     }
29.  
30.     skeletonDisplayManager.Draw(frame);
31.  
32.     stabilitiesList.ItemsSource = stabilities;
33.  
34.     currentPosture.Text = "Current posture: " + postureRecognizer.CurrentPosture.ToString();
35. }

回放&记录工具

在用Kinect SDK开发过程中另一个重要的一点是有效的测试。为了测试你必须站起来，到传感器面前摆出合适的姿势。除非你有一个助手，否则会变得很痛苦。

因此我们有一个kinect信息的记录和回放服务。

记录

记录部分非常地简单，因为我们只需要得到SkeletonFrame且遍历每一个骨骼将其中内容序列化：

1. public void Record(SkeletonFrame frame)
2. {
3.     if (writer == null)
4.         throw new Exception("You must call Start before calling Record");
5.  
6.     TimeSpan timeSpan = DateTime.Now.Subtract(referenceTime);
7.     referenceTime = DateTime.Now;
8.     writer.Write((long)timeSpan.TotalMilliseconds);
9.     writer.Write(frame.FloorClipPlane);
10.     writer.Write((int)frame.Quality);
11.     writer.Write(frame.NormalToGravity);
12.  
13.     writer.Write(frame.Skeletons.Length);
14.  
15.     foreach (SkeletonData skeleton in frame.Skeletons)
16.     {
17.         writer.Write((int)skeleton.TrackingState);
18.         writer.Write(skeleton.Position);
19.         writer.Write(skeleton.TrackingID);
20.         writer.Write(skeleton.EnrollmentIndex);
21.         writer.Write(skeleton.UserIndex);
22.         writer.Write((int)skeleton.Quality);
23.  
24.         writer.Write(skeleton.Joints.Count);
25.         foreach (Joint joint in skeleton.Joints)
26.         {
27.             writer.Write((int)joint.ID);
28.             writer.Write((int)joint.TrackingState);
29.             writer.Write(joint.Position);
30.         }
31.     }
32. }

回放

回放机制主要的问题是关于数据结构。事实上，kinect类是封装好的、不对外暴露它的构造函数。为了解决这个问题，我们复制模仿了kinect类层次，且增加了隐式强制操作符:

SkeletonReplay 类中的Start 方法来负责回放:

1. public void Start()
2. {
3.     context = SynchronizationContext.Current;
4.  
5.     CancellationToken token = cancellationTokenSource.Token;
6.  
7.     Task.Factory.StartNew(() =>
8.     {
9.         foreach (ReplaySkeletonFrame frame in frames)
10.         {
11.             Thread.Sleep(TimeSpan.FromMilliseconds(frame.TimeStamp));
12.  
13.             if (token.IsCancellationRequested)
14.                 return;
15.                                       
16.             ReplaySkeletonFrame closure = frame;
17.             context.Send(state =>
18.                             {
19.                                 if (SkeletonFrameReady != null)
20.                                     SkeletonFrameReady(this, new ReplaySkeletonFrameReadyEventArgs{SkeletonFrame = closure});
21.                             }, null);
22.         }
23.     }, token);
24. }

最终，我们可以记录和回放手势来调试我们的应用:

你可以在这里下载到回放的例子: http://www.catuhe.com/msdn/davca.replay.zip

知道何时开始

最后一个剩下需要解决的问题就是决定何时开始手势的分析。我们已经知道需在身体静止的时候，但这还不够。 

即使我保持静止，我说话时配合了大量手势，也会无意中触发了某个手势。

为了防止这种情况，需要增加一个姿势的条件。 

这是我们为什么打算用PostureDetector类:

1. public class PostureDetector
2. {
3.     const float Epsilon = 0.1f;
4.     const float MaxRange = 0.25f;
5.     const int AccumulatorTarget = 10;
6.  
7.     Posture previousPosture = Posture.None;
8.     public event Action<Posture> PostureDetected;
9.     int accumulator;
10.     Posture accumulatedPosture = Posture.None;
11.  
12.     public Posture CurrentPosture
13.     {
14.         get { return previousPosture; }
15.     }
16.  
17.     public void TrackPostures(ReplaySkeletonData skeleton)
18.     {
19.         if (skeleton.TrackingState != SkeletonTrackingState.Tracked)
20.             return;
21.  
22.         Vector3? headPosition = null;
23.         Vector3? leftHandPosition = null;
24.         Vector3? rightHandPosition = null;
25.  
26.         foreach (Joint joint in skeleton.Joints)
27.         {
28.             if (joint.Position.W < 0.8f || joint.TrackingState != JointTrackingState.Tracked)
29.                 continue;
30.  
31.             switch (joint.ID)
32.             {
33.                 case JointID.Head:
34.                     headPosition = joint.Position.ToVector3();
35.                     break;
36.                 case JointID.HandLeft:
37.                     leftHandPosition = joint.Position.ToVector3();
38.                     break;
39.                 case JointID.HandRight:
40.                     rightHandPosition = joint.Position.ToVector3();
41.                     break;
42.             }
43.         }
44.  
45.         // HandsJoined
46.         if (CheckHandsJoined(rightHandPosition, leftHandPosition))
47.             return;
48.  
49.         // LeftHandOverHead
50.         if (CheckHandOverHead(headPosition, leftHandPosition))
51.         {
52.             RaisePostureDetected(Posture.LeftHandOverHead);
53.             return;
54.         }
55.  
56.         // RightHandOverHead
57.         if (CheckHandOverHead(headPosition, rightHandPosition))
58.         {
59.             RaisePostureDetected(Posture.RightHandOverHead);
60.             return;
61.         }
62.  
63.         // LeftHello
64.         if (CheckHello(headPosition, leftHandPosition))
65.         {
66.             RaisePostureDetected(Posture.LeftHello);
67.             return;
68.         }
69.  
70.         // RightHello
71.         if (CheckHello(headPosition, rightHandPosition))
72.         {
73.             RaisePostureDetected(Posture.RightHello);
74.             return;
75.         }
76.  
77.         previousPosture = Posture.None;
78.         accumulator = 0;
79.     }
80.  
81.     bool CheckHandOverHead(Vector3? headPosition, Vector3? handPosition)
82.     {
83.         if (!handPosition.HasValue || !headPosition.HasValue)
84.             return false;
85.  
86.         if (handPosition.Value.Y < headPosition.Value.Y)
87.             return false;
88.  
89.         if (Math.Abs(handPosition.Value.X - headPosition.Value.X) > MaxRange)
90.             return false;
91.  
92.         if (Math.Abs(handPosition.Value.Z - headPosition.Value.Z) > MaxRange)
93.             return false;
94.  
95.         return true;
96.     }
97.  
98.  
99.     bool CheckHello(Vector3? headPosition, Vector3? handPosition)
100.     {
101.         if (!handPosition.HasValue || !headPosition.HasValue)
102.             return false;
103.  
104.         if (Math.Abs(handPosition.Value.X - headPosition.Value.X) < MaxRange)
105.             return false;
106.  
107.         if (Math.Abs(handPosition.Value.Y - headPosition.Value.Y) > MaxRange)
108.             return false;
109.  
110.         if (Math.Abs(handPosition.Value.Z - headPosition.Value.Z) > MaxRange)
111.             return false;
112.  
113.         return true;
114.     }
115.  
116.     bool CheckHandsJoined(Vector3? leftHandPosition, Vector3? rightHandPosition)
117.     {
118.         if (!leftHandPosition.HasValue || !rightHandPosition.HasValue)
119.             return false;
120.  
121.         float distance = (leftHandPosition.Value - rightHandPosition.Value).Length();
122.  
123.         if (distance > Epsilon)
124.             return false;
125.  
126.         RaisePostureDetected(Posture.HandsJoined);
127.         return true;
128.     }
129.  
130.     void RaisePostureDetected(Posture posture)
131.     {
132.         if (accumulator < AccumulatorTarget)
133.         {
134.             if (accumulatedPosture != posture)
135.             {
136.                 accumulator = 0;
137.                 accumulatedPosture = posture;
138.             }
139.             accumulator++;
140.             return;
141.         }
142.  
143.         if (previousPosture == posture)
144.             return;
145.  
146.         previousPosture = posture;
147.         if (PostureDetected != null)
148.             PostureDetected(posture);
149.  
150.         accumulator = 0;
151.     }
152. }

PostureDetector 类基于对每个节点位置的比较。例如，为了触发Hello 姿势, 我们必须知道是否手与头在同一高度，且手在一侧至少25cm.

此外，系统使用了一个累加器来确保在指定的几帧内姿势是静止的。

此外，此类是极富可扩展性的。

基于模版的搜索

基于算法搜索的主要确定就是所有的手势不都是易于用限制条件描述的。我们因此需考虑更加通用的方法。

我们假定一个手势是可以被记录的，且随后系统会决定当前的手势是否是已知的。

最终，我们的目标是有效地比较两个手势。

Compare the comparable

在我们开始写一个比较算法之前，我们必须先标准化数据。

 
事实上，一个手势是一系列点的集合Indeed（在这篇文章中我们将简单地比较2D手势如画圆)。点的坐标决定了离传感器的距离 ，我们把他们一起放到一个通用的参考，为此需做以下工作:

1. 用定义了数量的点集合生成一个新的手势
2. 旋转手势使得第一个点在0度位置
3. 缩放手势使得其在坐标轴上的值介于0和1之间
4. 将手势居中到(0,0)点

做了这些变化后，我们就能够比较点数组了。

为了使用这些技术来包装点数组，我们用以下代码：

1. public static List<Vector2> Pack(List<Vector2> positions, int samplesCount)
2. {
3.     List<Vector2> locals = ProjectListToDefinedCount(positions, samplesCount);
4.  
5.     float angle = GetAngleBetween(locals.Center(), positions[0]);
6.     locals = locals.Rotate(-angle);
7.  
8.     locals.ScaleToReferenceWorld();
9.     locals.CenterToOrigin();
10.  
11.     return locals;
12. }

GoldenSectionExtensions 静态类有一些用到的方法:

1. public static class GoldenSectionExtensions
2. {
3.     // Get length of path
4.     public static float Length(this List<Vector2> points)
5.     {
6.         float length = 0;
7.  
8.         for (int i = 1; i < points.Count; i++)
9.         {
10.             length += (points[i - 1] - points[i]).Length();
11.         }
12.  
13.         return length;
14.     }
15.  
16.     // Get center of path
17.     public static Vector2 Center(this List<Vector2> points)
18.     {
19.         Vector2 result = points.Aggregate(Vector2.Zero, (current, point) => current + point);
20.  
21.         result /= points.Count;
22.  
23.         return result;
24.     }
25.  
26.     // Rotate path by given angle
27.     public static List<Vector2> Rotate(this List<Vector2> positions, float angle)
28.     {
29.         List<Vector2> result = new List<Vector2>(positions.Count);
30.         Vector2 c = positions.Center();
31.  
32.         float cos = (float)Math.Cos(angle);
33.         float sin = (float)Math.Sin(angle);
34.  
35.         foreach (Vector2 p in positions)
36.         {
37.             float dx = p.X - c.X;
38.             float dy = p.Y - c.Y;
39.  
40.             Vector2 rotatePoint = Vector2.Zero;
41.             rotatePoint.X = dx * cos - dy * sin + c.X;
42.             rotatePoint.Y = dx * sin + dy * cos + c.Y;
43.  
44.             result.Add(rotatePoint);
45.         }
46.         return result;
47.     }
48.  
49.     // Average distance betweens paths
50.     public static float DistanceTo(this List<Vector2> path1, List<Vector2> path2)
51.     {
52.         return path1.Select((t, i) => (t - path2[i]).Length()).Average();
53.     }
54.  
55.     // Compute bounding rectangle
56.     public static Rectangle BoundingRectangle(this List<Vector2> points)
57.     {
58.         float minX = points.Min(p => p.X);
59.         float maxX = points.Max(p => p.X);
60.         float minY = points.Min(p => p.Y);
61.         float maxY = points.Max(p => p.Y);
62.  
63.         return new Rectangle(minX, minY, maxX - minX, maxY - minY);
64.     }
65.  
66.     // Check bounding rectangle size
67.     public static bool IsLargeEnough(this List<Vector2> positions, float minSize)
68.     {
69.         Rectangle boundingRectangle = positions.BoundingRectangle();
70.  
71.         return boundingRectangle.Width > minSize && boundingRectangle.Height > minSize;
72.     }
73.  
74.     // Scale path to 1x1
75.     public static void ScaleToReferenceWorld(this List<Vector2> positions)
76.     {
77.         Rectangle boundingRectangle = positions.BoundingRectangle();
78.         for (int i = 0; i < positions.Count; i++)
79.         {
80.             Vector2 position = positions[i];
81.  
82.             position.X *= (1.0f / boundingRectangle.Width);
83.             position.Y *= (1.0f / boundingRectangle.Height);
84.  
85.             positions[i] = position;
86.         }
87.     }
88.  
89.     // Translate path to origin (0, 0)
90.     public static void CenterToOrigin(this List<Vector2> positions)
91.     {
92.         Vector2 center = positions.Center();
93.         for (int i = 0; i < positions.Count; i++)
94.         {
95.             positions[i] -= center;
96.         }
97.     }
98. }

黄金分割

比较我们的数据可以通过一个简单的比较每个点之间的平均距离方法。但是，这种解决方案不够精确。 

所以我使用了一个更加好的算法:黄金分割法。http://www.math.uic.edu/~jan/mcs471/Lec9/gss.pdf

JavaScript版本的实现如下: http://depts.washington.edu/aimgroup/proj/dollar/

以下是C#版本的实现:

1. public static float Search(List<Vector2> current, List<Vector2> target, float a, float b, float epsilon)
2. {
3.     float x1 = ReductionFactor * a + (1 - ReductionFactor) * b;
4.     List<Vector2> rotatedList = current.Rotate(x1);
5.     float fx1 = rotatedList.DistanceTo(target);
6.  
7.     float x2 = (1 - ReductionFactor) * a + ReductionFactor * b;
8.     rotatedList = current.Rotate(x2);
9.     float fx2 = rotatedList.DistanceTo(target);
10.  
11.     do
12.     {
13.         if (fx1 < fx2)
14.         {
15.             b = x2;
16.             x2 = x1;
17.             fx2 = fx1;
18.             x1 = ReductionFactor * a + (1 - ReductionFactor) * b;
19.             rotatedList = current.Rotate(x1);
20.             fx1 = rotatedList.DistanceTo(target);
21.         }
22.         else
23.         {
24.             a = x1;
25.             x1 = x2;
26.             fx1 = fx2;
27.             x2 = (1 - ReductionFactor) * a + ReductionFactor * b;
28.             rotatedList = current.Rotate(x2);
29.             fx2 = rotatedList.DistanceTo(target);
30.         }
31.     }
32.     while (Math.Abs(b - a) > epsilon);
33.  
34.     float min = Math.Min(fx1, fx2);
35.  
36.     return 1.0f - 2.0f * min / Diagonal;
37. }

利用此算法，我们可以简单将一个模板与当前手势进行比较并且得到一个分数(0到1之间)

学习机

我们改善我们的成功比率，我们需要多个模板。为此，我们开发了LearningMachine 类 ，它的职责是存储我们的模板(例如：学习新的模型)并且与当前的姿势进行比较:

1. public class LearningMachine
2. {
3.     readonly List<RecordedPath> paths;
4.  
5.     public LearningMachine(Stream kbStream)
6.     {
7.         if (kbStream == null || kbStream.Length == 0)
8.         {
9.             paths = new List<RecordedPath>();
10.             return;
11.         }
12.  
13.         BinaryFormatter formatter = new BinaryFormatter();
14.  
15.         paths = (List<RecordedPath>)formatter.Deserialize(kbStream);
16.     }
17.  
18.     public List<RecordedPath> Paths
19.     {
20.         get { return paths; }
21.     }
22.  
23.     public bool Match(List<Vector2> entries, float threshold, float minimalScore, float minSize)
24.     {
25.         return Paths.Any(path => path.Match(entries, threshold, minimalScore, minSize));
26.     }
27.  
28.     public void Persist(Stream kbStream)
29.     {
30.         BinaryFormatter formatter = new BinaryFormatter();
31.  
32.         formatter.Serialize(kbStream, Paths);
33.     }
34.  
35.     public void AddPath(RecordedPath path)
36.     {
37.         path.CloseAndPrepare();
38.         Paths.Add(path);
39.     }
40. }

每个RecordedPath 有一种Match 方法的实现，这里会调用黄金分割搜索:

1. public bool Match(List<Vector2> positions, float threshold, float minimalScore, float minSize)
2. {
3.     if (positions.Count < samplesCount)
4.         return false;
5.  
6.     if (!positions.IsLargeEnough(minSize))
7.         return false;
8.  
9.     List<Vector2> locals = GoldenSection.Pack(positions, samplesCount);
10.  
11.     float score = GoldenSection.Search(locals, points, -MathHelper.PiOver4, MathHelper.PiOver4, threshold);
12.  
13.     return score > minimalScore;
14. }

多亏了我们的检测算法和状态机，我们可以用一个可靠且对kinect提供的信息以来非常小的系统来处理。

你可以找到一个以以上为基础的识别画圆的例子: http://www.catuhe.com/msdn/circleKB.zip.

总结

So we have at our disposal a set of tools for working with Kinect. In addition we have two systems to detect a large number of gestures. 

It's now your turn to use these services in your Kinect applications!

To go further

• Kinect on Channel9
• Coding4Fun Kinect
• Microsoft Research site

• © 2011 Microsoft Corporation.
 
• Terms of Use 
• Trademarks 
• Privacy Statement 
• Report Abuse