lightweightopenpose对数据处理

输入网络的图片大小是固定的，定长的数据结构节省空间。

1.transformer的convert是对关键生成neck坐标然后换索引，flip把左右关键点调换。

2.有归一化mean=128，std=256。

3.heatmap。

4.paf。

train.py是直接对数据用transformer，然后生成keypoint和paf。

1.看lightweight openpose处理过程，如果用其他代码训练的话需要确定关键点位置。
(1)convert是生成新的关键交换索引。
(2)flip也是交换位置。

val.py对数据的处理:

读取图片然后规范化mean=128，std=256。

交换轴因为pytorch输入都是chw。

keypoint的最后一个图是所有关键点的集合。

hrnet:

get_joint获取关键点。

transformer处理,flip那里翻折。

keypoin

paf

 主要关键点对，图片大小对，归一化大小对，keypoint和paf生成对。就行。

我一开始以为lightweight只能存储一个人的关键点之后我发现它可以存储多个，一张图片的第一个人存储在prepared_annotation中第一张图的其他人存储在prepared_other_annotations = []，修后存于一个prepared_annotation里。一张图片中可能有n个人，创建n个prepared_other_annotations = []，每个人都会被存储在相对应的prepared_other_annotations，其他人则会被存储在prepared_other_annotations。就相当于有n个prepared_annotation，每个prepared_annotation都会有所有人的关键点。

我输出lightweight中相邻的索引

with open(labels, 'rb') as f:
self._labels = pickle.load(f)

对labels进行输出

对索引

128940
128941

的labels进行展示得出

{'img_paths': '000000158567.jpg', 'img_width': 500, 'img_height': 334, 'objpos': [470.26, 272.06], 'image_id': 158567, 'bbox': [458.19, 221.37, 24.14, 101.38], 'segment_area': 1834.9875, 'scale_provided': 0.27548913043478257, 'num_keypoints': 11, 'segmentations': [{'counts': [1588, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3309, 6, 327, 11, 322, 13, 320, 15, 319, 16, 318, 16, 318, 16, 318, 16, 318, 16, 318, 16, 319, 15, 320, 13, 323, 10, 328, 4, 6339, 6, 327, 8, 325, 10, 323, 12, 322,
12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 11, 322, 11, 322, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 8, 6, 308, 12, 7,
8, 307, 12, 6, 10, 307, 10, 6, 12, 306, 10, 6, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12, 6, 10, 306, 12, 7, 8, 308, 10, 10, 4, 311, 8, 326, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 13, 322, 13, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 11, 324, 13, 320, 15, 319, 16, 318, 17, 317, 17, 317, 17, 317, 17, 315, 19, 314, 20, 313, 21, 312, 21, 313, 20, 314, 18, 316, 13, 321, 13, 321, 13, 321, 13, 321, 13, 322, 11, 324, 9, 327, 5, 1667, 5, 328, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 10, 327, 8, 322, 13, 320, 15, 318, 16, 317, 17, 317, 17, 317, 17, 317, 17, 317, 17, 317, 16, 318, 15, 319, 13, 320, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 324, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 7, 329, 3, 2673, 5, 327, 8, 325, 10, 323, 12, 322, 12, 322,
12, 322, 12, 322, 12, 322, 12, 321, 13, 321, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 2, 6, 320, 4, 3, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3992, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 321, 12, 321, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 2, 4, 2, 3331, 1, 332, 10, 323, 12, 322, 12, 322, 12, 322, 11, 323, 11, 323, 10, 324, 10, 325, 9, 325, 10, 1, 1, 322, 12, 322, 13, 321, 13, 322, 12, 322, 12, 322, 12, 322, 12, 322, 14, 321, 14, 321, 14, 320, 15, 319, 15, 319, 15, 318, 16, 318, 16, 318, 16, 318, 16, 318, 15, 319, 14, 320, 12, 323, 10, 325, 8, 331, 1, 2332, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 13, 321, 14, 320, 14, 321, 13, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 1331, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 322, 11, 322, 10, 323, 10, 323, 13, 321, 14, 320, 15, 319, 16, 318, 16, 318, 16, 318, 16, 318, 16, 319, 15, 317, 17, 316, 17, 316, 18, 315, 19, 315, 18, 316, 17, 317, 15, 319, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3005, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 14, 320, 15, 319, 16, 318, 17, 318, 16, 319, 15, 319, 15, 318, 16, 317, 17, 316, 18, 316, 17, 317, 16, 318, 14, 320, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3672, 2, 331, 7, 326, 9, 324, 11, 323, 12, 322, 12, 322, 12, 321, 13, 321, 13, 321, 13, 321, 12, 322, 12,
322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 663, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 1, 6, 315, 20, 314, 21, 313, 22, 312, 22, 311, 23, 310, 24, 310, 24, 310, 24, 310, 24, 310, 12, 1, 10, 311, 12, 2, 8, 312, 13, 3, 4, 315, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322,
12, 320, 13, 320, 13, 320, 12, 321, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 323, 9, 324, 8, 5, 6, 314, 10, 3, 8, 312, 12, 1, 10, 311, 24, 310, 24, 310, 24, 310, 24, 310, 24, 310, 24, 311, 10, 1, 12, 312, 8, 3, 10, 315, 4, 6, 8, 328, 1, 13133], 'size': [334, 500]}], 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [471, 228, 1], [464, 239, 1], [470, 238, 1], [0, 0, 2], [475, 252, 1], [0, 0, 2], [480, 263, 1], [464, 266, 1], [470, 266, 1], [466, 294, 1], [470, 295, 1], [467, 315, 1], [470, 316, 1]], 'processed_other_annotations': [{'objpos': [251.105, 258.46500000000003], 'bbox': [238.14, 221.34, 25.93, 74.25], 'segment_area': 1103.3586, 'scale_provided': 0.20176630434782608, 'num_keypoints': 9, 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [249, 227, 1], [242, 236, 1], [249, 236, 1], [0, 0, 2], [257, 244, 1], [0, 0, 2], [256, 236, 1], [245, 261, 1], [251, 262, 1], [0, 0, 2], [251, 276, 1], [0, 0, 2], [249, 291, 1]]}, {'objpos': [389.04499999999996, 269.555], 'bbox': [366.34, 221.48, 45.41, 96.15], 'segment_area': 2523.8144, 'scale_provided': 0.2612771739130435, 'num_keypoints': 12, 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [377, 237, 1], [400, 236, 1], [369, 246, 1], [407, 245, 1], [374, 259, 1], [404, 257, 1], [383, 263, 1], [395, 262, 1], [387, 290, 1], [401, 286, 1], [387, 310, 1], [403, 308, 1]]}, {'objpos': [443.65, 270.01], 'bbox': [425.46, 225.37, 36.38, 89.28], 'segment_area': 1887.11775, 'scale_provided': 0.2426086956521739, 'num_keypoints': 11, 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [437, 233, 0], [446, 233, 1], [432, 245, 1], [452, 244, 1],
[428, 252, 1], [0, 0, 2], [0, 0, 2], [0, 0, 2], [438, 270, 1], [453, 269, 1], [442, 288, 1], [455, 289, 1], [445, 307, 1], [457, 306, 1]]}, {'objpos': [486.805, 263.345], 'bbox': [473.61, 208.63, 26.39, 109.43], 'segment_area': 2053.05075, 'scale_provided': 0.2973641304347826,
'num_keypoints': 13, 'keypoints': [[495, 219, 1], [496, 216, 1], [491, 218, 1], [0, 0, 2], [488, 218, 1], [0, 0, 2], [480, 230, 1], [475, 245, 0], [0, 0, 2], [477, 263, 0], [0, 0, 2], [496, 264, 1], [483, 265, 1], [496, 291, 1], [482, 290, 1], [496, 308, 1], [478, 310, 1]]}]}
------
{'img_paths': '000000158567.jpg', 'img_width': 500, 'img_height': 334, 'objpos': [443.65, 270.01], 'image_id': 158567, 'bbox': [425.46, 225.37, 36.38, 89.28], 'segment_area': 1887.11775, 'scale_provided': 0.2426086956521739, 'num_keypoints': 11, 'segmentations': [{'counts': [1588, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3309, 6, 327, 11, 322, 13,
320, 15, 319, 16, 318, 16, 318, 16, 318, 16, 318, 16, 318, 16, 319, 15, 320, 13, 323, 10, 328, 4, 6339, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 11, 322, 11, 322, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 8, 6, 308, 12, 7, 8, 307, 12, 6, 10, 307, 10, 6, 12, 306, 10, 6, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12, 5, 12, 305, 12,
6, 10, 306, 12, 7, 8, 308, 10, 10, 4, 311, 8, 326, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 13, 322,
13, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 11, 324, 13, 320, 15, 319, 16, 318, 17, 317, 17, 317, 17, 317, 17, 315, 19,
314, 20, 313, 21, 312, 21, 313, 20, 314, 18, 316, 13, 321, 13, 321, 13, 321, 13, 321, 13, 322, 11, 324, 9, 327, 5, 1667, 5, 328, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 10, 327, 8, 322, 13, 320, 15, 318, 16, 317, 17, 317, 17, 317, 17, 317, 17, 317, 17, 317, 16, 318, 15, 319, 13, 320, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 324, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 7, 329, 3, 2673, 5, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 321, 13, 321, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 2, 6, 320, 4, 3, 8, 325, 10,
323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3992, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 321, 12, 321, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 2, 4, 2, 3331,
1, 332, 10, 323, 12, 322, 12, 322, 12, 322, 11, 323, 11, 323, 10, 324, 10, 325, 9, 325, 10, 1, 1, 322, 12, 322, 13, 321, 13, 322, 12, 322,
12, 322, 12, 322, 12, 322, 14, 321, 14, 321, 14, 320, 15, 319, 15, 319, 15, 318, 16, 318, 16, 318, 16, 318, 16, 318, 15, 319, 14, 320, 12,
323, 10, 325, 8, 331, 1, 2332, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 13, 321, 14, 320, 14, 321, 13, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 1331, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323,
10, 322, 11, 322, 10, 323, 10, 323, 13, 321, 14, 320, 15, 319, 16, 318, 16, 318, 16, 318, 16, 318, 16, 319, 15, 317, 17, 316, 17, 316, 18,
315, 19, 315, 18, 316, 17, 317, 15, 319, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3005, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 322, 14, 320, 15, 319, 16, 318, 17, 318, 16, 319, 15, 319, 15, 318, 16, 317, 17, 316, 18, 316, 17, 317, 16, 318, 14, 320, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 3672, 2, 331, 7, 326, 9, 324, 11, 323, 12, 322, 12, 322, 12, 321, 13, 321, 13, 321, 13, 321, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 328, 4, 663, 6, 327, 8, 325, 10, 323, 12, 322, 12, 322, 12, 1, 6, 315, 20, 314, 21, 313, 22, 312, 22, 311, 23, 310, 24, 310, 24, 310, 24, 310, 24, 310, 12, 1, 10, 311, 12, 2, 8, 312, 13, 3, 4, 315, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 320, 13, 320, 13, 320, 12, 321, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 322, 12, 323, 10, 325, 8, 323, 9, 324, 8, 5, 6, 314, 10, 3, 8, 312, 12, 1, 10, 311, 24, 310, 24, 310, 24, 310, 24, 310, 24, 310, 24, 311, 10, 1, 12, 312, 8, 3, 10, 315, 4, 6, 8, 328, 1, 13133],
'size': [334, 500]}], 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [437, 233, 0], [446, 233, 1], [432, 245, 1], [452, 244, 1], [428, 252, 1], [0, 0, 2], [0, 0, 2], [0, 0, 2], [438, 270, 1], [453, 269, 1], [442, 288, 1], [455, 289, 1], [445, 307, 1], [457, 306, 1]], 'processed_other_annotations': [{'objpos': [251.105, 258.46500000000003], 'bbox': [238.14, 221.34, 25.93, 74.25], 'segment_area': 1103.3586, 'scale_provided': 0.20176630434782608, 'num_keypoints': 9, 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [249, 227, 1], [242, 236, 1],
[249, 236, 1], [0, 0, 2], [257, 244, 1], [0, 0, 2], [256, 236, 1], [245, 261, 1], [251, 262, 1], [0, 0, 2], [251, 276, 1], [0, 0, 2], [249, 291, 1]]}, {'objpos': [389.04499999999996, 269.555], 'bbox': [366.34, 221.48, 45.41, 96.15], 'segment_area': 2523.8144, 'scale_provided':
0.2612771739130435, 'num_keypoints': 12, 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [377, 237, 1], [400, 236, 1], [369, 246, 1], [407, 245, 1], [374, 259, 1], [404, 257, 1], [383, 263, 1], [395, 262, 1], [387, 290, 1], [401, 286, 1], [387, 310, 1], [403, 308, 1]]}, {'objpos': [470.26, 272.06], 'bbox': [458.19, 221.37, 24.14, 101.38], 'segment_area': 1834.9875, 'scale_provided': 0.27548913043478257, 'num_keypoints': 11, 'keypoints': [[0, 0, 2], [0, 0, 2], [0, 0, 2], [0, 0, 2], [471, 228, 1], [464, 239, 1], [470, 238, 1], [0,
0, 2], [475, 252, 1], [0, 0, 2], [480, 263, 1], [464, 266, 1], [470, 266, 1], [466, 294, 1], [470, 295, 1], [467, 315, 1], [470, 316, 1]]}, {'objpos': [486.805, 263.345], 'bbox': [473.61, 208.63, 26.39, 109.43], 'segment_area': 2053.05075, 'scale_provided': 0.2973641304347826,
'num_keypoints': 13, 'keypoints': [[495, 219, 1], [496, 216, 1], [491, 218, 1], [0, 0, 2], [488, 218, 1], [0, 0, 2], [480, 230, 1], [475, 245, 0], [0, 0, 2], [477, 263, 0], [0, 0, 2], [496, 264, 1], [483, 265, 1], [496, 291, 1], [482, 290, 1], [496, 308, 1], [478, 310, 1]]}]}