inspect_bottle_label_360_degree.hdev 案例
这是一个halcon实现4个瓶子侧面拼接成360°全景图的案例,在上面吃了大亏,特此记录一下。
基本流程图

1. 主函数
* ============================== 主函数 ============================== * 瓶子标签360度展开检测主程序 * 目标:生成瓶身标签的无缝全景展开图 * 核心步骤:1)瓶体位姿估计 2)图像校正 3)图像拼接 * * 全局变量声明(用于中间结果可视化控制) global def tuple DisplayIntermediateResults * 控制中间结果显示的布尔值 global def tuple WindowWidthLimit * 显示窗口最大宽度限制 global def tuple WindowHeightLimit * 显示窗口最大高度限制 * ====================== 标准控制参数 ====================== PixelSizeInMM := 0.2 * 全景图的毫米级像素尺寸(决定输出图像分辨率) ColorMosaic := true * 是否生成彩色全景图(true=彩色,false=灰度) HighImageQuality := true * 高质量处理模式(true=双三次插值,false=双线性插值) BackgroundMayContainTexture := false * 背景是否可能包含纹理(影响轮廓提取策略) InteractivelyDefineRegion := false * 是否交互式定义标签区域(true=手动绘制,false=使用预设) DisplayIntermediateResultsFor := 'first' * 中间结果显示模式('first'=仅首组,'all'=所有组) * ====================== 精细调整参数 ====================== PerformFineAdjustment := true * 是否执行精细调整(true=启用图像配准) FineAdjustmentMatchingWidth := 30 * 配准区域宽度(像素,影响匹配精度) FineAdjustmentMaxShift := 30 * 最大允许偏移量(像素,限制调整范围) BlendingSeam := 10 * 图像融合接缝宽度(像素,影响过渡平滑度) * ====================== 轮廓提取参数 ====================== SilhouetteMeasureDistance := 10 * 测量点间距(像素,控制轮廓采样密度) SilhouetteMeasureLength2 := 30 * 测量区域半长度(像素,决定边缘检测范围) SilhouetteMeasureSigma := 0.5 * 高斯滤波Sigma值(影响边缘检测抗噪性) SilhouetteMeasureThreshold := 5 * 边缘检测阈值(值越小灵敏度越高) SilhouetteMaxTilt := rad(10) * 瓶子最大倾斜角度(弧度制,影响姿态估计容错) * 初始化显示设置 WindowWidthLimit := 800 * 窗口宽度上限 WindowHeightLimit := 600 * 窗口高度上限 dev_update_off () * 关闭图形更新加速处理 * 验证中间结果显示模式参数 if (DisplayIntermediateResultsFor != 'first' and DisplayIntermediateResultsFor != 'all') * 参数值错误处理 throw ('DisplayIntermediateResultsFor参数错误(必须为first或all)') endif DisplayIntermediateResults := true * 初始启用中间结果展示 * ====================== 数据准备阶段 ====================== * 设置数据路径 PathCsm := 'inspect_bottle_camera_setup_model.csm' * 相机标定模型文件路径 PathImg := 'bottle_label' * 图像文件夹路径 * 读取图像获取尺寸信息 list_image_files (PathImg, 'default', [], ImageFiles) * 获取文件夹内所有图像路径 * 筛选特定命名模式的图像(四组相机拍摄) RegExpression := '(freixenet|got2b|jever|wala)_0[1-2]_cam_[1-4].*' tuple_regexp_select (ImageFiles, RegExpression, ImageFiles) * 正则筛选图像 read_image (Image, ImageFiles[0]) * 读取第一张图像 get_image_size (Image, Width, Height) * 获取图像宽高 * 创建显示窗口 dev_close_window () * 关闭现有窗口 * 创建适应图像尺寸的窗口(受限于WindowWidthLimit/HeightLimit) dev_open_window_fit_image (Image, 0, 0, WindowWidthLimit, WindowHeightLimit, WindowHandle) set_display_font (WindowHandle, 16, 'mono', 'true', 'false') * 设置等宽字体 * 读取相机标定模型 read_camera_setup_model (PathCsm, CameraSetupModel) * 加载相机参数 * 设置相机模型原点为相机0投影中心 get_camera_setup_param (CameraSetupModel, 0, 'pose', CamPose0) * 获取相机0位姿 set_camera_setup_param (CameraSetupModel, 'general', 'coord_transf_pose', CamPose0) * 设置坐标系原点 * ====================== 主处理循环 ====================== * 遍历不同瓶子类型(jever, freixenet, got2b, wala) Objects := ['jever', 'freixenet', 'got2b', 'wala'] * 瓶型列表 for Obj := 0 to |Objects| - 1 by 1 Object := Objects[Obj] * 当前瓶型 * 根据瓶型设置参数 LabelMinCol := -1 * 标签区域左边界初始化 LabelMaxCol := -1 * 标签区域右边界初始化 if (Object == 'jever') CylinderRadiusInMM := 30.04 * 瓶子半径(mm) LabelMinCol := 100 * 标签左边界(像素) LabelMaxCol := 1250 * 标签右边界(像素) elseif (Object == 'freixenet') CylinderRadiusInMM := 31.3 LabelMinCol := 600 LabelMaxCol := 1150 elseif (Object == 'got2b') CylinderRadiusInMM := 24.935 LabelMinCol := 375 LabelMaxCol := 1025 elseif (Object == 'wala') CylinderRadiusInMM := 15.35 LabelMinCol := 660 LabelMaxCol := 1160 endif * 单位转换(毫米→米) PixelSize := 0.001 * PixelSizeInMM * 像素尺寸(米) CylinderRadius := 0.001 * CylinderRadiusInMM * 瓶子半径(米) * 解析图像集信息 PathExpression := Object + '_' * 构建瓶型匹配模式 tuple_regexp_select (ImageFiles, PathExpression, ImageSelection) * 筛选当前瓶型图像 LastTuple := split(ImageSelection[|ImageSelection| - 1],'/') * 分割路径 LastTupleParts := split(LastTuple,'_') * 分割文件名 NumSetsStr := LastTupleParts[|LastTupleParts| - 3] * 提取图像组数字符串 NumCamsStr := split(LastTupleParts[|LastTupleParts| - 1],'.')[0] * 提取相机数字符串 * 移除前导零(避免八进制解释) remove_leading_zeros (chr(ords(NumSetsStr)), NumSets) * 转换图像组数 remove_leading_zeros (chr(ords(NumCamsStr)), NumCams) * 转换相机数量 * ============== 交互式区域定义(可选) ============== if (InteractivelyDefineRegion) read_image (FirstImage, ImageSelection[0]) * 读取首张图像 * 调整窗口显示 dev_resize_window_fit_image (FirstImage, 0, 0, WindowWidthLimit, WindowHeightLimit) dev_display (FirstImage) * 显示图像 * 显示操作提示 disp_message (WindowHandle, ['定义需要展开的瓶子区域', '注意:仅使用矩形的左右边界'], 'window', 12, 12, 'black', 'true') * 绘制矩形区域 if (min([LabelMinCol,LabelMaxCol]) < 0) draw_rectangle1 (WindowHandle, Row1, LabelMinCol, Row2, LabelMaxCol) * 全新绘制 else * 在预设基础上调整 draw_rectangle1_mod (WindowHandle, 100, LabelMinCol, Height - 101, LabelMaxCol, Row1, LabelMinCol, Row2, LabelMaxCol) endif * 约束边界在图像范围内 LabelMinCol := max([0,LabelMinCol]) LabelMaxCol := min([Width - 1,LabelMaxCol]) endif * 计算有效边界 BorderLeft := LabelMinCol * 左边界(像素) BorderRight := Width - 1 - LabelMaxCol * 右边界(像素) * ============== 创建轮廓测量句柄 ============== MeasureHandles := [] * 初始化测量句柄数组 * 沿标签宽度方向创建垂直测量区域 for Col := LabelMinCol to LabelMaxCol by SilhouetteMeasureDistance * 创建垂直测量矩形(90度方向) gen_measure_rectangle2 (Height / 2, Col, rad(90), Height / 2, SilhouetteMeasureLength2, Width, Height, 'nearest_neighbor', MeasureHandle) MeasureHandles := [MeasureHandles,MeasureHandle] * 添加到数组 endfor * ============== 相机模型准备 ============== * 创建畸变校正映射 prepare_distortion_removal (RectificationMaps, CameraSetupModel, CameraSetupModelZeroDist, NumCameras) * 估算瓶子初始位姿(假设位于相机中心) determine_approximate_cylinder_pose_in_center_of_cameras (CameraSetupModelZeroDist, NumCameras, PoseCylinderApprox, HomMat3DCylinderApprox) * 计算圆柱模型所需高度范围 determine_required_cylinder_model_extent (BorderLeft, BorderRight, Width, Height, CameraSetupModelZeroDist, NumCameras, HomMat3DCylinderApprox, PixelSize, MinZ, MaxZ) * 计算轮廓间距范围 determine_min_max_silhouette_distance (CameraSetupModelZeroDist, NumCameras, PoseCylinderApprox, CylinderRadius, MinZ, MaxZ, Height, MinPairDist, MaxPairDist) * 生成圆柱3D模型 gen_cylinder_model (PixelSize, CylinderRadius, MinZ, MaxZ, NumSlices, NumPointsPerSlice, MinZI, MaxZI, ActualPixelSize, CylinderPointsX, CylinderPointsY, CylinderPointsZ) ActualPixelSizeInMM := 1000.0 * ActualPixelSize * 实际像素尺寸(mm) * 确定全景图尺寸 MosaicHeight := NumSlices * 高度=切片数 MosaicWidth := NumPointsPerSlice * 宽度=每片点数 * ============== 图像组处理循环 ============== for Set := 1 to NumSets by 1 * 读取当前图像组 tuple_regexp_select (ImageSelection, '_' + Set$'02d' + '_cam', ImageSet) * 筛选当前组图像 * 验证图像数量 if (|ImageSet| != NumCams) throw ('图像集' + Set$'02d' + '中的图像数量与相机数量不符') endif read_image (Images, ImageSet) * 读取图像组 * 开始计时 count_seconds (Seconds1) * 检查彩色处理可行性 if (ColorMosaic) select_obj (Images, ObjectSelected, 1) * 选择首张图像 count_channels (ObjectSelected, Channels) * 检查通道数 if (Channels == 1) ColorMosaic := false * 单通道图像强制灰度模式 elseif (Channels != 3) stop () * 通道数异常终止(必须1或3) endif endif * 消除镜头畸变 eliminate_radial_distortions (Images, RectificationMaps, ImagesRectified, ImagesGrayRectified, NumCameras) * 核心算法1:计算3D旋转轴 determine_rotation_axis_3d (ImagesRectified, ImagesGrayRectified, CylinderRadius, BackgroundMayContainTexture, MeasureHandles, Width, Height, CameraSetupModelZeroDist, NumCameras, MinPairDist, MaxPairDist, SilhouetteMeasureSigma, SilhouetteMeasureThreshold, SilhouetteMaxTilt, 0.1 * PixelSize, PoseCylinder, Quality, CameraSetupModelZeroDistInCylinderOrigin, RadiusEstimated) * 分配相机负责区域 determine_source_cameras_for_mosaic_parts (Regions, CameraSetupModelZeroDistInCylinderOrigin, NumCameras, CylinderPointsX, CylinderPointsY, CylinderPointsZ, NumPointsPerSlice, MinZI, MosaicWidth, MosaicHeight) * 核心算法2:图像拼接 stitch_images (Regions, ImagesRectified, ImagesGrayRectified, FinalMosaic, WindowHandle, ColorMosaic, HighImageQuality, PerformFineAdjustment, FineAdjustmentMaxShift, FineAdjustmentMatchingWidth, BlendingSeam, CameraSetupModelZeroDistInCylinderOrigin, NumCameras, MosaicWidth, MosaicHeight, CylinderPointsX, CylinderPointsY, CylinderPointsZ, NumSlices, NumPointsPerSlice, CylinderRadius, LabelMinCol, LabelMaxCol) * 结束计时 count_seconds (Seconds2) TimeMS := (Seconds2 - Seconds1) * 1000.0 * 计算耗时(ms) * 显示最终全景图 dev_resize_window_fit_image (FinalMosaic, 0, 0, WindowWidthLimit, WindowHeightLimit) dev_display (FinalMosaic) * 显示全景图 Message := '360度全景图' * 状态消息 if (not DisplayIntermediateResults) Message[1] := '耗时: ' + TimeMS$'.0f' + 'ms' * 添加耗时信息 endif disp_message (WindowHandle, Message, 'window', 12, 12, 'black', 'true') * 显示消息 * 显示继续提示(非最后一组时) if (Obj < |Objects| - 1 or Set < NumSets) disp_continue_message (WindowHandle, 'black', 'true') endif * 更新中间结果显示状态 if (DisplayIntermediateResultsFor == 'first') DisplayIntermediateResults := false * 后续组不显示中间结果 endif stop () * 暂停查看结果 endfor endfor
2.3D旋转轴计算 (determine_rotation_axis_3d)
* ====================== 3D旋转轴计算函数 ====================== * 功能:精确计算瓶体旋转轴心 * 输入: * ImagesRectified - 校正后图像 * ImagesGrayRectified - 校正后灰度图像 * CylinderRadius - 瓶子半径(米) * ... [其他参数] * 输出: * CameraSetupModelZeroDistInCylinderOrigin - 以瓶体为中心的相机模型 * 确定圆柱模型上需要使用的部分(根据标签区域) * 生成采样点索引(每个切片的起始点) Indices := [0,cumul(gen_tuple_const(NumSlices - 1,NumPointsPerSlice))] * 提取采样点3D坐标 SamplePX := CylinderPointsX[Indices] SamplePY := CylinderPointsY[Indices] SamplePZ := CylinderPointsZ[Indices] * 获取相机0参数 get_camera_setup_param (CameraSetupModelZeroDistInCylinderOrigin, 0, 'params', CamParam0) get_camera_setup_param (CameraSetupModelZeroDistInCylinderOrigin, 0, 'pose', CamPose0) * 坐标转换:3D点→相机坐标系→2D投影 pose_to_hom_mat3d (CamPose0, HomMat3D0) * 位姿→变换矩阵 hom_mat3d_invert (HomMat3D0, HomMat3D0Invert) * 求逆变换 affine_trans_point_3d (HomMat3D0Invert, SamplePX, SamplePY, SamplePZ, Qx, Qy, Qz) * 坐标转换 project_3d_point (Qx, Qy, Qz, CamParam0, ImageSampleRow, ImageSampleColumn) * 3D→2D投影 * 计算有效切片范围(基于标签区域) MinSlice := max([find(sgn(ImageSampleColumn - LabelMinCol),1)[0],0]) * 起始切片 MaxSlice := min([find(sgn(ImageSampleColumn - LabelMaxCol),1)[0],NumSlices - 1]) * 结束切片 * 更新全景图高度和3D点云 MosaicHeight := MaxSlice - MinSlice + 1 * 新高度 * 裁剪点云数据 CylinderPointsX := CylinderPointsX[MinSlice * NumPointsPerSlice : MaxSlice * NumPointsPerSlice + NumPointsPerSlice - 1] CylinderPointsY := CylinderPointsY[MinSlice * NumPointsPerSlice : MaxSlice * NumPointsPerSlice + NumPointsPerSlice - 1] CylinderPointsZ := CylinderPointsZ[MinSlice * NumPointsPerSlice : MaxSlice * NumPointsPerSlice + NumPointsPerSlice - 1] * ============== 无精细调整模式 ============== if (not PerformFineAdjustment) * 创建空全景图像 if (ColorMosaic) * 彩色模式:分别创建RGB通道 gen_image_const (ImageMosaicR, 'byte', MosaicWidth, MosaicHeight) gen_image_const (ImageMosaicG, 'byte', MosaicWidth, MosaicHeight) gen_image_const (ImageMosaicB, 'byte', MosaicWidth, MosaicHeight) compose3 (ImageMosaicR, ImageMosaicG, ImageMosaicB, ImageMosaic) * 合并为彩色 else * 灰度模式:创建单通道 gen_image_const (ImageMosaic, 'byte', MosaicWidth, MosaicHeight) endif * 裁剪区域到全景图尺寸 clip_region (Regions, Regions, 0, 0, MosaicHeight - 1, MosaicWidth - 1) * 遍历每个相机填充全景图 for Cam := 0 to NumCameras - 1 by 1 * 获取相机参数 get_camera_setup_param (CameraSetupModelZeroDistInCylinderOrigin, Cam, 'params', CamParam) get_cam_par_data (CamParam, 'image_width', ImageWidth) * 图像宽度 get_cam_par_data (CamParam, 'image_height', ImageHeight) * 图像高度 get_camera_setup_param (CameraSetupModelZeroDistInCylinderOrigin, Cam, 'pose', CamPose) * 坐标转换:世界坐标→相机坐标 pose_to_hom_mat3d (CamPose, HomMat3D) hom_mat3d_invert (HomMat3D, HomMat3DInvert) * 求逆变换 * 获取当前相机负责区域 select_obj (Regions, Region, Cam + 1) get_region_points (Region, Rows, Columns) * 获取区域点坐标 * 线性坐标计算(将2D网格映射到1D索引) LinCoord := Rows * NumPointsPerSlice + Columns * 3D点投影到2D图像 affine_trans_point_3d (HomMat3DInvert, subset(CylinderPointsX,LinCoord), subset(CylinderPointsY,LinCoord), subset(CylinderPointsZ,LinCoord), Qx, Qy, Qz) project_3d_point (Qx, Qy, Qz, CamParam, ImageRow, ImageColumn) * 创建有效点掩膜(在图像范围内) Mask := ImageRow [>=] 0 and ImageRow [<=] ImageHeight - 1 and ImageColumn [>=] 0 and ImageColumn [<=] ImageWidth - 1 ImageRowSub := select_mask(ImageRow,Mask) * 筛选有效行坐标 ImageColumnSub := select_mask(ImageColumn,Mask) * 筛选有效列坐标 * 根据颜色模式处理 if (ColorMosaic) select_obj (ImagesRectified, ObjectSelected, Cam + 1) * 选择当前相机图像 decompose3 (ObjectSelected, ImageR, ImageG, ImageB) * 分解RGB通道 if (HighImageQuality) * 高质量模式:双三次插值 if (BicubicInterpolation) get_grayval_interpolated (ImageR, ImageRowSub, ImageColumnSub, 'bicubic', Grayval) set_grayval (ImageMosaicR, select_mask(Rows,Mask), select_mask(Columns,Mask), Grayval) * 重复处理G/B通道... else * 双线性插值 get_grayval_interpolated (ImageR, ImageRowSub, ImageColumnSub, 'bilinear', Grayval) set_grayval (ImageMosaicR, select_mask(Rows,Mask), select_mask(Columns,Mask), Grayval) * 重复处理G/B通道... endif else * 低质量模式:直接采样 get_grayval (ObjectSelected, ImageRowSub, ImageColumnSub, Grayval1) set_grayval (ImageMosaic, select_mask(Rows,Mask), select_mask(Columns,Mask), Grayval1) endif else * 灰度图像处理 select_obj (ImagesGrayRectified, ImageGray, Cam + 1) if (HighImageQuality) get_grayval_interpolated (ImageGray, ImageRowSub, ImageColumnSub, 'bilinear', Grayval) set_grayval (ImageMosaic, select_mask(Rows,Mask), select_mask(Columns,Mask), Grayval) else get_grayval (ImageGray, ImageRowSub, ImageColumnSub, Grayval1) set_grayval (ImageMosaic, select_mask(Rows,Mask), select_mask(Columns,Mask), Grayval1) endif endif endfor copy_image (ImageMosaic, FinalMosaic) * 复制最终结果 * ============== 精细调整模式 ============== else * 扩展区域以创建重叠(用于配准) DilWidth := max([FineAdjustmentMatchingWidth + 2 * FineAdjustmentMaxShift, BlendingSeam + FineAdjustmentMaxShift]) / 2 * 2 + 1 dilation_rectangle1 (Regions, RegionsDilation, DilWidth, 1) * 矩形膨胀 clip_region (RegionsDilation, RegionsDilation, 0, 0, MosaicHeight - 1, MosaicWidth - 1) * 裁剪到边界 * 创建空对象存储展开图像 gen_empty_obj (UnrolledImages) * 遍历相机生成展开图像 for Cam := 0 to NumCameras - 1 by 1 * 创建空展开图像 if (ColorMosaic) * 彩色模式:分别创建RGB通道 gen_image_const (ImageUnrolledR, 'byte', MosaicWidth, MosaicHeight) gen_image_const (ImageUnrolledG, 'byte', MosaicWidth, MosaicHeight) gen_image_const (ImageUnrolledB, 'byte', MosaicWidth, MosaicHeight) else * 灰度模式:创建单通道 gen_image_const (ImageUnrolled, 'byte', MosaicWidth, MosaicHeight) endif * 获取相机参数(同上) ... * 坐标转换和投影(同上) ... * 可视化:显示即将展开的区域 if (DisplayIntermediateResults) * 生成平滑区域用于显示 for ClosingRad := 1.5 to 15.5 by 1 closing_circle (Region, RegionTrans, ClosingRad) * 闭运算平滑边界 connection (RegionTrans, ConnectedRegions) * 连接区域 count_obj (ConnectedRegions, Number) if (Number == 1) break * 直到形成单一区域 endfor * 准备显示图像 select_obj (ImagesRectified, Image, Cam + 1) copy_image (Image, ImagePart) reduce_domain (ImagePart, RegionTrans, ImageReduced) * 裁剪关注区域 scale_image (ImageReduced, ImageScaled, 0.25, 150) * 缩放增强对比度 copy_image (Image, ImageDisp) overpaint_gray (ImageDisp, ImageScaled) * 叠加显示 rotate_image (ImageDisp, ImageRotate, -90, 'constant') * 旋转便于查看 * 显示结果 dev_resize_window_fit_image (ImageRotate, 0, 0, WindowWidthLimit, WindowHeightLimit) dev_display (ImageRotate) disp_message (WindowHandle, ['相机 ' + (Cam + 1) + ':', '即将展开的图像区域'], 'window', 12, 12, 'black', 'true') stop () endif * 填充展开图像(类似无调整模式) ... * 可视化:显示展开结果 if (DisplayIntermediateResults) * 创建掩膜并填充背景 gen_region_points (RegionMaskOut, select_mask(Rows,Mask), select_mask(Columns,Mask)) get_domain (UnrolledOut, DomainUnrolledOut) difference (DomainUnrolledOut, RegionMaskOut, RegionDifference) * 设置背景色(彩色/灰度) if (ColorMosaic) FillColor := [255, 255, 255] else FillColor := 255 overpaint_region (UnrolledOut, RegionDifference, FillColor, 'fill') * 填充背景 * 显示结果 dev_resize_window_fit_image (UnrolledOut, 0, 0, WindowWidthLimit, WindowHeightLimit) dev_display (UnrolledOut) disp_message (WindowHandle, ['相机 ' + (Cam + 1) + ':', '展开后的图像'], 'window', 12, 12, 'black', 'true') stop () endif endfor * ============== 图像配准与融合 ============== * 确定图像重叠顺序(从左到右) smallest_rectangle1 (RegionsDilation, RowBB1, ColumnBB1, RowBB2, ColumnBB2) * 获取区域边界 From := sort_index(ColumnBB1) * 按左边界排序 To := [From[1:|From| - 1],From[0]] * 创建配对关系(当前→下一张) * 初始化配准变量 gen_empty_obj (UnrolledImagesShifted) * 存储配准后图像 ShiftAccumulated := 0 * 累计偏移量 hom_mat2d_identity (HomMat2DAccumulated) * 初始化单位变换矩阵 * 配准处理循环 for FT := 0 to |From| - 1 by 1 F := From[FT] * 当前图像索引 T := To[FT] * 目标图像索引 * 特殊处理最后一张图像(仅保留右侧) if (FT == |From| - 1) connection (RegionTo, ConnectedRegions) if (count_obj(ConnectedRegions) > 1) sort_region (ConnectedRegions, SortedRegions, 'character', 'true', 'column') select_obj (SortedRegions, RegionTo, 2) * 选择右侧区域 * 将非负责区域涂黑 get_domain (ImageTo, Domain) difference (Domain, RegionTo, RegionBlack) overpaint_region (ImageTo, RegionBlack, [0, 0, 0], 'fill') endif endif * 图像预处理(转换到实数类型) if (Channels == 3) rgb1_to_gray (ImageFrom, GrayFrom) * 彩色转灰度 convert_image_type (GrayFrom, ImageFrom, 'real') * 转实数 * 同样处理目标图像... else convert_image_type (ImageFrom, ImageFrom, 'real') * 同样处理目标图像... endif * 创建NCC配准模板 intersection (RegionFrom, RegionTo, RegionIntersection) * 获取重叠区域 smallest_rectangle1 (RegionIntersection, RowInters1, ColumnInters1, RowInters2, ColumnInters2) ColumnCut := (ColumnInters1 + ColumnInters2) / 2 * 中心列 * 定义旋转角度范围 Angles := [-3, 3] * ±3度 PaddingTopBottom := 20 * 上下边距 * 创建模板区域 gen_rectangle1 (Rectangle, PaddingTopBottom, ColumnCut - FineAdjustmentMatchingWidth / 2, MosaicHeight - 1 - PaddingTopBottom, ColumnCut + FineAdjustmentMatchingWidth / 2) convert_image_type (ImageFrom, ImageFromB, 'byte') * 转字节类型 reduce_domain (ImageFromB, Rectangle, ImageModel) * 裁剪模板区域 * 创建NCC模型 create_ncc_model (ImageModel, 1, rad(Angles[0]), rad(Angles[1] - Angles[0]), 'auto', 'use_polarity', ModelID) region_features (ImageModel, ['row', 'column', 'width'], Values) * 获取模板位置 * 创建搜索区域 gen_rectangle2 (RoiSearch, Values[0], Values[1], 0, FineAdjustmentMaxShift, 1) * 矩形区域 convert_image_type (ImageTo, ImageToB, 'byte') reduce_domain (ImageToB, RoiSearch, ImageSearch) * 裁剪搜索区域 * 执行NCC配准 find_ncc_model (ImageSearch, ModelID, rad(Angles[0]), rad(Angles[1] - Angles[0]), 0.3, 1, 1.0, 'true', 0, Row, Column, Angle, Score) if (|Score| == 0) * 匹配失败处理 Row := Values[0] Column := Values[1] Angle := 0 Score := 0 endif * 计算累计偏移 ShiftAccumulated := ShiftAccumulated + (Values[1] - Column) * 计算变换矩阵 vector_angle_to_rigid (Row, Column, Angle, Values[0], Values[1], 0, HomMat2DI) * 完整变换 * 分解变换(平移+剩余) vector_angle_to_rigid (0, Column, 0, 0, Values[1], 0, HomMat2DITrans) * 纯平移 hom_mat2d_invert (HomMat2DITrans, HomMat2DITransInvert) * 平移逆变换 hom_mat2d_compose (HomMat2DI, HomMat2DITransInvert, HomMat2DRemainder) * 剩余变换 * 应用剩余变换(局部形变) * 计算变换前后的列差异 DCol := ColOrig - ColTrans * 创建渐变向量场 Ramp := [gen_tuple_const(MorphingExtLeft,2.0), inverse(cumul(gen_tuple_const(MorphingWidth2 - 1,2.0 / MorphingWidth2))),0] * 应用矢量场变形 real_to_vector_field (ImageWR, ImageWC, VectorField, 'vector_field_relative') * 执行图像变形 if (Channels == 3) decompose3 (ImagePart, Image1, Image2, Image3) * 分解RGB unwarp_image_vector_field (Image1, VectorField, ImageUnwarped1) * 变形通道1 * 同样处理其他通道... compose3 (...) * 重新组合 else unwarp_image_vector_field (ImagePart, VectorField, ImageUnwarped) * 灰度图像变形 endif * 应用平移变换 hom_mat2d_compose (HomMat2DAccumulated, HomMat2DITrans, HomMat2DAccumulated) * 更新累计变换 affine_trans_image (MorphedImage, ImageToShift, HomMat2DAccumulated, 'constant', 'false') * 应用变换 * 处理最后一张图像的特殊情况 if (FT == |From| - 1) get_image_size (ImageToShift, MosaicWidthExt, MosaicHeightExt) * 获取最终尺寸 endif * 存储配准后图像 reduce_domain (ImageToShift, RegionToShift, ImageToShiftReduced) concat_obj (UnrolledImagesShifted, ImageToShiftReduced, UnrolledImagesShifted) endfor * ============== 亮度融合 ============== * 确定融合中心点 for I := 1 to NumberUnrolledImages - 1 by 1 * 计算相邻图像重叠区域中心 smallest_rectangle1 (RegionIntersection, RowI1, ColumnI1, RowI2, ColumnI2) ColumnBlendingCenter[I - 1] := int(round(0.5 * (ColumnI1 + ColumnI2))) endfor * 创建融合蒙版 gen_empty_obj (BlendingMasks) * 计算融合参数 BSEven := BlendingSeam / 2 * 2 * 确保偶数 if (BSEven > BlendingSeam) BSEven := BSEven + 2 BS2 := BSEven / 2 * 半宽 if (BSEven >= 2) BSStep := 1.0 / BSEven * 渐变步长 BSLeft := cumul(gen_tuple_const(BSEven,BSStep)) * 左渐变 BSRight := 1.0 + BSStep - BSLeft * 右渐变 endif * 生成融合蒙版 * 首张图像蒙版 MaskOneLine := [gen_tuple_const(ColumnBlendingCenter[0] - BS2,1),BSRight] gen_image_const (ImageM1, 'real', MosaicWidthExt, 1) set_grayval (ImageM1, ..., MaskOneLine) * 设置渐变值 zoom_image_size (ImageM1, ImageM, MosaicWidthExt, MosaicHeight, 'nearest_neighbor') * 缩放 concat_obj (BlendingMasks, ImageM, BlendingMasks) * 中间图像蒙版(类似处理) ... * 最后图像蒙版 ... * 应用融合 for I := 1 to NumberUnrolledImages by 1 * 图像对齐 if (I < NumberUnrolledImages) tile_images_offset (...) * 普通对齐 else tile_images_offset (...) * 末张特殊对齐 endif * 应用蒙版 if (Channels == 3) compose3 (Mas, Mas, Mas, MultiChannelMask) * 彩色蒙版 mult_image (ImgReal, MultiChannelMask, ImageForStitching, 1, 0) * 相乘 else mult_image (ImgReal, Mas, ImageForStitching, 1, 0) * 灰度蒙版 endif * 累加到全景图 add_image (ImageMosaic, ImageForStitching, ImageMosaic, 1, 0) endfor * 裁剪最终全景图 tile_images_offset (ImageMosaic, FinalMosaic, 0, 0, 0, 0, MosaicHeight - 1, RightBorder, RightBorder + 1, MosaicHeight) endif return ()
3. 图像拼接 (stitch_images)
* ====================== 图像拼接函数 ====================== * 功能:精确计算瓶体旋转轴心 * 输入:相机位置、轮廓点云等 * 输出:瓶体姿态(PoseCylinder) * 收集相机位置 for Cam := 0 to NumCameras - 1 by 1 get_camera_setup_param (CameraSetupModelZeroDist, Cam, 'pose', CamPose) XCam[Cam] := CamPose[0] * X坐标 YCam[Cam] := CamPose[1] * Y坐标 ZCam[Cam] := CamPose[2] * Z坐标 endfor * 计算瓶体初始位置(相机位置均值) CylinderPointApprox := [mean(XCam),mean(YCam),mean(ZCam)] LenI := 1.0 / sqrt(sum(CylinderPointApprox * CylinderPointApprox)) * 归一化因子 CylinderXTmp := CylinderPointApprox * LenI * X方向 * 拟合平面(估算瓶体方向) fit_plane (XCam, YCam, ZCam, NX, NY, NZ, C) * 拟合相机位置平面 CylinderZTmp := [NX,NY,NZ] * 法线方向为Z轴 * 调整坐标系方向 get_camera_setup_param (CameraSetupModelZeroDist, 0, 'pose', CamPose) pose_to_hom_mat3d (CamPose, HomMat3DCamPose0) DirTest := sum(CylinderZTmp * HomMat3DCamPose0[[0, 4, 7]]) * 方向测试 if (DirTest > 0) * 确保方向一致 CylinderZTmp := -CylinderZTmp endif * 构建坐标系 cross_product (CylinderZTmp, CylinderXTmp, CylinderYTmp) * 叉积计算Y轴 * 创建齐次变换矩阵 HomMat3DCylinderApprox := [CylinderXTmp[0],CylinderYTmp[0],CylinderZTmp[0],CylinderPointApprox[0], CylinderXTmp[1],CylinderYTmp[1],CylinderZTmp[1],CylinderPointApprox[1], CylinderXTmp[2],CylinderYTmp[2],CylinderZTmp[2],CylinderPointApprox[2]] hom_mat3d_to_pose (HomMat3DCylinderApprox, PoseCylinderApprox) * 矩阵→位姿 * 创建以瓶体为中心的相机模型 serialize_camera_setup_model (CameraSetupModelZeroDist, SerializedItemHandle) * 序列化 deserialize_camera_setup_model (SerializedItemHandle, CameraSetupModelZeroDistCylApprox) * 反序列化 set_camera_setup_param (CameraSetupModelZeroDistCylApprox, 'general', 'coord_transf_pose', PoseCylinderApprox) * 设置原点 * ============== 轮廓点采集 ============== * 初始化存储变量 A2x := []; A2y := []; A2z := [] * 视线起点 B2x := []; B2y := []; B2z := [] * 视线方向 SilhRow := []; SilhCol := []; SilhCam := [] * 图像坐标和相机索引 * 遍历每个相机采集轮廓点 for Cam := 0 to NumCameras - 1 by 1 * 获取当前相机图像和参数 select_obj (ImagesRectified, ImageRectified, Cam + 1) select_obj (ImagesGrayRectified, GrayImage, Cam + 1) get_camera_setup_param (CameraSetupModelZeroDistCylApprox, Cam, 'params', CamParam) get_camera_setup_param (CameraSetupModelZeroDistCylApprox, Cam, 'pose', CamPose) * 创建模糊函数(用于边缘对提取) create_funct_1d_pairs ([MinPairDist[Cam] - 10,MinPairDist[Cam],MaxPairDist[Cam] / cos(SilhouetteMaxTilt),MaxPairDist[Cam] / cos(SilhouetteMaxTilt) + 10], [0, 1, 1, 0], FuzzyFunction) * 遍历测量句柄提取边缘 Row := []; Column := [] * 存储边缘点 for I := 0 to |MeasureHandles| - 1 by 1 set_fuzzy_measure (MeasureHandles[I], 'size', FuzzyFunction) * 设置模糊函数 * 提取边缘对 fuzzy_measure_pairing (GrayImage, MeasureHandles[I], SilhouetteMeasureSigma, SilhouetteMeasureThreshold, 0.5, 'all', 'no_restriction', 0, RowEdgeFirst, ColumnEdgeFirst, AmplitudeFirst, RowEdgeSecond, ColumnEdgeSecond, AmplitudeSecond, RowPairCenter, ColumnPairCenter, FuzzyScore, IntraDistance) * 根据背景特性选择点 if (BackgroundMayContainTexture) * 纹理背景:使用所有边缘点 Row := [Row,RowEdgeFirst,RowEdgeSecond] Column := [Column,ColumnEdgeFirst,ColumnEdgeSecond] else * 纯色背景:仅使用首尾边缘点 if (|RowEdgeFirst| > 0) SIF := sort_index(-RowEdgeFirst)[0] * 最高点 SIS := sort_index(RowEdgeSecond)[0] * 最低点 Row := [Row,RowEdgeFirst[SIF],RowEdgeSecond[SIS]] Column := [Column,ColumnEdgeFirst[SIF],ColumnEdgeSecond[SIS]] endif endif endfor * 可视化:显示轮廓点 if (DisplayIntermediateResults) * 准备显示窗口 get_image_size (GrayImage, ImgWidth, ImgHeight) dev_resize_window_fit_size (0, 0, ImgWidth, ImgHeight + 300, WindowWidthLimit, WindowHeightLimit) dev_set_part (-300, 0, ImgHeight, ImgWidth) * 扩展显示区域 * 创建十字标记 gen_cross_contour_xld (Cross, Row, Column, 6, 0.785398) dev_clear_window () dev_display (GrayImage) dev_display (Cross) * 显示轮廓点 * 显示说明信息 Message := '用于瓶体位姿估计的轮廓点(必须位于瓶体轮廓上)' MessageWrapped := regexp_replace(Message + ' ',['(.{0,45})\\s', 'replace_all'],'$1\n') disp_message (WindowHandle, ['相机 ' + (Cam + 1) + ':',MessageWrapped], 'window', 12, 12, 'black', 'true') stop () * 暂停查看 endif * 存储点信息 SilhRow := [SilhRow,Row] * 行坐标 SilhCol := [SilhCol,Column] * 列坐标 SilhCam := [SilhCam,gen_tuple_const(|Row|,Cam)] * 相机索引 * 视线计算:2D点→3D射线 get_line_of_sight (Row, Column, CamParam, PX, PY, PZ, QX, QY, QZ) * 获取视线向量 pose_to_hom_mat3d (CamPose, HomMat3D) * 位姿→变换矩阵 * 提取相机位置 PX0 := HomMat3D[3] PY0 := HomMat3D[7] PZ0 := HomMat3D[11] * 转换视线方向到世界坐标系 affine_trans_point_3d (HomMat3D, QX, QY, QZ, QX0, QY0, QZ0) * 存储视线数据 A2x := [A2x,gen_tuple_const(|Row|,PX0)] * 起点X A2y := [A2y,gen_tuple_const(|Row|,PY0)] * 起点Y A2z := [A2z,gen_tuple_const(|Row|,PZ0)] * 起点Z B2x := [B2x,QX0 - PX0] * 方向向量X B2y := [B2y,QY0 - PY0] * 方向向量Y B2z := [B2z,QZ0 - PZ0] * 方向向量Z endfor * ============== 迭代优化旋转轴 ============== * 初始化轴心参数 A1x := 0.0; A1y := 0.0; A1z := 0 * 轴心起点 B1x := 0.0; B1y := 0.0; B1z := 1 * 轴心方向 * 优化控制参数 Dx := 99999 * 初始误差 ErrorLog := [] * 误差记录 Iter := 0; IterInter := 0 * 迭代计数器 MaxIter := 100; MaxIterInter := 5 * 最大迭代次数 LastError := 99999 * 上次误差 SDevFactor := 5.0 * 异常点剔除因子 * 主优化循环 while (Iter < MaxIter) * 计算当前轴心与所有轮廓视线的距离 distance_skew_lines (A1x, A1y, A1z, B1x, B1y, B1z, A2x, A2y, A2z, B2x, B2y, B2z, Dist) * 计算与瓶体半径的误差 E := CylinderRadius - Dist ErrorLog := [ErrorLog,mean(E)] * 记录平均误差 * 数值微分计算梯度 Delta := 0.001 * 微分步长 * 对A1x的偏导 distance_skew_lines (A1x + Delta, A1y, A1z, B1x, B1y, B1z, A2x, A2y, A2z, B2x, B2y, B2z, DistTmp) DistDAx := (DistTmp - Dist) / Delta * 对A1y的偏导(类似) ... * 对B1x的偏导(类似) ... * 对B1y的偏导(类似) ... * 构建雅可比矩阵 create_matrix (|A2x|, 4, 0, A) * 创建空矩阵 SeqR := [0:|A2x| - 1] * 行索引 SeqC := gen_tuple_const(|A2x|,0) * 列索引 * 填充偏导数值 set_value_matrix (A, SeqR, SeqC, DistDAx) * 第0列:dDist/dA1x set_value_matrix (A, SeqR, SeqC + 1, DistDAy) * 第1列:dDist/dA1y set_value_matrix (A, SeqR, SeqC + 2, DistDBx) * 第2列:dDist/dB1x set_value_matrix (A, SeqR, SeqC + 3, DistDBy) * 第3列:dDist/dB1y * 构建误差向量 create_matrix (|A2x|, 1, 0, y) set_value_matrix (y, SeqR, SeqC, CylinderRadius - Dist) * 误差值 * 求解最小二乘问题:(A^T A) x = A^T y solve_matrix (A, 'general', 0, y, X) get_full_matrix (X, Values) * 获取解向量 * 更新参数 A1x := A1x + Values[0] * 更新A1x A1y := A1y + Values[1] * 更新A1y B1x := B1x + Values[2] * 更新B1x B1y := B1y + Values[3] * 更新B1y * 更新迭代计数 Iter := Iter + 1 IterInter := IterInter + 1 * 异常点剔除(每5次迭代或收敛时) if (|ErrorLog| > 3) if (deviation(ErrorLog[|ErrorLog| - 3:|ErrorLog| - 1]) < 1e-5 or IterInter >= MaxIterInter) IterInter := 0 * 重置计数器 * 检查收敛条件 if (fabs(ErrorLog[|ErrorLog| - 1]) < MaxError and fabs(LastError - ErrorLog[|ErrorLog| - 1]) < 1e-5 and SDevFactor <= 3.0) MaxIter := -1 * 标记收敛 continue endif LastError := ErrorLog[|ErrorLog| - 1] * 记录当前误差 * 计算误差标准差 SDevE := deviation(fabs(E)) SDevFactor := max([3.0,SDevFactor - 0.5]) * 更新剔除因子 MaskUse := fabs(E) [<] SDevFactor * SDevE * 创建有效点掩膜 * 确保有足够点被剔除 while (min(MaskUse) == 1 and SDevFactor > 3.0) SDevFactor := max([3.0,SDevFactor - 0.1]) * 增大剔除范围 MaskUse := fabs(E) [<] SDevFactor * SDevE endwhile * 检查各相机剩余点数 SilhCamTmp := select_mask(SilhCam,MaskUse) for CamI := 0 to NumCameras - 1 by 1 NumPointsRemaining[CamI] := sum(find(SilhCamTmp,CamI) [!=] -1) endfor * 确保至少三个相机有30%以上的点 if (sum(NumPointsRemaining / real(NumPoints) [>=] 0.3) <= 3) MaxIter := -1 * 点分布不理想,终止 continue endif * 更新数据(剔除异常点) A2x := select_mask(A2x,MaskUse) A2y := select_mask(A2y,MaskUse) ... [其他坐标类似] ... * 更新可视化数据 SilhRowElim := [SilhRowElim,select_mask(SilhRow,1 - MaskUse)] * 被剔除点 ... [类似更新其他] ... SilhRow := select_mask(SilhRow,MaskUse) * 更新有效点 ... [类似更新其他] ... endif endif endwhile * ============== 最终位姿计算 ============== * 计算坐标系 get_camera_setup_param (CameraSetupModelZeroDistCylApprox, 0, 'pose', CamPose0) * 计算瓶体轴心到相机0的垂足 point_to_line_perpendicular_foot (CamPose0[0:2], [A1x,A1y,A1z], [B1x,B1y,B1z], Foot) * 计算X轴(垂足到相机0的向量) XAxis := Foot - CamPose0[0:2] XAxis := XAxis / sqrt(sum(XAxis * XAxis)) * 归一化 * 计算Z轴(瓶体方向) ZAxis := [B1x,B1y,B1z] ZAxis := ZAxis / sqrt(sum(ZAxis * ZAxis)) * 归一化 * 计算Y轴(叉积) cross_product (ZAxis, XAxis, YAxis) * 构建变换矩阵 HomMat3DCylinderEst := [XAxis[0],YAxis[0],ZAxis[0],A1x, XAxis[1],YAxis[1],ZAxis[1],A1y, XAxis[2],YAxis[2],ZAxis[2],A1z] * 转换为位姿 hom_mat3d_to_pose (HomMat3DCylinderEst, PoseCylinderEst) * 创建以瓶体为中心的相机模型 pose_compose (PoseCylinderEst, [0, 0, 0, 0, 0, 0, 0], PoseCylinder) * 组合位姿 serialize_camera_setup_model (CameraSetupModelZeroDistCylApprox, SerializedItemHandle) deserialize_camera_setup_model (SerializedItemHandle, CameraSetupModelZeroDistInCylinderOrigin) set_camera_setup_param (CameraSetupModelZeroDistInCylinderOrigin, 'general', 'coord_transf_pose', PoseCylinder) * 设置原点 * ============== 可视化验证 ============== if (DisplayIntermediateResults) * 创建圆柱3D模型 gen_cylinder_object_model_3d ([0, 0, 0, 0, 0, 0, 0], CylinderRadius, -0.1, 0.1, ObjectModel3DCylinder) * 为每个相机创建显示窗口 for Cam := 0 to NumCameras - 1 by 1 * 准备窗口 dev_open_window_fit_image (GrayImage, Cam / 2 * (300 + 63), Cam % 2 * (400 + 12), 400, 300, WindowHandleI) set_display_font (WindowHandleI, 16, 'mono', 'true', 'false') WH[Cam] := WindowHandleI * 存储窗口句柄 * 投影圆柱到图像 get_camera_setup_param (CameraSetupModelZeroDistInCylinderOrigin, Cam, 'params', CamParam) get_camera_setup_param (CameraSetupModelZeroDistInCylinderOrigin, Cam, 'pose', CamPose) pose_invert (CamPose, PoseInvert) * 求逆位姿 render_object_model_3d (ImageCyl, ObjectModel3DCylinder, CamParam, PoseInvert, [], []) * 渲染 * 提取轮廓 threshold (ImageCyl, Region, 0, 0) * 阈值处理 gen_contour_region_xld (Region, Contours, 'border') * 生成轮廓 * 显示结果 dev_display (Image) * 显示原图 dev_set_color ('blue') * 轮廓颜色 dev_display (Contours) * 显示投影轮廓 * 标记接受/拒绝的点 UsedId := find(SilhCam,Cam) * 接受点索引 if (UsedId != -1) dev_set_color ('green') * 接受点=绿色 gen_cross_contour_xld (CrossUsed, subset(SilhRow,UsedId), subset(SilhCol,UsedId), 6, 0.785398) dev_display (CrossUsed) endif RejectedId := find(SilhCamElim,Cam) * 拒绝点索引 if (RejectedId != -1) dev_set_color ('red') * 拒绝点=红色 gen_cross_contour_xld (CrossUsed, subset(SilhRowElim,RejectedId), subset(SilhColElim,RejectedId), 6, 0.785398) dev_display (CrossUsed) endif * 显示标题 disp_message (WindowHandleI, '相机 ' + (Cam + 1) + ':', 'window', 12, 12, 'black', 'true') endfor * 创建图例窗口 dev_open_window (0, 2 * (400 + 12), 400, 300, 'black', WindowHandleI) set_display_font (WindowHandleI, 16, 'mono', 'true', 'false') * 显示图例说明 disp_message (WindowHandleI, [ '绿色: 接受点(位于瓶体轮廓)', '红色: 拒绝点', '蓝色: 估计轮廓'], 'window', 12, 12, ['green', 'red', 'blue'], 'false') stop () * 暂停查看 * 关闭临时窗口 for Cam := 0 to NumCameras - 1 by 1 dev_set_window (WH[Cam]) dev_close_window () endfor dev_set_window (WindowHandleI) dev_close_window () endif return ()

浙公网安备 33010602011771号