实用指南：基于Halcon深度学习之分类

********环境准备******系统：win7以上系统***显卡：算力3.0以上***显卡驱动：10.1以上版本（nvidia-smi查看指令） ***读取深度学习模型***read_dl_model ('pretrained_dl_classifier_compact.hdl', DLModelHandle)***获取模型中的图象张量及对应的灰度值范围get_dl_model_param (DLModelHandle, 'image_width', ImgWidth)get_dl_model_param (DLModelHandle, 'image_height', ImgHeight)get_dl_model_param (DLModelHandle, 'image_num_channels', ImgChannel)get_dl_model_param (DLModelHandle, 'image_range_min', ImgRangMin)get_dl_model_param (DLModelHandle, 'image_range_max', ImgRangMax)  ************预处理数据准备及拆分**********classNames := ['1-2','1-3','1-4','2-4','3-5','4-7','5-6']rawImageFolders := '../images/' + classNames***把原图像数据转换为对应的Dataset格式***read_dl_dataset_classification (rawImageFolders, 'last_folder', DLDataset)***对数据集进行拆分***trainPrecent := 60 // 用于训练的比例validdatatPrecent := 20 // 用于测试的比例GenSplitParam := dict{overwrite_split:'true'}split_dl_dataset (DLDataset, trainPrecent, validdatatPrecent, GenSplitParam)***预处理数据****1、创建预处理参数** 预处理数据存储路径processFolder := 'ProcessData'** 定义预处理参数文件路径dlProcessFileName := processFolder + '/dl_preprocess_param.hdict'** 检查文件是否存在file_exists (processFolder, FileExists)if(not FileExists)    make_dir (processFolder)endif** 创建预处理参数create_dl_preprocess_param ('classification', ImgWidth, ImgHeight, ImgChannel, \                            ImgRangMin, ImgRangMax, 'none', 'full_domain',\                            [], [], [], [], DLPreprocessParam)** 对数据进行预处理GenParam := dict{overwrite_files:'true'}preprocess_dl_dataset (DLDataset, processFolder, DLPreprocessParam, \                       GenParam, DLDatasetFileName)** 保存预处理参数文件write_dict (DLPreprocessParam, dlProcessFileName, [], []) ************使用预处理数据对模型进行训练************** 总周期（所有图片训练一次为一个周期）numEpochs := 100** 批次数（一个周期本分割的份数）batchSize := 5** 评估周期(每训练几个周期进行一次评估)evluationIntervalEphochs := 2** 学习率learningRate := 0.001**** 设置模型参数set_dl_model_param(DLModelHandle, 'class_names', classNames)set_dl_model_param(DLModelHandle, 'image_dimensions', [ImgWidth,ImgHeight,ImgChannel])set_dl_model_param(DLModelHandle, 'learning_rate', learningRate)set_dl_model_param (DLModelHandle, 'batch_size', batchSize)** 设置增强参数GenParamName := []GenParamValue := []** 定义参数字典,增强参数augmentParam := dict{}augmentParam.augmentation_percentage := 100augmentParam.mirror := 'rc'GenParamName := [GenParamName,'augment']GenParamValue := [GenParamValue,augmentParam] ** 定义存储最佳模型路径和最终模型路径augmentBest := dict{}augmentBest.type := 'best'augmentBest.basename := processFolder + '/model_best'GenParamName := [GenParamName,'serialize']GenParamValue := [GenParamValue,augmentBest] augmentFinal := dict{}augmentFinal.type := 'final'augmentFinal.basename := processFolder + '/model_final'GenParamName := [GenParamName,'serialize']GenParamValue := [GenParamValue,augmentFinal]** 创建训练参数及执行训练create_dl_train_param (DLModelHandle, numEpochs, evluationIntervalEphochs, \                       'true', 32, GenParamName, GenParamValue, TrainParam)** 执行模型训练train_dl_model (DLDataset, DLModelHandle, TrainParam, 0,\                TrainResults, TrainInfos, EvaluationInfos)** 释放资源clear_dl_model (DLModelHandle)** 关闭训练窗口stop()dev_close_window()dev_close_window() ***********模型评估************* 定义评估模型文件路径retainModelFile := processFolder + '/model_best.hdl'** 读取训练后的模型read_dl_model (retainModelFile, RetainDLModelHandle)** 定义评估的指标evaluateGenParam := ['f_score','recall','precision','absolute_confusion_matrix',\                     'relative_confusion_matrix']evaluateDict := dict{measures:evaluateGenParam}** 执行模型评估evaluate_dl_model(DLDataset, RetainDLModelHandle, 'split', 'test', \                  evaluateDict, EvaluationResult, EvalParams)** 显示评估结果GenDispParam := dict{}GenDispParam.display_mode := ['measures','pie_charts_recall','pie_charts_precision',\                              'absolute_confusion_matrix','relative_confusion_matrix']WindowHandleDict := dict{}dev_display_classification_evaluation(EvaluationResult, EvalParams,\                                      GenDispParam, WindowHandleDict)stop()dev_close_window_dict(WindowHandleDict)  ***********模型推断************* 读取推断数据list_image_files('../test_image','bmp',[],ImageFiles)** 读取预处理参数read_dict(dlProcessFileName,[],[],DLPreprocessParam)** 设置显示字体大小dev_get_window(WindowHandle)set_display_font(WindowHandle, 26, 'mono', 'true', 'false')** 循环检测每一张图像for Index :=0 to |ImageFiles| -1 by 1    ** 获取当前图像    read_image(Image,ImageFiles[Index])    ** 生产样本图像    gen_dl_samples_from_images(Image, DLSampleBatch)    ** 把推断图像处理成预处理图像数据    preprocess_dl_samples(DLSampleBatch, DLPreprocessParam)    **  使用模型进行推断类别    apply_dl_model(RetainDLModelHandle, DLSampleBatch, [], DLResultBatch)    ** 评估结果显示    className := DLResultBatch.classification_class_names[0]    score := DLResultBatch.classification_confidences[0]    ** 显示图像上面    dev_disp_text('类别：'+className+',分数：'+score, 'window',\                  'top', 'left', 'white', 'box_color', 'forest green')    stop()endfor clear_dl_model(RetainDLModelHandle)

以上为halcon源码，联合C#编程操作界面

C#源码

///         /// 分类类别名称        ///         private HTuple classNames;         ///         /// 模型文件        ///         public string[] ModelFiles { get; private set; }         #region 主界面相关事件        public HomePage()        {            InitializeComponent();        }         private void HomePage_FormClosing(object sender, FormClosingEventArgs e)        {        }        private void HomePage_Load(object sender, System.EventArgs e)        {                       // UI 显示日志            ViewAppender viewAppender = new ViewAppender(dgvLog)            {                Location = "",            };            LogCombiner.Ins.Append(viewAppender);            // 初始化网络模型            InitializeModels();             // 模型评估默认参数            // recall            cbMetric.SelectedIndex = 0;            // test            cbSampleSelectValues.SelectedIndex = 0;        }        #endregion         #region 设置HWindowControl控件的显示方式        private void DispImage(HImage image, HWindow window)        {            int imageWidth, imageHeight, winRow, winCol, winWidth, winHeight, partWidth, partHeight;            try            {                image.GetImageSize(out imageWidth, out imageHeight);                window.GetWindowExtents(out winRow, out winCol, out winWidth, out winHeight);                if (winWidth > winHeight)                {                    partWidth = imageWidth;                    partHeight = imageWidth * winHeight / winWidth;                }                else                {                    partWidth = imageHeight * winWidth / winHeight;                    partHeight = imageHeight;                }                 // 设置显示图像在窗口中间                var offsetX = (imageWidth - partWidth) / 2;                var offsetY = (imageHeight - partHeight) / 2;                // 设置窗口显示区域                HOperatorSet.SetPart(window, offsetY, offsetX, partHeight + offsetY - 1, partWidth + offsetX - 1);                // 显示图形                HOperatorSet.DispObj(image, window);            }            catch (HalconException hEx)            {                MessageBox.Show(hEx.Message);            }        }         ///         /// 把HObject转化为对应的HImage方法        ///         ///         ///         private HImage HObject2HImage(HObject hObject)        {            HImage image = new HImage();            // 获取当前传人对象的通道数【灰度/彩色】            HOperatorSet.CountChannels(hObject, out var channels);            if (channels.I == 0)            {                return image;            }            // 判断通道数            if (channels.I == 1)            {                // 获取传人图形的数据                HOperatorSet.GetImagePointer1(hObject,                    out var pointer, out var type, out var width, out var height);                // 生成当通道图形                image.GenImage1(type, width, height, pointer);            }            else            {                // 彩色图                HOperatorSet.GetImagePointer3(hObject, out var pointerRed, out var pointerGreen,                    out var pointerBlue, out var type, out var width, out var height);                image.GenImage3(type, width, height, pointerRed, pointerGreen, pointerBlue);            }             return image;        }        #endregion         #region 加载模型文件选择下拉框        private void InitializeModels()        {            // 获取项目编译路径            ModelFiles = Directory.GetFiles(Path.Combine(Directory.GetCurrentDirectory(), "hdl"),"*.hdl");            // 把模型文件添加至下拉框中            foreach(string modelFile in ModelFiles)            {                // 获取模型文件名                string modelName = Path.GetFileNameWithoutExtension(modelFile);                // 添加到下拉列表                cbBackBone.Items.Add(modelName);            }            // 判断是否有模型文件            if(ModelFiles.Length > 0)            {                // 默认选择第一个文件                cbBackBone.SelectedIndex = 0;                Logger.Notify("model load finished .", "Preprocess");            }            else            {                // 如果没有模型文件,下拉框禁用                cbBackBone.Enabled = false;                Logger.Warn("model files not found .", "Preprocess");            }        }        #endregion         #region 读取模型参数        private void cbBackBone_SelectedIndexChanged(object sender, EventArgs e)        {            try            {                // 读取选择的网络模型文件                string modelFile = ModelFiles[cbBackBone.SelectedIndex];                // 加载模型 read_dl_model                HOperatorSet.ReadDlModel(modelFile, out var dLModelHandle);                // 判断模型不为空                if(dLModelHandle == null)                {                    Logger.Warn("read_dl_model failure .", "Preprocess");                    return;                }                // 读取模型张量数                HOperatorSet.GetDlModelParam(dLModelHandle, "image_width", out HTuple imageWidth);                HOperatorSet.GetDlModelParam(dLModelHandle, "image_height", out HTuple imageHeight);                HOperatorSet.GetDlModelParam(dLModelHandle, "image_num_channels", out HTuple imageChannels);                HOperatorSet.GetDlModelParam(dLModelHandle, "image_range_min", out HTuple imageRangMin);                HOperatorSet.GetDlModelParam(dLModelHandle, "image_range_max", out HTuple imageRangMax);                // 数据填充至对应文本框                tbImgWidth.Text = imageWidth.ToString();                tbImgHeight.Text = imageHeight.ToString();                tbChannel.Text = imageChannels.ToString();                tbGrayMin.Text = imageRangMin.ToString();                tbGrayMax.Text = imageRangMax.ToString();                 // 释放加载的模型                HOperatorSet.ClearDlModel(dLModelHandle);                dLModelHandle.Dispose();                dLModelHandle = null;            }            catch (Exception ex)            {                Logger.Error(ex.Message, "Preprocess");            }        }        #endregion         #region 选择样本路径和存储路径        private void btnSelectRawPath_Click(object sender, EventArgs e)        {            using (FolderBrowserDialog dlg = new FolderBrowserDialog())            {                if (dlg.ShowDialog() == DialogResult.OK)                {                    // 重新创建，防止重复添加                    classNames = new HTuple();                    tbRawPath.Text = dlg.SelectedPath;                    // 获取选择文件夹下面的所有文件夹名称                    string[] folder = Directory.GetDirectories(tbRawPath.Text);                    // 把文件夹名添加到元组中                    foreach (var item in folder)                    {                        classNames.Append(new DirectoryInfo(item).Name);                    }                }            }        }         private void btnSelectPrePath_Click(object sender, EventArgs e)        {            using (FolderBrowserDialog dlg = new FolderBrowserDialog())            {                if (dlg.ShowDialog() == DialogResult.OK)                {                    tbPreStorPath.Text = dlg.SelectedPath;                }            }        }        #endregion         #region 执行数据预处理操作        private async void btnExecPreprocess_Click(object sender, EventArgs e)        {            // 获取预处理对象            var preprocess = Classification.ProcessAction;            // 绑定更新方法            preprocess.UpdateImageAction += Preprocess_UpdateImageAction;             // 获取界面的参数            var imageWith = Convert.ToInt32(tbImgWidth.Text);            var imageHeight = Convert.ToInt32(tbImgHeight.Text);            var imageNumChannels = Convert.ToInt32(tbChannel.Text);            var imageRangeMin = Convert.ToInt32(tbGrayMin.Text);            var imageRangeMax = Convert.ToInt32(tbGrayMax.Text);            var backBoneFileName = ModelFiles[cbBackBone.SelectedIndex];            var rawImagePath = tbRawPath.Text;            var preprocessPath = tbPreStorPath.Text;             // 设置到预处理对象属性中            preprocess.ImageHeight = imageHeight;            preprocess.ImageWidth = imageWith;            preprocess.ImageNumChannels = imageNumChannels;            preprocess.ImageRangeMin = imageRangeMin;            preprocess.ImageRangeMax = imageRangeMax;            preprocess.BackBonePath = backBoneFileName;            preprocess.PreProcessPath = preprocessPath;            preprocess.SamplesPath = rawImagePath;            // 设置分类名称            preprocess.ClassNames = classNames;            // 界面显示信息            HalconDLTool.set_display_font(hWinPre.HalconWindow, 20, "mono", "true", "false");            // 调用预处理对象的预处理方法            var res = await preprocess.ExecutePreProcess();            // 判断执行结果            if (res > 0)            {                HOperatorSet.DispText(hWinPre.HalconWindow, "Preprocess finished .", "window", 12, 12, "white", "box_color", "forest green");                Logger.Notify("Preprocess finished .", "Preprocess");            }            else            {                HOperatorSet.DispText(hWinPre.HalconWindow, "Preprocess failure .", "window", 12, 12, "white", "box_color", "red");                Logger.Error("Preprocess failure .", "Preprocess");            }        }        ///         ///         ///         ///         ///         private void Preprocess_UpdateImageAction(HObject obj)        {            try            {                if (obj != null)                {                    var hw = hWinPre.HalconWindow;                    hw.ClearWindow();                    hw.SetWindowParam("flush", "false");                    DispImage(HObject2HImage(obj), hw);                    hw.FlushBuffer();                }            }            catch (Exception)            {                 throw;            }        }        #endregion         #region 选择训练的数据集文件        private string SelectFile(string filter)        {            using(OpenFileDialog dlg = new OpenFileDialog())            {                dlg.Filter = filter;                if(dlg.ShowDialog() == DialogResult.OK)                {                    return dlg.FileName;                }                return null;            }        }        private void btnSelectData_Click(object sender, EventArgs e)        {            // 获取选择的数据集文件路径            string datasetFile = SelectFile("Dataset(*.hdict)|*.hdict");            // 把数据集路径存放至控件的Tag属性,并显示文件名称            tbDataPath.Tag = datasetFile;            if(datasetFile != null)            {                // 将数据集名称显示至文本框                tbDataPath.Text = Path.GetFileNameWithoutExtension(datasetFile);            }            //  获取当前文件所在的路径，模型文件也在当前目录下            Classification.TrainingAction.ModelFilePath = Path.GetDirectoryName(datasetFile);        }        #endregion         #region 执行模型训练        private async void btnTrain_Click(object sender, EventArgs e)        {            // 获取训练对象            var training = Classification.TrainingAction;            // 获取界面的数据            var batchSize = Convert.ToInt32(tbBatchSize.Text);            var learningRate = Convert.ToDouble(tbLearningRate.Text);            var numEpochs = Convert.ToInt32(tbNumEpochs.Text);            var evaluateNumEpochs = Convert.ToInt32(tbEvluationIntervalEphochs.Text);            var trainingPrecent = Convert.ToInt32(tbTrainPrecent.Text);            var validationPercent = Convert.ToInt32(tbValiddatatPrecent.Text);            // 设置训练对象的属性            training.TraingPercent = trainingPrecent;            training.ValidationPercent = validationPercent;            training.BatchSize = batchSize;            training.LearningRate = learningRate;            training.NumEpochs = numEpochs;            training.EvaluateNumEpoch = evaluateNumEpochs;            // 设置显示窗口的的背景            var hw = hWinTrain.HalconWindow;            hw.SetWindowParam("background_color", "light gray");            training.TrainWindow = hw;             // 调用模型训练的方法            int res = await training.ExecuteTraining();             if (res > 0)            {                Logger.Notify("training finished .", "Training");            }            else            {                Logger.Error("training exception .", "Training");            }        }        #endregion         #region 执行评估        private void btnEvaluate_Click(object sender, EventArgs e)        {            // 获取评估对象并给属性赋值            var evaluate = Classification.EvaluateAction;            evaluate.EvaluateMetric = cbMetric.Text;            evaluate.EvaluateDataName = cbSampleSelectValues.Text;            evaluate.EvaluateWindow = hWinEvaluate.HalconWindow;             // 执行模型评估            int res = evaluate.ExecuteEvaluate();             if (res > 0)            {                Logger.Notify("evaluate finished .", "Evaluate");            }            else            {                Logger.Error("evaluate error .", "Evaluate");            }        }        private void btnSelectEvaluateModel_Click(object sender, EventArgs e)        {            // 获取选择的数据集文件路径            string datasetFile = SelectFile("Model(*.hdl)|*.hdl");            if (datasetFile != null)            {                // 将数据集名称显示至文本框                tbEvaluateModelFile.Text = Path.GetFileNameWithoutExtension(datasetFile);            }            // 设置完成的模型路径            Classification.EvaluateAction.EvaluateModelFileName = datasetFile;        }        #endregion         #region 执行推断        private void btnSelectInferModel_Click(object sender, EventArgs e)        {            // 获取选择的数据集文件路径            string datasetFile = SelectFile("Model(*.hdl)|*.hdl");            // 把数据集路径存放至控件的Tag属性,并显示文件名称            tbInferModel.Tag = datasetFile;            if (datasetFile != null)            {                // 将数据集名称显示至文本框                tbInferModel.Text = Path.GetFileNameWithoutExtension(datasetFile);            }            // 设置推断使用的模型文件            Classification.InferenceAction.ModelTypeName = datasetFile;        }        private void btnSelectInferImg_Click(object sender, EventArgs e)        {             var dialog = new FolderBrowserDialog();            if (dialog.ShowDialog() == DialogResult.OK)            {                // 获取用户选择的目录                var inferDataPath = dialog.SelectedPath;                // 给文本框赋值                tbInferPath.Text = inferDataPath;                // 设置推断文件，这里可以进行优化文件后缀，避免放置非图像文件                Classification.InferenceAction.InferImages = Directory.GetFiles(inferDataPath).ToList();            }        }        private async void btnSigInfer_Click(object sender, EventArgs e)        {            // 获取推断对象            var infer = Classification.InferenceAction;            infer.UpdateInvoker += InferOnece_UpdateInvoker;             // 调用推断方法            int res = await infer.ExecuteInfer(InferType.ONECE);             if (res > 0)            {                // 赋值推断结果                tbClass.Text = infer.Class;                tbScore.Text = (infer.Score*100).ToString("F2");                tbTicks.Text = (infer.Ticks * 1000).ToString("F4");                Logger.Log($"{infer.Index}.Type:{infer.Class},Score:{(infer.Score * 100).ToString("F2")}", "Inference");            }            else            {                Logger.Log("infer method error.", "Inference");            }        }         private void InferOnece_UpdateInvoker(HObject obj)        {            if (obj != null)            {                var hw = hWinInfer.HalconWindow;                // 关闭窗口缓存更新                hw.SetWindowParam("flush", "false");                // 把上一次窗口显示数据清空                hw.ClearWindow();                // 显示采集图像                DispImage(HObject2HImage(obj), hw);                // 更新创建数据                hw.FlushBuffer();            }        }         private async void btnContinueInfer_Click(object sender, EventArgs e)        {            // 获取推断对象            var infer = Classification.InferenceAction;            infer.ContinueUpdateInvoker += InferContinue_UpdateInvoker;             // 调用推断方法            int res = await infer.ExecuteInfer(InferType.CONTINUE);             if (res > 0)            {                // 赋值推断结果                Logger.Notify("continue inference finished.", "Inference");            }            else            {                Logger.Log("continue infer method error.", "Inference");            }        }         private void InferContinue_UpdateInvoker(HObject obj, Inference inference)        {            if (obj != null)            {                var hw = hWinInfer.HalconWindow;                // 关闭窗口缓存更新                hw.SetWindowParam("flush", "false");                // 把上一次窗口显示数据清空                hw.ClearWindow();                // 显示采集图像                DispImage(HObject2HImage(obj), hw);                // 更新创建数据                hw.FlushBuffer();                 // 多线程中调用必须使用异步调用方式                BeginInvoke(new Action(() =>                {                    // 赋值推断结果                    tbClass.Text = inference.Class;                    tbScore.Text = (inference.Score*100).ToString("F2");                    tbTicks.Text = (inference.Ticks * 1000).ToString("F4");                }));                Logger.Log($"{inference.Index}.Type:{inference.Class},Score:{(inference.Score * 100).ToString("F2")}", "Inference");            }        }        #endregion

完整打包资料资料包​​​​​​​