[OpenR8 solution] Image_PCB_SSD512_Caffe (Using Caffe SSD512 for object detection on PCB)
  1. Image_PCB_SSD512_Caffe

 

This solution of Image_PCB_SSD512_Caffe is implemented by Caffe framework and SSD512 (Single Shot MultiBox Detector). The main objective is: train the dataset by using SSD512 model. Then implement the trained model to detect the capacitors on PCB.

 

Firstly, we need to prepare images for training. The size of training image is 512 x 512. Select the objects  in the image and label them with the class name. The purpose is to let the model learn and recognize what objects are in the image.

 

Secondly, we need to prepare some txt files, such as train.txt and test.txt, for configuration of training. Then we do the annoset_to_lmdb to transfer the training dataset into LMDB format which is used by Caffe framework. Afterward, the training is executed.

 

The final step is the inference. The trained model is implemented for object recognition. In this solution, the model is trained to recognize the capacitors on PCB.

 

Here is the flow of this solution shown in Fig. 1.

 

Fig. 1 Flow chart.png

Fig.1 Flow chart.

 

 

  1. Step1: Labeling

 

  1. Objective:

Label the objects to be learned by SSD model. In this solution, the detected objects are capacitors on PCB.

 

  1. Tool:

LabelImg is an open source tool which is used for preparing labels for training. The labelImg.exe is attached in the solution folder. The following link is the reference for using labelImg. https://tw.openrobot.club/article/index?sn=10976

 

  1. After labeling the objects (capacitor). The image files and label files (.xml) should be place in specific folder as below.

Images for training: openR8/solution/Image_PCB_SSD512_Caffe/data/train_image

Labels for training: openR8/solution/Image_PCB_SSD512_Caffe/data/train_annotation

Images for testing: openR8/solution/Image_PCB_SSD512_Caffe/data/test_image

Labels for testing: openR8/solution/Image_PCB_SSD512_Caffe/data/test_annotation

 

Fig. 2 labelImg.png

Fig.2  labelImg.exe.

 

Fig. 3 Labeling the object.png

Fig. 3 Labeling the object.

 

Fig. 4 Training samples and xml files.png

Fig. 4 Training samples and xml files.

 

 

  1. Step2: Generate train.txt and test.txt

 

  1. Objective:

The txt files are for defining which data would be used for training and testing respectively.

 

  1. Procedures:

(1) Open R8_Python3.6_CPU.bat or R8_Python3.6_GPU.bat. Then open “1_prepare_train_txt.py”.

 

Fig. 5 R8 Software.png

Fig. 5 R8 Software.

 

Fig. 6 Open 1_prepare_train_txt.png

Fig. 6 Open 1_prepare_train_txt.py.

 

(2) Make sure that the image and label files are set correctly. Then press run, the program will generate train.txt under openR8/solution/Image_PCB_SSD512_Caffe/data.

 

Fig. 7 Run 1_prepare_train_txt.png

Fig.7 Run 1_prepare_train_txt.py.

 

(3) Open “2_prepare_test_txt.py”.

 

Fig. 8 Open 2_prepare_test_txt.png

Fig.8  Open 2_prepare_test_txt.py.

 

(4) Make sure that the image and label files are set correctly. Then press run, the program will generate test.txt under “openR8/solution/Image_PCB_SSD512_Caffe/data”.

 

Fig. 9 Run 2_prepare_test_txt.png

Fig. 9 Run 2_prepare_test_txt.py.

 

Fig. 10 train.txt and test.txt.png

Fig. 10 train.txt and test.txt.

 

Fig. 11 train.png

Fig.11  train.txt.

 

Fig. 12 test.png

Fig.12  test.txt.

 

 

  1. Step3: Generate labelmap.prototxt

 

  1. Objective:

The labelmap.prototxt is for defining the classes for object detection.

 

  1. Open the labelmap.prototxt under the following path.

openR8/solution/Image_PCB_SSD512_Caffe/data

The format of labelmap.prototxt is as below. Edit the name and display name of the classes for object detection.

 

item {

name: "none_of_the_above"

label: 0

display_name: "background"

item {

name: “name of class"

label: 1

display_name: " name of class"

……

item {

name: "name of class"

label: n

display_name: "name of class"

 

[Notice] The first item belongs to the background which is the default setting in SSD. 

 

Fig. 13 labelmap.prototxt.png

Fig. 13 labelmap.proto.

 

  1. In this solution, the class names are set to capacitor1, capacitor2, capacitor3.

 

Fig. 14 labelmap.prototxt in this solution.png

Fig. 14 labelmap.prototxt in this solution.

 

 

  1. Step4: annoset_to_lmdb

 

This step aims to transfer the images and labels into lmdb format used for training.

 

  1. Open annoset_to_lmdb.py

 

Fig. 15 Open 3_annoset_to_lmdb.png

Fig. 15 Open 3_annoset_to_lmdb.py.

 

  1. Run annoset_to_lmdb to generate train_lmdb and test_lmdb.

 

Fig. 16 Run 3_annoset_to_lmdb.png

Fig. 16 Run 3_annoset_to_lmdb.py.

 

The following is the explanation of functions used in this flow.

 

►Caffe_Init: Initialize Caffe framework.

 

►File_DeleteDir: When there is an old lmdb file in the folder, you need to remove the old file before starting to generate a new lmdb file.

Press File_DeleteDir in Flow area. The function details will be shown in the Function area. Press Edit to edit the value of dirName in the Variable area.  

 

Fig. 17 File_DeleteDir.png

Fig. 17 File_DeleteDir.

 

►Caffe_ObjectDetect_CreateTrainData: generate lmdb file.

 

Fig. 18 Caffe_ObjectDetect_CreateTrainData.png

Fig. 18 Caffe_ObjectDetect_CreateTrainData.

 

 

  1. Step5: Training

 

  1. Open 4_train.py. Make sure that if the computer has GPU. Then press Run to start training. After training, the result files are as below.

Path: OpenR8/solution/Image_PCB_SSD512_Caffe/data/

File 1: xxx.caffemodel

File 2: xxx. solverstate

 

Fig. 19 Open 4_train.png

Fig. 19 Open 4_train.

 

Fig. 20 Run 4_train.png

Fig. 20 Run 4_train.

 

Fig. 21 The trained models.png

Fig. 21 The trained models.

 

  1. Function details:

►Caffe_Init: Initialize Caffe framework.

 

►Caffe_Train:

【GPU】This variable is for setting GPU acceleration. Set the value to “all” to use GPU or set to “null” not to use GPU. (If you fill in “all”, please confirm whether the device has GPU)

【solverPath】Set the path of solver.prototxt which is the configuration of training.

【continueTrainModelPath】Set the path of pre-trained model.

 

Fig. 22 solver.prototxt.png

Fig. 22 solver.prototxt.

 

 

  1. Step6: Inference

 

  1. Procedure

►Select the image to be tested, as shown in Figure 25 below.

 

►Verify that your computer supports GPU acceleration. Click [Caffe_ObjectDetect_ReadNet] GPU [Edit], change the [all] of the variable area [value] to [] to cancel the use of GPU acceleration.

 

►Click [Caffe_ObjectDetect_ReadNet] caffeModelPath [...] to select the training model (.caffemodel file) to be used.

 

►View the results on images. If you want to see the detected bounding box on the image, press Debug to show the result of object detection.

 

[Notice] You can use “5_train_result.py” and “6_test_result.py” to measure the classification results of train_image and test_image in all Caffe models, and you can decide which training model to be select based on the classification result.

 

Fig. 23 inference.png

Fig. 23 inference.py.

 

Fig. 24 Open 8_inference.png

Fig. 24 Open 8_inference.py.

 

Fig. 25 Select image for inference.png

Fig. 25 Select image for inference.

 

  1. Function details:

►Caffe_Init: Initialize Caffe.

 

►Image_Open:

【imageFileName (String) 】The path of inference image.

【image (Image) 】After reading the image for inference according to the image file name above, save it to this variable.

 

►Caffe_ObjectDetect_ReadNet

【CaffeObject (Object) 】The Caffe object.

【GPU (String) 】Whether to enable GPU acceleration.

【deployPath (String) 】Read data/deploy.prototxt.

【caffeModelPath (String) 】Read trained model. (.caffemodel)

【labelPath (String) 】Read the class file data/predefined_classes.txt.

 

►Caffe_ObjectDetect_InferenceImage

【CaffeObject (Object) 】The Caffe object.

【OutputResult (Json) 】The outcome of inference.

【GPU (Int) 】Whether to enable GPU acceleration.

【image (Image) 】

【ConfidenceThreshold (Float) 】The confidence threshold of object recognition.

 

►Json_Print

【json (Json) 】Print the outcome of inference.

 

►Image_DrawRectJson

【image (Image) 】

【json (Json) 】The outcome of inference.

【imageDrawRectJson (Image) 】Draw the result to the inference image.

 

►Debug_Image

【image (Image) 】Show the inference image.

【displayPecentage (Int) 】The showing percentage of inference image.

 

 

  1. Step7: Using webcam for inference

 

After the model has been trained, the trained model can also be tested by capturing the image from the webcam in this step which is “8_inference_webcam.”.

 

  1. Procedure:

►[OpenCV_VideoCapture_Open] Make sure the deviceNumber is set (default is 0).

 

►Check if the computer supports GPU acceleration.

 

►Press Run to detect the objects through the webcam. 

 

Fig. 26 8_inference_webcam.py.png

Fig. 26 8_inference_webcam.py.

 

Fig. 27 Open 8_inference_webcam.png

Fig. 27 Open 8_inference_webcam.py.

 

Fig. 28 The device number of the webcam.png

Fig. 28 The device number of the webcam.

 

  1. Function details:

 

►OpenCV_VideoCapture_Init: Initialize video capture.

 

►OpenCV_VideoCapture_Open: Open selected webcam.

 

►OpenCV_VideoCapture_Grab: Grab the image from the webcam.

 

►OpenCV_VideoCapture_Retrieve: Retrieve the image from the webcam.

 

►OpenCV_VideoCapture_Release:Close the webcam. 

 


Recommended Article

1.
OpenR8 Community Edition - AI Software for Everyone (Free Download)