[OpenR8 solution] Image_PCB_SSD300_Caffe (Using Caffe SSD300 for object detection on PCB)
  1. Image_PCB_SSD300_Caffe

 

This Image_PCB_300_Caffe is the use of the Caffe framework. Speed is more than twice times faster than Keras. First Use the SSD (Single Shot multibox detector) deep learning algorithm to train the model, and then through the trained model to detect the capacitor above on PCB. This training image size is 300x300.

 

The main process is shown in Fig. 1 below.

 

First, we need to prepare the image that we want the model to learn, select the Target box in the image and record its category. The goal is to let the model know which category of objects in this image belong to.

 

Then use OpenR8 to execute the flow file to automatically generate two TXT list files to let the model know which files are being trained and tested, and which categories are grouped in these files.

 

After the pre-processing is ready. Then through six flow files in Image_PCB_SSD300_Caffe. The training and testing can be completed by executing sequentially.

 

Fig. 1. Flow chart..png

Fig. 1. Flow chart.

 

 

  1. Step1:Pre_processing - label the area of interest

 

Fig. 1. 1Flow chart..png

 

  1. Objective :

Box to select the preparation with the labeled region of interest. 

 

  1. Description :

In the Image box, select the object that you want the model to learn. For example, you want the model to learn the capacitance to prepare the capacitance image, and then the Capacitance box in the image to be selected, marked as a capacitor. As a result, the model will know that the area marked by this image is capacitance. And so on, if you want the model to learn something, prepare its image and markup category box.

 

So the files we need to prepare contain the following two items

(1) Images (images that you want the model to learn, images that you want to test the accuracy of the model)

(2) An XML file that stores the target location of an image (files that are automatically generated when the box is selected)

 

  1. Example :

In this Image_PCB_SSD300_Caffe solution example:

To detect the capacitor in the PCB, we mark the capacitor in the PCB image through LabelImg.exe (Fig. 2 below), as shown in Fig. 3 red box is the box to select the capacitor and mark it as the [capacitor] category. When the box is selected, the. xml file is automatically generated when stored.

Places the image in the specified folder location. Fig. 4 below.

【Training】

Image path : openR8/solution/Image_PCB_SSD300_Caffe/data/train_image

Xml path : openR8/solution/Image_PCB_SSD300_Caffe/data/train_annotation

 

  1. Additional notes :

The tagged software labelImg.exe has been attached with the OPENR8 software and its file path is OpenR8 > Solution > Image_SSD300_Caffe > LabelImg.exe. Fig. 2 below, which can be used refer to the Open Source Robot Club [Ezai simple AI] labelimg usage method (Windows version).

 

Fig. 2. The file path of LabelImg.exe..jpg

Fig. 2. The file path of LabelImg.exe.

 

Fig. 3. Mark the target object in the image..png

Fig. 3. Mark the target object in the image.

 

Fig. 4. The file path of image and xml training..png

Fig. 4. The file path of image and xml training.

 

Fig. 5. The file path of image and xml inference. .png

Fig. 5. The file path of image and xml inference.

 

 

  1. Step 2 : Pre-processing- establish train.txt files to record file locations

 

Fig. 5. 1The file path of image and xml inference. .png

 

  1. Objective :

The txt files are for defining which data would be used for training and testing respectively.

 

  1. How to use it

【Training】train.txt

 

Step1: Open OpenR8,Run R8.exe.

 

Fig. 6. Run R8.exe

Fig. 6. Run R8.png

 

Step2 : Choosing and opening a solution.

 

Fig. 7. Open Solution.png

Fig. 7. Open Solution.

 

Step3 : Run flow,Generate train.txt.

 

Fig. 8. Run Solution.png

Fig. 8. Run Solution.

 

Fig. 9. Open Solution.png

Fig. 9. Open Solution.

 

【Testing】

Step1: Open OpenR8,Run R8.exe.

Step2 : Choosing and opening a solution. See Fig. 10.

 

Fig. 10. Generate test.png

Fig. 10. Generate test.txt.

 

  1. Run flow,Generate test.txt.

 

Fig. 11. Run Solution.png

Fig. 11. Run Solution

 

Fig. 12. Generate test.txt .png

Fig. 12. Generate test.txt

 

 

  1. Step 3: Pre-processing- establish a TXT file for the Record Volume label category

 

Fig. 12. 1Generate test.txt .png

 

  1. Objective :

Create a category TXT to let the model know what categories the data is divided into, so that the model can learn to classify the images on its own.

 

  1. Procedures:

Please create a "labelmap.prototxt" file under the openR8/solution/Image_PCB_SSD300_Caffe/data path.

The file content format is as follows:

       item {

name: "none_of_the_above"

label: 0

display_name: "background"

}

item {

name: " The name of the category"

label: 1

display_name: "display the name of category"

}

……

item {

name: " The name of the category "

label: n

display_name: " display the name of category "

}

 

Please note that the name and display_name above must be associated with the The category name and number of classifications within the XML file.

 

  1. Example:

In the openR8/solution/Image_PCB_SSD300_Caffe/data folder Labelmap.prototxt, you can open the file by notepad++ or Notepad, as shown in Fig. 13 below.

 

The contents of the file are shown in Fig. 14 below, and we can see that we classify the images into none_of_the_above (none of the above) and capacitor two categories, of which individual volume labels are numbered 0 and 1.

 

If you want to add a new category yourself, please add the category name and label number, as shown in Fig. 14 (right).    

 

Fig. 13. The file path of Labelmap.prototxt..png

Fig. 13. The file path of Labelmap.prototxt.

 

Fig. 14.1 Labelmap content Diagram left add category diagram right ..png

Fig. 14. Labelmap content Diagram (left), add category diagram (right).

 

 

  1. Step 4:Pre_processing - annoset_to_lmdb

 

Fig. 14.1 Labelmap content Diagram left add category diagram right ..png

This step is the Lmdb file required to train and test the model for the TXT transformation that has been defined in the previous two steps.

 

  1. Description: Open and Run Annoset_to_lmdb flow

(1) Please "mouse double-click R8.exe" => click "File" => "open" => "OpenR8 > solution Image_PCB_SSD300_Caffe" => "3_annoset_to_ Lmdb.flow. " The diagram is shown in Fig. 15Fig. 16Fig. 17.

 

(2) Please click Execute. If there is an old Lmdb file, you will jump out and remove it, click Yes (Y), as shown in Fig. 18 below. When the execution is complete, train_lmdb and Test_lmdb are generated. See Fig. 19, Fig. 20 below.

For any questions about turning on the software to the load solution, refer to the "OpenR8 manual".

 

Fig. 15. R8.png

Fig. 15. R8.exe.

 

Fig. 16. Open the annoset_to_lmdb.flow..png

Fig. 16. Open the annoset_to_lmdb.flow.

 

Fig. 17. Load the annoset_to_lmdb..png

Fig. 17. Load the annoset_to_lmdb.

 

Fig. 18. delete the lmdb..png

Fig. 18. delete the lmdb.

 

Fig. 19. Running complete..png

Fig. 19. Running complete.

 

Fig. 20. Create the train_lmdb and test_lmdb..png

Fig. 20. Create the train_lmdb and test_lmdb.

 

  1. annoset_to_lmdb Parameter description

 

Fig. 21. Input file left turn to output file right ..png

Fig. 21. Input file (left) turn to output file (right).

 

The following is the explanation of functions used in this flow.

 

Caffe_Init : Initialize Caffe framework.

 

File_DeleteDir (Fig. 22) : When there is an ol d lmdb file in the folder, you need to remove the old file before starting to generate a new lmdb file. Press File_DeleteDir in Flow area. The function details will be shown in the Function area. Press Edit to edit the value of dirName in the Variable area.

 

Fig. 22. File_DeleteDir..png

Fig. 22. File_DeleteDir.

 

Caffe_ObjectDetect_CreateTrainData: generate lmdb file.

 

Fig. 23. Caffe_ObjectDetect_CreateTrainData..png

Fig. 23. Caffe_ObjectDetect_CreateTrainData.

 

 

  1. Step 5: Training

 

Fig. 23.1 Caffe_ObjectDetect_CreateTrainData..png

The introduction is divided into two parts, the 1th first describes how to start training, and then if you are interested in knowing that each process block can be viewed in the 2nd to describe the details of this process.

 

  1. Description : Training model.

First, use the OpenR8 "open 4_train.flow" file and load the 4_train.flow. See Fig. 24, Fig. 25.

 

Next, press "Run" to start the training model, which takes a little time to wait for the program to build the model. See Fig. 26.

 

When this run is complete, a trained model is generated, and its model file generation is placed, as shown in Fig. 27 below.

 

Path : OpenR8/solution/Image_PCB_SSD300_Caffe/data/

File name 1 : xxx.Caffemodel

File name2 : xxx. Solverstate

 

If the hardware does not support the NVIDIA GPU. An error message appears. Follow Fig. 28, Fig. 29, below.

 

Fig. 24. The path of 4_train.flow..png

Fig. 24. The path of 4_train.flow.

 

Fig. 25. Load the 4_train.flow..png

Fig. 25. Load the 4_train.flow.

 

Fig. 26. Run 4_train.flow..png

Fig. 26. Run 4_train.flow.

 

Fig. 27. The production model is finished after training..png

Fig. 27. The production model is finished after training.

 

Fig. 28. No GPU error message-Training.png

Fig. 28. No GPU error message-Training

 

Fig. 29. How to set up without NVIDIA GPU..png

Fig. 29. How to set up without NVIDIA GPU.

 

  1. The Parameters Description of 4_train.flow:

►Caffe_Init: Initialize Caffe framework.

 

►Caffe_Train:

GPU】This variable is for setting GPU acceleration. Set the value to “all” to use GPU or set to “null” not to use GPU. (If you fill in “all”, please confirm whether the device has GPU)

 

solverPath】Set the path of solver.prototxt which is the configuration of training. See Fig. 30, Fig. 31.

 

continueTrainModelPath】Set the path of pre-trained model.

 

Fig. 30. The Caffe_Train of 4_train.flow..png

Fig. 30. The Caffe_Train of 4_train.flow.

 

Fig. 31. solver.prototxt..png

Fig. 31. solver.prototxt.

 

 

  1. Step 6: Test the model of each training stage

 

  1. Objective : Used to test the accuracy of each stage model, from which to find the appropriate model, to avoid over fitting of training samples during training, resulting in poor test sample results.

  1. Description :

(1) Training sample: "Turn on OpenR8" => "Open and load 5_train_result.flow" => "Run Flow" => "train_result.txt generation".

 

(2) Testing sample: "Turn on OpenR8" => "Open and load 6_test_result.flow" => "Run Flow" => "test_result.txt generation".

 

(3) Select the appropriate training number model:

As shown in Fig. 32, it can be observed from the content of train_result.txt/test_result.txt that the results of the model at a certain training number stage are acceptable or have reached convergence after a certain stage. Therefore, it is more appropriate for us to choose the model of this number.

 

Fig. 32. Detail of train_result.txt..png

Fig. 32. Detail of train_result.txt.

 

 

  1. Step 7: Inference using a trained model - inference

 

Fig. 32.1 Detail of train_result.txt..png

 

When the model has been trained, you can use this step to infer using a trained model.

  1. How to use the process :

►First use OpenR8 to open "7_inference" and load the file. See Fig. 33, Fig. 34.

 

►Select the image you want to test. See Fig. 35.

 

►Choosing a model that has been trained. See Fig. 36.

 

►View the results of the test message. Press run to detect the target coordinate position and its category in the image through a trained model. See Fig. 37.

 

►View the resulting image after the inference. If you want to see the results box detected above the image, press Debug. Displays the results of object detection. PCB images are marked with detected capacitors. See Fig. 38.

 

If the hardware does not support the NVIDIA GPU. An error message appears. Follow Fig. 39, Fig. 40 below.

 

Fig. 33. 7_inference.png

Fig. 33. 7_inference.

 

Fig. 34. Load 7_inference.png

Fig. 34. Load 7_inference.

 

Fig. 35. 7_inference - Select test Images..png

Fig. 35. 7_inference - Select test Images.

 

Fig. 36. 7_inference - select the model you want to infer.png

Fig. 36. 7_inference - select the model you want to infer.

 

Fig. 37. 7_inference running result..png

Fig. 37. 7_inference running result.

 

Fig. 38. 7_inference - Debugging results..png

Fig. 38. 7_inference - Debugging results.

 

Fig. 39. Error message-show no GPU..png

Fig. 39. Error message-show no GPU.

 

Fig. 40. How to set up without NVIDIA GPU..png

Fig. 40. How to set up without NVIDIA GPU.

 

  1. Function details:

►Caffe_Init: Initialize Caffe.

 

►Image_Open:

【imageFileName (String) 】The path of inference image.

【image (Image) 】After reading the image for inference according to the image file name above, save it to this variable.

 

►Caffe_ObjectDetect_ReadNet

【CaffeObject (Object) 】The Caffe object.

【GPU (String) 】Whether to enable GPU acceleration.

【deployPath (String) 】Read data/deploy.prototxt.

【caffeModelPath (String) 】Read trained model. (.caffemodel)

【labelPath (String) 】Read the class file data/predefined_classes.txt.

【meanFilePath (String)】

 

►Caffe_ObjectDetect_InferenceImage

【CaffeObject (Object) 】The Caffe object.

【OutputResult (Json) 】The outcome of inference.

【GPU (Int) 】Whether to enable GPU acceleration.

【image (Image) 】

【ConfidenceThreshold (Float) 】The confidence threshold of object recognition.

 

►Json_Print

【json (Json) 】Print the outcome of inference.

 

►Image_DrawRectJson

【image (Image) 】

【json (Json) 】The outcome of inference.

【imageDrawRectJson (Image) 】Draw the result to the inference image.

 

►Debug_Image

【image (Image) 】Show the inference image.

►【displayPecentage (Int) 】The showing percentage of inference image. A value of 200 means 200 (the picture size is magnified one times), 50 means 50 (the picture size is halved), and so on, zoom in and out.

 

 

  1. Step8: Using webcam for inference

 

Fig. 40. 1How to set up without NVIDIA GPU..png

 

  1. How to use the process :

When the model has been trained, we can real-time image object detection and Identification with webcam through the “8_inference_webcam.flow”.

 

First use OpenR8 to open "8_inference_webcam". Loading flow. See Fig. 41,  Fig. 42.

 

Make sure the deviceNumber of “OpenCV_VideoCapture_Open” is set (default is 0). See Fig. 43.

 

Press Run. See Fig. 44.

 

If the appliance does not support the NVIDIA GPU. An error message appears. Please refer to Fig. 39, Fig. 40 above.

 

Fig. 41. Load 8_inference_webcam.png

Fig. 41. Load 8_inference_webcam.

 

Fig. 42. Complete the loading flow..jpg

Fig. 42. Complete the loading flow.

 

Fig. 43. 6_inference_webcam - Check the deviceNumber of webcam..png

Fig. 43. 6_inference_webcam - Check the deviceNumber of webcam.

 

Fig. 44. Run 8_inference_webcam..png

Fig. 44. Run 8_inference_webcam.


Recommended Article

1.
OpenR8 - AI Software for Everyone (Download)