[OpenR8 solution] Image_PCB_SSD300_Keras (Using SSD 300 algorithm and Keras for PCB object detection)
  1. Image_PCB_SSD300_Keras


This Image_PCB_SSD300_Keras is through Keras library, using the SSD (Single Shot Multibox Detector) method to detect the capacitance above the PCB.


This article is a training test using the size of the 300 x 300 image, and a training sample of the size of the 512 x 512 can be referenced (see Solution SSD512 description file). The image size of 300x300 is faster than 512x512, but the accuracy is slightly worse than 512 x 512.


Our main process is shown in  Fig. 1 below.


First, we need to prepare the image that we want the model to learn, select the Target box in the image and record its category. The goal is to let the model know which category of objects in this image belong to.


After the pre-processing is ready. Then through three flow files in Image_PCB_SSD300_Keras. The training and testing can be completed by executing sequentially.


Fig. 1. Flow chart..png

Fig. 1. Flow chart.



  1. Introduction

The contents of this solution Image_PCB_SSD300_Keras folder are shown in Fig. 2 below.


The main content is pre-processing, training and testing of three uses.


Fig. 2. Folder diagram..png

Fig. 2. Folder diagram.


  1. Pre-processing

Pre-processing mainly for the marking box to select the file and perform the .py file production training required files.


【tags and box selection of files】

data/annotation : The path location of the XML file. Ex : 1.xml、2.xml…etc.

data/image : The path you want to train the image. Ex : 1.png、2.png…etc.

get_data_from_XML_PCB.py : A file that sets the type name of the classification and the number of classifications.


【Generate the files needed for training】

1_prepare.flow : Convert the labeled file into the .pkl file required for training.

data/PCB.pkl : Files generated for the above-mentioned implementation of the 1_prepare.flow.


  1. Training Models

The training model contains two parts, the file that the user needs to execute and the file that the training runs.


【Files that the user needs to perform or modify】 2_train.flow : Files used in the OPENR8 training model. Enter a command prompt message to run the PY file.

src/ssd_run_training.py : Set the quantity to categorize.

data/output/weights.30-1.39.hdf5 : It's a model file from the end of training.


【source code of the training model】

ssd.py、ssd_layers.py、ssd_utils.py : SSD algorithm of the Keras library and the training file.

data folder.


  1. Test models


【Files required to be executed by the user】

3_inference.flow : File of the OpenR8 test mode.

data/pics : Place the picture you need for your test. Ex : 1.png、2.png、3.png…etc.

data/weights_PCB.hdf5 : The model you want to test. Through this model, the detection results of image are obtained.

src/ssd_inference.py : Set categories for classifications.

【File of test run】

src/ssd_run_inference.py : The file for the test run.




  1. Step1:Pre_processing - label the area of interest


Select the object you want the model to learn in the image. For example, If you want the model to learn capacitance, prepare a picture of the capacitor, then select the Capacitance box in the image and mark the box as a capacitor. As a result, the model will know that the area of the image is capacitance. And so on, to get the model to learn something, prepare the image and box to mark and label it. So the files we need to prepare contain the following two items.


(1) Images (images that you want the model to learn, images that you want to test the accuracy of the model)

File path : openR8/solution/Image_PCB_SSD300_Keras/data/image (According to the current batch size and other network parameters set, it is recommended that the picture is greater than 11 sheets. In addition the picture channel is 3 channels, if not please change yourself to RGB 3 channel)


(2) An XML file that stores the target location of an image (files that are  automatically generated when the box is selected)

File path : openR8/solution/Image_PCB_SSD300_Keras/data/annotation


Fig. 3. The file path of image and xml training..png

Fig. 3. The file path of image and xml training.


In this Image_PCB_SSD300_Keras solution example:


To detect the capacitor in the PCB, we mark the capacitor in the PCB image through LabelImg.exe (Fig. 4 below), as shown in Fig. 9 green box is the box to select the capacitor and mark it as the [capacitor] category. When the box is selected, the. xml file is automatically generated when stored.


The step method for labeling is as follows :


First step : Turn on the label software

The path to the label software : openR8/tool/Deep Learning/labelImg. SeeFig. 4. Click twice with the mouse. Open the screen as shown in Fig. 5:


Fig. 4. The file path of LabelImg.exe..png

Fig. 4. The file path of LabelImg.exe.


Fig. 5. Turn on the label tool..jpg

Fig. 5. Turn on the label tool.


Second Step : Set the path of the label image sample. See Fig. 6. Turn on display when selected. See Fig. 7. 


Fig. 6. Select the path of the image sample..png

Fig. 6. Select the path of the image sample.


Fig. 7. Open the path of the image sample..png

Fig. 7. Open the path of the image sample.


The third step : Select a folder to store xml categories. See Fig. 8. When the selection is turned on, the following Fig. 9 is displayed.


Fig. 8. Select the path of the xml..png

Fig. 8. Select the path of the xml.


Fig. 9. Loading files..png

Fig. 9. Loading files.


Step Fourth : Label samples with tags for their categories.

Press the Create RectBox to select the sample range. Then select a category. One picture does not limit the box to the same category. When the box is ready, you can press save to store the tag category of the file. Then press next image to continue the box to select the next sample picture until all sample pictures are marked with a good category.


Fig. 10. Press the Create rectbox box to select the sample..png

Fig. 10. Press the Create rectbox box to select the sample.


Fig. 11. Select the category to which the box is selected..png

Fig. 11. Select the category to which the box is selected.


The tagged software labelImg.exe has been attached with the OPENR8 software and its file path is OpenR8 > Solution > Image_PCB_SSD300_Caffe > labelImg.exe (Fig. 4), which can be used refer to the Open Source Robot Club [Ezai simple AI] labelimg usage method (Windows version).



  1. Step two: Pre-processing-Define category names and number of classifications.


This step is to define which categories we divide the data into and the number of classifications that the model can use to learn the classification through these data. Mainly divided into three processes : 


  1. Open the get_data_from_xml_pcb.py file. Please use "notepad++" to open "get_data_from_xml_pcb.py" as shown in Fig. 12, Fig. 13.

Path : solution/Image_PCB_SSD300_Keras/src/get_data_from_XML_PCB.py

  1. Modify the encoding of the notepad++. Please click "Encoding" => "Encoding in UTF-8". See Fig. 14 below.

  1. Sets the number of data that you want to categorize. Fig. 15 below. Please change the number of self.num_classes = 1 in the file to the classification number "1".

For example, we classify the data into two categories: capacitance and LED, then self.num_classes = 2.


  1. Set the categories you want to categorize your data. Fig. 16 below.

​Found it in _to_one_hot (self, name) :

if name == 'capacitor':

one_hot_vector[0] = 1

Set the category format for data classification, as follows:

if name == ' Category name a':

one_hot_vector[0] = 1

elif name == ' Category name b':

one_hot_vector[1] = 1

…..And so on

For example: We are divided into two categories, the names of which are capacitor and LEDs. You need to modify it to:

if name == 'capacitor':

one_hot_vector[0] = 1

elif name == 'LED':

one_hot_vector[1] = 1


Fig. 12. The path of the get_data_from_XML_PCB.py..png

Fig. 12. The path of the get_data_from_XML_PCB.py.


Fig. 13. Content of the get_data_from_XML_PCB.py..png

Fig. 13. Content of the get_data_from_XML_PCB.py.


Fig. 14. Modify the notepad encoding to UTF-8..png

Fig. 14. Modify the notepad++ encoding to UTF-8.


Fig. 15. Set the number of classifications..png

Fig. 15. Set the number of classifications.


Fig. 16. Set classification category names..png

Fig. 16. Set classification category names.



  1. Step three: Install Python and Kit


If there is no hardware that supports GPU acceleration operations and the appliance installs Python. Please set Fig. 17 below. Adds the path of OpenR8 python within the path in the environment variable. Conversely, the setting is shown in Fig. 18 below. 


Fig. 17. The Diagram of Python Installation..png

Fig. 17. The Diagram of Python Installation.


Fig. 18. The Diagram of Python Installation..png

Fig. 18. The Diagram of Python Installation.



  1. Step four: Pre-processing -1_prepare


Then the remaining steps require only the use of OpenR8 software to run the flow file.


This step takes only two steps to load the process file and click run to turn the previously prepared information into the information required for training.


  1. Loading flow files.

Please "mouse double-click R8.exe" => click "File" => "open" => "solution" Image_PCB300_SSD_Keras "in the folder =>" "1_prepare.flow". The diagram is shown in Fig. 19, Fig. 20Fig. 21.


For any questions about turning on the software load solution, refer to the "OpenR8 operating manual".


Fig. 19. R8.png

Fig. 19. R8.exe.


Fig. 20. File Open..png

Fig. 20. File Open.


Fig. 21. Select Image_PCB_SSD300_Keras Folder..png

Fig. 21. Select Image_PCB_SSD300_Keras Folder.


Fig. 22. Select 1_prepare.flow..png

Fig. 22. Select 1_prepare.flow.


Fig. 23. Load 1_prepare.flow..png

Fig. 23. Load 1_prepare.flow.


  1. Run the process file. Simply click "Run" to start running the flow file and use the instructions to perform the previously prepared get_data_from_xml_pcb.py. See Fig. 24, Fig. 25.


Fig. 24. Run 1_prepare.flow..png

Fig. 24. Run 1_prepare.flow.


Fig. 25. Run complete 1_prepare.flow produce PCB.pkl..png

Fig. 25. Run complete 1_prepare.flow produce PCB.pkl.



  1. Step five: Training model -2_train


The introduction is mainly divided into two parts, the first to introduce how to start training.

When the pre-preparation data is complete, start the training model.


  1. Fill in the parameters of the training. Please turn on C:/Users/ai/Desktop/OpenR8/solution/Image_PCB_SSD300_Keras/src/ssd_run_training.py

Find num_classes = 1 + 1 in content. The preceding 1 is represented as the quantity to categorize, and the following 1 represents the background.

For example: Sub-capacitance and resistor two categories, the previous 1 need to be modified to 2, that is, num_classes = 2 + 1.


  1. Open the flow.

Use the OpenR8 "open 2_train.flow" file and load the 2_train. See Fig. 26, Fig. 27.  

  1. Execution flow process.

Press Run to start the training model, and this step takes a little time to wait for the program to build a model. See Fig. 28.


  1. The execution is completed and the model is generated.

When this execution is completed, a trained model is generated, and the model file is generated and placed, as shown in Figure 29 below.

The path is: the file root

directory/solution/Image_PCB_SSD300_Keras/data/output/file name 1:xxx. hdf5


Fig. 26. The file path of 2_train..png

Fig. 26. The file path of 2_train.


Fig. 27. Open 2_train..png

Fig. 27. Open 2_train.


Fig. 28. Run 2_train and its running process diagram..png

Fig. 28. Run 2_train and its running process diagram.


Fig. 29. Running completed 2_train generated model file..png

Fig. 29. Running completed 2_train generated model file.



  1. Step Six: Test the trained model-inference_image


When the model has been trained, you can test the trained model with this step.


  1. Open flow file.

Use OpenR8 to turn on "3_inference.flow" and load the file. Fig. 30, Fig. 31 below.


  1. Set up test images.

Then set the image that needs to be tested. The method is shown in Fig. 32 below.

The images tested should be placed in:

Folder root directory/solution/Image_PCB_SSD_Keras/data/pics.


  1. Set the model for the test.

Please rename the/data/output trained WEIGHTS.30-1.39.HDF5 model to WEIGHTS_PCB.HDF5, and then Replace Dataweights_PCB.hdf5.


  1. Sets the categories that are required for testing. File Location


Please modify the content voc_classes = ['capacitor'] to the category you define voc_classes = [‘class1’, ‘class2’, ‘class3’….],class1、class2……is the name of the category that was previously defined.

For example : Want to classify capacitors with LEDs in a total of two categories, voc_classes = ['capacitor', 'LED']. 


  1. Run and display the results.

Click to run. See Fig. 33.


Fig. 30. 3_inference..png

Fig. 30. 3_inference.


Fig. 31. Load 3_inference..png

Fig. 31. Load 3_inference.


Fig. 32. 3_inference Select test image..png

 Fig. 32. 3_inference Select test image.


Fig. 33. 3_inference result.png

Fig. 33. 3_inference result.



Recommended Article

OpenR8 - AI Software for Everyone (Download)