[OpenR8 solution] Image_PCB_FasterRCNN_Keras (Using Keras FasterRCNN for object detection on PCB)
  1. Image_PCB_FasterRCNN_Keras

 

Through the Keras function library, the FASTERRCNN method is used to detect the capacitance above the PCB.

 

First, we need to prepare the image that we want to learn from the model, select the target frame in the image, and mark the selected box as the category. The purpose is to let the model know what the object in the image belongs to Object.

 

 

  1. Folder Introduction

 

The contents of this solution Image_PCB_FasterRCNN_Keras folder are shown in Fig.

 

1, Fig. 2 below. The content is mainly divided into pre-processing, training and testing of three uses.

 

Fig. 1. Folder diagram.png

Fig. 1. Folder diagram.

 

Fig. 2. Data folder diagram.png

Fig. 2. Data folder diagram.

 

  1. Pre-processing

The pre-processing is mainly for the tag box to select the file and execute. The py file produces two parts of the file required for training. 

 

【Record boxes and their categories of files】

 

data folder :

requirements.txt : Record the software version required for this solution.

annotation : The path to the XML file. Example: 1.xml, 2.xml, etc.

image : The path of the image you want to train. For example: 1.png, 2.png and so on.

trainval.txt : Record those files and their XML is to be trained.

test.txt : Record which files and their XML are to be tested.

 

  1. Training Models

The training model contains the files that the user needs to execute as well as the files of the training operation.

 

【Files required to be executed by the user - training】

1_train.flow. The flow file of the training model.

 

【Training required files】

keras_frcnn : Folder.     config.pickle、measure_map.py、train_frcnn.

 

  1. Test models

 

【Files required to be executed by the user - testing】

2_inference.flow. Flow files for OpenR8 test models. Run the Py file with command prompt characters.

 

【File of test run】

model_frcnn_pcb.hdf5 : A model of training production that can be used to infer images.   

test_one_image.py. Files needed to test image.

 

 

  1. Step one:Pre_processing - label the area of interest

 

Select the object you want the model to learn in the image. For example, If you want the model to learn capacitance, prepare a picture of the capacitor, then select the Capacitance box in the image and mark the box as a capacitor. As a result, the model will know that the area of the image is capacitance. And so on, to get the model to learn something, prepare the image and box to mark and label it.

 

So the files we need to prepare contain the following two items.

 

(1) Images (images that you want the model to learn, images that you want to test the accuracy of the model)

File path : openR8/solution/Image_PCB_FasterRCNN_Keras/data/image

(In addition the picture channel is 3 channels, if not please change yourself to RGB 3 channel)

 

(2) An XML file that stores the target location of an image (files that are  automatically generated when the box is selected)

File path : openR8/solution/Image_PCB_FasterRCNN_Keras/data/annotation

 

The lable method can refer to the Image_PCB_SSD_Keras solution example by simply changing the path to the image and XML path of this solution.

 

To detect the capacitor in the PCB, we mark the capacitor in the PCB image through LabelImg.exe (Fig. 3 below), as shown in Fig. 8 green box is the box to select the capacitor and mark it as the [capacitor] category. When the box is selected, the. xml file is automatically generated when stored.

 

The step method for labeling is as follows :  

First step : Turn on the label software

The path to the label software : OpenR8/solution/Image_PCB_SSD512_Caffe/labelImg. See Fig. 3.

Click twice with the mouse. Open the screen as shown in Fig. 4:

 

Fig. 3. The file path of LabelImg.exe..png

Fig. 3. The file path of LabelImg.exe.

 

Fig. 4. Turn on the label tool..jpg

Fig. 4. Turn on the label tool.

 

Second Step : Set the path of the label image sample. See Fig. 5. Turn on display when selected. See Fig. 6. 

 

Fig. 5. Select the path of the image sample..png

Fig. 5. Select the path of the image sample.

 

Fig. 6. Open the path of the image sample..png

Fig. 6. Open the path of the image sample.

 

The third step : Select a folder to store xml categories. See Fig. 7. When the selection is turned on, the following Fig. 8 is displayed.

 

Fig. 7. Select the path of the xml..png

Fig. 7. Select the path of the xml.

 

Fig. 8. Loading files..png

Fig. 8. Loading files.

 

Step Fourth : Label samples with tags for their categories.

 

Press the Create RectBox to select the sample range. Then select a category. One picture does not limit the box to the same category. When the box is ready, you can press save to store the tag category of the file. Then press next image to continue the box to select the next sample picture until all sample pictures are marked with a good category.

 

Fig. 9. Press the Create rectbox box to select the sample..png

Fig. 9. Press the Create rectbox box to select the sample.

 

Fig. 10. Select the category to which the box is selected..png

Fig. 10. Select the category to which the box is selected.

 

The tagged software labelImg.exe has been attached with the OPENR8 software and its file path is OpenR8 > Solution > Image_PCB_SSD300_Caffe > labelImg.exe (Fig. 3), which can be used refer to the Open Source Robot Club [Ezai simple AI] labelimg usage method (Windows version).

 

 

  1. Step two: Pre-processing-Define category names and number of classifications.

 

This step is to define which image are intended for the machine to learn and which image are to be used to test the model.

 

  1. For training purposes : solution/Image_PCB_FasterRCNN_Keras/data/trainval.txt. See Fig. 11.

 

Fig. 11. trainval.png

Fig. 11. trainval.txt.

 

  1. For testing purposes : solution/Image_PCB_FasterRCNN_Keras/data/test.txt. See Fig. 12.

 

Fig. 12. test.png

Fig. 12. test.txt.

 

 

  1. ​Step three: Install Python and Kit

 

If there is no hardware that supports GPU acceleration operations and the appliance installs Python. Please set Fig. 13 below. Adds the path of OpenR8 python within the path in the environment variable. Conversely, the setting is shown in Fig. 14 below.

 

※The software version used by this example is related to Fig. 15 below. 

 

Fig. 13. The Diagram of Python Installation..png

Fig. 13. The Diagram of Python Installation.

 

Fig. 14. The Diagram of Python Installation..png

Fig. 14. The Diagram of Python Installation.

 

Fig. 15. Software versions..png

Fig. 15. Software versions.

 

 

  1. Step four: Training model -2_train

 

When the pre-preparation data is complete, start the training model.

 

  1. Open the flow.

Use the OpenR8 "open 1_train.flow" file and load the 1_train. See Fig. 16, Fig. 17.

  1. Execution flow process.

Press Run to start the training model, and this step takes a little time to wait for the program to build a model. See Fig. 18.

  1. The execution is completed and the model is generated.

When this execution is completed, a trained model is generated, and the model file is generated and placed, as shown in Fig. 19 below. The path is: the file root directory/solution/Image_PCB_FasterRCNN_Keras/data/file name 1:xxx. hdf5

 

Fig. 16. The path of 2_train.flow..png

Fig. 16. The path of 2_train.flow.

 

Fig. 17. Open 1_train.flow.jpg

Fig. 17. Open 1_train.flow.

 

Fig. 18. Run 1_train and its running process diagram..png

Fig. 18. Run 1_train and its running process diagram.

 

Fig. 19. Running completed 1_train generated model file..png

Fig. 19. Running completed 1_train generated model file.

 

 

  1. Step Five: Test the trained model - inference_image

 

When the model has been trained, you can test the trained model with this step.

 

  1. Open flow file.

Use OpenR8 to turn on "3_inference.flow" and load the file. Fig. 20, Fig. 21 below.

  1. Set up test images.

Then set the image that needs to be tested. The method is shown in Fig. 22 below.

The images tested should be placed in:

Folder root directory/solution/Image_PCB_FasterRCNN_Keras/data/image.

  1. et the model for the test. Please refer to Fig. 23 below. 4.    Run and display the results. Click to run. See Fig. 24.

 

Fig. 20. 2_inference.flow..png

Fig. 20. 2_inference.flow.

 

Fig. 21. Load 2_inference..jpg

Fig. 21. Load 2_inference.

 

Fig. 22. 2_inference Select test image..png

Fig. 22. 2_inference Select test image.

 

Fig. 23. 2_inference Select the model to do the test..png

Fig. 23. 2_inference Select the model to do the test.

 

Fig. 24. 2_inference test results..png

Fig. 24. 2_inference test results.


Recommended Article

1.
OpenR8 - AI Software for Everyone (Download)