[OpenR8 solution] Image_FaceRecognitionCelebrity (Face recognition of famous people)
  1. Image_FaceRecognitionCelebrity

 

Using the Caffe framework, the human face in the image is detected and identified through the SSD (Single Shot Multibox detector) method. The main process is shown in Fig. 1 below.

 

  1. Box selection and markup : we need to prepare the image that we want the model to learn, select the target box in the image, and mark the box with the box, in order to let the model know who the characters in the image are.
  2. Perform flow files to complete training and testing: Through the three flow files in OpenR8 Image_ facerecognitioncelebrity, the training and testing can be completed sequentially. 

 

Fig. 1. Training Model Flowchart..png

Fig. 1. Training Model Flowchart.

 

 

  1. Folder Introduction 

 

The contents of this solution Image_ Facerecognitioncelebrity folder are shown in Fig. 2, Fig. 3 below.

 

The content is mainly divided into pre-processing, training and testing of three uses. 

 

Fig. 2. Folder diagram..jpg

Fig. 2. Folder diagram.

 

Fig. 3. Data folder diagram..jpg

Fig. 3. Data folder diagram.

 

  1. Pre-processing

 

Filename

Use

Box selection related files required by the user.

labelImg.exe

Label software.

data/predefined_classes.txt

Record the categories that let the model learn.

data/annotation

The markup category file where you want the model to learn the image.

data/image

Image that you want the model to learn.

data/image_test

Images used to test the training model.

data/trainval.txt

Record the filename of all training images and tag files.

Flow files that the user needs to perform.

1_annoset_to_lmdb.flow

Turn the data (picture and its tag file) into the Lmdb file required for the next training.

Files generated by running flow.

test_lmdb (data.mdb、lock.mdb)

Run 1_annoset_to_lmdb.flow to produce the Lmdb file required for testing.

trainval_lmdb (data.mdb、lock.mdb)

Lmdb files required for the execution of training generated by 1_annoset_to_lmdb.flow.

 

  1. Training

 

Filename

Use

Flow files that the user needs to perform.

2_train.flow

The flow of the OPENR8 training model.

The files produced by the completion of the training.

snapshot_iter_80000.caffemodel

File name .caffemodel is a model file produced by training.

The files needed to run the training.

deploy.prototxt

labelmap.prototxt

solver.prototxt

test.prototxt

train.prototxt

Network settings and parameters required for training.

 

  1. Test models

 

Filename

Use

Flow files that the user needs to perform.

3_inference_image.flow

Open R8 flow required to test the model.

4_inference_webcam.flow

OpenR8 flow file required to detect and identify webcam real-time sequences.

 

 

  1. Step 1 : Pre_processing - label the area of interest

 

In the Image box, select the object that you want the model to learn. For example, if you want the model to learn which face to prepare its image, then frame the face in the image and mark it with a person's name. As a result, the model will know who is in the area of this image box. And so on, if you want the model to learn who, prepare the image and box to mark.

 

So the files we need to prepare contain the following three items

  1. Images (images that you want the model to learn, images that you want to test the accuracy of the model)

Image path : openR8/solution/Image_FaceRecognitionCelebrity/data/image

 

  1. An XML file that stores the target location of an image (files that are automatically generated when the box is selected)

Xml path : openR8/solution/Image_FaceRecognitionCelebrity/data/annotation

 

  1. Record the TXT file for the classification category. Please use notepad++ to open, the name of the person to be identified from the upper to bottom input (the first must be background do not delete). Fig. 4 below.

File path : openR8/solution/Image_FaceRecognitionCelebrity/data/predefined_classes.txt

 

Fig. 4. predefined_classes.jpg

Fig. 4. predefined_classes.txt.

 

The markup method can be referred to as the Image_PCB_SSD_Caffe solution example, which replaces the path of the image, XML file of this solution.

To detect the capacitor in the PCB, we mark the capacitor in the PCB image through LabelImg.exe (Fig. 5 below), as shown in Fig. 12 green box is the box to select the capacitor and mark it as the [capacitor] category. When the box is selected, the. xml file is automatically generated when stored.

 

The markup step method is as follows:

First step : Turn on the label software

The path to the label software: OpenR8 > Solution > Image_SSD300_Caffe > LabelImg.exe. See Fig. 5. Click twice with the mouse. Open the screen as shown in Fig. 6 :

 

Fig. 5. The file path of LabelImg.exe..jpg

Fig. 5. The file path of LabelImg.exe.

 

Fig. 6. Turn on the label tool..jpg

Fig. 6. Turn on the label tool.

 

Second Step : Set the path of the label image sample. See Fig. 7. Turn on display when selected. See Fig. 8. 

 

Fig. 7. Select the path of the image sample..png

Fig. 7. Select the path of the image sample.

 

Fig. 8. Open the path of the image sample..png

Fig. 8. Open the path of the image sample.

 

The third step : Select a folder to store xml categories. See Fig. 9. When the selection is turned on, the following Fig. 10 is displayed.

 

Fig. 9. Select the path of the xml..png

Fig. 9. Select the path of the xml.

 

Fig. 10. Loading files..png

Fig. 10. Loading files.

 

Step Fourth : Label samples with tags for their categories.

Press the Create RectBox to select the sample range. Then select a category. One image does not limit the box to the same category. When the box is ready, you can press save to store the tag category of the file. Then press next image to continue the box to select the next sample picture until all sample pictures are marked with a good category. See Fig. 11, Fig. 12 below.

 

Fig. 11. Press the Create rectbox box to select the sample..png

Fig. 11. Press the Create rectbox box to select the sample.

 

Fig. 12. Select the category to which the box is selected..png

Fig. 12. Select the category to which the box is selected.

 

The tagged software labelImg.exe has been attached with the OpenR8 software and its file path is OpenR8 > Solution > Image_PCB_SSD300_Caffe > labelImg.exe (Fig. 5), which can be used refer to the Open Source Robot Club [Ezai simple AI] labelimg usage method (Windows version).

 

 

  1. Step2 : Pre-processing- establish train.txt files to record file locations

 

  1. Objective :

The txt files are for defining which data would be used for training and testing respectively.

  1. How to use it :

Use Labelimage to compose a list of two folder files Data/image and data/annotation.

Press Customized > Export file list > Generate filelist.txt > Rename the filelist.txt to Trainval.txt replace the original file. See Fig. 13, Fig. 14 below.

 

Fig. 13. Generate train.txt 1 ..jpg

Fig. 13. Generate train.txt (1).

 

Fig. 14. Generate train.txt 2 ..png

Fig. 14. Generate train.txt (2).

 

 

  1. Step 3: Pre-processing- establish a TXT file for the Record Volume label category

 

  1. Objective :

Create a category TXT to let the model know what categories the data is divided into, so that the model can learn to classify the images on its own.

  1. Procedures :

Please create a "labelmap.prototxt" file under the path : OpenR8/solution/Image_FaceRecognitionCelebrity/data.

The file content format is as follows: 

item {

name: "none_of_the_above"

label: 0

display_name: "background"

}

item {

name: " The name of the category "

label: 1

display_name: "display the name of category "

}

......

item {

name: " The name of the category "

label: n

display_name: "display the name of category "

}

 

Please note that the name and display_name above must be associated with the The category name and number of classifications within the XML file.

 

  1. Example:

In the openR8/solution/Image_ FaceRecognitionCelebrity/data folder labelmap.prototxt, you can open the file by notepad++ or Notepad, as shown in Fig. 15 below.

The contents of the file are shown in Fig. 16 below, and we can see that we classify the images into none_of_the_above (none of the above) and 14 person names, of which individual volume labels are numbered 0~14.

If you want to add a new category yourself, please add the category name and label number, as shown in Fig. 16 (right).

 

Fig. 15. The file path of Labelmap.prototxt..png

Fig. 15. The file path of Labelmap.prototxt.

 

Fig. 16. Labelmap content Diagram left add category diagram right ..png

Fig. 16. Labelmap content Diagram (left), add category diagram (right).

 

 

  1. Step 4 : Pre_processing - annoset_to_lmdb

 

The remaining steps are followed by using only OpenR8 software to run the flow file.

 

This step takes only two steps to load the flow file with click Run to convert the previously prepared file into the Lmdb file required for training.

 

  1. Loading flow files.

Please "mouse double-click R8.exe" => click "File" => "open" => "OpenR8 > solution>Image_FaceRecognitionCelebrity " => "1_annoset_to_ Lmdb.flow. " The diagram is shown in Fig. 17, Fig. 18, Fig. 19, Fig. 20.

For any questions about turning on the software to the load solution, refer to the "OpenR8 manual".

 

Fig. 17. R8.png

Fig. 17. R8.exe.

 

Fig. 18. Open File..jpg

Fig. 18. Open File.

 

Fig. 19. Select the 1_annoset_to_lmdb..png

Fig. 19. Select the 1_annoset_to_lmdb.

 

Fig. 20. Load the 1_prepare.flow..jpg

Fig. 20. Load the 1_prepare.flow.

 

  1. Run the flow file. Just click "Run" to start running the flow file to generate the Lmdb file required for training. See Fig. 21, Fig. 22 below.

 

Fig. 21. Run the 1_annoset_to_lmdb.flow..png

Fig. 21. Run the 1_annoset_to_lmdb.flow.

 

Fig. 22. Complete the execution 1_annoset_to_lmdb.flow. Generate Lmdb Files..png

Fig. 22. Complete the execution 1_annoset_to_lmdb.flow. Generate Lmdb Files.

 

 

  1. Step 5 : Training - 2_train

 

When the pre-preparation data is complete, start training the model.

 

  1. Open the flow.
  2. Use OpenR8 to "open the 2_train.flow" file and load the 2_train, as shown in Fig. 23,  Fig. 24 below.
  3. Running the flow.
  4. Press "execute" to start the training model, and this step takes a little time to wait for the program to build the model. See Fig. 25 below.

*ERROR! No GPU! The GPU string value must be empty! If the hardware does not support NVIDIA, change the GPU of the [Caffe_ training] function from [all] to [].

  1. The run is complete and the model is generated.
  2. When this run is complete, a trained model is generated, and the model file generation path is shown in Fig. 26 below. The path is:

solution/image_facerecognitioncelebrity/data/file name: XXX. Caffemodel

 

Fig. 23. The path of 2_train..png

Fig. 23. The path of 2_train.

 

Fig. 24. Open the 2_train.flow..jpg

Fig. 24. Open the 2_train.flow.

 

Fig. 25. Run the 2_train.flow..png

Fig. 25. Run the 2_train.flow.

 

Fig. 26. Run complete 2_train generated model file..png

Fig. 26. Run complete 2_train generated model file.

 

 

  1. Step 7: Inference using a trained model - inference

 

When the model has been trained, you can use this step to infer using a trained model.

 

  1. Open the flow file.

Use OpenR8 to turn on "3_inference_image.flow" and load the file. See Fig. 27, Fig. 28 below.

  1. Set up test images.

Then set the image that needs to be tested. The method is shown in Fig. 29 below.

The image path of the test: folder root directory /solution/Image _ FaceRecognitionCelebrity/data/image_test.

  1. Select the model you have trained. See Fig. 30 below.
  2. Run and display the results. See Fig. 31, Fig. 32 below.

*ERROR! No GPU! The GPU string value must be empty! If the hardware does not support NVIDIA, change the GPU of the two functions of the [Caffe_ ObjectDetect_ReadNet] and the [Caffe_ObjectDetect_InferenceImage] to [].

  1. If the hardware does not have a GPU, fill the GPU parameters with null values. SeeFig. 33, Fig. 34 below.

 

Fig. 27. 3_inference_image.png

Fig. 27. 3_inference_image.

 

Fig. 28. Load 3_inference_image.flow..jpg

Fig. 28. Load 3_inference_image.flow.

 

Fig. 29. 3_inference_image - Select the image you want to test..png

Fig. 29. 3_inference_image - Select the image you want to test.

 

Fig. 30. Select the model you want to test..png

Fig. 30. Select the model you want to test.

 

Fig. 31. 3_inference_image run test result..png

Fig. 31. 3_inference_image run test result.

 

Fig. 32. 3_inference_image debug test result..png

Fig. 32. 3_inference_image debug test result.

 

Fig. 33. GPU - Set all..png

Fig. 33. GPU - Set all.

 

Fig. 34. GPU Set null..png

Fig. 34. GPU Set null.

 

 

  1. Step8: Using webcam for inference

 

When the model has been trained, we can real-time image object detection and Identification with webcam through the “3_inference_webcam.flow”.

 

  1. Open the flow file

First use OpenR8 to open "8_inference_webcam". Loading flow. See Fig. 35, Fig. 36 below.

  1. Choosing a trained model. See Fig. 37 below.
  2. Running and Display results. See Fig. 38 below.

*ERROR! No GPU! The GPU string value must be empty! If the hardware does not support NVIDIA, change the GPU of the two functions of the [Caffe_ ObjectDetect_ReadNet] and the [Caffe_ObjectDetect_InferenceImage] to [].

 

Fig. 35. 3_inference_ webcam..png

Fig. 35. 3_inference_ webcam.

 

Fig. 36. Load 3_inference_webcam..jpg

Fig. 36. Load 3_inference_webcam.

 

Fig. 37. Select the model you want to test..png

Fig. 37. Select the model you want to test.

 

Fig. 38. 3_inference_webcam..png

Fig. 38. 3_inference_webcam.


Recommended Article

1.
OpenR8 Community Edition - AI Software for Everyone (Free Download)