[OpenR8 solution] Image_MoveMouseThenScreenShot (Mouse control and image screenshot analysis)
  1. Introduction

 

This section describes how users can load an existing solution through the R8 software to automatically click on the next image, get each image for analysis and detection, and finally save the analysis results image to the native side.

Step 1. Set the window you want to get.

 

  1. Open the sample 1.png of the input folder with the Windows built-in Photos app. Move the window screen to the upper-left corner. See Fig. 1, Fig. 2 below.
  2. Locate the [next page] button coordinates of the Photos App (x, y coordinates on the full screen). See Fig. 3 below. (You can use Windows built-in Snipping tool to make screenshots and then measure coordinates with windows built-in Paint).
  3. Follow step 2) to confirm the position of the image on the full screen, the upper-left coordinate x and y, and image size (width、height). See Fig. 4, Fig. 5 below. 

 

Note: Use the screenshot software to get the coordinates, multiply by a screen zoom magnification. Therefore, when the screen display scale is not 100%, the actual screen coordinates need to be divided by that value. For example, if the screen display scale size is 120%, the coordinate and width and height must be divided by 1.2.

 

Fig. 1. Open image..png

Fig. 1. Open image.

 

Fig. 2. The image window is placed in the upper-left corner of the screen..png

Fig. 2. The image window is placed in the upper-left corner of the screen.

 

Fig. 3. Get the Next button position in the picture on the screen..png

Fig. 3. Get the Next button position in the picture on the screen.

 

Fig. 4. .png

Fig. 4. Confirm the position of the open picture in the screen window (upper-left corner).

 

Fig. 5..png

Fig. 5. Confirm the position of the open picture in the screen window (width and height).

 

Step 2. Sets the position of the command prompt on the screen. So that the command prompt window does not cover the Photos App. See Fig. 6, Fig. 7, Fig. 8 below.

 

Fig. 6. ..png

Fig. 6. Open dos window.

 

Fig. 7...png

Fig. 7. Properties of the DOS window.

 

Fig. 8. ..png

Fig. 8. Adjust DOS window size and position.

 

Step 3. Run R8.exe,Open the flow.

 

  1. Please double-click to turn on R8. The picture is shown in Fig. 9 below.
  2. Click "File" => "open" above R8. The picture is shown in Fig. 10 below.
  3. Select the Image_movemousethenscreenshot folder under solution. See Fig. 11 below.
  4. Select and open the flow file. See Fig. 12, Fig. 13 below.

 

Fig. 9. ..png

Fig. 9. 【Run R8.exe】

 

Fig. 10. ..png

Fig. 10. 【File】=>【Open】.

 

Fig. 11..png

Fig. 11. Solution Folder.

 

Fig. 12. ..png

Fig. 12. Select a flow file.

 

Fig. 13. ..jpg

Fig. 13. Open flow file.

 

Step 4. Fill in the X and Y coordinates of the step 1 measurement [next page] button into the values of Variable positionX and positionY, respectively. See Fig. 14 below.

 

 

Fig. 14. .png

Fig. 14. Fill in the simulation mouse click Position.

 

Step 5. Change the x, y, w, and h values of the “Image_Crop” to the upper-left coordinate and the width and height of the Step 1 measurement detection area. See Fig. 15 below.

 

Fig. 15...png

Fig. 15. Diagram of detecting area coordinates.

 

Step 6. Press run in OpenR8 to get the program running. Note that OpenR8 can not be full screen, otherwise it will cover the Photos App window, in addition to the program run do not move the mouse. See Fig. 16 below.

 

Fig. 16. .png

Fig. 16. Run flow.

 

Step 7. The image results are stored in the data folder, and the analysis results are stored in the Inference folder. See Fig. 17 below.

 

Fig. 17. . .png

Fig. 17. Stored result.

 

 

  1. Flow

 

The process for this solution. Fig. 18 below.

If you want to modify the process yourself, you can modify it through the corresponding function box.

 

Fig. 18. .png

Fig. 18. Flow chart.

 

 


Recommended Article

1.
OpenR8 Community Edition - AI Software for Everyone (Free Download)