i need python libraries to make a bot that will recognize an image of the button and click on it, button’s position is randomized every click.
it should collect the image and click by itself, without user interaction
i have used OpenCV in the past, but i do not know how to automate the clicking on the screen
>Solution :
To automate clicking on the screen based on the recognition of a button in an image, you can use the pyautogui library to simulate mouse clicks. Here’s a simple example using Python, OpenCV for image recognition, and pyautogui for automating the mouse clicks:
import cv2
import pyautogui
import numpy as np
import time
# Function to find the position of the button in the screenshot
def find_button(screen, button_template):
result = cv2.matchTemplate(screen, button_template, cv2.TM_CCOEFF_NORMED)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result)
return max_loc
# Function to simulate a mouse click at the specified position
def click_button(position):
pyautogui.click(position[0], position[1])
# Path to the screenshot of the screen
screenshot_path = 'path/to/screenshot.png'
# Path to the template image of the button
button_template_path = 'path/to/button_template.png'
# Load the images
screenshot = cv2.imread(screenshot_path)
button_template = cv2.imread(button_template_path)
# Find the button in the screenshot
button_position = find_button(screenshot, button_template)
# Click the button
click_button(button_position)