A very light Python package to send mouse and keyboard actions to wayland and get screenshots as numpy arrays, throught Rust.
It supports deterministic actions:
import logging
from wdotool import Wdotool
logging.basicConfig(level=logging.INFO)
w = Wdotool()
w.move_mouse(x_extent=2560, y_extent=1440, x=100, y=100)
w.right_click(duration_ms=10)
screen_image = w.screenshot()
assert screen_image.shape == (1440, 2560, 4)Or statistical actions:
import logging
from wdotool import Wdotool
logging.basicConfig(level=logging.INFO)
w = Wdotool("wayland-2")
w.move_mouse(x_extent=2560, y_extent=1440, x=100, y=100, x_max=120, y_max=120)
w.right_click(duration_ms=10, duration_ms_max=20)
screen_image = w.screenshot("HDMI-A-1")
assert screen_image.shape == (1440, 2560, 4)If parameters {p_name} and {p_name}_max are defined, it will draw a random value in the range [{p_name}, {p_name}_max], following a normal distribution of mean ({p_name} + {p_name}_max)/2 and standard variation ({p_name}_max - {p_name})/2.
The python package doesn't have any dependencies, except numpy, if you wish to do screenshots.
Your Wayland compositor need to support 3 protocols:
- virtual-keyboard-unstable-v1, version 1
- wlr-screencopy-unstable-v1, version 3
- wlr-virtual-pointer-unstable-v1 version 2
This solution has been tested under Hyprland and Cage, but according to the compatibility lists, should also work under Sway and Mir
This package follows NEP 29 and supports Python versions 3.11+ and Numpy versions 2.0+