Selenium is an open source automated testing suite for web applications across different browsers and platforms. Selenium is not a single tool but four different software applications:
- Selenium Integrated Development Environment (IDE)
- Selenium Remote Control (RC)
- WebDriver
- Selenium Grid
This tutorial demonstrates how to use the Webdriver with Python. In the tutorial you will see commands that allow you navigate to predefined url's and links that on the website, scroll the page, send text to fields, and open a new tab.
It is also important to note good scraping protocols:
- Don't submit requests that will overload their servers
- Larger websites will most likely have scraping defenses
- Read the site owner rules for robots at mainURL/robots.txt
https://selenium-python.readthedocs.io/
https://www.guru99.com/introduction-to-selenium.html