It appears to be currently impossible to add a benchmark that uses a feature added in Python >3.7. Even if you add the requires-python field in the pyproject.toml file for a specific benchmark, the benchmark is run on all Python versions in CI. This means that the CI will fail if the benchmark includes a feature that is new in 3.8, for example.

This issue currently blocks both of these open PRs:

• Add a benchmark for typeshed-stats #268
• Add a benchmark for runtime-checkable protocols #280