Most of our config is in pyproject.toml, and so shouldn't need to be specified as arguments.

fyi to run I usually do:

mypy lib/matplotlib

(though mypy -m matplotlib will also work, that gets the runtime, and is closer to what stubtest does)

✔️ I edited a stub file to make sure that if I broke something there it flagged

❌ I edited a backends py file to see that it got ignored (it flagged that file, despite being in the excludes in mypy config)

❓ : I edited a gallery entry which did not previously have an error to introduce an error, wanting it to not check (currently a lot of of our galleries would fail, there has been no effort towards typing those files)

• it marks the test as Passed (not Skipped, which might be ideal, but okay with Passed)
• if I just run mypy on the file directly, I fail...
• I don't understand how it knows not to run on that, there is nothing actually in the pyproject.toml that says to run on only those paths (there are exclusions

⁉️ : I edited a gallery file that did previously have errors

• when I had multiple files edited, it did flag (for all three of the previous files...), but when I only had the one file edited, no flag

We could be explicit about which files we include on the hook, which would probably resolve all(?) of these (but I'm not a fan of duplicating the info there and in mypy's own config... just not sure how to have one source of truth) using a combination of files and exclude: https://pre-commit.com/#config-exclude

Finally, since its running in its own venv, we probably need to tell it to install dependencies that are referred to in stubs (it ignores them if they are missing, but that will deviate from behavior either running outside of pre-commit locally or the CI check, which is just confusing.

Not sure if you can just do additional_dependencies: ["."] to get all of the runtime dependencies without listing them individually... but also we want these ones at least:

which we interact with (some more than others)

We also pin mypy version right now, because there is a bug (that I reported) in recent versions of mypy that affects one file... leaning towards #type: ignoreing it because there hasn't been progress on the report, but current CI is on mypy==1.1.1, so that should be the rev we use here.

I guess it is also true that the hook, which only checks edited files, will not flag things if you cause e.g. child classes in other files to be be broken (CI will still see it, but it is a balancing act a bit there) this is likely relatively rare, but something that could be confusing

My major reason for wanting to check this locally (via pre-commit) is so that I don't have to rebase b/c every push is a new commit.
Also confusingly this is what I get when I start editing random files to induce error:

lib\matplotlib\colors.pyi:81: error: Overloaded function signature 3 will never be matched: signature 1's parameter type(s) are the same or broader  [misc]
lib\matplotlib\colors.pyi:165: error: Overloaded function signature 3 will never be matched: signature 2's parameter type(s) are the same or broader  [misc]
lib\matplotlib\colors.pyi:171: error: Overloaded function signature 3 will never be matched: signature 2's parameter type(s) are the same or broader  [misc]

I've gotten it to use the .toml config, but I have no idea how to deal with this error:

mypy: "lib\matplotlib\typing.py" shadows library module "typing"
note: A user-defined top-level module with name "typing" is not supported
mypy: "lib\matplotlib\typing.py" shadows library module "typing"
note: A user-defined top-level module with name "typing" is not supported

Ok, using the cuda mypy config showed me that I can just pass in where to look and not mess around with a mypy_path, which fixes that error.

Which means that this now is pointed to "matplotlib/lib" and pulls it's configuration from pyproject.toml

This form has the advantage of catching errors caused in other files, but the disadvantage of always taking the time to run on every file

Judging from the CI run, the time is not bad (~38 s without mypy, ~43 with mypy) I'm guessing checks run in parallel to some degree.

Overall I'd say i'm +0.5 on including this running against the whole codebase like it is (modulo config cleanup), but would like to hear if others have objections to the hook time going up.

This form has the advantage of catching errors caused in other files, but the disadvantage of always taking the time to run on every file

Yeah, that also seems like it's best practice and also the only way I've sorted to avoid:

lib\matplotlib\__init__.pyi: error: Duplicate module named "matplotlib" (also at "lib\matplotlib\__init__.py")
lib\matplotlib\__init__.pyi: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules for more info
lib\matplotlib\__init__.pyi: note: Common resolutions include: a) using `--exclude` to avoid checking one of them, b) adding `__init__.py` somewhere, c) using `--explicit-package-bases` or adjusting MYPYPATH
Found 1 error in 1 file (errors prevented further checking)

but also mypy only gets run if a .py file gets changed and I think (via some hack testing) setting files: lib/matplotlib causes it to only run when files in lib/matplotlib are changed.

Um did I break the stubtests?

eta: never mind, I broke pyproject.toml locally