Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit c84e643

Browse filesBrowse files
DaveLakDaveLak
committed
Replace the suboptimal fuzz_tree harness with a better alternative
As discussed in the initial fuzzing integration PR[^1], `fuzz_tree.py`'s implementation was not ideal in terms of coverage and its reading/writing to hard-coded paths inside `/tmp` was problematic as (among other concerns), it causes intermittent crashes on ClusterFuzz[^2] when multiple workers execute the test at the same time on the same machine. The changes here replace `fuzz_tree.py` completely with a completely new `fuzz_repo.py` fuzz target which: - Uses `tempfile.TemporaryDirectory()` to safely manage tmpdir creation and tear down, including during multi-worker execution runs. - Retains the same feature coverage as `fuzz_tree.py`, but it also adds considerably more from much smaller data inputs and with less memory consumed (and it doesn't even have a seed corpus or target specific dictionary yet.) - Can likely be improved further in the future by exercising additional features of `Repo` to the harness. Because `fuzz_tree.py` was removed and `fuzz_repo.py` was not derived from it, the Apache License call outs in the docs were also updated as they only apply to the singe `fuzz_config.py` file now. [^1]: #1901 (comment) [^2]: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68355
1 parent 797009d commit c84e643
Copy full SHA for c84e643

File tree

5 files changed

+57
-90
lines changed
Filter options

5 files changed

+57
-90
lines changed

‎README.md

Copy file name to clipboardExpand all lines: README.md
+2-2Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -240,8 +240,8 @@ Please have a look at the [contributions file][contributing].
240240

241241
[3-Clause BSD License](https://opensource.org/license/bsd-3-clause/), also known as the New BSD License. See the [LICENSE file][license].
242242

243-
Two files exclusively used for fuzz testing are subject to [a separate license, detailed here](./fuzzing/README.md#license).
244-
These files are not included in the wheel or sdist packages published by the maintainers of GitPython.
243+
One file exclusively used for fuzz testing is subject to [a separate license, detailed here](./fuzzing/README.md#license).
244+
This file is not included in the wheel or sdist packages published by the maintainers of GitPython.
245245

246246
[contributing]: https://github.com/gitpython-developers/GitPython/blob/main/CONTRIBUTING.md
247247
[license]: https://github.com/gitpython-developers/GitPython/blob/main/LICENSE

‎fuzzing/README.md

Copy file name to clipboardExpand all lines: fuzzing/README.md
+8-8Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -225,14 +225,14 @@ to [the official OSS-Fuzz documentation][oss-fuzz-docs].
225225
## LICENSE
226226

227227
All files located within the `fuzzing/` directory are subject to [the same license](../LICENSE)
228-
as [the other files in this repository](../README.md#license) with two exceptions:
229-
230-
Two files located in this directory, [`fuzz_config.py`](./fuzz-targets/fuzz_config.py)
231-
and [`fuzz_tree.py`](./fuzz-targets/fuzz_tree.py), have been migrated here from the OSS-Fuzz project repository where
232-
they were originally created. As such, these two files retain their original license and copyright notice (Apache
233-
License, Version 2.0 and Copyright 2023 Google LLC respectively.) Each file includes a notice in their respective header
234-
comments stating that they have been modified. [LICENSE-APACHE](./LICENSE-APACHE) contains the original license used by
235-
the OSS-Fuzz project repository at the time they were migrated.
228+
as [the other files in this repository](../README.md#license) with one exception:
229+
230+
[`fuzz_config.py`](./fuzz-targets/fuzz_config.py) was migrated to this repository from the OSS-Fuzz project's repository
231+
where it was originally created. As such, [`fuzz_config.py`](./fuzz-targets/fuzz_config.py) retains its original license
232+
and copyright notice (Apache License, Version 2.0 and Copyright 2023 Google LLC respectively) as in a header
233+
comment, followed by a notice stating that it has have been modified contributors to GitPython.
234+
[LICENSE-APACHE](./LICENSE-APACHE) contains the original license used by the OSS-Fuzz project repository at the time the
235+
file was migrated.
236236

237237
[oss-fuzz-repo]: https://github.com/google/oss-fuzz
238238

‎fuzzing/dictionaries/fuzz_tree.dict

Copy file name to clipboardExpand all lines: fuzzing/dictionaries/fuzz_tree.dict
-13Lines changed: 0 additions & 13 deletions
This file was deleted.

‎fuzzing/fuzz-targets/fuzz_repo.py

Copy file name to clipboard
+47Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
import atheris
2+
import io
3+
import sys
4+
import os
5+
import tempfile
6+
7+
if getattr(sys, "frozen", False) and hasattr(sys, "_MEIPASS"):
8+
path_to_bundled_git_binary = os.path.abspath(os.path.join(os.path.dirname(__file__), "git"))
9+
os.environ["GIT_PYTHON_GIT_EXECUTABLE"] = path_to_bundled_git_binary
10+
11+
with atheris.instrument_imports():
12+
import git
13+
14+
15+
def TestOneInput(data):
16+
fdp = atheris.FuzzedDataProvider(data)
17+
18+
with tempfile.TemporaryDirectory() as temp_dir:
19+
repo = git.Repo.init(path=temp_dir)
20+
21+
# Generate a minimal set of files based on fuzz data to minimize I/O operations.
22+
file_paths = [os.path.join(temp_dir, f"File{i}") for i in range(min(3, fdp.ConsumeIntInRange(1, 3)))]
23+
for file_path in file_paths:
24+
with open(file_path, "wb") as f:
25+
# The chosen upperbound for count of bytes we consume by writing to these
26+
# files is somewhat arbitrary and may be worth experimenting with if the
27+
# fuzzer coverage plateaus.
28+
f.write(fdp.ConsumeBytes(fdp.ConsumeIntInRange(1, 512)))
29+
30+
repo.index.add(file_paths)
31+
repo.index.commit(fdp.ConsumeUnicodeNoSurrogates(fdp.ConsumeIntInRange(1, 80)))
32+
33+
fuzz_tree = git.Tree(repo, git.Tree.NULL_BIN_SHA, 0, "")
34+
35+
try:
36+
fuzz_tree._deserialize(io.BytesIO(data))
37+
except IndexError:
38+
return -1
39+
40+
41+
def main():
42+
atheris.Setup(sys.argv, TestOneInput)
43+
atheris.Fuzz()
44+
45+
46+
if __name__ == "__main__":
47+
main()

‎fuzzing/fuzz-targets/fuzz_tree.py

Copy file name to clipboardExpand all lines: fuzzing/fuzz-targets/fuzz_tree.py
-67Lines changed: 0 additions & 67 deletions
This file was deleted.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.