Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit dd80942

Browse filesBrowse files
JoshuaKGoldbergJoshuaKGoldberg
committed
docs: reorganize README.md with comparisons/ directory
1 parent 28a1970 commit dd80942
Copy full SHA for dd80942
Expand file treeCollapse file tree

6 files changed

+204
-205
lines changed

‎README.md

Copy file name to clipboardExpand all lines: README.md
+10-205Lines changed: 10 additions & 205 deletions
Original file line numberDiff line numberDiff line change
@@ -40,25 +40,29 @@ The `caseEntries` values in `src/data.ts` can be modified to test:
4040

4141
## Results
4242

43-
Right now, `parserOptions.project` _with_ single-run inference outperforms `parserOptions.projectService`.
43+
Right now, `parserOptions.project` with single-run inference outperforms `parserOptions.projectService`.
4444
This is a performance issue and we are investigating it as a bug.
4545

4646
```plaintext
4747
┌───────┬───────────────────────┬───────────────────────┐
4848
│ files │ project (even layout) │ service (even layout) │
49-
┼───────┼───────────────────────┼───────────────────────┤
50-
│ 1024 │ '1.750 s ± 0.008 s' │ '2.473 s ± 0.011 s' │
51-
┴───────┴───────────────────────┴───────────────────────┘
49+
├───────┼───────────────────────┼───────────────────────┤
50+
│ 1024 │ '2.371 s ± 0.029 s' │ '2.724 s ± 0.049 s' │
51+
└───────┴───────────────────────┴───────────────────────┘
5252
```
5353

5454
See [typescript-eslint/typescript-eslint#9571 Performance: parserOptions.projectService no longer outperforms parserOptions.project](https://github.com/typescript-eslint/typescript-eslint/issues/9571) in typescript-eslint.
55-
Also see the 📌 pinned issues later in this file.
5655

5756
### Result Measurement Notes
5857

59-
- Example measurements taken on an M1 Max Mac Studio with Node.js 22.4.1
58+
- Example measurements taken on an M1 Max Mac Studio with Node.js 22.12.0
6059
- These results are similar across TypeScript versions: 5.0.4, 5.4.5, and 5.5.3
6160

61+
## Comparisons
62+
63+
The `comparisons/` directory contains details on more specific comparisons.
64+
See each `comparisons/*.md` file for details on what's being measured.
65+
6266
## Traces
6367

6468
The `traces/` directory contains more specific traces for investigations.
@@ -67,205 +71,6 @@ The `traces/` directory contains more specific traces for investigations.
6771
6872
All comparisons were run on a common shape of linting: 1024 files with the "even" (triangle-shaped) imports layout.
6973

70-
### Comparison: Globals in Scopes
71-
72-
> 📌 Filed on typescript-eslint as [⚡ Performance: Overhead of populateGlobalsFromLib in scope-manager](https://github.com/typescript-eslint/typescript-eslint/issues/9575).
73-
74-
This trace shows the impact of `@typescript-eslint/scope-manager`'s `populateGlobalsFromLib`.
75-
76-
See `traces/globals-scope-manager/`:
77-
78-
- `baseline.cpuprofile`: Baseline measurement with no changes
79-
- `skipping.cpuprofile`: Commenting out the contents of `populateGlobalsFromLib`
80-
81-
They were generated with:
82-
83-
```shell
84-
cd files-1024-layout-even-singlerun-true-types-service
85-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
86-
# clear ../../node_modules/@typescript-eslint/scope-manager/dist/referencer/Referencer.js > populateGlobalsFromLib
87-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
88-
```
89-
90-
Hyperfine measurements show a ~20% improvement in lint time:
91-
92-
| Variant | Measurement | User Time |
93-
| -------- | ----------------- | --------- |
94-
| Baseline | 3.137 s ± 0.024 s | 4.417 s |
95-
| Skipping | 2.477 s ± 0.014 s | 3.501 s |
96-
97-
### Comparison: Project and Project Service
98-
99-
This is a preliminary trace to start debugging their differences.
100-
101-
See `traces/Project 1 - Service 2.cpuprofile`.
102-
103-
- Trace #1: `parserOptions.project`
104-
- Trace #2: `parserOptions.projectService`
105-
106-
It was generated with:
107-
108-
```shell
109-
cd cases/files-1024-layout-even-singlerun-true-types-project
110-
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
111-
cd ../files-1024-layout-even-singlerun-true-types-service
112-
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
113-
```
114-
115-
Comparing equivalent code paths:
116-
117-
| Code Path | Project | Service |
118-
| ----------------- | ------- | ------- |
119-
| All `verifyText`s | 2040ms | 2859ms |
120-
| `parseForESLint` | 993ms | 1090ms |
121-
122-
### Comparison: Project Service Client File Cleanups
123-
124-
> 📌 Filed on TypeScript as [⚡ Performance: Project service spends excess time cleaning client files when called synchronously](https://github.com/microsoft/TypeScript/issues/59335).
125-
126-
This comparison shows the cost of the TypeScript project service calling `cleanupProjectsAndScriptInfos`.
127-
128-
See `traces/service-file-cleanup/`:
129-
130-
- `baseline.cpuprofile`: Baseline measurement with no changes
131-
- `skipping.cpuprofile`: Commenting out the contents of TypeScript's `cleanupProjectsAndScriptInfos`
132-
133-
They were generated with:
134-
135-
```shell
136-
cd files-1024-layout-even-singlerun-true-types-service
137-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
138-
# clear ../../node_modules/typescript/lib/typescript.js > cleanupProjectsAndScriptInfos
139-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
140-
```
141-
142-
Hyperfine measurements show a ~15-20% improvement in lint time:
143-
144-
| Variant | Measurement | User Time |
145-
| -------- | ----------------- | --------- |
146-
| Baseline | 3.215 s ± 0.041 s | 4.483 s |
147-
| Skipping | 2.501 s ± 0.017 s | 3.758 s |
148-
149-
### Comparison: Project Service Uncached File System Stats
150-
151-
> 📌 Filed on TypeScript as [⚡ Performance: Project service doesn't cache all fs.statSync](https://github.com/microsoft/TypeScript/issues/59338).
152-
153-
This comparison shows the cost uncached `fs.statSync` calls inside the project service.
154-
155-
See `traces/service-uncached-stats/`:
156-
157-
- `baseline.cpuprofile`: Baseline measurement with no changes
158-
- `caching.cpuprofile`: Adding a caching `Map` to TypeScript's `statSync`
159-
160-
They were generated with:
161-
162-
```shell
163-
cd files-1024-layout-even-singlerun-true-types-service
164-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
165-
# edit ../../node_modules/typescript/lib/typescript.js > statSync (see diff below)
166-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=caching.cpuprofile ../../node_modules/eslint/bin/eslint.js
167-
```
168-
169-
<details>
170-
<summary><code>diff</code> patch to switch to the <em>Caching</em> variant...</summary>
171-
172-
```diff
173-
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
174-
index 4baad59..44639d5 100644
175-
--- a/node_modules/typescript/lib/typescript.js
176-
+++ b/node_modules/typescript/lib/typescript.js
177-
@@ -8546,9 +8546,15 @@ var sys = (() => {
178-
}
179-
}
180-
};
181-
+ const statCache = new Map();
182-
return nodeSystem;
183-
function statSync(path) {
184-
- return _fs.statSync(path, { throwIfNoEntry: false });
185-
+ if (statCache.has(path)) {
186-
+ return statCache.get(path);
187-
+ }
188-
+ const result = _fs.statSync(path, { throwIfNoEntry: false });
189-
+ statCache.set(path, result);
190-
+ return result;
191-
}
192-
function enableCPUProfiler(path, cb) {
193-
if (activeSession) {
194-
```
195-
196-
</details>
197-
198-
Hyperfine measurements show a ~7-12% improvement in lint time:
199-
200-
| Variant | Measurement | User Time |
201-
| -------- | ----------------- | --------- |
202-
| Baseline | 3.112 s ± 0.033 s | 4.382 |
203-
| Caching | 2.740 s ± 0.030 s | 4.032 |
204-
205-
### Comparison: Project Service Uncached File System Path Reads
206-
207-
> 📌 Filed on TypeScript as [⚡ Performance: Project service doesn't cache all fs.realpath](https://github.com/microsoft/TypeScript/issues/59342).
208-
209-
This comparison shows the cost uncached `fs.realpath` calls inside the project service.
210-
211-
See `traces/service-uncached-realpaths/`:
212-
213-
- `baseline.cpuprofile`: Baseline measurement with no changes
214-
- `caching.cpuprofile`: Adding a caching `Map` to TypeScript's `realpath`
215-
216-
They were generated with:
217-
218-
```shell
219-
cd files-1024-layout-even-singlerun-true-types-service
220-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
221-
# edit ../../node_modules/typescript/lib/typescript.js > realpath (see diff below)
222-
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=caching.cpuprofile ../../node_modules/eslint/bin/eslint.js
223-
```
224-
225-
<details>
226-
<summary><code>diff</code> patch to switch to the <em>Caching</em> variant...</summary>
227-
228-
```diff
229-
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
230-
index 4baad59..e53476d 100644
231-
--- a/node_modules/typescript/lib/typescript.js
232-
+++ b/node_modules/typescript/lib/typescript.js
233-
@@ -13,6 +13,8 @@ See the Apache Version 2.0 License for specific language governing permissions
234-
and limitations under the License.
235-
***************************************************************************** */
236-
237-
+var realpathCache = new Map();
238-
+
239-
var ts = {}; ((module) => {
240-
"use strict";
241-
var __defProp = Object.defineProperty;
242-
@@ -8798,6 +8800,15 @@ var sys = (() => {
243-
return path.length < 260 ? _fs.realpathSync.native(path) : _fs.realpathSync(path);
244-
}
245-
function realpath(path) {
246-
+ const cached = realpathCache.get(path);
247-
+ if (cached) {
248-
+ return cached;
249-
+ }
250-
+ const result = realpathWorker(path);
251-
+ realpathCache.set(path, result);
252-
+ return result;
253-
+ }
254-
+ function realpathWorker(path) {
255-
try {
256-
return fsRealpath(path);
257-
} catch {
258-
```
259-
260-
</details>
261-
262-
Hyperfine measurements with `--runs 50` show a ~0.5-2.5% improvement in lint time:
263-
264-
| Variant | Measurement | User Time |
265-
| -------- | ----------------- | --------- |
266-
| Baseline | 3.153 s ± 0.039 s | 4.403 s |
267-
| Caching | 3.073 s ± 0.048 s | 4.377 s |
268-
26974
## Contributors
27075

27176
<!-- spellchecker: disable -->

‎comparisons/globals-in-scopes.md

Copy file name to clipboard
+26Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Comparison: Globals in Scopes
2+
3+
> 📌 Filed on typescript-eslint as [⚡ Performance: Overhead of populateGlobalsFromLib in scope-manager](https://github.com/typescript-eslint/typescript-eslint/issues/9575).
4+
5+
This trace shows the impact of `@typescript-eslint/scope-manager`'s `populateGlobalsFromLib`.
6+
7+
See `traces/globals-scope-manager/`:
8+
9+
- `baseline.cpuprofile`: Baseline measurement with no changes
10+
- `skipping.cpuprofile`: Commenting out the contents of `populateGlobalsFromLib`
11+
12+
They were generated with:
13+
14+
```shell
15+
cd files-1024-layout-even-singlerun-true-types-service
16+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
17+
# clear ../../node_modules/@typescript-eslint/scope-manager/dist/referencer/Referencer.js > populateGlobalsFromLib
18+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
19+
```
20+
21+
Hyperfine measurements show a ~20% improvement in lint time:
22+
23+
| Variant | Measurement | User Time |
24+
| -------- | ----------------- | --------- |
25+
| Baseline | 3.137 s ± 0.024 s | 4.417 s |
26+
| Skipping | 2.477 s ± 0.014 s | 3.501 s |

‎comparisons/project-and-project-service.md

Copy file name to clipboard
+24Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# Comparison: Project and Project Service
2+
3+
This is a preliminary trace to start debugging their differences.
4+
5+
See `traces/Project 1 - Service 2.cpuprofile`.
6+
7+
- Trace #1: `parserOptions.project`
8+
- Trace #2: `parserOptions.projectService`
9+
10+
It was generated with:
11+
12+
```shell
13+
cd cases/files-1024-layout-even-singlerun-true-types-project
14+
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
15+
cd ../files-1024-layout-even-singlerun-true-types-service
16+
node --inspect-brk ../../node_modules/eslint/bin/eslint.js
17+
```
18+
19+
Comparing equivalent code paths:
20+
21+
| Code Path | Project | Service |
22+
| ----------------- | ------- | ------- |
23+
| All `verifyText`s | 2040ms | 2859ms |
24+
| `parseForESLint` | 993ms | 1090ms |

‎comparisons/project-service-client-file-cleanups.md

Copy file name to clipboard
+26Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
# Comparison: Project Service Client File Cleanups
2+
3+
> 📌 Filed on TypeScript as [⚡ Performance: Project service spends excess time cleaning client files when called synchronously](https://github.com/microsoft/TypeScript/issues/59335).
4+
5+
This comparison shows the cost of the TypeScript project service calling `cleanupProjectsAndScriptInfos`.
6+
7+
See `traces/service-file-cleanup/`:
8+
9+
- `baseline.cpuprofile`: Baseline measurement with no changes
10+
- `skipping.cpuprofile`: Commenting out the contents of TypeScript's `cleanupProjectsAndScriptInfos`
11+
12+
They were generated with:
13+
14+
```shell
15+
cd files-1024-layout-even-singlerun-true-types-service
16+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
17+
# clear ../../node_modules/typescript/lib/typescript.js > cleanupProjectsAndScriptInfos
18+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=skipping.cpuprofile ../../node_modules/eslint/bin/eslint.js
19+
```
20+
21+
Hyperfine measurements show a ~15-20% improvement in lint time:
22+
23+
| Variant | Measurement | User Time |
24+
| -------- | ----------------- | --------- |
25+
| Baseline | 3.215 s ± 0.041 s | 4.483 s |
26+
| Skipping | 2.501 s ± 0.017 s | 3.758 s |

‎comparisons/project-service-uncached-file-system-path-reads.md

Copy file name to clipboard
+63Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
# Comparison: Project Service Uncached File System Path Reads
2+
3+
> 📌 Filed on TypeScript as [⚡ Performance: Project service doesn't cache all fs.realpath](https://github.com/microsoft/TypeScript/issues/59342).
4+
5+
This comparison shows the cost uncached `fs.realpath` calls inside the project service.
6+
7+
See `traces/service-uncached-realpaths/`:
8+
9+
- `baseline.cpuprofile`: Baseline measurement with no changes
10+
- `caching.cpuprofile`: Adding a caching `Map` to TypeScript's `realpath`
11+
12+
They were generated with:
13+
14+
```shell
15+
cd files-1024-layout-even-singlerun-true-types-service
16+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=baseline.cpuprofile ../../node_modules/eslint/bin/eslint.js
17+
# edit ../../node_modules/typescript/lib/typescript.js > realpath (see diff below)
18+
node --cpu-prof --cpu-prof-interval=100 --cpu-prof-name=caching.cpuprofile ../../node_modules/eslint/bin/eslint.js
19+
```
20+
21+
<details>
22+
<summary><code>diff</code> patch to switch to the <em>Caching</em> variant...</summary>
23+
24+
```diff
25+
diff --git a/node_modules/typescript/lib/typescript.js b/node_modules/typescript/lib/typescript.js
26+
index 4baad59..e53476d 100644
27+
--- a/node_modules/typescript/lib/typescript.js
28+
+++ b/node_modules/typescript/lib/typescript.js
29+
@@ -13,6 +13,8 @@ See the Apache Version 2.0 License for specific language governing permissions
30+
and limitations under the License.
31+
***************************************************************************** */
32+
33+
+var realpathCache = new Map();
34+
+
35+
var ts = {}; ((module) => {
36+
"use strict";
37+
var __defProp = Object.defineProperty;
38+
@@ -8798,6 +8800,15 @@ var sys = (() => {
39+
return path.length < 260 ? _fs.realpathSync.native(path) : _fs.realpathSync(path);
40+
}
41+
function realpath(path) {
42+
+ const cached = realpathCache.get(path);
43+
+ if (cached) {
44+
+ return cached;
45+
+ }
46+
+ const result = realpathWorker(path);
47+
+ realpathCache.set(path, result);
48+
+ return result;
49+
+ }
50+
+ function realpathWorker(path) {
51+
try {
52+
return fsRealpath(path);
53+
} catch {
54+
```
55+
56+
</details>
57+
58+
Hyperfine measurements with `--runs 50` show a ~0.5-2.5% improvement in lint time:
59+
60+
| Variant | Measurement | User Time |
61+
| -------- | ----------------- | --------- |
62+
| Baseline | 3.153 s ± 0.039 s | 4.403 s |
63+
| Caching | 3.073 s ± 0.048 s | 4.377 s |

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.