Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 2b41bf8

Browse filesBrowse files
authored
devops(driver): assemble driver from npm package and Node.js builds (#3122)
1 parent f75d727 commit 2b41bf8
Copy full SHA for 2b41bf8

16 files changed

+434-292Lines changed: 434 additions & 292 deletions

File tree

Expand file treeCollapse file tree
Open diff view settings
Filter options
Expand file treeCollapse file tree
Open diff view settings
Collapse file

‎.claude/skills/playwright-roll/SKILL.md‎

Copy file name to clipboardExpand all lines: .claude/skills/playwright-roll/SKILL.md
+47-30Lines changed: 47 additions & 30 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: Roll Playwright Python to a new driver version. Walks the upstream
55

66
# Rolling Playwright Python
77

8-
The goal of a roll is to move the driver pin in `DRIVER_SHA` to a new release, port every public API change introduced upstream during that interval, and suppress the rest, so that `./scripts/update_api.sh` runs clean and the test suite still passes.
8+
The goal of a roll is to move the driver pin in `DRIVER_VERSION` to a new release, port every public API change introduced upstream during that interval, and suppress the rest, so that `./scripts/update_api.sh` runs clean and the test suite still passes.
99

1010
The previous human-facing summary lives in `../../../ROLLING.md`. This skill is the operational playbook — read it end to end before starting.
1111

@@ -15,7 +15,7 @@ The Python port is hand-written code in `playwright/_impl/`, plus a generator (`
1515

1616
1. introspects the Python `_impl` classes via `inspect`,
1717
2. emits typed wrapper classes into `playwright/{async,sync}_api/_generated.py`, and
18-
3. diffs the introspected surface against `playwright/driver/package/api.json` (built into the new driver from source).
18+
3. diffs the introspected surface against the Playwright `api.json` (generated from the upstream docs — see step 2).
1919

2020
Anything in `api.json` that is missing or differently typed in `_impl/` causes generation to fail. Three resolutions:
2121

@@ -37,37 +37,49 @@ The upstream documentation source of truth is `docs/src/api/*.md` in the playwri
3737
- If `python3-venv` is missing system-wide, use `uv venv env` instead, then `uv pip install --python env/bin/python --upgrade pip`. Don't try to `apt install` — sudo is denied in the harness.
3838
- Always activate the venv before any `pip`, `pytest`, `mypy`, or `pre-commit` invocation.
3939

40-
### 2. Bump the driver and build it from source
40+
### 2. Bump the driver pin, download it, and generate api.json
41+
42+
You need a nearby `microsoft/playwright` checkout for the docs walk and for
43+
`api.json` generation. Point `PW_SRC_DIR` at it and check out the new tag there:
4144

4245
```sh
43-
# Edit DRIVER_SHA (repo root): replace with the microsoft/playwright commit SHA
44-
# for the new release, e.g. the commit that v<new> points at.
45-
# 87bb9ddbd78f329df18c2b24847bc9409240cd07
46-
# Update the "# microsoft/playwright @ v<new>" comment in scripts/build_driver.sh too.
46+
export PW_SRC_DIR=../playwright
47+
git -C "$PW_SRC_DIR" fetch --tags origin
48+
git -C "$PW_SRC_DIR" checkout v<new> # e.g. v1.62.0
49+
```
50+
51+
Then bump the pins and assemble the driver:
52+
53+
```sh
54+
# Edit DRIVER_VERSION (repo root): the playwright-core npm version for the new
55+
# release, no "v" prefix, e.g. 1.62.0
56+
python scripts/update_node_version.py # refresh NODE_VERSION to the current LTS
4757

4858
source env/bin/activate
49-
python -m build --wheel # clones microsoft/playwright @ DRIVER_SHA and builds the driver from source
59+
python -m build --wheel # downloads playwright-core @ DRIVER_VERSION + Node.js, assembles the driver
5060
playwright install chromium # NOT --with-deps; sudo is denied
61+
62+
# api.json isn't in the bundle, and the walk below inspects `langs` from it.
63+
# Generate a copy to inspect (update_api.sh generates its own temp copy in step 6):
64+
API_JSON_MODE=1 node "$PW_SRC_DIR/utils/doclint/generateApiJson.js" > /tmp/api.json
5165
```
5266

53-
The wheel build clones `microsoft/playwright` at the commit in `DRIVER_SHA`
54-
into `driver/playwright-src`, runs `npm ci && npm run build`, and runs upstream's
55-
`utils/build/build-playwright-driver.sh` to produce the per-platform driver
56-
bundles (`driver/playwright-<sha>-*.zip`), then unpacks the driver under
57-
`playwright/driver/package/`. From this point,
58-
`playwright/driver/package/api.json` reflects the new release. This requires
59-
**Node.js, npm, git and bash** on PATH; the first build is slow (full upstream
60-
build + per-platform Node downloads).
67+
The wheel build just downloads the `playwright-core` npm package at
68+
`DRIVER_VERSION` and the matching Node.js binary (no source build, no Node/npm/git
69+
toolchain), and unpacks the driver under `playwright/driver/`. `api.json` is the
70+
one piece not shipped in the bundle — it's generated from `$PW_SRC_DIR` on demand
71+
(here to `/tmp/api.json` for the walk, and into a temp file passed via
72+
`PW_API_JSON` by `./scripts/update_api.sh` during codegen).
6173

6274
### 3. Identify the commit range
6375

64-
The build step (step 2) clones the upstream monorepo into `driver/playwright-src`.
76+
Use the nearby `microsoft/playwright` checkout at `$PW_SRC_DIR` (from step 2).
6577
Bring it up to date and ensure release branches/tags are present before walking
6678
the range:
6779

6880
```sh
69-
git -C driver/playwright-src fetch --tags
70-
git -C driver/playwright-src fetch origin 'release-*:release-*'
81+
git -C "$PW_SRC_DIR" fetch --tags
82+
git -C "$PW_SRC_DIR" fetch origin 'release-*:release-*'
7183
```
7284

7385
There is sometimes no `vX.Y.0` tag for the latest release (the bots cut release branches first and tag later). Anchor on commits, not tags.
@@ -76,14 +88,14 @@ The diff range is "every commit on the new release branch since the previous rel
7688

7789
- **Previous release end**: the `chore: bump version to vX.Y.0-next` commit on `main`. That commit is the first commit *after* the previous release (X.Y-1) was cut. Use its parent (`<sha>~1`) as the lower bound.
7890
```sh
79-
git -C driver/playwright-src log --all --grep="bump version to v" --oneline | head
91+
git -C "$PW_SRC_DIR" log --all --grep="bump version to v" --oneline | head
8092
```
8193
- **New release end**: the tip of `release-<new>` (or the matching tag if it exists).
8294

8395
Save the commit list, oldest first, scoped to `docs/src/api/`:
8496

8597
```sh
86-
git -C driver/playwright-src log <prev-anchor>~1..release-<new> --oneline --reverse -- docs/src/api > /tmp/roll-<new>-commits.md
98+
git -C "$PW_SRC_DIR" log <prev-anchor>~1..release-<new> --oneline --reverse -- docs/src/api > /tmp/roll-<new>-commits.md
8799
```
88100

89101
A normal roll yields 50–100 commits. If you see 0 or thousands, the range is wrong.
@@ -95,7 +107,7 @@ Format the file as a markdown checklist and add the standard preamble (status le
95107
For each commit, in chronological order:
96108

97109
```sh
98-
git -C driver/playwright-src show <sha> -- docs/src/api/
110+
git -C "$PW_SRC_DIR" show <sha> -- docs/src/api/
99111
```
100112

101113
Look for:
@@ -113,7 +125,7 @@ Before tagging anything as MISMATCH or N/A based on appearance, dump the actual
113125

114126
```python
115127
import json
116-
data = json.load(open("playwright/driver/package/api.json"))
128+
data = json.load(open("/tmp/api.json"))
117129
classes = {c["name"]: c for c in data}
118130
for cls_name in ["Page", "BrowserContext", "Screencast", "Debugger"]:
119131
cls = classes.get(cls_name)
@@ -140,7 +152,7 @@ A few rules of thumb that catch most "actually a PORT" cases:
140152

141153
#### PORT
142154

143-
Implement the change in `playwright/_impl/<module>.py`. Use the upstream JS implementation as a reference: `driver/playwright-src/packages/playwright-core/src/client/<module>.ts`. Translate idioms:
155+
Implement the change in `playwright/_impl/<module>.py`. Use the upstream JS implementation as a reference: `$PW_SRC_DIR/packages/playwright-core/src/client/<module>.ts`. Translate idioms:
144156

145157
| Upstream JS | Python |
146158
|---|---|
@@ -205,13 +217,18 @@ Tick the box in `/tmp/roll-<new>-commits.md` with one line: `[x] <sha> <subject>
205217
### 5. Regenerate
206218

207219
```sh
208-
./scripts/update_api.sh
220+
PW_SRC_DIR=../playwright ./scripts/update_api.sh # PW_SRC_DIR already exported in step 2
209221
```
210222

211223
The script does, in order:
212-
1. `git checkout HEAD -- playwright/{async,sync}_api/_generated.py` (resets to last committed),
213-
2. runs `scripts/generate_{sync,async}_api.py` which dumps to `.x` then renames into place,
214-
3. invokes `pre-commit run --files` on the generated files.
224+
1. generates `api.json` from `$PW_SRC_DIR` into a temp file and exports `PW_API_JSON`,
225+
2. `git checkout HEAD -- playwright/{async,sync}_api/_generated.py` (resets to last committed),
226+
3. runs `scripts/generate_{sync,async}_api.py` (they read `api.json` via `PW_API_JSON`), dumping to `.x` then renaming into place,
227+
4. invokes `pre-commit run --files` on the generated files.
228+
229+
**CI no longer verifies that `_generated.py` is in sync** (the Lint job dropped the
230+
"Verify generated API is up to date" step so it needn't check out upstream). So
231+
regenerating here and committing the result is on you — don't skip it.
215232

216233
Failure modes and fixes:
217234

@@ -245,7 +262,7 @@ For each PORT, add one async test and a matching sync test. Conventions:
245262

246263
### 7. Update existing high-touch artifacts
247264

248-
- `DRIVER_SHA` (and the version comment in `scripts/build_driver.sh`): already done in step 2.
265+
- `DRIVER_VERSION` and `NODE_VERSION`: already done in step 2.
249266
- `README.md`: gets the chromium/firefox/webkit version table updated automatically by `scripts/update_versions.py` (called from `update_api.sh`). Don't edit by hand.
250267
- The "Backport changes" tracking issue on GitHub (filed by `microsoft-playwright-automation`) is the *intent* tracker, but it's frequently out of sync with what's actually been ported. Treat it as a starting point, not the source of truth — the `docs/src/api/` commit walk is authoritative.
251268

@@ -281,7 +298,7 @@ Class names use the upstream PascalCase (`BrowserContext`, `BrowserType`); metho
281298
- **A cluster of suppressions on the same class is a smell.** If you're about to add five `Method not implemented: Foo.*` lines, you're almost certainly looking at a class that needs to be implemented. Implement the whole thing once and the suppressions disappear.
282299
- **Watch for revert pairs in the same range.** 1.59 added and reverted `Browser.isRemote` (#39613 / #39620) inside the same release. Walking chronologically lets you skip the add when you see the revert later.
283300
- **Watch for rename-revert pairs.** 1.59 had `Locator.normalize``Locator.toCode` (#39648) → `Locator.normalize` (#39754). Final state wins; only port the last.
284-
- **Doc renames almost always include a wire-protocol rename.** Whenever you see `### param: X.y.oldName``### param: X.y.newName` in a doc commit, also `git -C driver/playwright-src show <sha> -- packages/protocol/src/protocol.yml` and the corresponding `*Dispatcher.ts` file. If the wire field changed too, the channel-send dict key in `_impl/` must change. Suppressing the doc-side mismatch is hiding a real bug — the previous Python code is silently sending an unknown field that the new server ignores.
301+
- **Doc renames almost always include a wire-protocol rename.** Whenever you see `### param: X.y.oldName``### param: X.y.newName` in a doc commit, also `git -C "$PW_SRC_DIR" show <sha> -- packages/protocol/src/protocol.yml` and the corresponding `*Dispatcher.ts` file. If the wire field changed too, the channel-send dict key in `_impl/` must change. Suppressing the doc-side mismatch is hiding a real bug — the previous Python code is silently sending an unknown field that the new server ignores.
285302
- **TypedDicts beat `Dict[str, X]` for any structured return.** When the docs describe a return as `[Object]` with named fields (or even `[Object=Foo]`), define a `TypedDict` in `_api_structures.py`, re-export from both public `__init__.py` files, and use it. Zero runtime cost (it's still a `dict`), and the doc generator's type comparator matches by structure via `get_type_hints`.
286303
- **Positional renames are free.** A param with no default before any `*` separator is positional-or-keyword in Python, but realistic call sites pass it positionally. Renaming such a param doesn't break callers.
287304
- **The "Backport changes" GitHub issue can be misleading.** In the 1.59 roll its checkboxes were all marked `[x]` with annotations like "✅ IMPLEMENTED", but several of those features had not actually been merged into the Python port. Trust the `docs/src/api/` walk over the issue.
Collapse file

‎.github/workflows/ci.yml‎

Copy file name to clipboardExpand all lines: .github/workflows/ci.yml
+1-47Lines changed: 1 addition & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -17,61 +17,26 @@ concurrency:
1717
cancel-in-progress: true
1818

1919
jobs:
20-
build-driver:
21-
name: Build driver
22-
runs-on: ubuntu-latest
23-
timeout-minutes: 60
24-
steps:
25-
- uses: actions/checkout@v6
26-
- name: Set up Node.js
27-
uses: actions/setup-node@v6
28-
with:
29-
node-version: '24'
30-
- name: Build driver bundles from source
31-
run: bash scripts/build_driver.sh
32-
- name: Upload driver bundles
33-
uses: actions/upload-artifact@v7
34-
with:
35-
name: driver-bundles
36-
path: driver/playwright-*.zip
37-
if-no-files-found: error
38-
# The bundles are already-compressed zips; skip re-compression.
39-
compression-level: 0
40-
retention-days: 1
41-
4220
infra:
4321
name: Lint
44-
needs: build-driver
4522
runs-on: ubuntu-latest
4623
steps:
4724
- uses: actions/checkout@v6
4825
- name: Set up Python
4926
uses: actions/setup-python@v6
5027
with:
5128
python-version: "3.10"
52-
- name: Download driver bundles
53-
uses: actions/download-artifact@v8
54-
with:
55-
name: driver-bundles
56-
path: driver/
57-
- name: Install dependencies & browsers
29+
- name: Install dependencies
5830
run: |
5931
python -m pip install --upgrade pip
6032
pip install -r local-requirements.txt
6133
pip install -r requirements.txt
6234
pip install -e .
63-
python -m build --wheel
64-
python -m playwright install --with-deps
6535
- name: Lint
6636
run: pre-commit run --show-diff-on-failure --color=always --all-files
67-
- name: Generate APIs
68-
run: bash scripts/update_api.sh
69-
- name: Verify generated API is up to date
70-
run: git diff --exit-code
7137

7238
build:
7339
name: Build
74-
needs: build-driver
7540
timeout-minutes: 45
7641
strategy:
7742
fail-fast: false
@@ -125,11 +90,6 @@ jobs:
12590
uses: actions/setup-python@v6
12691
with:
12792
python-version: ${{ matrix.python-version }}
128-
- name: Download driver bundles
129-
uses: actions/download-artifact@v8
130-
with:
131-
name: driver-bundles
132-
path: driver/
13393
- name: Install dependencies & browsers
13494
run: |
13595
python -m pip install --upgrade pip
@@ -159,7 +119,6 @@ jobs:
159119

160120
test-stable:
161121
name: Stable
162-
needs: build-driver
163122
timeout-minutes: 45
164123
strategy:
165124
fail-fast: false
@@ -178,11 +137,6 @@ jobs:
178137
uses: actions/setup-python@v6
179138
with:
180139
python-version: "3.10"
181-
- name: Download driver bundles
182-
uses: actions/download-artifact@v8
183-
with:
184-
name: driver-bundles
185-
path: driver/
186140
- name: Install dependencies & browsers
187141
run: |
188142
python -m pip install --upgrade pip
Collapse file

‎CLAUDE.md‎

Copy file name to clipboardExpand all lines: CLAUDE.md
+7-5Lines changed: 7 additions & 5 deletions
  • Display the source diff
  • Display the rich diff
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,13 @@ Python bindings for [Playwright](https://playwright.dev). The Python client talk
1010

1111
- `playwright/_impl/` — hand-written client implementation (one module per object: `_browser.py`, `_page.py`, `_locator.py`, `_network.py`, etc.). Edit these to add or change behavior.
1212
- `playwright/async_api/_generated.py`, `playwright/sync_api/_generated.py`**auto-generated**. Never edit by hand; rerun `./scripts/update_api.sh` after changing `_impl/` or the driver.
13-
- `scripts/generate_api.py`, `scripts/generate_async_api.py`, `scripts/generate_sync_api.py`, `scripts/documentation_provider.py` — codegen and validation. They diff the Python implementation against the driver's `playwright/driver/package/api.json` and abort if either side is out of sync.
13+
- `scripts/generate_api.py`, `scripts/generate_async_api.py`, `scripts/generate_sync_api.py`, `scripts/documentation_provider.py` — codegen and validation. They diff the Python implementation against Playwright's `api.json` (provided via the `PW_API_JSON` env var; see `scripts/update_api.sh`) and abort if either side is out of sync.
1414
- `scripts/expected_api_mismatch.txt` — explicit allowlist of "documented in JS, not in Python" or "named differently in Python" gaps. Lines that no longer apply must be removed.
1515
- `tests/async/`, `tests/sync/` — pytest suites. Most new tests are added to the async file with a sync mirror.
16-
- `DRIVER_SHA` — the single source of truth for which Playwright commit the driver is built from (one line, the 40-char `microsoft/playwright` commit SHA). Read by `setup.py`, `scripts/build_driver.sh`, and CI. The wheel build clones `microsoft/playwright` at this commit and builds the driver from source (via `scripts/build_driver.sh` + upstream's `utils/build/build-playwright-driver.sh`). The SHA is baked into the staged bundle filenames (`driver/playwright-<sha>-<suffix>.zip`), so it doubles as the build cache key.
17-
- `scripts/build_driver.sh` — clones and builds the upstream driver bundles into `driver/`. A portable bash script (shareable with the other language forks) that needs Node.js, npm, git and bash; invoked from `setup.py`'s `bdist_wheel`. Reads the pin from `DRIVER_SHA`; takes no arguments.
16+
- `DRIVER_VERSION` — the single source of truth for which Playwright release the driver is assembled from (one line, the `playwright-core` npm version, e.g. `1.61.0`, no `v` prefix). Read by `setup.py`, `scripts/build_driver.py`, and CI. The wheel build downloads `playwright-core` at this version from npm plus the matching Node.js binary and assembles the per-platform bundles — no source build. The version is baked into the staged bundle filenames (`driver/playwright-<version>-<suffix>.zip`), so it doubles as the build cache key.
17+
- `NODE_VERSION` — the Node.js version bundled with the driver (one line, e.g. `24.16.0`). Maintained at roll time by `scripts/update_node_version.py` (latest LTS, mirroring upstream's `utils/build/update-playwright-node.mjs`).
18+
- `scripts/build_driver.py` — assembles the per-platform driver bundles into `driver/` by downloading the `playwright-core` npm package (`DRIVER_VERSION`) and the official Node.js binaries (`NODE_VERSION`). Pure Python stdlib (no Node/npm/git); invoked from `setup.py`'s `bdist_wheel` with the target platform's suffix (no arg builds all six).
19+
- `api.json` is **not** shipped in the bundle and is never written into the driver — `scripts/update_api.sh` generates it from a nearby `microsoft/playwright` checkout (`$PW_SRC_DIR`) into a temp file and passes it to codegen via `PW_API_JSON` (read by `scripts/documentation_provider.py`). Needed only when regenerating the API, never at runtime.
1820
- `ROLLING.md`, `CONTRIBUTING.md` — human-facing setup and roll docs.
1921

2022
## Setup
@@ -26,7 +28,7 @@ python3 -m venv env && source env/bin/activate
2628
pip install --upgrade pip
2729
pip install -r local-requirements.txt
2830
pip install -e .
29-
python -m build --wheel # clones microsoft/playwright @ DRIVER_SHA and builds the driver from source
31+
python -m build --wheel # downloads playwright-core @ DRIVER_VERSION + Node.js and assembles the driver
3032
pre-commit install
3133
```
3234

@@ -39,7 +41,7 @@ If the system lacks `python3-venv`, `uv venv env` is an acceptable substitute (t
3941
- Type-check: `mypy playwright`.
4042
- Run tests: `pytest --browser chromium [-k name]`. Browsers are installed via `playwright install chromium` (do **not** use `--with-deps`, which requires sudo).
4143

42-
When changing public API, edit `_impl/`, then run `./scripts/update_api.sh`. The script regenerates `_generated.py` and validates against the driver's `api.json`. If validation fails, fix the mismatch in `_impl/`, in `expected_api_mismatch.txt`, or in `documentation_provider.py` — not by hand-editing `_generated.py`.
44+
When changing public API, edit `_impl/`, then run `./scripts/update_api.sh`. The script regenerates `_generated.py` and validates against Playwright's `api.json` (which it generates from `$PW_SRC_DIR`). If validation fails, fix the mismatch in `_impl/`, in `expected_api_mismatch.txt`, or in `documentation_provider.py` — not by hand-editing `_generated.py`.
4345

4446
## Rolling Playwright to a new version
4547

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.