Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Editor parity: TypeScript-source debugging (M0-M2) and document symbols - #1308

#1308
Merged
nickna merged 9 commits into
mainnickna/SharpTS:mainfrom
worktree-epic-1306nickna/SharpTS:worktree-epic-1306Copy head branch name to clipboard
Jul 28, 2026
Merged

Editor parity: TypeScript-source debugging (M0-M2) and document symbols#1308
nickna merged 9 commits into
mainnickna/SharpTS:mainfrom
worktree-epic-1306nickna/SharpTS:worktree-epic-1306Copy head branch name to clipboard

Conversation

@nickna

@nickna nickna commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Delivers the debugger half of #1306. sharpts --compile app.ts -g now produces an assembly a
debugger steps through in the original .ts source: breakpoints bind, stepping follows executable
statements, locals show under their source names within their real scopes, and Just My Code steps
over the bundled stdlib.

Milestones M0, M1 and M2 complete; M3 started. Full suite green after every commit
(15881 tests). #1306 stays open — see what's left, below.

Commits

3a14dc6f Track 0 — portable PDBs surviving assembly-reference rewriting
2c5b226b Track A — source spans through parsing and AST transforms
5f535503 Track B1 — breakpoints bound to TypeScript statements
720a7bf2 Track B2 — named locals and lexical scopes
e902b137 Track B4 — "SharpTS: Debug Current File"
e3e4ad57 Track B3 — Just My Code over the bundled stdlib
7b0577cc Track C1 + first of C3 — feature mode and document symbols
0d2536cc Acceptance criterion asserted in one test

Two findings that shaped the design

The go/no-go gate was passable, but not the obvious way. PEPacker.AssemblyReferenceRewriter
destroys the debug directory and exposes no hook — but it preserves MethodDef row identity,
which is what makes re-attaching symbols to the finished bytes valid. DebugDirectoryInjector
appends the directory into .text's file-alignment slack, changing no RVA, section count or header
size; PdbEmitter.VerifyMethodMappingPreserved proves the assumption per build and fails the
compile rather than shipping symbols that point a debugger at the wrong methods.

A .NET 10 sharp edge. PersistedAssemblyBuilder.GenerateMetadata(out pdbBuilder) adds a
MethodDebugInformation row only for methods that have an IL body, so the table is not parallel
to MethodDef as the format requires. SharpTS emits interfaces, so it always hits this — 894 rows
for 905 methods, and every sequence point lands on the wrong method. SharpTS therefore builds the
PDB tables itself.

A performance regression, found and fixed

Attributing lowered statements re-walked shared subtrees, turning a two-file sharpts -p . from
698 ms into 19,286 ms. Three CLI integration tests caught it, but they passed in isolation and
failed only under suite load, so AttributingNestedLoweringsStaysLinear guards it directly. The fix
also cut the whole suite from ~20 min to ~14.5 min.

Still open on #1306

  • C2 semantic binding index — the large remaining piece, and a prerequisite for the rest of C3
    (definition, references, rename). Deliberately not started: the epic prefers integrating binding
    capture with checker/environment resolution over a divergent scope implementation, and
    TypeSystem/ is ~35.5k lines across 43 TypeChecker*.cs files. A partly-correct binder yields
    wrong "go to definition", which is worse than the feature being absent.
  • Track D — versioned caches, debounce/cancellation, reverse module invalidation.
  • Track A tail — expression and type-node spans (deferred by the epic; C2/C3 will need them).

Known verification gap

Symbol metadata is asserted thoroughly. No automated test observes a debugger actually stopping.
vsdbg cannot be scripted for it — it is licensed for use only with Visual Studio and VS Code and
enforces that with a handshake its own clients answer — so automating this means netcoredbg, a
separate install. Until then docs/debugging-typescript.md carries a manual smoke checklist, which
has not yet been run by a human. The VS Code command typechecks but was likewise not exercised
end-to-end.

Accepted v1 behaviour, documented rather than left as surprises: module top-level bindings appear as
static fields of $Program (they outlive the initializer), and async/generator locals appear as
state-machine fields (they must survive suspension) — restoring the latter as locals needs
portable-PDB state-machine custom debug information.

Refs #1306

nickna added 9 commits July 27, 2026 15:12
Adds `--debug`/`-g` to compile mode: the assembly is written with a debug
directory and a matching portable PDB beside it. This is the symbol pipeline
the TypeScript-source debugger is built on (epic #1306, track 0); sequence
points arrive with the statement source model.

Two properties of the toolchain made the obvious approach wrong.

`PEPacker.AssemblyReferenceRewriter` rebuilds the final PE to retarget
System.Private.CoreLib references, and the rebuilt image has no debug
directory — it exposes only Rewrite/Save/Dispose, so there is no hook to
preserve one. Symbols are therefore attached to the finished bytes:
DebugDirectoryInjector appends the directory into a section's file-alignment
slack, growing VirtualSize inside the section's own virtual page so no RVA,
section count, or header size changes. The rewriter does preserve MethodDef
row identity, which is what keeps a PDB valid across it, but
PdbEmitter.VerifyMethodMappingPreserved proves that per build and fails the
compile rather than shipping symbols that point a debugger at the wrong
methods. The PDB is serialized against the finished image's row counts, since
rewriting changes TypeRef/MemberRef/AssemblyRef counts.

`PersistedAssemblyBuilder.GenerateMetadata` can populate a PDB metadata
builder, but it adds a MethodDebugInformation row only for methods that have
an IL body. That table is defined to be parallel to MethodDef, so every
abstract or interface method shifts the rows after it — in a real build, 894
rows for 905 methods, and line information lands on the wrong method. SharpTS
emits interfaces, so it always hits this. DebugInfoCollector builds the
Document and MethodDebugInformation tables directly, keeping the parallel
invariant exact and leaving room for hidden points and local scopes.

SaveToBytes() keeps its contract for in-memory and test callers; SaveArtifacts()
is the new PE-plus-PDB result, and Save(path, pdbPath) lets EXE builds put
symbols beside the executable rather than the temporary DLL that gets bundled.
Builds without --debug take none of this path and emit no debug directory.

Tests cover the gate directly: an assembly shaped to expose the misalignment
trap goes through the real rewriter, and the resulting PDB is checked for
matching CodeView identity, checksum, documents, and sequence points landing
on the methods they were recorded against.
Introduces the per-source model that TypeScript-source debugging and editor
navigation both need: where every statement came from, carried intact across
the rewrites that happen between parsing and emission (epic #1306, track A).

SourceSpan holds half-open UTF-16 offsets — the form Token.Start already
carries, and the one that nests and compares with plain integer arithmetic.
Spans live in a SpanTable keyed by object reference rather than on the AST
records: a span field would drag position into record equality, so two
`return x;` statements on different lines would stop comparing equal.
SourceDocument ties a file's text, checksum, line index, and span table
together; ParsedModule now carries one, with the embedded stdlib marked
virtual since it has no file a debugger could open.

Span capture is centralized in Declaration() and Statement() rather than
spread across the productions they dispatch to. Every declaration and
statement in the language funnels through those two methods, so wrapping them
covers all of them and cannot drift as productions are added. First-write-wins
means the innermost production — which knows the tightest extent — is the one
kept.

Transforms have to state provenance explicitly, because they rebuild nodes:
VarHoister rewrites every statement enclosing a `var`, and GeneratorArrowLifter
rewrites everything around a lifted generator. Both now copy the original's
position onto its replacement at the single point all rewrites pass through,
and mark genuinely synthetic nodes hidden — the hoisted `var x;` declarations
sit at the top of the body, not where the user wrote them, so pointing them at
the original text would make stepping jump backwards. Destructuring lowerings
attribute each part of the expansion to the declaration it came from.

Token gains End and Span. The lexer builds every lexeme as the exact slice it
consumed, so the end is the start plus the lexeme length, quotes and delimiters
included. Splitting `>>`/`>>>` when closing nested generics now carries the
offset forward instead of dropping it, which had been costing a position for
everything parsed after a nested generic.

Attributing a lowering's parts descends only into parts that lack a span.
Without that guard the walk compounds — statement nodes are shared between
sequences and a sequence is re-recorded by every enclosing production — which
turned a two-file project check into a 19-second run. AttributingNestedLowerings
StaysLinear guards it directly, since the CLI integration tests only caught it
under parallel load and would have missed it on a faster machine.

Expressions, type nodes, and the compilation-side NestedFunctionLifter are not
instrumented yet; the lifter needs ILCompiler to receive documents, which
arrives with sequence-point emission.
Emits sequence points for executable statements, so `--compile app.ts -g`
produces an assembly a debugger steps through in the original `.ts` source
rather than in emitted IL (epic #1306, track B1).

Statement dispatch was duplicated: ILEmitter overrode EmitStatement with its
own switch, and the state-machine emitters used the base one, so there was no
single place to mark a statement. Both now override EmitStatementCore and share
one EmitStatement, which is where the mark happens — after the dead-code check
and before any IL, so the offset is the first instruction the statement
produces. Async and generator bodies get line information from that alone,
without either emitter knowing symbols exist.

Two paths still needed attention, both found by tracing what actually reached
the dispatcher rather than assuming. Top-level expression statements go through
EmitExpressionWithAsyncWait, which wraps them in top-level-await plumbing and
drives expression emission itself; it marks explicitly via MarkStatementStart.
And containers were taking the offset of the statement inside them: a block or
a `try` emits nothing of its own, so it shared an offset with its first real
statement, and since two points cannot share an offset the container won —
stepping landed on a brace. Blocks, lowering sequences, `try`, and labels no
longer take a point; conditions and loop headers still do, because their test
really does execute there. Type-only and separately-emitted declarations are
skipped as well.

Which document a statement belongs to is resolved from the module path emission
already tracks, rather than a parallel "current document" every phase would have
to maintain. CompilationContext gains the owning method and the debug scope,
supplied by the context factories — the one place contexts are built.

Debug builds are marked [Debuggable(Default | DisableOptimizations)] so the JIT
keeps the shape the PDB describes. Builds without --debug set neither the scope
nor the attribute and emit no points.

Also completes the transform provenance deferred from the span work:
NestedFunctionLifter copies positions onto the statements it rebuilds and marks
the aliases it leaves behind hidden, which is possible now that documents reach
ILCompiler.

Verified by hand across multi-module, async, generator, loop, try/catch, class
and hoisted-var programs — including that an imported module's methods resolve
to their own file. docs/debugging-typescript.md carries the editor recipes and
the manual smoke checklist; automated coverage asserts the line sets directly.
Emits the LocalVariable and LocalScope tables, so a debugger stopped in
compiled TypeScript shows `doubled` rather than a slot number, and offers a
binding only where it is actually in scope (epic #1306, track B2).

LocalsManager is the seam. Every binding the user wrote is declared through it,
while spill and scratch slots are declared straight on the ILGenerator and never
reach it — so listening in that one place names user-visible locals and nothing
else, rather than emitting everything and classifying afterwards. It reports
declarations and scope boundaries through ILocalSymbolSink, which keeps it from
depending on the symbol types; the context factories attach the sink, being the
one place emission contexts are built.

Scopes nest, so MethodLocalSymbols tracks them with a stack: entering records
the IL offset a scope opens at, exiting records where it closes. The scope
LocalsManager starts with is the method body itself and is never explicitly
closed, so its length comes from the finished method's IL size, read back from
the emitted image. Resolving a name by walking outward from the innermost scope
is what makes shadowing work, and what keeps a `for` binding out of the locals
window outside its loop.

The format requires LocalScope to be sorted by method, then start ascending,
then length descending, with each scope's variables in a contiguous run of
LocalVariable rows; both are established when the tables are written, and a test
pins the ordering because a reader silently resolves against the wrong enclosing
scope if it drifts. Temporaries the lowerings introduce are marked
DebuggerHidden.

Two categories deliberately do not appear as locals, and the docs say so rather
than leaving them as surprises: module top-level bindings are static fields of
$Program because they outlive the initializer that assigns them, and async and
generator locals are state-machine fields because they must survive suspension.
Presenting the latter as locals again needs the portable-PDB state-machine
hoisted-variable custom debug information, which is a follow-up.
One command to go from an open `.ts` file to a debug session: save if dirty,
compile that saved source with `-g`, and start debugging the result (epic
#1306, track B4).

Saving first matters. The PDB records a SHA-256 checksum of the text that was
compiled, so debugging unsaved-then-stale source is exactly the case a debugger
refuses to bind breakpoints for; compiling what is on disk keeps the checksum
describing what the editor is showing.

No SharpTS debug adapter is involved. A compiled program is an ordinary .NET
assembly, so the `coreclr` adapter from the C# extension drives it. That
extension is a separate install and `startDebugging` fails unhelpfully without
it, so the command checks for it up front and offers to install it. The docs
also carry the `launch.json` form for driving it manually, since a launch
configuration cannot be contributed for a debug type another extension owns.

Output goes beside the source rather than into a temporary directory: that is
where the runtimeconfig, co-located dependencies, and imported modules resolve
from, and it leaves nothing to accumulate.
Marks methods compiled from embedded stdlib modules [DebuggerNonUserCode], so
stepping over `path.join(...)` lands on the next line the user wrote instead of
descending into `path.ts` (epic #1306, track B3).

Emitting sequence points made this necessary. A Node-compatible module is real
TypeScript compiled into the program, so once every statement carried line
information the stdlib became exactly as steppable as the user's own files —
a three-line program importing `path` gained a 43 KB PDB and a stack of
stdlib frames to step through.

Line information for those methods is deliberately kept rather than suppressed:
a stack trace through the stdlib should still name a file and line, and turning
Just My Code off should still let you step in. Only the classification changes.

Emitted runtime helpers need no marking. They carry no debug information at all,
which already places them outside user code — adding attributes there would be
belt-and-braces on a rule the debugger already applies.
Adds a feature-mode contract to the language server and the first navigation
capability behind it: `textDocument/documentSymbol` (epic #1306, track C1 and
the first step of C3).

The two audiences want opposite things. In VS Code these files are already
served by tsserver, which does ordinary navigation better than SharpTS could and
would collide with it; what SharpTS knows that tsserver does not is the .NET
interop surface. A standalone client — Neovim, Helix, anything without a
TypeScript server of its own — has no such alternative. So `sharpts-lsp` takes
`--language-features interop-only|full` and defaults to `full`, while the VS Code
extension launches it explicitly as `interop-only`. Interop diagnostics, hover,
completion and signature help are served in both.

The mode is fixed when the server starts, and deliberately so: registering a
handler is what advertises its capability during initialization, and retracting
one afterwards needs dynamic registration/unregistration, which this server does
not implement. Reading a setting later could not take back a capability the
client has already been told about, so changing the mode means a restart.

Outline ranges come from the parser's span table, so a symbol covers exactly the
text its declaration occupies and selecting one puts the cursor on the name
rather than the whole declaration. Building it surfaced two declarations that
never passed through the statement dispatcher and so carried no span at all:
`export class C {}` parses its declaration directly, and namespace bodies have
their own member dispatcher. Both are fixed where the spans are recorded rather
than worked around in the outline, since every future consumer of positions
would have hit the same hole.

A declaration with no recorded span is skipped rather than given an invented
range, and an unparseable file yields no symbols instead of throwing — a client
asks for symbols on every keystroke, so half-typed input is the normal case.
The pieces were each covered, but nothing asserted them together on a program
with imports, which is the shape the criterion is actually about: two TypeScript
documents with matching checksums, sequence points attributed to the file each
method was written in, named locals, nested lexical scopes, and a CodeView
identity that still matches after the reference rewriter has rebuilt the
assembly.

Multi-document attribution had only been checked by hand until now — the
existing assertion was Assert.Single on a one-document fixture, which cannot
catch a method resolving to the wrong file.

Also records in the docs why the remaining verification gap stays manual. No
automated test observes a debugger actually stopping; `vsdbg` cannot be scripted
for it, being licensed for use only with Visual Studio and VS Code and enforcing
that with a handshake its own clients answer. Automating it means `netcoredbg`,
which is a separate install, so the smoke checklist stays manual for now — and
says so, rather than leaving the next person to rediscover it.
Both sides extended the same parser fluent chain in ModuleResolver.LoadModule:
this branch added .WithSourceDocument(document) so each module keeps the text,
checksum and statement spans that debug symbols and editor navigation resolve
against; main added .WithFilePath(absolutePath) and the JSX dialect setup.

All of them are kept — they configure different things and none supersedes
another. WithSourceDocument only defaults the file path when none was set, so
main's explicit WithFilePath still wins regardless of order.
@nickna
nickna merged commit 2b4a471 into main Jul 28, 2026
2 checks passed
@nickna
nickna deleted the worktree-epic-1306 branch July 28, 2026 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Morty Proxy This is a proxified and sanitized view of the page, visit original site.