Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Add support for external file references in source maps#15935

Merged
rbuckton merged 1 commit into
microsoft:mastermicrosoft/TypeScript:masterfrom
chuckjaz:external-file-source-mapchuckjaz/TypeScript:external-file-source-mapCopy head branch name to clipboard
Jun 7, 2017
Merged

Add support for external file references in source maps#15935
rbuckton merged 1 commit into
microsoft:mastermicrosoft/TypeScript:masterfrom
chuckjaz:external-file-source-mapchuckjaz/TypeScript:external-file-source-mapCopy head branch name to clipboard

Conversation

@chuckjaz

Copy link
Copy Markdown
Contributor

A proposed implementation of #15934 that allows transformers to generate source map references to external files for nodes that are added to the AST that represent code from a different file and, potentially, a different source language.

Fixes #15934

@mhegazy

mhegazy commented May 22, 2017

Copy link
Copy Markdown
Contributor

@rbuckton can you please review this change?

@chuckjaz

Copy link
Copy Markdown
Contributor Author

@rbuckton I wanted to add tests for this but it was unclear to me where they should go. Recommendations?

@rbuckton rbuckton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few minor comments, but generally this looks good.

Comment thread src/compiler/factory.ts Outdated
/**
* Create an external source map source file reference
*/
export function createSourceMapSource(fileName: string, text: string, skipTrivia?: (pos: number) => number) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should have an explicit return type annotation, to associate it with the definition of SourceMapSource in types.ts.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving the result untyped avoids a catch-22 caused by https://github.com/Microsoft/TypeScript/blob/v2.3.4/src/services/types.ts#L70-L72.

I added return type and as any which I don't really like but there is no good way I found around this. Including services/types.ts (e.g. when building tsserer.js) requires this literal have getLineAndCharacterOfPosition() but not include services/types.ts (e.g. when building tsc.js) requires it not have it. This is caused indirectly because SourceMapSource needs to extend SourceFileLIke (which I changed to be more clear by explicitly extending the interface).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, why must it extend SourceFileLike? Could you not use a union in the places where this matters in either sourcemap.ts or utilities.ts?

@chuckjaz chuckjaz Jun 6, 2017

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sourcemap.ts calls getLineAndCharacterOfPosition() from scanner.ts with a SourceMapSource which then requires SourceFileSource to be SourceFileLike.

I don't want to infect scanner.ts with SourceMapSource as SourceFileLike is intended to be the minimal specification of a content and cached lines. The fact it is extended to include getLineAndCharacterOfPosition() in services/types.ts seems troubling and unnecessary.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chuckjaz The constructor for a SourceFile is actually created by the objectAllocator. This allows us to keep nodes relatively light for the commandline compiler, but also allows us to create richer nodes for the language service.

You could consider adding a constructor factory for SourceMapSource to the object allocator, and hook in the getLineAndCharacterOfPosition method with a separate constructor in services.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make this clean I believe there are three options:

  1. Remove the extension of SourceFileLike in service/types.ts and fix all calls to the method getLineAndCharacterofPosition() to call the function getLineAndCharacterOfPosition().
  2. Introduce a new sub-type of SourceFileLIke in service/types.ts, such as ServiceSource, that extends SourceFileLIke and move all references to SourceFileLIke in services to use ServiceSource.
  3. Introduce a new type in compiler/types.ts called something like SourceWithLines that SourceFileLike extends from and change all other references to SourceFileLike in compiler to SourceWithLines.

I like the first one best. It is faster than using a method and I could not find any cases in service that overrides getCharacterAndLineOfPosition in a source file that would required either (2) or (3).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rbuckton Sorry. I was writing my last response before I saw yours. I will update the PR to your recommendation.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please note that this method is also used in src/harness/typeWriter.ts. Please also note that while SourceFileLike and SourceFileLike#getLineAndCharacterOfPosition are public, the computeLineAndCharacterOfPosition function is internal.

I think my comment above stands as the best approach at this point.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread src/compiler/sourcemap.ts Outdated
const emitNode = node.emitNode;
const emitFlags = emitNode && emitNode.flags;
const { pos, end } = emitNode && emitNode.sourceMapRange || node;
let source = emitNode && emitNode.sourceMapRange && emitNode.sourceMapRange.source;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To cut down on lookups, consider changing the above to:

const range = emitNode && emitNode.sourceMapRange;
const { pos, end } = range || node;
let source = range && range.source;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@chuckjaz chuckjaz force-pushed the external-file-source-map branch 3 times, most recently from a96b88b to ba8adea Compare June 6, 2017 22:19

@rbuckton rbuckton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few more minor comments.

Comment thread src/compiler/types.ts Outdated
export interface SourceMapSource {
fileName: string;
text: string;
lineMap: number[];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

line maps are internal only.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Comment thread src/compiler/core.ts Outdated
function SourceMapSource(this: SourceMapSource, fileName: string, text: string, skipTrivia?: (pos: number) => number) {
this.fileName = fileName;
this.text = text;
if (skipTrivia) this.skipTrivia = skipTrivia;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You lost the default for skipTrivia with this change. Previously you had pos => pos. Was this intended?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Comment thread src/services/types.ts Outdated
}

export interface SourceMapSource {
/* @internal */ getLineAndCharacterOfPosition(pos: number): LineAndCharacter;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getLineAndCharacterOfPosition is generally public.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was intentional to prevent getLineAndCharacterOfPosition from being written to typescript.d.ts as part of SourceMapSource and making supplying a getLineAndCharacerOfPosition() required when creating a SourceMapRange for calls to setSourceMapRange().

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source is optional on SourceMapRange and a SourceMapSource should always be created by calling createSourceMapSource in factory.ts. Marking it internal here means that SourceMapSource can't be used as a SourceFileLike in the public API, only in the compiler.

@rbuckton rbuckton Jun 6, 2017

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SourceFileLike isn't really used in the public API anyways so maybe this doesn't matter.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing /* @internal */ since removing it then it will essentially force calling createSoruceMapSource() which is, as you point out, a good thing.

@chuckjaz chuckjaz force-pushed the external-file-source-map branch from ba8adea to 34e8f38 Compare June 6, 2017 22:49
@chuckjaz chuckjaz force-pushed the external-file-source-map branch from 34e8f38 to ce1d1c8 Compare June 7, 2017 00:45
@rbuckton rbuckton merged commit b217c39 into microsoft:master Jun 7, 2017
@rbuckton

rbuckton commented Jun 7, 2017

Copy link
Copy Markdown
Contributor

This is a great change! Thanks for the contribution!

@microsoft microsoft locked and limited conversation to collaborators Jun 14, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Morty Proxy This is a proxified and sanitized view of the page, visit original site.