discussion/proposal: merging some lower-level racket/base stuff together for perf reasons#5406
discussion/proposal: merging some lower-level racket/base stuff together for perf reasons#5406jesboat wants to merge 15 commits intoracket:masterracket/racket:masterfrom jesboat:merge-lower-layers-squashedjesboat/racket:merge-lower-layers-squashedCopy head branch name to clipboard
Conversation
|
|
||
| (module reader syntax/module-reader | ||
| #:language 'racket/kernel)) | ||
|
|
There was a problem hiding this comment.
this file's changes are from #5402. They're mostly orthogonal to the rest of the branch, except that I wrote the new files using #lang racket/kernel instead of (module foo '#%kernel . _) syntax.
| stx)) | ||
| stx))) | ||
|
|
||
| (begin-for-syntax |
There was a problem hiding this comment.
Written from scratch. This should (I think) handle everything the define-like forms from define.rkt handle (i.e. everything except keyword and optional arguments) as well as the smaller -define and -define-syntax.
| @@ -3,216 +3,10 @@ | ||
| ;; -define, when, unless, let/ec, define-struct |
There was a problem hiding this comment.
this file just exists as a shim and should probably be killed. same with a couple others.
| @@ -3,7 +3,8 @@ | ||
| ;; but the JIT generates faster code, especially for the common cases. |
There was a problem hiding this comment.
Is this still true in CS? If it's only true in BC, is it worth keeping the second implementation around, or removing it for code simplicitly (at the cost of slightly worse BC perf?)
d86539b to
c294261
Compare
| (define-values (struct:break-paramz make-break-paramz break-paramz? break-paramz-ref break-paramz-set!) | ||
| (make-struct-type 'break-parameterization #f 1 0 #f)) | ||
|
|
||
| (-define-struct break-parameterization (cell)) |
There was a problem hiding this comment.
Turns out this struct has been sitting here unused since at least before v4. I don't know if/when it was ever used.
c294261 to
b254f37
Compare
|
This seems like a good idea. I have been worried about (1) the compatibility issues (as you mention); (2) whether |
|
@mflatt awesome. I'll start on breaking it up into reviewable chunks, adding the stubs, etc. |
Summary:
There is a very common pattern in the implementation of low-level macros:
(if something
(void)
(raise-syntax-error #f constant-string local-variable))
Introduce the appropriate abstraction to handle them.
This does not introduce uses; rather, uses will be added as more of the core
syntax is centralized and some of the implementations are slightly modernized.
See racket#5406 for more on that effort.
This is quite straightforward; mostly it's just changing a cond where
the first pile of branches do error checking to a handful of
`raise-syntax-error-if` and `raise-syntax-error-unless` invocations.
I also took advantage of the opportunity to improve some error messages,
going from:
bad syntax
missing test expression and body
missing body
to
bad syntax (illegal use of `.')
bad syntax (missing test expression and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see racket#5406 for more on that.
Quite straightforward. This also improves the error messages from
bad syntax
to
bad syntax (illegal use of `.')
bad syntax (missing identifier and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see racket#5406 for more on that.
Minimal modifications to the `cond` implementation to only use kernel syntax. Note that it still uses non-kernel functions. This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Summary:
There is a very common pattern in the implementation of low-level macros:
(if something
(void)
(raise-syntax-error #f constant-string local-variable))
Introduce the appropriate abstraction to handle them.
This does not introduce uses; rather, uses will be added as more of the core
syntax is centralized and some of the implementations are slightly modernized.
See racket#5406 for more on that effort.
This is quite straightforward; mostly it's just changing a cond where
the first pile of branches do error checking to a handful of
`raise-syntax-error-if` and `raise-syntax-error-unless` invocations.
I also took advantage of the opportunity to improve some error messages,
going from:
bad syntax
missing test expression and body
missing body
to
bad syntax (illegal use of `.')
bad syntax (missing test expression and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see racket#5406 for more on that.
Quite straightforward. This also improves the error messages from
bad syntax
to
bad syntax (illegal use of `.')
bad syntax (missing identifier and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see racket#5406 for more on that.
Minimal modifications to the `cond` implementation to only use kernel syntax. Note that it still uses non-kernel functions. This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Summary: Test Plan:
Summary: TODO See racket#5406 for more on that effort. Depends on something.
Summary: Test Plan:
This rename makes twofold sense. First, `qq-and-or.rkt` is already more than just what it says on the tin-- it also defines `let`, `let*`, and `letrec`. Second, as part of the effort to reduce startup time, we plan on merging more core macros into the same file; for more info about the overall effort, see racket#5406 Testing: manually reviewed `git grep qq-and-or` in this repo; the only hits are #%requires which this commit adjusts. Searched all `*.rkt` and `*.ss` files in my copy of the package archive; the hits are `stxparse-info` which appears to include a copy of a decent chunk of racket/base's implementation and is broken anyway, and one hit in TR which will need a coordinated change. TODO coordinated change in TR
Summary: Test Plan:
No code changes, just moving the definitions and provides. Adjust all other files which required define-et-al to require core-syntax instead. This means that some files now require core-syntax twice; that clutter will be cleaned up in a later commit. Testing: no hits in the repo for `define-et-al` or in any 1p/3p packages (except stxparse-info, which includes multiple copies of racket/base's implementation and already fails to build in any case.) This is part of an overall effort to blah blah blah, see link to giant PR goes here.
No changes to the implementation. For provides, instead of having core-syntax directly provide `old-cond`, `old-cond` is now provided by a submodule; this means that all the places where require `core-syntax` won't get an `old-cond` they don't want (and won't have to hide it from exporting to clients who shouldn't get it.) Adjust all other files which required cond.rkt to require core-syntax instead. This means that some files now require core-syntax twice; that clutter will be cleaned up in a later commit. Testing: no hits in the repo for `cond.rkt` or `private/cond`; no relevant hits for `private/cond` in any 1p/3p packages (except stxparse-info, which includes multiple copies of racket/base's implementation and already fails to build in any case.) This is part of an overall effort to blah blah blah, see link to giant PR goes here.
This has been unused since 6b1dad2 in 2019, which comments that the file is being "kept for backwards-compatibility". However, no 1p or 3p packages refer to `small-scheme` (in any `*.ss` or `*.rkt` files), so I think it's safe to remove at this point.
Minimal modifications to the `cond` implementation to only use kernel syntax. Note that it still uses non-kernel functions. This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
There is a very common pattern in the implementation of low-level macros:
(if something
(void)
(raise-syntax-error #f constant-string local-variable))
Introduce the appropriate abstraction to handle them.
This does not introduce uses; rather, uses will be added as more of the core
syntax is centralized and some of the implementations are slightly modernized.
See racket#5406 for more on that effort.
This is quite straightforward; mostly it's just changing a cond where
the first pile of branches do error checking to a handful of
`raise-syntax-error-if` and `raise-syntax-error-unless` invocations.
I also took advantage of the opportunity to improve some error messages,
going from:
bad syntax
missing test expression and body
missing body
to
bad syntax [when used as identifier or in improper list]
bad syntax (missing test expression and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see racket#5406 for more on that.
Quite straightforward. This also improves the error messages from
bad syntax
to
bad syntax [when used as identifier macro or in an improper list]
bad syntax (missing identifier and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see racket#5406 for more on that.
Minimal modifications to the `cond` implementation to only use kernel syntax. Note that it still uses non-kernel functions. This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
There is a very common pattern in the implementation of low-level macros:
(if something
(void)
(raise-syntax-error #f constant-string local-variable))
Introduce the appropriate abstraction to handle them.
This does not introduce uses; rather, uses will be added as more of the core
syntax is centralized and some of the implementations are slightly modernized.
See #5406 for more on that effort.
This is quite straightforward; mostly it's just changing a cond where
the first pile of branches do error checking to a handful of
`raise-syntax-error-if` and `raise-syntax-error-unless` invocations.
I also took advantage of the opportunity to improve some error messages,
going from:
bad syntax
missing test expression and body
missing body
to
bad syntax [when used as identifier or in improper list]
bad syntax (missing test expression and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see #5406 for more on that.
Quite straightforward. This also improves the error messages from
bad syntax
to
bad syntax [when used as identifier macro or in an improper list]
bad syntax (missing identifier and body)
bad syntax (missing body)
This is part of a larger effort to centralize and modernize some core syntax
constructs; see #5406 for more on that.
Minimal modifications to the `cond` implementation to only use kernel syntax. Note that it still uses non-kernel functions. This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
This rename makes twofold sense. First, `qq-and-or.rkt` is already more than just what it says on the tin-- it also defines `let`, `let*`, and `letrec`. Second, as part of the effort to reduce startup time, we plan on merging more core macros into the same file; for more info about the overall effort, see racket#5406 Testing: manually reviewed `git grep qq-and-or` in this repo; the only hits are #%requires which this commit adjusts. Searched all `*.rkt` and `*.ss` files in my copy of the package archive; the hits are `stxparse-info` which appears to include a copy of a decent chunk of racket/base's implementation and is broken anyway, and one hit in TR which will need a coordinated change. Note that this requires a coordinated (backwards-compatible) change in Typed Racket which should be merged at least a couple hours before this change is merged. That's racket/typed-racket#1499
This rename makes twofold sense. First, `qq-and-or.rkt` is already more than just what it says on the tin-- it also defines `let`, `let*`, and `letrec`. Second, as part of the effort to reduce startup time, we plan on merging more core macros into the same file; for more info about the overall effort, see racket#5406 Testing: manually reviewed `git grep qq-and-or` in this repo; the only hits are #%requires which this commit adjusts. Searched all `*.rkt` and `*.ss` files in my copy of the package archive; the hits are `stxparse-info` which appears to include a copy of a decent chunk of racket/base's implementation and is broken anyway, and one hit in TR which will need a coordinated change. Note that this requires a coordinated (backwards-compatible) change in Typed Racket which should be merged at least a couple hours before this change is merged. That's racket/typed-racket#1499
|
Status update:
After those, all of qq-and-or.rkt, define-et-al.rkt, cond.rkt, define.rkt, top-int.rkt, and performance-hint.rkt will use only kernel syntax, and can be merged into an omnibus file, tentatively named core-syntax.rkt, completing this series. |
Previously, racket/base had three `define` implementations: 1. In `define-et-al.rkt`, an implementation with limited feature support. 2. In `define.rkt`, an heavyweight implementation supporting curried functions, but still not optional parameters or defaults, which is used throughout much of racket/base (and, until recently, was indirectly public via an export to `#lang mzscheme`.) 3. In `kw.rkt`, the final "public" define, with full feature support. This situation is unsatisfactory. After discussion (see racket#5473), we're merging the first two `defines` into a single implementation which supports everything except keywords and optional arguments, is unsuitable for export, and lightweight enough to be easily written using only kernel syntax. This commit: - implements the new define in define-et-al.rkt, provides it - modifies files which used the old `-define` to use the new one via a `rename` require (found by searching the repo for `(?<![a-z0-9-])-define`) - replaces the `define.rkt` implementations with a re-export of the new one This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Replace the `define-values-for-syntax` implementation with one using only kernel syntax, eliminating many requires from `racket/private/define.rkt`. This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Previously, racket/base had three `define` implementations: 1. In `define-et-al.rkt`, an implementation with limited feature support. 2. In `define.rkt`, an heavyweight implementation supporting curried functions, but still not optional parameters or defaults, which is used throughout much of racket/base (and, until recently, was indirectly public via an export to `#lang mzscheme`.) 3. In `kw.rkt`, the final "public" define, with full feature support. This situation is unsatisfactory. After discussion (see racket#5473), we're merging the first two `defines` into a single implementation which supports everything except keywords and optional arguments, is unsuitable for export, and lightweight enough to be easily written using only kernel syntax. This commit: - implements the new define in define-et-al.rkt, provides it - modifies files which used the old `-define` to use the new one via a `rename` require (found by searching the repo for `(?<![a-z0-9-])-define`) - replaces the `define.rkt` implementations with a re-export of the new one This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Replace the `define-values-for-syntax` implementation with one using only kernel syntax, eliminating many requires from `racket/private/define.rkt`. This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Previously, racket/base had three `define` implementations: 1. In `define-et-al.rkt`, an implementation with limited feature support. 2. In `define.rkt`, an heavyweight implementation supporting curried functions, but still not optional parameters or defaults, which is used throughout much of racket/base (and, until recently, was indirectly public via an export to `#lang mzscheme`.) 3. In `kw.rkt`, the final "public" define, with full feature support. This situation is unsatisfactory. After discussion (see racket#5473), we're merging the first two `defines` into a single implementation which supports everything except keywords and optional arguments, is unsuitable for export, and lightweight enough to be easily written using only kernel syntax. This commit: - implements the new define in define-et-al.rkt, provides it - modifies files which used the old `-define` to use the new one via a `rename` require (found by searching the repo for `(?<![a-z0-9-])-define`) - replaces the `define.rkt` implementations with a re-export of the new one This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Replace the `define-values-for-syntax` implementation with one using only kernel syntax, eliminating many requires from `racket/private/define.rkt`. This is part of a larger effort to centralize and modernize some core syntax constructs; see racket#5406 for more on that.
Previously, racket/base had three `define` implementations: 1. In `define-et-al.rkt`, an implementation with limited feature support. 2. In `define.rkt`, an heavyweight implementation supporting curried functions, but still not optional parameters or defaults, which is used throughout much of racket/base (and, until recently, was indirectly public via an export to `#lang mzscheme`.) 3. In `kw.rkt`, the final "public" define, with full feature support. This situation is unsatisfactory. After discussion (see #5473), we're merging the first two `defines` into a single implementation which supports everything except keywords and optional arguments, is unsuitable for export, and lightweight enough to be easily written using only kernel syntax. This commit: - implements the new define in define-et-al.rkt, provides it - modifies files which used the old `-define` to use the new one via a `rename` require (found by searching the repo for `(?<![a-z0-9-])-define`) - replaces the `define.rkt` implementations with a re-export of the new one This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
Replace the `define-values-for-syntax` implementation with one using only kernel syntax, eliminating many requires from `racket/private/define.rkt`. This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
This rename makes twofold sense. First, `qq-and-or.rkt` is already more than just what it says on the tin-- it also defines `let`, `let*`, and `letrec`. Second, as part of the effort to reduce startup time, we plan on merging more core macros into the same file; for more info about the overall effort, see racket#5406 Testing: manually reviewed `git grep qq-and-or` in this repo; the only hits are #%requires which this commit adjusts. Searched all `*.rkt` and `*.ss` files in my copy of the package archive; the hits are `stxparse-info` which appears to include a copy of a decent chunk of racket/base's implementation and is broken anyway, and one hit in TR which will need a coordinated change. Note that this requires a coordinated (backwards-compatible) change in Typed Racket which should be merged at least a couple hours before this change is merged. That's racket/typed-racket#1499
This rename makes twofold sense. First, `qq-and-or.rkt` is already more than just what it says on the tin-- it also defines `let`, `let*`, and `letrec`. Second, as part of the effort to reduce startup time, we plan on merging more core macros into the same file; for more info about the overall effort, see #5406 Testing: manually reviewed `git grep qq-and-or` in this repo; the only hits are #%requires which this commit adjusts. Searched all `*.rkt` and `*.ss` files in my copy of the package archive; the hits are `stxparse-info` which appears to include a copy of a decent chunk of racket/base's implementation and is broken anyway, and one hit in TR which will need a coordinated change. Note that this requires a coordinated (backwards-compatible) change in Typed Racket which should be merged at least a couple hours before this change is merged. That's racket/typed-racket#1499
This is just `s/-define-syntax/define-syntax/` in a couple places in letstx-scheme.rkt. The motivation is to remove the import of `-define-syntax` which causes unpleasant behavior with `all-from` (or `all-from-except`) provides. This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
Move cond.rkt's definitions (with no changes) into core-syntax.rkt, adding a section header. Remove cond.rkt. Adjust the requires of other files appropriately. Note that this leads to some files requiring `core-syntax.rkt` more than once; this is tidied in a future commit in this PR. This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
Move define-et-al.rkt's definitions (with no changes) into core-syntax.rkt, adding a section header. Remove define-et-al.rkt. Adjust the requires of other files appropriately. Note that this leads to some files requiring `core-syntax.rkt` more than once; this is tidied in a future commit in this PR. This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
Move define.rkt's definitions (with no changes) into core-syntax.rkt, adding a section header. Remove define.rkt. Adjust the requires of other files appropriately. Note that this leads to some files requiring `core-syntax.rkt` more than once; this is tidied in a future commit in this PR. This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
The previous three commits introduced duplicate requires of core-syntax.rkt. This commit tidies them all up. This is part of a larger effort to centralize and modernize some core syntax constructs; see #5406 for more on that.
Making this PR mostly as a discussion point for @mflatt, @samth, and others who are interested.
This code is not intended for review as-is.
Background
For the past $time, I've been hacking around with the Racket implementation, specifically the parts below
#lang racket/baseand above Chez, with an idea towards finding potential performance improvements in things that matter to people. In discord, @samth suggested a couple possible places where performance improvements could be helpfulAs a first pass, I was poking through the lower layers of racket/base's in-Racket implementation. It turns out some of the low-level modules can be merged together, with relatively little loss of expressiveness. Specifically:
can be combined into one
core-macros.rktand cross-phase-persistentpico.rkt.This branch is approximately what I benchmarked using https://gist.github.com/jesboat/0ef53a4c93ea0f1a154c1460d5b5ecdb to get https://docs.google.com/spreadsheets/d/1Lb5_tRu825SazTYaPPONXDSCHIDY79rcNLNcgYc5Its/edit?gid=778923661#gid=778923661 (see also discord), although it has been rebased since then. It also appears to show significant improvements in startup time (e.g.
time racket -I racket/base </dev/null).This PR
This commit is a snapshot of my working tree, which includes:
#lang racket/kernelentirely kernel" which is maybe not going to be merged at all(#%declare #:require=define)in racket/base dependencies for unrelated reasonsThe development history is squashed out, and I would not consider this PR as an actual request to pull anything. Rather, it's (a) a place for me to attach the working tree and (b) a place to discuss proposed changes in the comments.
What next?
My suggestion would be for reviewers (/me glances awkwardly at @mflatt?) to:
If the perf gains are worth the loss of expressiveness from combining the various lower-layer modules, then I would start breaking the changes into smaller logical commits which could be reviewed and (hopefully) merged.
As a final note, the main distribution does not currently compile with this because compatibility-lib's mzscheme imports some private stuff from racket/base. Minimal changes to make it work with this branch are https://gist.github.com/jesboat/9c987896b94deb1d554597fbfe384cd5, which is also something I'd need to tidy up before potential upstreaming.
Overview of new files
core-macros.rkt
let,let*, let-loop, and/or, qq unfoldeddefineforms in private/define.rkt and also the-defineforms in define-et-alpico.rkt: