Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 0b031b2

Browse filesBrowse files
kateinoigakukunkateinoigakukun
authored
Merge pull request #490 from ruby/katei/bundle-install-target-rbconfig
Use `bundle install --target-rbconfig` to install gems for Wasm target
2 parents eb1ebaa + 4ddccfc commit 0b031b2
Copy full SHA for 0b031b2

File tree

Expand file treeCollapse file tree

30 files changed

+241
-70
lines changed
Filter options
Expand file treeCollapse file tree

30 files changed

+241
-70
lines changed

‎.github/workflows/build.yml

Copy file name to clipboardExpand all lines: .github/workflows/build.yml
+11-11Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ jobs:
3131
fetch-depth: 0
3232
- uses: ruby/setup-ruby@v1
3333
with:
34-
ruby-version: "3.3"
34+
ruby-version: "head"
3535
bundler-cache: true
3636
- run: ./bin/setup
3737
- run: bundle exec rake check:type
@@ -77,7 +77,7 @@ jobs:
7777
- uses: actions/checkout@v4
7878
- uses: ruby/setup-ruby@v1
7979
with:
80-
ruby-version: "3.3"
80+
ruby-version: "head"
8181
bundler-cache: true
8282
- run: ./bin/setup
8383
- run: rake ci:pin_build_manifest
@@ -90,8 +90,8 @@ jobs:
9090
- name: Set matrix
9191
id: set-matrix
9292
run: |
93-
rake ci:rake_task_matrix > matrix.json
94-
echo "entries=$(cat matrix.json)" >> $GITHUB_OUTPUT
93+
rake ci:rake_task_matrix
94+
echo "entries=$(cat ci_matrix.json)" >> $GITHUB_OUTPUT
9595
9696
rake-tasks:
9797
strategy:
@@ -142,21 +142,21 @@ jobs:
142142
if: ${{ inputs.prerel_name != '' && matrix.entry.prerelease != '' }}
143143
- name: rake ${{ matrix.entry.task }}
144144
run: ./build-exec rake --verbose ${{ matrix.entry.task }}
145+
- uses: actions/upload-artifact@v4
146+
if: ${{ matrix.entry.artifact }}
147+
with:
148+
name: ${{ matrix.entry.artifact_name }}
149+
path: ${{ matrix.entry.artifact }}
145150
- uses: ruby/setup-ruby@v1
146151
if: ${{ matrix.entry.test != '' }}
147152
with:
148-
ruby-version: "3.3"
153+
ruby-version: "head"
149154
bundler-cache: false
150155
- name: rake ${{ matrix.entry.test }}
151156
run: |
152157
bundle install --with=check --without=development
153158
rake ${{ matrix.entry.test }}
154159
if: ${{ matrix.entry.test != '' }}
155-
- uses: actions/upload-artifact@v4
156-
if: ${{ matrix.entry.artifact }}
157-
with:
158-
name: ${{ matrix.entry.artifact_name }}
159-
path: ${{ matrix.entry.artifact }}
160160

161161
release-artifacts:
162162
needs: [rake-tasks]
@@ -176,7 +176,7 @@ jobs:
176176
registry-url: https://registry.npmjs.org/
177177
- uses: ruby/setup-ruby@v1
178178
with:
179-
ruby-version: "3.3"
179+
ruby-version: "head"
180180
bundler-cache: true
181181
- run: ./bin/setup
182182
- run: echo "PREREL_NAME=${{ inputs.prerel_name }}" >> $GITHUB_ENV

‎CONTRIBUTING.md

Copy file name to clipboardExpand all lines: CONTRIBUTING.md
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ $ gh run download <run-id>
117117
$ for pkg in cross-gem/pkg/ruby_wasm-*; do gem push $pkg; done
118118
$ gem build && gem push ruby_wasm-*.gem && rm ruby_wasm-*.gem
119119
$ (cd packages/gems/js/ && gem build && gem push js-*.gem && rm js-*.gem)
120+
$ rake bump_dev_version
120121
```
121122

122123
## Release Channels

‎Gemfile

Copy file name to clipboardExpand all lines: Gemfile
+2-1Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,8 @@ group :development do
1111
end
1212

1313
group :check do
14-
gem "webrick"
14+
# Use the latest version of webrick for URI change in Ruby 3.4
15+
gem "webrick", github: "ruby/webrick", ref: "0c600e169bd4ae267cb5eeb6197277c848323bbe"
1516
gem "syntax_tree", "~> 3.5"
1617
gem "steep"
1718
end

‎Rakefile

Copy file name to clipboardExpand all lines: Rakefile
+3-3Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,20 +28,20 @@ NPM_PACKAGES = [
2828
{
2929
name: "ruby-head-wasm-wasi",
3030
ruby_version: "head",
31-
gemfile: "packages/npm-packages/ruby-wasm-wasi/Gemfile",
31+
gemfile: "packages/npm-packages/ruby-head-wasm-wasi/Gemfile",
3232
target: "wasm32-unknown-wasip1",
3333
enable_component_model: true,
3434
},
3535
{
3636
name: "ruby-3.3-wasm-wasi",
3737
ruby_version: "3.3",
38-
gemfile: "packages/npm-packages/ruby-wasm-wasi/Gemfile",
38+
gemfile: "packages/npm-packages/ruby-3.3-wasm-wasi/Gemfile",
3939
target: "wasm32-unknown-wasip1"
4040
},
4141
{
4242
name: "ruby-3.2-wasm-wasi",
4343
ruby_version: "3.2",
44-
gemfile: "packages/npm-packages/ruby-wasm-wasi/Gemfile",
44+
gemfile: "packages/npm-packages/ruby-3.2-wasm-wasi/Gemfile",
4545
target: "wasm32-unknown-wasip1"
4646
},
4747
{ name: "ruby-wasm-wasi", target: "wasm32-unknown-wasip1" }

‎bin/setup

Copy file name to clipboardExpand all lines: bin/setup
-1Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,6 @@ set -vx
66
root="$(cd "$(dirname "$0")/.." && pwd)"
77

88
env BUNDLE_GEMFILE="$root/Gemfile" bundle install
9-
env BUNDLE_GEMFILE="$root/packages/npm-packages/ruby-wasm-wasi/Gemfile" bundle install
109

1110
# Build vendored jco if Rust toolchain is available and submodule is checked out
1211
if command -v rustc && [ -f vendor/jco/package.json ]; then

‎builders/wasm32-unknown-wasip1/Dockerfile

Copy file name to clipboardExpand all lines: builders/wasm32-unknown-wasip1/Dockerfile
+13Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,19 @@ RUN set -eux pipefail; \
3434
sh -s -- -y --no-modify-path --profile minimal --default-toolchain $RUST_VERSION; \
3535
chmod -R a+w $RUSTUP_HOME $CARGO_HOME
3636

37+
# Install the latest Ruby to use the latest Bundler for cross-building C extension gems.
38+
ADD --keep-git-dir=true https://github.com/ruby/ruby.git /buildruby
39+
RUN set -eux; \
40+
cd /buildruby; \
41+
./autogen.sh; \
42+
mkdir -p /opt/ruby; \
43+
./configure --prefix=/opt/ruby --disable-install-doc; \
44+
make -j$(nproc); \
45+
make install; \
46+
cd /; \
47+
rm -rf /buildruby
48+
ENV PATH=/opt/ruby/bin:$PATH
49+
3750
ENV BUNDLE_PATH=/usr/local/gems
3851
RUN set -eux; \
3952
mkdir -p $BUNDLE_PATH; \

‎ext/ruby_wasm/extconf.rb

Copy file name to clipboardExpand all lines: ext/ruby_wasm/extconf.rb
+4-1Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,4 +3,7 @@
33
require "mkmf"
44
require "rb_sys/mkmf"
55

6-
create_rust_makefile("ruby_wasm/ruby_wasm")
6+
create_rust_makefile("ruby_wasm/ruby_wasm") do |r|
7+
# We require head Ruby, so we need to fallback to compiled API
8+
r.use_stable_api_compiled_fallback = true
9+
end

‎ext/ruby_wasm/src/lib.rs

Copy file name to clipboardExpand all lines: ext/ruby_wasm/src/lib.rs
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,9 @@ impl WasiVirt {
252252
// Disable sockets for now since `sockets/ip-name-lookup` is not
253253
// supported by @bytecodealliance/preview2-shim yet
254254
virt.sockets(false);
255+
// Disable http for now since `http` is not supported by
256+
// wasmtime yet
257+
virt.http(false);
255258
Ok(())
256259
})
257260
}

‎lib/ruby_wasm/build/product/crossruby.rb

Copy file name to clipboardExpand all lines: lib/ruby_wasm/build/product/crossruby.rb
+7-3Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -71,14 +71,14 @@ def do_extconf(executor, crossruby)
7171
return
7272
end
7373
objdir = product_build_dir crossruby
74-
rbconfig_rb = Dir.glob(File.join(crossruby.dest_dir, "usr/local/lib/ruby/*/wasm32-wasi/rbconfig.rb")).first
74+
rbconfig_rb = crossruby.rbconfig_rb
7575
raise "rbconfig.rb not found" unless rbconfig_rb
7676
extconf_args = [
7777
"-C", objdir,
7878
"#{@srcdir}/extconf.rb",
7979
"--target-rbconfig=#{rbconfig_rb}",
8080
]
81-
extconf_args << "--enable-component-model" if @features.support_component_model?
81+
extconf_args << "--disable-component-model" unless @features.support_component_model?
8282
executor.system crossruby.baseruby_path, *extconf_args
8383
end
8484

@@ -111,7 +111,7 @@ def do_legacy_extconf(executor, crossruby)
111111
"-I#{crossruby.build_dir}",
112112
"--",
113113
]
114-
extconf_args << "--enable-component-model" if @features.support_component_model?
114+
extconf_args << "--disable-component-model" unless @features.support_component_model?
115115
# Clear RUBYOPT to avoid loading unrelated bundle setup
116116
executor.system crossruby.baseruby_path,
117117
*extconf_args,
@@ -301,6 +301,10 @@ def extinit_c_erb
301301
File.expand_path("../crossruby/extinit.c.erb", __FILE__)
302302
end
303303

304+
def rbconfig_rb
305+
Dir.glob(File.join(dest_dir, "usr/local/lib/ruby/*/wasm32-wasi/rbconfig.rb")).first
306+
end
307+
304308
def baseruby_path
305309
File.join(@baseruby.install_dir, "bin/ruby")
306310
end

‎lib/ruby_wasm/cli.rb

Copy file name to clipboardExpand all lines: lib/ruby_wasm/cli.rb
+9-3Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -310,7 +310,10 @@ def self.bundled_patches_path
310310

311311
def derive_packager(options)
312312
__skip__ = definition = nil
313-
__skip__ = if defined?(Bundler) && !options[:disable_gems]
313+
features = RubyWasm::FeatureSet.derive_from_env
314+
# The head ruby & dynamic linking uses "bundle" command to build gems instead of in-process integration.
315+
use_in_process_gem_building = !(options[:ruby_version] == "head" && features.support_dynamic_linking?)
316+
__skip__ = if defined?(Bundler) && !options[:disable_gems] && use_in_process_gem_building
314317
begin
315318
# Silence Bundler UI if --print-ruby-cache-key is specified not to bother the JSON output.
316319
level = options[:print_ruby_cache_key] ? :silent : Bundler.ui.level
@@ -321,17 +324,20 @@ def derive_packager(options)
321324
Bundler.ui.level = old_level
322325
end
323326
end
324-
RubyWasm.logger.info "Using Gemfile: #{definition.gemfiles}" if definition
327+
RubyWasm.logger.info "Using Gemfile: #{definition.gemfiles.map(&:to_s).join(", ")}" if definition
325328
RubyWasm::Packager.new(
326329
root, build_config(options), definition,
327-
features: RubyWasm::FeatureSet.derive_from_env
330+
features: features,
328331
)
329332
end
330333

331334
def do_print_ruby_cache_key(packager)
332335
ruby_core_build = packager.ruby_core_build
333336
require "digest"
334337
digest = Digest::SHA256.new
338+
# The build system key is used to invalidate the cache when the build system is updated.
339+
build_system_key = 1
340+
digest.update(build_system_key.to_s)
335341
ruby_core_build.cache_key(digest)
336342
hexdigest = digest.hexdigest
337343
require "json"

‎lib/ruby_wasm/packager.rb

Copy file name to clipboardExpand all lines: lib/ruby_wasm/packager.rb
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ def package(executor, dest_dir, options)
3434

3535
ruby_core.build_gem_exts(executor, fs.bundle_dir)
3636

37-
fs.package_gems
37+
fs.package_gems unless features.support_component_model?
3838
fs.remove_non_runtime_files(executor)
3939
if options[:stdlib]
4040
options[:without_stdlib_components].each do |component|

‎lib/ruby_wasm/packager/core.rb

Copy file name to clipboardExpand all lines: lib/ruby_wasm/packager/core.rb
+51-35Lines changed: 51 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
require "forwardable"
2+
require "pathname"
23

34
class RubyWasm::Packager::Core
45
def initialize(packager)
@@ -19,7 +20,6 @@ def build(executor, options)
1920
def build_strategy
2021
@build_strategy ||=
2122
begin
22-
has_exts = @packager.specs.any? { |spec| spec.extensions.any? }
2323
if @packager.features.support_dynamic_linking?
2424
DynamicLinking.new(@packager)
2525
else
@@ -59,14 +59,6 @@ def specs_with_extensions
5959
end
6060
end
6161

62-
def wasi_exec_model
63-
# TODO: Detect WASI exec-model from binary exports (_start or _initialize)
64-
use_js_gem = @packager.specs.any? do |spec|
65-
spec.name == "js"
66-
end
67-
use_js_gem ? "reactor" : "command"
68-
end
69-
7062
def with_unbundled_env(&block)
7163
__skip__ = if defined?(Bundler)
7264
Bundler.with_unbundled_env(&block)
@@ -138,12 +130,16 @@ def _link_gem_exts(executor, build, ruby_root, gem_home, module_bytes)
138130
wasi_sdk_path = toolchain.wasi_sdk_path
139131
libraries << File.join(wasi_sdk_path, "share/wasi-sysroot/lib/wasm32-wasi", lib)
140132
end
141-
wasi_adapter = RubyWasm::Packager::ComponentAdapter.wasi_snapshot_preview1(wasi_exec_model)
142-
adapters = [wasi_adapter]
143133
dl_openable_libs = []
144134
dl_openable_libs << [File.dirname(ruby_root), Dir.glob(File.join(ruby_root, "lib", "ruby", "**", "*.so"))]
145135
dl_openable_libs << [gem_home, Dir.glob(File.join(gem_home, "**", "*.so"))]
146136

137+
has_js_so = dl_openable_libs.any? do |root, libs|
138+
libs.any? { |lib| lib.end_with?("/js.so") }
139+
end
140+
wasi_adapter = RubyWasm::Packager::ComponentAdapter.wasi_snapshot_preview1(has_js_so ? "reactor" : "command")
141+
adapters = [wasi_adapter]
142+
147143
linker = RubyWasmExt::ComponentLink.new
148144
linker.use_built_in_libdl(true)
149145
linker.stub_missing_functions(false)
@@ -187,31 +183,43 @@ def _build_gem_exts(executor, build, gem_home)
187183
baseruby.build(executor)
188184
end
189185

190-
exts = specs_with_extensions.flat_map do |spec, exts|
191-
exts.map do |ext|
192-
ext_feature = File.dirname(ext) # e.g. "ext/cgi/escape"
193-
ext_srcdir = File.join(spec.full_gem_path, ext_feature)
194-
ext_relative_path = File.join(spec.full_name, ext_feature)
195-
prod = RubyWasm::CrossRubyExtProduct.new(
196-
ext_srcdir,
197-
build.toolchain,
198-
features: @packager.features,
199-
ext_relative_path: ext_relative_path
200-
)
201-
[prod, spec]
202-
end
203-
end
186+
crossruby = build.crossruby
187+
rbconfig_rb = crossruby.rbconfig_rb
204188

205-
exts.each do |prod, spec|
206-
libdir = File.join(gem_home, "gems", spec.full_name, spec.raw_require_paths.first)
207-
extra_mkargs = [
208-
"sitearchdir=#{libdir}",
209-
"sitelibdir=#{libdir}",
210-
]
211-
executor.begin_section prod.class, prod.name, "Building"
212-
prod.build(executor, build.crossruby, extra_mkargs)
213-
executor.end_section prod.class, prod.name
214-
end
189+
options = @packager.full_build_options
190+
target_triplet = options[:target]
191+
192+
local_path = File.join("bundle", target_triplet)
193+
env = {
194+
"BUNDLE_APP_CONFIG" => File.join(".bundle", target_triplet),
195+
"BUNDLE_PATH" => local_path,
196+
"BUNDLE_WITHOUT" => "build",
197+
# Do not auto-switch bundler version by Gemfile.lock
198+
"BUNDLE_VERSION" => "system",
199+
# FIXME: BUNDLE_PATH is set as a installation destination here, but
200+
# it is also used as a source of gems to be loaded by RubyGems itself.
201+
# RubyGems loads "psych" gem and if Gemfile includes "psych" gem,
202+
# RubyGems tries to load "psych" gem from BUNDLE_PATH at the second
203+
# time of "bundle install" command. But the extension of "psych" gem
204+
# under BUNDLE_PATH is built for Wasm target, not for host platform,
205+
# so it fails to load the extension.
206+
#
207+
# Thus we preload psych from the default LOAD_PATH here to avoid
208+
# loading Wasm version of psych.so via `Kernel#require` patched by
209+
# RubyGems.
210+
"RUBYOPT" => "-rpsych",
211+
}
212+
213+
args = [
214+
File.join(baseruby.install_dir, "bin", "bundle"),
215+
"install",
216+
"--standalone",
217+
"--target-rbconfig",
218+
rbconfig_rb,
219+
]
220+
221+
executor.system(*args, env: env)
222+
executor.cp_r(local_path, gem_home)
215223
end
216224

217225
def cache_key(digest)
@@ -337,6 +345,14 @@ def build_gem_exts(executor, gem_home)
337345
# No-op because we already built extensions as part of the Ruby build
338346
end
339347

348+
def wasi_exec_model
349+
# TODO: Detect WASI exec-model from binary exports (_start or _initialize)
350+
use_js_gem = @packager.specs.any? do |spec|
351+
spec.name == "js"
352+
end
353+
use_js_gem ? "reactor" : "command"
354+
end
355+
340356
def link_gem_exts(executor, ruby_root, gem_home, module_bytes)
341357
return module_bytes unless @packager.features.support_component_model?
342358

‎lib/ruby_wasm/packager/file_system.rb

Copy file name to clipboardExpand all lines: lib/ruby_wasm/packager/file_system.rb
-1Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,6 @@ def remove_stdlib_component(executor, component)
4444
when "enc"
4545
# Remove all encodings except for encdb.so and transdb.so
4646
enc_dir = File.join(@ruby_root, "lib", "ruby", ruby_version, "wasm32-wasi", "enc")
47-
puts File.join(enc_dir, "**/*.so")
4847
Dir.glob(File.join(enc_dir, "**/*.so")).each do |entry|
4948
next if entry.end_with?("encdb.so", "transdb.so")
5049
RubyWasm.logger.debug "Removing stdlib encoding: #{entry}"

‎lib/ruby_wasm/version.rb

Copy file name to clipboard
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
module RubyWasm
2-
VERSION = "2.6.2"
2+
VERSION = "2.6.2.dev"
33
end

‎packages/gems/js/ext/js/extconf.rb

Copy file name to clipboardExpand all lines: packages/gems/js/ext/js/extconf.rb
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111

1212
$objs = %w[js-core.o witapi-core.o]
1313

14-
use_component_model = enable_config("component-model", false)
14+
use_component_model = enable_config("component-model", true)
1515
$stderr.print "Building with component model: "
1616
$stderr.puts use_component_model ? "\e[1;32myes\e[0m" : "\e[1;31mno\e[0m"
1717
if use_component_model

‎packages/gems/js/lib/js/version.rb

Copy file name to clipboard
+1-1Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
module JS
2-
VERSION = "2.6.2"
2+
VERSION = "2.6.2.dev"
33
end

‎packages/npm-packages/ruby-3.2-wasm-wasi/.gitignore

Copy file name to clipboard
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,4 @@
11
*.tgz
2+
/tmp
3+
/bundle
4+
/vendor

‎packages/npm-packages/ruby-wasm-wasi/Gemfile renamed to ‎packages/npm-packages/ruby-3.2-wasm-wasi/Gemfile

Copy file name to clipboard
File renamed without changes.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.