fluencelabs/wasm-bindgen
1
0
Fork 0
mirror of https://github.com/fluencelabs/wasm-bindgen synced 2025-06-13 13:01:22 +00:00
Code Issues Projects Releases Wiki Activity

Add reference output tests for JS operations (#1894)

Browse Source 
* Add reference output tests for JS operations

This commit starts adding a test suite which checks in, to the
repository, test assertions for both the JS and wasm file outputs of a
Rust crate compiled with `#[wasm_bindgen]`. These aren't intended to be
exhaustive or large scale tests, but rather micro-tests to help observe
the changes in `wasm-bindgen`'s output over time.

The motivation for this commit is basically overhauling how all the GC
passes work in `wasm-bindgen` today. The reorganization is also included
in this commit as well.

Previously `wasm-bindgen` would, in an ad-hoc fashion, run the GC passes
of `walrus` in a bunch of places to ensure that less "garbage" was seen
by future passes. This not only was a source of slowdown but it also was
pretty brittle since `wasm-bindgen` kept breaking if extra iteams leaked
through.

The strategy taken in this commit is to have one precise location for a
GC pass, and everything goes through there. This is achieved by:

* All internal exports are removed immediately when generating the
  nonstandard wasm interface types section. Internal exports,
  intrinsics, and runtime support are all referenced by the various
  instructions and/or sections that use them. This means that we now
  have precise tracking of what an adapter uses.

* This in turn enables us to implement the `add_gc_roots` function for
  `walrus` custom sections, which in turn allows walrus GC passes to do
  what `unexport_unused_intrinsics` did before. That function is now no
  longer necessary, but effectively works the same way. All intrinsics
  are unexported at the beginning and then they're selectively
  re-imported and re-exported through the JS glue generation pass as
  necessary and defined by the bindings.

* Passes like the `anyref` pass are now much more precise about the
  intrinsics that they work with. The `anyref` pass also deletes any
  internal intrinsics found and also does some rewriting of the adapters
  aftewards now to hook up calls to the heap count import to the heap
  count intrinsic in the wasm module.

* Fix handling of __wbindgen_realloc

The final user of the `require_internal_export` function was
`__wbindgen_realloc`. This usage has now been removed by updating how we
handle usage of the `realloc` function.

The wasm interface types standard doesn't have a `realloc` function
slot, nor do I think it ever will. This means that as a polyfill for
wasm interface types we'll always have to support the lack of `realloc`.
For direct Rust to JS, however, we can still optionally handle
`realloc`. This is all handled with a few internal changes.

* Custom `StringToMemory` instructions now exist. These have an extra
  `realloc` slot to store an intrinsic, if found.
* Our custom instructions are lowered to the standard instructions when
  generating an interface types section.
* The `realloc` function, if present, is passed as an argument like the
  malloc function when passing strings to wasm. If it's not present we
  use a slower fallback, but if it's present we use the faster
  implementation.

This should mean that there's little-to-no impact on existing users of
`wasm-bindgen`, but this should continue to still work for wasm
interface types polyfills and such. Additionally the GC passes now work
in that they don't delete `__wbindgen_realloc` which we later try to
reference.

* Add an empty test for the anyref pass

* Precisely track I32FromOptionAnyref's dependencies

This depends on the anyref table and a function to allocate an index if
the anyref pass is running, so be sure to track that in the instruction
itself for GC rooting.

* Trim extraneous exports from nop anyref module

Or if you're otherwise not using anyref slices, don't force some
intrinsics to exist.

* Remove globals from reference tests

Looks like these values adjust in slight but insignificant ways over
time

* Update the anyref xform tests
This commit is contained in:
Alex Crichton
2019-12-04 12:01:39 -06:00
committed by GitHub
parent 9469c1641b
commit d7a4a772cf
52 changed files with 1263 additions and 357 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
crates/anyref-xform/src/lib.rs
View File

@ -48,6 +48,7 @@ pub struct Meta {
pub table: TableId, pub table: TableId,
pub alloc: Option<FunctionId>, pub alloc: Option<FunctionId>,
pub drop_slice: Option<FunctionId>, pub drop_slice: Option<FunctionId>,
pub live_count: Option<FunctionId>,
} }
struct Transform<'a> { struct Transform<'a> {
@ -178,20 +179,27 @@ impl Context {
let mut heap_alloc = None; let mut heap_alloc = None;
let mut heap_dealloc = None; let mut heap_dealloc = None;
let mut drop_slice = None; let mut drop_slice = None;
let mut live_count = None;
// Find exports of some intrinsics which we only need for a runtime // Find exports of some intrinsics which we only need for a runtime
// implementation. // implementation.
let mut to_delete = Vec::new();
for export in module.exports.iter() { for export in module.exports.iter() {
let f = match export.item { let f = match export.item {
walrus::ExportItem::Function(f) => f, walrus::ExportItem::Function(f) => f,
_ => continue, _ => continue,
}; };
match export.name.as_str() { match export.name.as_str() {
"__wbindgen_anyref_table_alloc" => heap_alloc = Some(f), "__anyref_table_alloc" => heap_alloc = Some(f),
"__wbindgen_anyref_table_dealloc" => heap_dealloc = Some(f), "__anyref_table_dealloc" => heap_dealloc = Some(f),
"__wbindgen_drop_anyref_slice" => drop_slice = Some(f), "__anyref_drop_slice" => drop_slice = Some(f),
"__anyref_heap_live_count" => live_count = Some(f),
_ => continue, _ => continue,
} }
to_delete.push(export.id());
}
for id in to_delete {
module.exports.delete(id);
} }
let mut clone_ref = None; let mut clone_ref = None;
if let Some(heap_alloc) = heap_alloc { if let Some(heap_alloc) = heap_alloc {
@ -240,6 +248,7 @@ impl Context {
table, table,
alloc: heap_alloc, alloc: heap_alloc,
drop_slice, drop_slice,
live_count,
}) })
} }
} }

9
crates/anyref-xform/tests/anyref-param-owned.wat
View File

@ -2,9 +2,9 @@
(module (module
(func $foo (export "foo") (param i32)) (func $foo (export "foo") (param i32))
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
@ -23,9 +23,6 @@
(func $alloc (type 0) (result i32) (func $alloc (type 0) (result i32)
i32.const 0) i32.const 0)
(func $foo (type 1) (param i32)) (func $foo (type 1) (param i32))
(func $dealloc (type 1) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func $foo anyref shim)) (export "foo" (func $foo anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

20
crates/anyref-xform/tests/anyref-param.wat
View File

@ -2,17 +2,16 @@
(module (module
(func $foo (export "foo") (param i32)) (func $foo (export "foo") (param i32))
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
(module (module
(type (;0;) (func (result i32))) (type (;0;) (func (param i32)))
(type (;1;) (func (param i32))) (type (;1;) (func (param anyref)))
(type (;2;) (func (param anyref))) (func $foo anyref shim (type 1) (param anyref)
(func $foo anyref shim (type 2) (param anyref)
(local i32) (local i32)
global.get 0 global.get 0
i32.const 1 i32.const 1
@ -31,13 +30,8 @@
i32.const 1 i32.const 1
i32.add i32.add
global.set 0) global.set 0)
(func $alloc (type 0) (result i32) (func $foo (type 0) (param i32))
i32.const 0)
(func $foo (type 1) (param i32))
(func $dealloc (type 1) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(global (;0;) (mut i32) (i32.const 32)) (global (;0;) (mut i32) (i32.const 32))
(export "foo" (func $foo anyref shim)) (export "foo" (func $foo anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

8
crates/anyref-xform/tests/clone-ref-intrinsic.wat
View File

@ -6,9 +6,9 @@
(func $foo (export "foo") (param i32) (result i32) (func $foo (export "foo") (param i32) (result i32)
local.get 0 local.get 0
call $clone) call $clone)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
@ -44,7 +44,5 @@
i32.const 0) i32.const 0)
(func $dealloc (type 1) (param i32)) (func $dealloc (type 1) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func $foo anyref shim)) (export "foo" (func $foo anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

8
crates/anyref-xform/tests/drop-ref-intrinsic.wat
View File

@ -6,9 +6,9 @@
(func $foo (export "foo") (param i32) (func $foo (export "foo") (param i32)
local.get 0 local.get 0
call $drop) call $drop)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
@ -31,7 +31,5 @@
i32.const 0) i32.const 0)
(func $dealloc (type 1) (param i32)) (func $dealloc (type 1) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func $foo anyref shim)) (export "foo" (func $foo anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

21
crates/anyref-xform/tests/import-anyref-owned.wat
View File

@ -5,19 +5,18 @@
(func (export "foo") (func (export "foo")
i32.const 0 i32.const 0
call $a) call $a)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
(module (module
(type (;0;) (func)) (type (;0;) (func))
(type (;1;) (func (result i32))) (type (;1;) (func (param i32)))
(type (;2;) (func (param i32))) (type (;2;) (func (param anyref)))
(type (;3;) (func (param anyref))) (import "" "a" (func $a (type 2)))
(import "" "a" (func $a (type 3))) (func $a anyref shim (type 1) (param i32)
(func $a anyref shim (type 2) (param i32)
local.get 0 local.get 0
table.get 0 table.get 0
local.get 0 local.get 0
@ -26,11 +25,7 @@
(func (;2;) (type 0) (func (;2;) (type 0)
i32.const 0 i32.const 0
call $a anyref shim) call $a anyref shim)
(func $alloc (type 1) (result i32) (func $dealloc (type 1) (param i32))
i32.const 0)
(func $dealloc (type 2) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func 2)) (export "foo" (func 2)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

10
crates/anyref-xform/tests/import-anyref-ret.wat
View File

@ -4,16 +4,15 @@
(import "" "a" (func $a (result i32))) (import "" "a" (func $a (result i32)))
(func (export "foo") (result i32) (func (export "foo") (result i32)
call $a) call $a)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
(module (module
(type (;0;) (func (result i32))) (type (;0;) (func (result i32)))
(type (;1;) (func (result anyref))) (type (;1;) (func (result anyref)))
(type (;2;) (func (param i32)))
(import "" "a" (func $a (type 1))) (import "" "a" (func $a (type 1)))
(func $a anyref shim (type 0) (result i32) (func $a anyref shim (type 0) (result i32)
(local i32 anyref) (local i32 anyref)
@ -28,9 +27,6 @@
call $a anyref shim) call $a anyref shim)
(func $alloc (type 0) (result i32) (func $alloc (type 0) (result i32)
i32.const 0) i32.const 0)
(func $dealloc (type 2) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func 2)) (export "foo" (func 2)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

20
crates/anyref-xform/tests/import-anyref.wat
View File

@ -5,30 +5,24 @@
(func (export "foo") (func (export "foo")
i32.const 0 i32.const 0
call $a) call $a)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
(module (module
(type (;0;) (func)) (type (;0;) (func))
(type (;1;) (func (result i32))) (type (;1;) (func (param i32)))
(type (;2;) (func (param i32))) (type (;2;) (func (param anyref)))
(type (;3;) (func (param anyref))) (import "" "a" (func $a (type 2)))
(import "" "a" (func $a (type 3))) (func $a anyref shim (type 1) (param i32)
(func $a anyref shim (type 2) (param i32)
local.get 0 local.get 0
table.get 0 table.get 0
call $a) call $a)
(func (;2;) (type 0) (func (;2;) (type 0)
i32.const 0 i32.const 0
call $a anyref shim) call $a anyref shim)
(func $alloc (type 1) (result i32)
i32.const 0)
(func $dealloc (type 2) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func 2)) (export "foo" (func 2)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

18
crates/anyref-xform/tests/mixed-export.wat
View File

@ -2,18 +2,17 @@
(module (module
(func $a (export "a") (param f32 i32 i64 i32 i32)) (func $a (export "a") (param f32 i32 i64 i32 i32))
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
(module (module
(type (;0;) (func (result i32))) (type (;0;) (func (result i32)))
(type (;1;) (func (param i32))) (type (;1;) (func (param f32 i32 i64 i32 i32)))
(type (;2;) (func (param f32 i32 i64 i32 i32))) (type (;2;) (func (param f32 anyref i64 anyref i32)))
(type (;3;) (func (param f32 anyref i64 anyref i32))) (func $a anyref shim (type 2) (param f32 anyref i64 anyref i32)
(func $a anyref shim (type 3) (param f32 anyref i64 anyref i32)
(local i32 i32) (local i32 i32)
global.get 0 global.get 0
i32.const 1 i32.const 1
@ -42,11 +41,8 @@
global.set 0) global.set 0)
(func $alloc (type 0) (result i32) (func $alloc (type 0) (result i32)
i32.const 0) i32.const 0)
(func $a (type 2) (param f32 i32 i64 i32 i32)) (func $a (type 1) (param f32 i32 i64 i32 i32))
(func $dealloc (type 1) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(global (;0;) (mut i32) (i32.const 32)) (global (;0;) (mut i32) (i32.const 32))
(export "a" (func $a anyref shim)) (export "a" (func $a anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

23
crates/anyref-xform/tests/mixed.wat
View File

@ -9,20 +9,19 @@
i32.const 4 i32.const 4
i32.const 5 i32.const 5
call $a) call $a)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
(module (module
(type (;0;) (func)) (type (;0;) (func))
(type (;1;) (func (result i32))) (type (;1;) (func (param i32)))
(type (;2;) (func (param i32))) (type (;2;) (func (param f32 i32 i64 i32 i32)))
(type (;3;) (func (param f32 i32 i64 i32 i32))) (type (;3;) (func (param f32 anyref i64 anyref i32)))
(type (;4;) (func (param f32 anyref i64 anyref i32))) (import "" "a" (func $a (type 3)))
(import "" "a" (func $a (type 4))) (func $a anyref shim (type 2) (param f32 i32 i64 i32 i32)
(func $a anyref shim (type 3) (param f32 i32 i64 i32 i32)
local.get 0 local.get 0
local.get 1 local.get 1
table.get 0 table.get 0
@ -40,11 +39,7 @@
i32.const 4 i32.const 4
i32.const 5 i32.const 5
call $a anyref shim) call $a anyref shim)
(func $alloc (type 1) (result i32) (func $dealloc (type 1) (param i32))
i32.const 0)
(func $dealloc (type 2) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func 2)) (export "foo" (func 2)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

10
crates/anyref-xform/tests/ret-anyref.wat
View File

@ -4,9 +4,9 @@
(func $foo (export "foo") (result i32) (func $foo (export "foo") (result i32)
i32.const 0) i32.const 0)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
@ -23,11 +23,7 @@
call $dealloc) call $dealloc)
(func $foo (type 0) (result i32) (func $foo (type 0) (result i32)
i32.const 0) i32.const 0)
(func $alloc (type 0) (result i32)
i32.const 0)
(func $dealloc (type 2) (param i32)) (func $dealloc (type 2) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func $foo anyref shim)) (export "foo" (func $foo anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

9
crates/anyref-xform/tests/table-grow-intrinsic.wat
View File

@ -7,9 +7,9 @@
i32.const 0 i32.const 0
call $grow call $grow
drop) drop)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
@ -32,9 +32,6 @@
drop) drop)
(func $alloc (type 0) (result i32) (func $alloc (type 0) (result i32)
i32.const 0) i32.const 0)
(func $dealloc (type 1) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func $foo anyref shim)) (export "foo" (func $foo anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

9
crates/anyref-xform/tests/table-set-null-intrinsic.wat
View File

@ -6,9 +6,9 @@
(func $foo (export "foo") (param i32) (func $foo (export "foo") (param i32)
local.get 0 local.get 0
call $set-null) call $set-null)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
@ -30,9 +30,6 @@
table.set 0) table.set 0)
(func $alloc (type 0) (result i32) (func $alloc (type 0) (result i32)
i32.const 0) i32.const 0)
(func $dealloc (type 1) (param i32))
(table (;0;) 32 anyref) (table (;0;) 32 anyref)
(export "foo" (func $foo anyref shim)) (export "foo" (func $foo anyref shim)))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc)))
;) ;)

7
crates/anyref-xform/tests/table.wat
View File

@ -4,9 +4,9 @@
(func $foo (param i32)) (func $foo (param i32))
(table (export "func") 0 funcref) (table (export "func") 0 funcref)
(elem (i32.const 0) 0) (elem (i32.const 0) 0)
(func $alloc (export "__wbindgen_anyref_table_alloc") (result i32) (func $alloc (export "__anyref_table_alloc") (result i32)
i32.const 0) i32.const 0)
(func $dealloc (export "__wbindgen_anyref_table_dealloc") (param i32)) (func $dealloc (export "__anyref_table_dealloc") (param i32))
) )
(; CHECK-ALL: (; CHECK-ALL:
@ -25,11 +25,8 @@
(func $alloc (type 0) (result i32) (func $alloc (type 0) (result i32)
i32.const 0) i32.const 0)
(func $foo (type 1) (param i32)) (func $foo (type 1) (param i32))
(func $dealloc (type 1) (param i32))
(table (;0;) 2 funcref) (table (;0;) 2 funcref)
(table (;1;) 32 anyref) (table (;1;) 32 anyref)
(export "func" (table 0)) (export "func" (table 0))
(export "__wbindgen_anyref_table_alloc" (func $alloc))
(export "__wbindgen_anyref_table_dealloc" (func $dealloc))
(elem (;0;) (i32.const 0) $foo $closure0 anyref shim)) (elem (;0;) (i32.const 0) $foo $closure0 anyref shim))
;) ;)

125
crates/cli-support/src/anyref.rs
View File

@ -1,3 +1,6 @@
use crate::descriptor::VectorKind;
use crate::intrinsic::Intrinsic;
use crate::wit::AuxImport;
use crate::wit::{AdapterKind, Instruction, NonstandardWitSection}; use crate::wit::{AdapterKind, Instruction, NonstandardWitSection};
use crate::wit::{AdapterType, InstructionData, StackChange, WasmBindgenAux}; use crate::wit::{AdapterType, InstructionData, StackChange, WasmBindgenAux};
use anyhow::Error; use anyhow::Error;
@ -17,7 +20,7 @@ pub fn process(module: &mut Module) -> Result<(), Error> {
.implements .implements
.iter() .iter()
.cloned() .cloned()
.map(|(core, adapter)| (adapter, core)) .map(|(core, _, adapter)| (adapter, core))
.collect::<HashMap<_, _>>(); .collect::<HashMap<_, _>>();
// Transform all exported functions in the module, using the bindings listed // Transform all exported functions in the module, using the bindings listed
@ -45,13 +48,73 @@ pub fn process(module: &mut Module) -> Result<(), Error> {
let meta = cfg.run(module)?; let meta = cfg.run(module)?;
let mut aux = module
.customs
.delete_typed::<WasmBindgenAux>()
.expect("wit custom section should exist");
let section = module let section = module
.customs .customs
.get_typed_mut::<WasmBindgenAux>() .get_typed_mut::<NonstandardWitSection>()
.expect("wit custom section should exist"); .expect("wit custom section should exist");
section.anyref_table = Some(meta.table);
section.anyref_alloc = meta.alloc; // If the module looks like it's going to use some of these exports, store
section.anyref_drop_slice = meta.drop_slice; // them in the aux section to get used.
//
// FIXME: this is not great, we should ideally have precise tracking of what
// requires what. These are used by catch clauses and anyref slices going
// in/out of wasm. The catch clauses are a bit weird but anyref slices
// should ideally track in their own instructions what table/functions
// they're referencing. This doesn't fit well in today's model of
// slice-related instructions, though, so let's just cop out and only enable
// these coarsely.
aux.anyref_table = Some(meta.table);
if module_needs_anyref_metadata(&aux, section) {
aux.anyref_alloc = meta.alloc;
aux.anyref_drop_slice = meta.drop_slice;
}
// Additonally we may need to update some adapter instructions other than
// those found for the anyref pass. These are some general "fringe support"
// things necessary to get absolutely everything working.
for (_, adapter) in section.adapters.iter_mut() {
let instrs = match &mut adapter.kind {
AdapterKind::Local { instructions } => instructions,
AdapterKind::Import { .. } => continue,
};
for instr in instrs {
match instr.instr {
// Calls to the heap live count intrinsic are now routed to the
// actual wasm function which keeps track of this.
Instruction::CallAdapter(adapter) => {
let id = match meta.live_count {
Some(id) => id,
None => continue,
};
let import = match aux.import_map.get(&adapter) {
Some(import) => import,
None => continue,
};
match import {
AuxImport::Intrinsic(Intrinsic::AnyrefHeapLiveCount) => {}
_ => continue,
}
instr.instr = Instruction::Standard(wit_walrus::Instruction::CallCore(id));
}
// Optional anyref values are now managed in the wasm module, so
// we need to store where they're managed.
Instruction::I32FromOptionAnyref {
ref mut table_and_alloc,
} => {
*table_and_alloc = meta.alloc.map(|id| (meta.table, id));
}
_ => continue,
};
}
}
module.customs.add(*aux);
Ok(()) Ok(())
} }
@ -267,3 +330,55 @@ fn export_xform(cx: &mut Context, export: Export, instrs: &mut Vec<InstructionDa
instrs.remove(idx); instrs.remove(idx);
} }
} }
/// This function shouldn't need to exist, see the fixme at the call-site.
fn module_needs_anyref_metadata(aux: &WasmBindgenAux, section: &NonstandardWitSection) -> bool {
use Instruction::*;
// our `handleError` intrinsic uses a few pieces of metadata to store
// indices directly into the wasm module.
if aux.imports_with_catch.len() > 0 {
return true;
}
// Look for any instructions which may use `VectorKind::Anyref`. If there
// are any then we'll need our intrinsics/tables/etc, otherwise we shouldn't
// ever need them.
section.adapters.iter().any(|(_, adapter)| {
let instructions = match &adapter.kind {
AdapterKind::Local { instructions } => instructions,
AdapterKind::Import { .. } => return false,
};
instructions.iter().any(|instr| match instr.instr {
VectorToMemory {
kind: VectorKind::Anyref,
..
}
| MutableSliceToMemory {
kind: VectorKind::Anyref,
..
}
| OptionVector {
kind: VectorKind::Anyref,
..
}
| VectorLoad {
kind: VectorKind::Anyref,
..
}
| OptionVectorLoad {
kind: VectorKind::Anyref,
..
}
| View {
kind: VectorKind::Anyref,
..
}
| OptionView {
kind: VectorKind::Anyref,
..
} => true,
_ => false,
})
})
}

21
crates/cli-support/src/descriptors.rs
View File

@ -37,27 +37,6 @@ pub fn execute(module: &mut Module) -> Result<WasmBindgenDescriptorsSectionId, E
section.execute_exports(module, &mut interpreter)?; section.execute_exports(module, &mut interpreter)?;
section.execute_closures(module, &mut interpreter)?; section.execute_closures(module, &mut interpreter)?;
// Delete all descriptor functions and imports from the module now that
// we've executed all of them.
//
// Note though that during this GC pass it's a bit aggressive in that it can
// delete the function table entirely. We don't actually know at this point
// whether we need the function table or not. The bindings generation may
// need to export the table so the JS glue can call functions in it, and
// that's only discovered during binding selection. For now we just add
// synthetic root exports for all tables in the module, and then we delete
// the exports just after GC. This should keep tables like the function
// table alive during GC all the way through to the bindings generation
// where we can either actually export it or gc it out since it's not used.
let mut exported_tables = Vec::new();
for table in module.tables.iter() {
exported_tables.push(module.exports.add("foo", table.id()));
}
walrus::passes::gc::run(module);
for export in exported_tables {
module.exports.delete(export);
}
Ok(module.customs.add(section)) Ok(module.customs.add(section))
} }

98
crates/cli-support/src/js/binding.rs
View File

@ -107,7 +107,12 @@ impl<'a, 'b> Builder<'a, 'b> {
instructions: &[InstructionData], instructions: &[InstructionData],
explicit_arg_names: &Option<Vec<String>>, explicit_arg_names: &Option<Vec<String>>,
) -> Result<String, Error> { ) -> Result<String, Error> {
if self.cx.aux.imports_with_assert_no_shim.contains(&adapter.id) { if self
.cx
.aux
.imports_with_assert_no_shim
.contains(&adapter.id)
{
bail!("generating a shim for something asserted to have no shim"); bail!("generating a shim for something asserted to have no shim");
} }
@ -414,6 +419,35 @@ impl<'a, 'b> JsBuilder<'a, 'b> {
arg, arg,
)); ));
} }
fn string_to_memory(
&mut self,
mem: walrus::MemoryId,
malloc: walrus::FunctionId,
realloc: Option<walrus::FunctionId>,
) -> Result<(), Error> {
self.typescript_required("string");
let pass = self.cx.expose_pass_string_to_wasm(mem)?;
let val = self.pop();
let malloc = self.cx.export_name_of(malloc);
let i = self.tmp();
let realloc = match realloc {
Some(f) => format!(", wasm.{}", self.cx.export_name_of(f)),
None => String::new(),
};
self.prelude(&format!(
"var ptr{i} = {f}({0}, wasm.{malloc}{realloc});",
val,
i = i,
f = pass,
malloc = malloc,
realloc = realloc,
));
self.prelude(&format!("var len{} = WASM_VECTOR_LEN;", i));
self.push(format!("ptr{}", i));
self.push(format!("len{}", i));
Ok(())
}
} }
fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) -> Result<(), Error> { fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) -> Result<(), Error> {
@ -512,21 +546,15 @@ fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) ->
} }
Instruction::Standard(wit_walrus::Instruction::StringToMemory { mem, malloc }) => { Instruction::Standard(wit_walrus::Instruction::StringToMemory { mem, malloc }) => {
js.typescript_required("string"); js.string_to_memory(*mem, *malloc, None)?;
let pass = js.cx.expose_pass_string_to_wasm(*mem)?; }
let val = js.pop();
let malloc = js.cx.export_name_of(*malloc); Instruction::StringToMemory {
let i = js.tmp(); mem,
js.prelude(&format!( malloc,
"var ptr{i} = {f}({0}, wasm.{malloc});", realloc,
val, } => {
i = i, js.string_to_memory(*mem, *malloc, *realloc)?;
f = pass,
malloc = malloc,
));
js.prelude(&format!("var len{} = WASM_VECTOR_LEN;", i));
js.push(format!("ptr{}", i));
js.push(format!("len{}", i));
} }
Instruction::Retptr => js.stack.push(retptr_val.to_string()), Instruction::Retptr => js.stack.push(retptr_val.to_string()),
@ -679,16 +707,16 @@ fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) ->
js.push(format!("high{}", i)); js.push(format!("high{}", i));
} }
Instruction::I32FromOptionAnyref => { Instruction::I32FromOptionAnyref { table_and_alloc } => {
js.typescript_optional("any"); js.typescript_optional("any");
let val = js.pop(); let val = js.pop();
js.cx.expose_is_like_none(); js.cx.expose_is_like_none();
match (js.cx.aux.anyref_table, js.cx.aux.anyref_alloc) { match table_and_alloc {
(Some(table), Some(alloc)) => { Some((table, alloc)) => {
let alloc = js.cx.expose_add_to_anyref_table(table, alloc)?; let alloc = js.cx.expose_add_to_anyref_table(*table, *alloc)?;
js.push(format!("isLikeNone({0}) ? 0 : {1}({0})", val, alloc)); js.push(format!("isLikeNone({0}) ? 0 : {1}({0})", val, alloc));
} }
_ => { None => {
js.cx.expose_add_heap_object(); js.cx.expose_add_heap_object();
js.push(format!("isLikeNone({0}) ? 0 : addHeapObject({0})", val)); js.push(format!("isLikeNone({0}) ? 0 : addHeapObject({0})", val));
} }
@ -756,6 +784,34 @@ fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) ->
js.push(format!("len{}", i)); js.push(format!("len{}", i));
} }
Instruction::OptionString {
mem,
malloc,
realloc,
} => {
js.typescript_optional("string");
let func = js.cx.expose_pass_string_to_wasm(*mem)?;
js.cx.expose_is_like_none();
let i = js.tmp();
let malloc = js.cx.export_name_of(*malloc);
let val = js.pop();
let realloc = match realloc {
Some(f) => format!(", wasm.{}", js.cx.export_name_of(*f)),
None => String::new(),
};
js.prelude(&format!(
"var ptr{i} = isLikeNone({0}) ? 0 : {f}({0}, wasm.{malloc}{realloc});",
val,
i = i,
f = func,
malloc = malloc,
realloc = realloc,
));
js.prelude(&format!("var len{} = WASM_VECTOR_LEN;", i));
js.push(format!("ptr{}", i));
js.push(format!("len{}", i));
}
Instruction::OptionVector { kind, mem, malloc } => { Instruction::OptionVector { kind, mem, malloc } => {
js.typescript_optional(kind.js_ty()); js.typescript_optional(kind.js_ty());
let func = js.cx.pass_to_wasm_function(*kind, *mem)?; let func = js.cx.pass_to_wasm_function(*kind, *mem)?;

171
crates/cli-support/src/js/mod.rs
View File

@ -20,7 +20,6 @@ pub struct Context<'a> {
imports_post: String, imports_post: String,
typescript: String, typescript: String,
exposed_globals: Option<HashSet<Cow<'static, str>>>, exposed_globals: Option<HashSet<Cow<'static, str>>>,
required_internal_exports: HashSet<Cow<'static, str>>,
next_export_idx: usize, next_export_idx: usize,
config: &'a Bindgen, config: &'a Bindgen,
pub module: &'a mut Module, pub module: &'a mut Module,
@ -87,7 +86,6 @@ impl<'a> Context<'a> {
imports_post: String::new(), imports_post: String::new(),
typescript: "/* tslint:disable */\n".to_string(), typescript: "/* tslint:disable */\n".to_string(),
exposed_globals: Some(Default::default()), exposed_globals: Some(Default::default()),
required_internal_exports: Default::default(),
imported_names: Default::default(), imported_names: Default::default(),
js_imports: Default::default(), js_imports: Default::default(),
defined_identifiers: Default::default(), defined_identifiers: Default::default(),
@ -169,33 +167,12 @@ impl<'a> Context<'a> {
Ok(()) Ok(())
} }
fn require_internal_export(&mut self, name: &'static str) -> Result<(), Error> {
if !self.required_internal_exports.insert(name.into()) {
return Ok(());
}
if self.module.exports.iter().any(|e| e.name == name) {
return Ok(());
}
bail!(
"the exported function `{}` is required to generate bindings \
but it was not found in the wasm file, perhaps the `std` feature \
of the `wasm-bindgen` crate needs to be enabled?",
name
);
}
pub fn finalize(&mut self, module_name: &str) -> Result<(String, String), Error> { pub fn finalize(&mut self, module_name: &str) -> Result<(String, String), Error> {
// Finalize all bindings for JS classes. This is where we'll generate JS // Finalize all bindings for JS classes. This is where we'll generate JS
// glue for all classes as well as finish up a few final imports like // glue for all classes as well as finish up a few final imports like
// `__wrap` and such. // `__wrap` and such.
self.write_classes()?; self.write_classes()?;
// We're almost done here, so we can delete any internal exports (like
// `__wbindgen_malloc`) if none of our JS glue actually needed it.
self.unexport_unused_internal_exports();
// Initialization is just flat out tricky and not something we // Initialization is just flat out tricky and not something we
// understand super well. To try to handle various issues that have come // understand super well. To try to handle various issues that have come
// up we always remove the `start` function if one is present. The JS // up we always remove the `start` function if one is present. The JS
@ -203,22 +180,6 @@ impl<'a> Context<'a> {
// previously present). // previously present).
let needs_manual_start = self.unstart_start_function(); let needs_manual_start = self.unstart_start_function();
// After all we've done, especially
// `unexport_unused_internal_exports()`, we probably have a bunch of
// garbage in the module that's no longer necessary, so delete
// everything that we don't actually need. Afterwards make sure we don't
// try to emit bindings for now-nonexistent imports by pruning our
// `wasm_import_definitions` set.
walrus::passes::gc::run(self.module);
let remaining_imports = self
.module
.imports
.iter()
.map(|i| i.id())
.collect::<HashSet<_>>();
self.wasm_import_definitions
.retain(|id, _| remaining_imports.contains(id));
// Cause any future calls to `should_write_global` to panic, making sure // Cause any future calls to `should_write_global` to panic, making sure
// we don't ask for items which we can no longer emit. // we don't ask for items which we can no longer emit.
drop(self.exposed_globals.take().unwrap()); drop(self.exposed_globals.take().unwrap());
@ -739,25 +700,6 @@ impl<'a> Context<'a> {
Ok(()) Ok(())
} }
fn unexport_unused_internal_exports(&mut self) {
let mut to_remove = Vec::new();
for export in self.module.exports.iter() {
match export.name.as_str() {
// Otherwise only consider our special exports, which all start
// with the same prefix which hopefully only we're using.
n if n.starts_with("__wbindgen") => {
if !self.required_internal_exports.contains(n) {
to_remove.push(export.id());
}
}
_ => {}
}
}
for id in to_remove {
self.module.exports.delete(id);
}
}
fn expose_drop_ref(&mut self) { fn expose_drop_ref(&mut self) {
if !self.should_write_global("drop_ref") { if !self.should_write_global("drop_ref") {
return; return;
@ -962,8 +904,6 @@ impl<'a> Context<'a> {
} }
} }
self.require_internal_export("__wbindgen_realloc")?;
// A fast path that directly writes char codes into WASM memory as long // A fast path that directly writes char codes into WASM memory as long
// as it finds only ASCII characters. // as it finds only ASCII characters.
// //
@ -975,10 +915,18 @@ impl<'a> Context<'a> {
// charCodeAt on ASCII strings is usually optimised to raw bytes. // charCodeAt on ASCII strings is usually optimised to raw bytes.
let encode_as_ascii = format!( let encode_as_ascii = format!(
"\ "\
if (realloc === undefined) {{
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length);
{mem}().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}}
let len = arg.length; let len = arg.length;
let ptr = malloc(len); let ptr = malloc(len);
const mem = {}(); const mem = {mem}();
let offset = 0; let offset = 0;
@ -988,7 +936,7 @@ impl<'a> Context<'a> {
mem[ptr + offset] = code; mem[ptr + offset] = code;
}} }}
", ",
mem mem = mem,
); );
// TODO: // TODO:
@ -997,14 +945,14 @@ impl<'a> Context<'a> {
// looping over the string to calculate the precise size, or perhaps using // looping over the string to calculate the precise size, or perhaps using
// `shrink_to_fit` on the Rust side. // `shrink_to_fit` on the Rust side.
self.global(&format!( self.global(&format!(
"function {name}(arg, malloc) {{ "function {name}(arg, malloc, realloc) {{
{debug} {debug}
{ascii} {ascii}
if (offset !== len) {{ if (offset !== len) {{
if (offset !== 0) {{ if (offset !== 0) {{
arg = arg.slice(offset); arg = arg.slice(offset);
}} }}
ptr = wasm.__wbindgen_realloc(ptr, len, len = offset + arg.length * 3); ptr = realloc(ptr, len, len = offset + arg.length * 3);
const view = {mem}().subarray(ptr + offset, ptr + len); const view = {mem}().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view); const ret = encodeString(arg, view);
{debug_end} {debug_end}
@ -1585,7 +1533,11 @@ impl<'a> Context<'a> {
if !self.should_write_global("handle_error") { if !self.should_write_global("handle_error") {
return Ok(()); return Ok(());
} }
self.require_internal_export("__wbindgen_exn_store")?; let store = self
.aux
.exn_store
.ok_or_else(|| anyhow!("failed to find `__wbindgen_exn_store` intrinsic"))?;
let store = self.export_name_of(store);
match (self.aux.anyref_table, self.aux.anyref_alloc) { match (self.aux.anyref_table, self.aux.anyref_alloc) {
(Some(table), Some(alloc)) => { (Some(table), Some(alloc)) => {
let add = self.expose_add_to_anyref_table(table, alloc)?; let add = self.expose_add_to_anyref_table(table, alloc)?;
@ -1593,21 +1545,22 @@ impl<'a> Context<'a> {
" "
function handleError(e) {{ function handleError(e) {{
const idx = {}(e); const idx = {}(e);
wasm.__wbindgen_exn_store(idx); wasm.{}(idx);
}} }}
", ",
add, add, store,
)); ));
} }
_ => { _ => {
self.expose_add_heap_object(); self.expose_add_heap_object();
self.global( self.global(&format!(
" "
function handleError(e) { function handleError(e) {{
wasm.__wbindgen_exn_store(addHeapObject(e)); wasm.{}(addHeapObject(e));
} }}
", ",
); store,
));
} }
} }
Ok(()) Ok(())
@ -1943,9 +1896,9 @@ impl<'a> Context<'a> {
let kind = match self.aux.export_map.get(&id) { let kind = match self.aux.export_map.get(&id) {
Some(export) => Kind::Export(export), Some(export) => Kind::Export(export),
None => { None => {
let core = self.wit.implements.iter().find(|pair| pair.1 == id); let core = self.wit.implements.iter().find(|pair| pair.2 == id);
match core { match core {
Some((core, _)) => Kind::Import(*core), Some((core, _, _)) => Kind::Import(*core),
None => Kind::Adapter, None => Kind::Adapter,
} }
} }
@ -2708,35 +2661,22 @@ impl<'a> Context<'a> {
Intrinsic::AnyrefHeapLiveCount => { Intrinsic::AnyrefHeapLiveCount => {
assert_eq!(args.len(), 0); assert_eq!(args.len(), 0);
if self.config.anyref { self.expose_global_heap();
// Eventually we should add support to the anyref-xform to prelude.push_str(
// re-write calls to the imported "
// `__wbindgen_anyref_heap_live_count` function into calls to let free_count = 0;
// the exported `__wbindgen_anyref_heap_live_count_impl` let next = heap_next;
// function, and to un-export that function. while (next < heap.length) {
// free_count += 1;
// But for now, we just bounce wasm -> js -> wasm because it is next = heap[next];
// easy. }
self.require_internal_export("__wbindgen_anyref_heap_live_count_impl")?; ",
"wasm.__wbindgen_anyref_heap_live_count_impl()".into() );
} else { format!(
self.expose_global_heap(); "heap.length - free_count - {} - {}",
prelude.push_str( INITIAL_HEAP_OFFSET,
" INITIAL_HEAP_VALUES.len(),
let free_count = 0; )
let next = heap_next;
while (next < heap.length) {
free_count += 1;
next = heap[next];
}
",
);
format!(
"heap.length - free_count - {} - {}",
INITIAL_HEAP_OFFSET,
INITIAL_HEAP_VALUES.len(),
)
}
} }
Intrinsic::InitAnyrefTable => { Intrinsic::InitAnyrefTable => {
@ -2953,15 +2893,32 @@ impl<'a> Context<'a> {
} }
}); });
if let Some(export) = export { if let Some(export) = export {
self.required_internal_exports
.insert(export.name.clone().into());
return export.name.clone(); return export.name.clone();
} }
let name = format!("__wbindgen_export_{}", self.next_export_idx); let default_name = format!("__wbindgen_export_{}", self.next_export_idx);
self.next_export_idx += 1; self.next_export_idx += 1;
let name = match id {
walrus::ExportItem::Function(f) => match &self.module.funcs.get(f).name {
Some(s) => to_js_identifier(s),
None => default_name,
},
_ => default_name,
};
self.module.exports.add(&name, id); self.module.exports.add(&name, id);
self.required_internal_exports.insert(name.clone().into());
return name; return name;
// Not really an exhaustive list, but works for our purposes.
fn to_js_identifier(name: &str) -> String {
name.chars()
.map(|c| {
if c.is_ascii() && (c.is_alphabetic() || c.is_numeric()) {
c
} else {
'_'
}
})
.collect()
}
} }
fn adapter_name(&self, id: AdapterId) -> String { fn adapter_name(&self, id: AdapterId) -> String {

81
crates/cli-support/src/lib.rs
View File

@ -1,14 +1,13 @@
#![doc(html_root_url = "https://docs.rs/wasm-bindgen-cli-support/0.2")] #![doc(html_root_url = "https://docs.rs/wasm-bindgen-cli-support/0.2")]
use anyhow::{bail, Context, Error}; use anyhow::{bail, Context, Error};
use std::collections::{BTreeMap, BTreeSet, HashMap}; use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet};
use std::env; use std::env;
use std::fs; use std::fs;
use std::mem; use std::mem;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::str; use std::str;
use walrus::Module; use walrus::Module;
use wasm_bindgen_wasm_conventions as wasm_conventions;
mod anyref; mod anyref;
mod decode; mod decode;
@ -280,15 +279,6 @@ impl Bindgen {
} }
}; };
// Our threads and multi-value xforms rely on the presence of the stack
// pointer, so temporarily export it so that our many GC's don't remove
// it before the xform runs.
let mut exported_shadow_stack_pointer = false;
if self.multi_value || self.threads.is_enabled(&module) {
wasm_conventions::export_shadow_stack_pointer(&mut module)?;
exported_shadow_stack_pointer = true;
}
// This isn't the hardest thing in the world too support but we // This isn't the hardest thing in the world too support but we
// basically don't know how to rationalize #[wasm_bindgen(start)] and // basically don't know how to rationalize #[wasm_bindgen(start)] and
// the actual `start` function if present. Figure this out later if it // the actual `start` function if present. Figure this out later if it
@ -346,10 +336,29 @@ impl Bindgen {
// This is only done if the anyref pass is enabled, which it's // This is only done if the anyref pass is enabled, which it's
// currently off-by-default since `anyref` is still in development in // currently off-by-default since `anyref` is still in development in
// engines. // engines.
//
// If the anyref pass isn't necessary, then we blanket delete the
// export of all our anyref intrinsics which will get cleaned up in the
// GC pass before JS generation.
if self.anyref { if self.anyref {
anyref::process(&mut module)?; anyref::process(&mut module)?;
} else {
let ids = module
.exports
.iter()
.filter(|e| e.name.starts_with("__anyref"))
.map(|e| e.id())
.collect::<Vec<_>>();
for id in ids {
module.exports.delete(id);
}
} }
// We've done a whole bunch of transformations to the wasm module, many
// of which leave "garbage" lying around, so let's prune out all our
// unnecessary things here.
gc_module_and_adapters(&mut module);
let aux = module let aux = module
.customs .customs
.delete_typed::<wit::WasmBindgenAux>() .delete_typed::<wit::WasmBindgenAux>()
@ -382,17 +391,6 @@ impl Bindgen {
} }
} }
// If we exported the shadow stack pointer earlier, remove it from the
// export set now.
if exported_shadow_stack_pointer {
wasm_conventions::unexport_shadow_stack_pointer(&mut module)?;
// The shadow stack pointer is potentially unused now, but since it
// most likely _is_ in use, we don't pay the cost of a full GC here
// just to remove one potentially unnecessary global.
//
// walrus::passes::gc::run(&mut module);
}
Ok(Output { Ok(Output {
module, module,
stem: stem.to_string(), stem: stem.to_string(),
@ -720,3 +718,42 @@ impl Output {
reset_indentation(&shim) reset_indentation(&shim)
} }
} }
fn gc_module_and_adapters(module: &mut Module) {
// First up we execute walrus's own gc passes, and this may enable us to
// delete entries in the `implements` section of the nonstandard wasm
// interface types section. (if the import is GC'd, then the implements
// annotation is no longer needed).
//
// By deleting adapter functions that may enable us to further delete more
// functions, so we run this in a loop until we don't actually delete any
// adapter functions.
loop {
walrus::passes::gc::run(module);
let imports_remaining = module
.imports
.iter()
.map(|i| i.id())
.collect::<HashSet<_>>();
let section = module
.customs
.get_typed_mut::<wit::NonstandardWitSection>()
.unwrap();
let mut deleted_implements = Vec::new();
section.implements.retain(|pair| {
if imports_remaining.contains(&pair.0) {
true
} else {
deleted_implements.push(pair.2);
false
}
});
if deleted_implements.len() == 0 {
break;
}
for id in deleted_implements {
section.adapters.remove(&id);
}
}
}

41
crates/cli-support/src/wit/incoming.rs
View File

@ -98,13 +98,13 @@ impl InstructionBuilder<'_, '_> {
Descriptor::Option(d) => self.incoming_option(d)?, Descriptor::Option(d) => self.incoming_option(d)?,
Descriptor::String | Descriptor::CachedString => { Descriptor::String | Descriptor::CachedString => {
let std = wit_walrus::Instruction::StringToMemory {
malloc: self.cx.malloc()?,
mem: self.cx.memory()?,
};
self.instruction( self.instruction(
&[AdapterType::String], &[AdapterType::String],
Instruction::Standard(std), Instruction::StringToMemory {
malloc: self.cx.malloc()?,
realloc: self.cx.realloc(),
mem: self.cx.memory()?,
},
&[AdapterType::I32, AdapterType::I32], &[AdapterType::I32, AdapterType::I32],
); );
} }
@ -163,13 +163,13 @@ impl InstructionBuilder<'_, '_> {
} }
Descriptor::String | Descriptor::CachedString => { Descriptor::String | Descriptor::CachedString => {
// This allocation is cleaned up once it's received in Rust. // This allocation is cleaned up once it's received in Rust.
let std = wit_walrus::Instruction::StringToMemory {
malloc: self.cx.malloc()?,
mem: self.cx.memory()?,
};
self.instruction( self.instruction(
&[AdapterType::String], &[AdapterType::String],
Instruction::Standard(std), Instruction::StringToMemory {
malloc: self.cx.malloc()?,
realloc: self.cx.realloc(),
mem: self.cx.memory()?,
},
&[AdapterType::I32, AdapterType::I32], &[AdapterType::I32, AdapterType::I32],
); );
} }
@ -218,7 +218,9 @@ impl InstructionBuilder<'_, '_> {
Descriptor::Anyref => { Descriptor::Anyref => {
self.instruction( self.instruction(
&[AdapterType::Anyref], &[AdapterType::Anyref],
Instruction::I32FromOptionAnyref, Instruction::I32FromOptionAnyref {
table_and_alloc: None,
},
&[AdapterType::I32], &[AdapterType::I32],
); );
} }
@ -272,7 +274,22 @@ impl InstructionBuilder<'_, '_> {
); );
} }
Descriptor::String | Descriptor::CachedString | Descriptor::Vector(_) => { Descriptor::String | Descriptor::CachedString => {
let malloc = self.cx.malloc()?;
let mem = self.cx.memory()?;
let realloc = self.cx.realloc();
self.instruction(
&[AdapterType::Anyref],
Instruction::OptionString {
malloc,
mem,
realloc,
},
&[AdapterType::I32; 2],
);
}
Descriptor::Vector(_) => {
let kind = arg.vector_kind().ok_or_else(|| { let kind = arg.vector_kind().ok_or_else(|| {
format_err!( format_err!(
"unsupported optional slice type for calling Rust function from JS {:?}", "unsupported optional slice type for calling Rust function from JS {:?}",

75
crates/cli-support/src/wit/mod.rs
View File

@ -83,6 +83,8 @@ pub fn process(
cx.verify()?; cx.verify()?;
cx.unexport_intrinsics();
let adapters = cx.module.customs.add(cx.adapters); let adapters = cx.module.customs.add(cx.adapters);
let aux = cx.module.customs.add(cx.aux); let aux = cx.module.customs.add(cx.aux);
Ok((adapters, aux)) Ok((adapters, aux))
@ -502,9 +504,15 @@ impl<'a> Context<'a> {
// itself but to the adapter shim we generated, so fetch that shim id // itself but to the adapter shim we generated, so fetch that shim id
// and flag it as catch here. This basically just needs to be kept in // and flag it as catch here. This basically just needs to be kept in
// sync with `js/mod.rs`. // sync with `js/mod.rs`.
let adapter = self.adapters.implements.last().unwrap().1; //
// For `catch` once we see that we'll need an internal intrinsic later
// for JS glue generation, so be sure to find that here.
let adapter = self.adapters.implements.last().unwrap().2;
if *catch { if *catch {
self.aux.imports_with_catch.insert(adapter); self.aux.imports_with_catch.insert(adapter);
if self.aux.exn_store.is_none() {
self.find_exn_store();
}
} }
if *assert_no_shim { if *assert_no_shim {
self.aux.imports_with_assert_no_shim.insert(adapter); self.aux.imports_with_assert_no_shim.insert(adapter);
@ -983,7 +991,7 @@ impl<'a> Context<'a> {
}; };
self.adapters self.adapters
.implements .implements
.push((import_id, walrus2us[&i.adapter_func])); .push((import_id, i.core_func, walrus2us[&i.adapter_func]));
} }
Ok(()) Ok(())
} }
@ -995,7 +1003,7 @@ impl<'a> Context<'a> {
// `$PLACEHOLDER_MODULE` are connected to an adapter via the // `$PLACEHOLDER_MODULE` are connected to an adapter via the
// `implements` section. // `implements` section.
let mut implemented = HashMap::new(); let mut implemented = HashMap::new();
for (core, adapter) in self.adapters.implements.iter() { for (core, _, adapter) in self.adapters.implements.iter() {
implemented.insert(core, adapter); implemented.insert(core, adapter);
} }
for import in self.module.imports.iter() { for import in self.module.imports.iter() {
@ -1006,6 +1014,15 @@ impl<'a> Context<'a> {
walrus::ImportKind::Function(_) => {} walrus::ImportKind::Function(_) => {}
_ => bail!("import from `{}` was not a function", PLACEHOLDER_MODULE), _ => bail!("import from `{}` was not a function", PLACEHOLDER_MODULE),
} }
// These are special intrinsics which were handled in the descriptor
// phase, but we don't have an implementation for them. We don't
// need to error about them in this verification pass though,
// having them lingering in the module is normal.
if import.name == "__wbindgen_describe" || import.name == "__wbindgen_describe_closure"
{
continue;
}
if implemented.remove(&import.id()).is_none() { if implemented.remove(&import.id()).is_none() {
bail!("import of `{}` doesn't have an adapter listed", import.name); bail!("import of `{}` doesn't have an adapter listed", import.name);
} }
@ -1065,6 +1082,10 @@ impl<'a> Context<'a> {
let import = self.module.imports.get(import); let import = self.module.imports.get(import);
let (import_module, import_name) = (import.module.clone(), import.name.clone()); let (import_module, import_name) = (import.module.clone(), import.name.clone());
let import_id = import.id(); let import_id = import.id();
let core_id = match import.kind {
walrus::ImportKind::Function(f) => f,
_ => bail!("bound import must be assigned to function"),
};
// Process the returned type first to see if it needs an out-pointer. This // Process the returned type first to see if it needs an out-pointer. This
// happens if the results of the incoming arguments translated to wasm take // happens if the results of the incoming arguments translated to wasm take
@ -1131,7 +1152,7 @@ impl<'a> Context<'a> {
.cx .cx
.adapters .adapters
.append(args.input, results, AdapterKind::Local { instructions }); .append(args.input, results, AdapterKind::Local { instructions });
args.cx.adapters.implements.push((import_id, id)); args.cx.adapters.implements.push((import_id, core_id, id));
Ok(f) Ok(f)
} }
@ -1233,6 +1254,13 @@ impl<'a> Context<'a> {
.ok_or_else(|| anyhow!("failed to find declaration of `__wbindgen_malloc` in module")) .ok_or_else(|| anyhow!("failed to find declaration of `__wbindgen_malloc` in module"))
} }
fn realloc(&self) -> Option<FunctionId> {
self.function_exports
.get("__wbindgen_realloc")
.cloned()
.map(|p| p.1)
}
fn free(&self) -> Result<FunctionId, Error> { fn free(&self) -> Result<FunctionId, Error> {
self.function_exports self.function_exports
.get("__wbindgen_free") .get("__wbindgen_free")
@ -1245,6 +1273,45 @@ impl<'a> Context<'a> {
self.memory self.memory
.ok_or_else(|| anyhow!("failed to find memory declaration in module")) .ok_or_else(|| anyhow!("failed to find memory declaration in module"))
} }
/// Removes the export item for all `__wbindgen` intrinsics which are
/// generally only purely internal helpers.
///
/// References to these functions are preserved through adapter instructions
/// if necessary, otherwise they can all be gc'd out. By the time this
/// function is called our discovery of these intrinsics has completed and
/// there's no need to keep around these exports.
fn unexport_intrinsics(&mut self) {
let mut to_remove = Vec::new();
for export in self.module.exports.iter() {
match export.name.as_str() {
n if n.starts_with("__wbindgen") => {
to_remove.push(export.id());
}
_ => {}
}
}
for id in to_remove {
self.module.exports.delete(id);
}
}
/// Attempts to locate the `__wbindgen_exn_store` intrinsic and stores it in
/// our auxiliary information.
///
/// This is only invoked if the intrinsic will actually be needed for JS
/// glue generation somewhere.
fn find_exn_store(&mut self) {
self.aux.exn_store = self
.module
.exports
.iter()
.find(|e| e.name == "__wbindgen_exn_store")
.and_then(|e| match e.item {
walrus::ExportItem::Function(f) => Some(f),
_ => None,
});
}
} }
fn extract_programs<'a>( fn extract_programs<'a>(

18
crates/cli-support/src/wit/nonstandard.rs
View File

@ -52,6 +52,9 @@ pub struct WasmBindgenAux {
pub anyref_table: Option<walrus::TableId>, pub anyref_table: Option<walrus::TableId>,
pub anyref_alloc: Option<walrus::FunctionId>, pub anyref_alloc: Option<walrus::FunctionId>,
pub anyref_drop_slice: Option<walrus::FunctionId>, pub anyref_drop_slice: Option<walrus::FunctionId>,
/// Various intrinsics used for JS glue generation
pub exn_store: Option<walrus::FunctionId>,
} }
pub type WasmBindgenAuxId = TypedCustomSectionId<WasmBindgenAux>; pub type WasmBindgenAuxId = TypedCustomSectionId<WasmBindgenAux>;
@ -357,4 +360,19 @@ impl walrus::CustomSection for WasmBindgenAux {
fn data(&self, _: &walrus::IdsToIndices) -> Cow<[u8]> { fn data(&self, _: &walrus::IdsToIndices) -> Cow<[u8]> {
panic!("shouldn't emit custom sections just yet"); panic!("shouldn't emit custom sections just yet");
} }
fn add_gc_roots(&self, roots: &mut walrus::passes::Roots) {
if let Some(id) = self.anyref_table {
roots.push_table(id);
}
if let Some(id) = self.anyref_alloc {
roots.push_func(id);
}
if let Some(id) = self.anyref_drop_slice {
roots.push_func(id);
}
if let Some(id) = self.exn_store {
roots.push_func(id);
}
}
} }

22
crates/cli-support/src/wit/section.rs
View File

@ -45,13 +45,14 @@ pub fn add(
anyref_table: _, // not relevant anyref_table: _, // not relevant
anyref_alloc: _, // not relevant anyref_alloc: _, // not relevant
anyref_drop_slice: _, // not relevant anyref_drop_slice: _, // not relevant
exn_store: _, // not relevant
} = aux; } = aux;
let adapter_context = |id: AdapterId| { let adapter_context = |id: AdapterId| {
if let Some((name, _)) = nonstandard.exports.iter().find(|p| p.1 == id) { if let Some((name, _)) = nonstandard.exports.iter().find(|p| p.1 == id) {
return format!("in function export `{}`", name); return format!("in function export `{}`", name);
} }
if let Some((core, _)) = nonstandard.implements.iter().find(|p| p.1 == id) { if let Some((core, _, _)) = nonstandard.implements.iter().find(|p| p.2 == id) {
let import = module.imports.get(*core); let import = module.imports.get(*core);
return format!( return format!(
"in function import from `{}::{}`", "in function import from `{}::{}`",
@ -107,12 +108,8 @@ pub fn add(
us2walrus.insert(*us, walrus); us2walrus.insert(*us, walrus);
} }
for (core, adapter) in nonstandard.implements.iter() { for (_, core, adapter) in nonstandard.implements.iter() {
let core = match module.imports.get(*core).kind { section.implements.add(us2walrus[adapter], *core);
walrus::ImportKind::Function(f) => f,
_ => bail!("cannot implement a non-function"),
};
section.implements.add(us2walrus[adapter], core);
} }
for (name, adapter) in nonstandard.exports.iter() { for (name, adapter) in nonstandard.exports.iter() {
@ -231,6 +228,14 @@ fn translate_instruction(
_ => bail!("expected to find an element of the function table"), _ => bail!("expected to find an element of the function table"),
} }
} }
StringToMemory {
mem,
malloc,
realloc: _,
} => Ok(wit_walrus::Instruction::StringToMemory {
mem: *mem,
malloc: *malloc,
}),
StoreRetptr { .. } | LoadRetptr { .. } | Retptr => { StoreRetptr { .. } | LoadRetptr { .. } | Retptr => {
bail!("return pointers aren't supported in wasm interface types"); bail!("return pointers aren't supported in wasm interface types");
} }
@ -250,7 +255,7 @@ fn translate_instruction(
bail!("64-bit integers aren't supported in wasm-bindgen"); bail!("64-bit integers aren't supported in wasm-bindgen");
} }
I32SplitOption64 { .. } I32SplitOption64 { .. }
| I32FromOptionAnyref | I32FromOptionAnyref { .. }
| I32FromOptionU32Sentinel | I32FromOptionU32Sentinel
| I32FromOptionRust { .. } | I32FromOptionRust { .. }
| I32FromOptionBool | I32FromOptionBool
@ -258,6 +263,7 @@ fn translate_instruction(
| I32FromOptionEnum { .. } | I32FromOptionEnum { .. }
| FromOptionNative { .. } | FromOptionNative { .. }
| OptionVector { .. } | OptionVector { .. }
| OptionString { .. }
| OptionRustFromI32 { .. } | OptionRustFromI32 { .. }
| OptionVectorLoad { .. } | OptionVectorLoad { .. }
| OptionView { .. } | OptionView { .. }

91
crates/cli-support/src/wit/standard.rs
View File

@ -1,7 +1,7 @@
use crate::descriptor::VectorKind; use crate::descriptor::VectorKind;
use std::borrow::Cow; use std::borrow::Cow;
use std::collections::HashMap; use std::collections::HashMap;
use walrus::{ImportId, TypedCustomSectionId}; use walrus::{FunctionId, ImportId, TypedCustomSectionId};
#[derive(Default, Debug)] #[derive(Default, Debug)]
pub struct NonstandardWitSection { pub struct NonstandardWitSection {
@ -9,7 +9,7 @@ pub struct NonstandardWitSection {
pub adapters: HashMap<AdapterId, Adapter>, pub adapters: HashMap<AdapterId, Adapter>,
/// A list of pairs for adapter functions that implement core wasm imports. /// A list of pairs for adapter functions that implement core wasm imports.
pub implements: Vec<(ImportId, AdapterId)>, pub implements: Vec<(ImportId, FunctionId, AdapterId)>,
/// A list of adapter functions and the names they're exported under. /// A list of adapter functions and the names they're exported under.
pub exports: Vec<(String, AdapterId)>, pub exports: Vec<(String, AdapterId)>,
@ -149,7 +149,11 @@ pub enum Instruction {
}, },
/// Pops an `anyref` from the stack, pushes either 0 if it's "none" or and /// Pops an `anyref` from the stack, pushes either 0 if it's "none" or and
/// index into the owned wasm table it was stored at if it's "some" /// index into the owned wasm table it was stored at if it's "some"
I32FromOptionAnyref, I32FromOptionAnyref {
/// Set to `Some` by the anyref pass of where to put it in the wasm
/// module, otherwise it's shoved into the JS shim.
table_and_alloc: Option<(walrus::TableId, walrus::FunctionId)>,
},
/// Pops an `anyref` from the stack, pushes either a sentinel value if it's /// Pops an `anyref` from the stack, pushes either a sentinel value if it's
/// "none" or the integer value of it if it's "some" /// "none" or the integer value of it if it's "some"
I32FromOptionU32Sentinel, I32FromOptionU32Sentinel,
@ -186,6 +190,19 @@ pub enum Instruction {
mem: walrus::MemoryId, mem: walrus::MemoryId,
}, },
/// Pops a string, pushes pointer/length or all zeros
OptionString {
malloc: walrus::FunctionId,
mem: walrus::MemoryId,
realloc: Option<walrus::FunctionId>,
},
/// Pops a string, pushes pointer/length
StringToMemory {
malloc: walrus::FunctionId,
mem: walrus::MemoryId,
realloc: Option<walrus::FunctionId>,
},
/// Pops an anyref, pushes pointer/length or all zeros /// Pops an anyref, pushes pointer/length or all zeros
OptionVector { OptionVector {
kind: VectorKind, kind: VectorKind,
@ -356,4 +373,72 @@ impl walrus::CustomSection for NonstandardWitSection {
fn data(&self, _: &walrus::IdsToIndices) -> Cow<[u8]> { fn data(&self, _: &walrus::IdsToIndices) -> Cow<[u8]> {
panic!("shouldn't emit custom sections just yet"); panic!("shouldn't emit custom sections just yet");
} }
fn add_gc_roots(&self, roots: &mut walrus::passes::Roots) {
use Instruction::*;
for (_, adapter) in self.adapters.iter() {
let instrs = match &adapter.kind {
AdapterKind::Local { instructions } => instructions,
AdapterKind::Import { .. } => continue,
};
for instr in instrs {
match instr.instr {
Standard(wit_walrus::Instruction::DeferCallCore(f))
| Standard(wit_walrus::Instruction::CallCore(f)) => {
roots.push_func(f);
}
StoreRetptr { mem, .. }
| LoadRetptr { mem, .. }
| View { mem, .. }
| OptionView { mem, .. }
| Standard(wit_walrus::Instruction::MemoryToString(mem)) => {
roots.push_memory(mem);
}
VectorToMemory { malloc, mem, .. }
| OptionVector { malloc, mem, .. }
| Standard(wit_walrus::Instruction::StringToMemory { mem, malloc }) => {
roots.push_memory(mem);
roots.push_func(malloc);
}
MutableSliceToMemory {
free, malloc, mem, ..
} => {
roots.push_memory(mem);
roots.push_func(malloc);
roots.push_func(free);
}
VectorLoad { free, mem, .. }
| OptionVectorLoad { free, mem, .. }
| CachedStringLoad { free, mem, .. } => {
roots.push_memory(mem);
roots.push_func(free);
}
OptionString {
mem,
malloc,
realloc,
}
| StringToMemory {
mem,
malloc,
realloc,
} => {
roots.push_memory(mem);
roots.push_func(malloc);
if let Some(id) = realloc {
roots.push_func(id);
}
}
I32FromOptionAnyref { table_and_alloc } => {
if let Some((table, alloc)) = table_and_alloc {
roots.push_table(table);
roots.push_func(alloc);
}
}
_ => {}
}
}
}
}
} }

9
crates/cli/Cargo.toml
View File

@ -30,7 +30,16 @@ wasm-bindgen-shared = { path = "../shared", version = "=0.2.55" }
[dev-dependencies] [dev-dependencies]
assert_cmd = "0.11" assert_cmd = "0.11"
diff = "0.1"
predicates = "1.0.0" predicates = "1.0.0"
rayon = "1.0"
tempfile = "3.0"
walrus = "0.14"
wasmprinter = "0.2"
[[test]]
name = "reference"
harness = false
[features] [features]
vendored-openssl = ['openssl/vendored'] vendored-openssl = ['openssl/vendored']

225
crates/cli/tests/reference.rs Normal file
View File

@ -0,0 +1,225 @@
//! A test suite to check the reference JS and wasm output of the `wasm-bindgen`
//! library.
//!
//! This is intended as an end-to-end integration test where we can track
//! changes to the JS and wasm output.
//!
//! Tests are located in `reference/*.rs` files and are accompanied with sibling
//! `*.js` files and `*.wat` files with the expected output of the `*.rs`
//! compilation. Use `BLESS=1` in the environment to automatically update all
//! tests.
use anyhow::{bail, Result};
use assert_cmd::prelude::*;
use rayon::prelude::*;
use std::env;
use std::fs;
use std::path::{Path, PathBuf};
use std::process::Command;
fn main() -> Result<()> {
let filter = env::args().nth(1);
let mut tests = Vec::new();
let dir = env::current_dir()?.join("tests/reference");
for entry in dir.read_dir()? {
let path = entry?.path();
if path.extension().and_then(|s| s.to_str()) != Some("rs") {
continue;
}
if let Some(filter) = &filter {
if !path.display().to_string().contains(filter) {
continue;
}
}
tests.push(path);
}
tests.sort();
let errs = tests
.par_iter()
.filter_map(|t| runtest(t).err().map(|e| (t, e)))
.collect::<Vec<_>>();
if errs.len() == 0 {
println!("{} tests passed", tests.len());
return Ok(());
}
eprintln!("failed tests:\n");
for (test, err) in errs {
eprintln!("{} failure\n{}", test.display(), tab(&format!("{:?}", err)));
}
bail!("tests failed");
}
fn runtest(test: &Path) -> Result<()> {
let contents = fs::read_to_string(test)?;
let td = tempfile::TempDir::new()?;
let manifest = format!(
"
[package]
name = \"reference-test\"
authors = []
version = \"1.0.0\"
edition = '2018'
[dependencies]
wasm-bindgen = {{ path = '{}' }}
[lib]
crate-type = ['cdylib']
path = '{}'
",
repo_root().display(),
test.display(),
);
fs::write(td.path().join("Cargo.toml"), manifest)?;
let target_dir = target_dir();
exec(
Command::new("cargo")
.current_dir(td.path())
.arg("build")
.arg("--target")
.arg("wasm32-unknown-unknown")
.env("CARGO_TARGET_DIR", &target_dir),
)?;
let wasm = target_dir
.join("wasm32-unknown-unknown")
.join("debug")
.join("reference_test.wasm");
let mut bindgen = Command::cargo_bin("wasm-bindgen")?;
bindgen
.arg("--out-dir")
.arg(td.path())
.arg(&wasm)
.arg("--no-typescript");
if contents.contains("// enable-anyref") {
bindgen.env("WASM_BINDGEN_ANYREF", "1");
}
exec(&mut bindgen)?;
let js = fs::read_to_string(td.path().join("reference_test.js"))?;
let wat = sanitize_wasm(&td.path().join("reference_test_bg.wasm"))?;
let js_assertion = test.with_extension("js");
let wat_assertion = test.with_extension("wat");
if env::var("BLESS").is_ok() {
fs::write(js_assertion, js)?;
fs::write(wat_assertion, wat)?;
return Ok(());
}
let js_expected = fs::read_to_string(&js_assertion)?;
let wat_expected = fs::read_to_string(&wat_assertion)?;
diff(&js_expected, &js)?;
diff(&wat_expected, &wat)?;
Ok(())
}
fn sanitize_wasm(wasm: &Path) -> Result<String> {
// Clean up the wasm module by removing all function
// implementations/instructions, data sections, etc. This'll help us largely
// only deal with exports/imports which is all we're really interested in.
let mut module = walrus::Module::from_file(wasm)?;
for func in module.funcs.iter_mut() {
let local = match &mut func.kind {
walrus::FunctionKind::Local(l) => l,
_ => continue,
};
local.block_mut(local.entry_block()).instrs.truncate(0);
}
let ids = module.data.iter().map(|d| d.id()).collect::<Vec<_>>();
for id in ids {
module.data.delete(id);
}
for mem in module.memories.iter_mut() {
mem.data_segments.drain();
}
let ids = module
.exports
.iter()
.filter(|e| match e.item {
walrus::ExportItem::Global(_) => true,
_ => false,
})
.map(|d| d.id())
.collect::<Vec<_>>();
for id in ids {
module.exports.delete(id);
}
walrus::passes::gc::run(&mut module);
let mut wat = wasmprinter::print_bytes(&module.emit_wasm())?;
wat.push_str("\n");
Ok(wat)
}
fn diff(a: &str, b: &str) -> Result<()> {
if a == b {
return Ok(());
}
let mut s = String::new();
for result in diff::lines(a, b) {
match result {
diff::Result::Both(l, _) => {
s.push_str(" ");
s.push_str(l);
}
diff::Result::Left(l) => {
s.push_str("-");
s.push_str(l);
}
diff::Result::Right(l) => {
s.push_str("+");
s.push_str(l);
}
}
s.push_str("\n");
}
bail!("found a difference:\n\n{}", s);
}
fn target_dir() -> PathBuf {
let mut dir = PathBuf::from(env::current_exe().unwrap());
dir.pop(); // current exe
if dir.ends_with("deps") {
dir.pop();
}
dir.pop(); // debug and/or release
return dir;
}
fn repo_root() -> PathBuf {
let mut repo_root = env::current_dir().unwrap();
repo_root.pop(); // remove 'cli'
repo_root.pop(); // remove 'crates'
repo_root
}
fn exec(cmd: &mut Command) -> Result<()> {
let output = cmd.output()?;
if output.status.success() {
return Ok(());
}
let mut err = format!("command failed {:?}", cmd);
err.push_str(&format!("\nstatus: {}", output.status));
err.push_str(&format!(
"\nstderr:\n{}",
tab(&String::from_utf8_lossy(&output.stderr))
));
err.push_str(&format!(
"\nstdout:\n{}",
tab(&String::from_utf8_lossy(&output.stdout))
));
bail!("{}", err);
}
fn tab(s: &str) -> String {
format!(" {}", s.replace("\n", "\n "))
}

22
crates/cli/tests/reference/add.js Normal file
View File

@ -0,0 +1,22 @@
import * as wasm from './reference_test_bg.wasm';
/**
* @param {number} a
* @param {number} b
* @returns {number}
*/
export function add_u32(a, b) {
var ret = wasm.add_u32(a, b);
return ret >>> 0;
}
/**
* @param {number} a
* @param {number} b
* @returns {number}
*/
export function add_i32(a, b) {
var ret = wasm.add_i32(a, b);
return ret;
}

11
crates/cli/tests/reference/add.rs Normal file
View File

@ -0,0 +1,11 @@
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn add_u32(a: u32, b: u32) -> u32 {
a + b
}
#[wasm_bindgen]
pub fn add_i32(a: i32, b: i32) -> i32 {
a + b
}

8
crates/cli/tests/reference/add.wat Normal file
View File

@ -0,0 +1,8 @@
(module
(type (;0;) (func (param i32 i32) (result i32)))
(func $add_u32 (type 0) (param i32 i32) (result i32))
(func $add_i32 (type 0) (param i32 i32) (result i32))
(memory (;0;) 17)
(export "memory" (memory 0))
(export "add_u32" (func $add_u32))
(export "add_i32" (func $add_i32)))

15
crates/cli/tests/reference/anyref-empty.js Normal file
View File

@ -0,0 +1,15 @@
import * as wasm from './reference_test_bg.wasm';
export const __wbindgen_init_anyref_table = function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
};
wasm.__wbindgen_start();

1
crates/cli/tests/reference/anyref-empty.rs Normal file
View File

@ -0,0 +1 @@
// enable-anyref

8
crates/cli/tests/reference/anyref-empty.wat Normal file
View File

@ -0,0 +1,8 @@
(module
(type (;0;) (func))
(import "./reference_test.js" "__wbindgen_init_anyref_table" (func (;0;) (type 0)))
(table (;0;) 32 anyref)
(memory (;0;) 17)
(export "memory" (memory 0))
(export "__wbindgen_export_0" (table 0))
(export "__wbindgen_start" (func 0)))

65
crates/cli/tests/reference/anyref-import-catch.js Normal file
View File

@ -0,0 +1,65 @@
import * as wasm from './reference_test_bg.wasm';
const lTextDecoder = typeof TextDecoder === 'undefined' ? require('util').TextDecoder : TextDecoder;
let cachedTextDecoder = new lTextDecoder('utf-8', { ignoreBOM: true, fatal: true });
cachedTextDecoder.decode();
let cachegetUint8Memory0 = null;
function getUint8Memory0() {
if (cachegetUint8Memory0 === null || cachegetUint8Memory0.buffer !== wasm.memory.buffer) {
cachegetUint8Memory0 = new Uint8Array(wasm.memory.buffer);
}
return cachegetUint8Memory0;
}
function getStringFromWasm0(ptr, len) {
return cachedTextDecoder.decode(getUint8Memory0().subarray(ptr, ptr + len));
}
function addToAnyrefTable0(obj) {
const idx = wasm.__anyref_table_alloc();
wasm.__wbindgen_export_0.set(idx, obj);
return idx;
}
function handleError(e) {
const idx = addToAnyrefTable0(e);
wasm.__wbindgen_exn_store(idx);
}
/**
*/
export function exported() {
wasm.exported();
}
export const __wbg_foo_8d66ddef0ff279d6 = function() {
try {
foo();
} catch (e) {
handleError(e)
}
};
export const __wbindgen_throw = function(arg0, arg1) {
throw new Error(getStringFromWasm0(arg0, arg1));
};
export const __wbindgen_rethrow = function(arg0) {
throw arg0;
};
export const __wbindgen_init_anyref_table = function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
};
wasm.__wbindgen_start();

14
crates/cli/tests/reference/anyref-import-catch.rs Normal file
View File

@ -0,0 +1,14 @@
// enable-anyref
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(catch)]
fn foo() -> Result<(), JsValue>;
}
#[wasm_bindgen]
pub fn exported() -> Result<(), JsValue> {
foo()
}

16
crates/cli/tests/reference/anyref-import-catch.wat Normal file
View File

@ -0,0 +1,16 @@
(module
(type (;0;) (func))
(type (;1;) (func (result i32)))
(type (;2;) (func (param i32)))
(import "./reference_test.js" "__wbindgen_init_anyref_table" (func (;0;) (type 0)))
(func $__wbindgen_exn_store (type 2) (param i32))
(func $exported (type 0))
(func $__anyref_table_alloc (type 1) (result i32))
(table (;0;) 32 anyref)
(memory (;0;) 17)
(export "memory" (memory 0))
(export "exported" (func $exported))
(export "__wbindgen_export_0" (table 0))
(export "__wbindgen_exn_store" (func $__wbindgen_exn_store))
(export "__anyref_table_alloc" (func $__anyref_table_alloc))
(export "__wbindgen_start" (func 0)))

21
crates/cli/tests/reference/anyref-nop.js Normal file
View File

@ -0,0 +1,21 @@
import * as wasm from './reference_test_bg.wasm';
/**
*/
export function foo() {
wasm.foo();
}
export const __wbindgen_init_anyref_table = function() {
const table = wasm.__wbindgen_export_0;
const offset = table.grow(4);
table.set(0, undefined);
table.set(offset + 0, undefined);
table.set(offset + 1, null);
table.set(offset + 2, true);
table.set(offset + 3, false);
;
};
wasm.__wbindgen_start();

6
crates/cli/tests/reference/anyref-nop.rs Normal file
View File

@ -0,0 +1,6 @@
// enable-anyref
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn foo() {}

10
crates/cli/tests/reference/anyref-nop.wat Normal file
View File

@ -0,0 +1,10 @@
(module
(type (;0;) (func))
(import "./reference_test.js" "__wbindgen_init_anyref_table" (func (;0;) (type 0)))
(func $foo (type 0))
(table (;0;) 32 anyref)
(memory (;0;) 17)
(export "memory" (memory 0))
(export "foo" (func $foo))
(export "__wbindgen_export_0" (table 0))
(export "__wbindgen_start" (func 0)))

2
crates/cli/tests/reference/empty.js Normal file
View File

@ -0,0 +1,2 @@
import * as wasm from './reference_test_bg.wasm';

0
crates/cli/tests/reference/empty.rs Normal file
View File

3
crates/cli/tests/reference/empty.wat Normal file
View File

@ -0,0 +1,3 @@
(module
(memory (;0;) 17)
(export "memory" (memory 0)))

58
crates/cli/tests/reference/import-catch.js Normal file
View File

@ -0,0 +1,58 @@
import * as wasm from './reference_test_bg.wasm';
const heap = new Array(32);
heap.fill(undefined);
heap.push(undefined, null, true, false);
function getObject(idx) { return heap[idx]; }
let heap_next = heap.length;
function dropObject(idx) {
if (idx < 36) return;
heap[idx] = heap_next;
heap_next = idx;
}
function takeObject(idx) {
const ret = getObject(idx);
dropObject(idx);
return ret;
}
function addHeapObject(obj) {
if (heap_next === heap.length) heap.push(heap.length + 1);
const idx = heap_next;
heap_next = heap[idx];
heap[idx] = obj;
return idx;
}
function handleError(e) {
wasm.__wbindgen_exn_store(addHeapObject(e));
}
/**
*/
export function exported() {
wasm.exported();
}
export const __wbindgen_object_drop_ref = function(arg0) {
takeObject(arg0);
};
export const __wbg_foo_8d66ddef0ff279d6 = function() {
try {
foo();
} catch (e) {
handleError(e)
}
};
export const __wbindgen_rethrow = function(arg0) {
throw takeObject(arg0);
};

12
crates/cli/tests/reference/import-catch.rs Normal file
View File

@ -0,0 +1,12 @@
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
extern "C" {
#[wasm_bindgen(catch)]
fn foo() -> Result<(), JsValue>;
}
#[wasm_bindgen]
pub fn exported() -> Result<(), JsValue> {
foo()
}

9
crates/cli/tests/reference/import-catch.wat Normal file
View File

@ -0,0 +1,9 @@
(module
(type (;0;) (func))
(type (;1;) (func (param i32)))
(func $__wbindgen_exn_store (type 1) (param i32))
(func $exported (type 0))
(memory (;0;) 17)
(export "memory" (memory 0))
(export "exported" (func $exported))
(export "__wbindgen_exn_store" (func $__wbindgen_exn_store)))

8
crates/cli/tests/reference/nop.js Normal file
View File

@ -0,0 +1,8 @@
import * as wasm from './reference_test_bg.wasm';
/**
*/
export function nop() {
wasm.nop();
}

4
crates/cli/tests/reference/nop.rs Normal file
View File

@ -0,0 +1,4 @@
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn nop() {}

6
crates/cli/tests/reference/nop.wat Normal file
View File

@ -0,0 +1,6 @@
(module
(type (;0;) (func))
(func $nop (type 0))
(memory (;0;) 17)
(export "memory" (memory 0))
(export "nop" (func $nop)))

89
crates/cli/tests/reference/string-arg.js Normal file
View File

@ -0,0 +1,89 @@
import * as wasm from './reference_test_bg.wasm';
const lTextDecoder = typeof TextDecoder === 'undefined' ? require('util').TextDecoder : TextDecoder;
let cachedTextDecoder = new lTextDecoder('utf-8', { ignoreBOM: true, fatal: true });
cachedTextDecoder.decode();
let cachegetUint8Memory0 = null;
function getUint8Memory0() {
if (cachegetUint8Memory0 === null || cachegetUint8Memory0.buffer !== wasm.memory.buffer) {
cachegetUint8Memory0 = new Uint8Array(wasm.memory.buffer);
}
return cachegetUint8Memory0;
}
function getStringFromWasm0(ptr, len) {
return cachedTextDecoder.decode(getUint8Memory0().subarray(ptr, ptr + len));
}
let WASM_VECTOR_LEN = 0;
const lTextEncoder = typeof TextEncoder === 'undefined' ? require('util').TextEncoder : TextEncoder;
let cachedTextEncoder = new lTextEncoder('utf-8');
const encodeString = (typeof cachedTextEncoder.encodeInto === 'function'
? function (arg, view) {
return cachedTextEncoder.encodeInto(arg, view);
}
: function (arg, view) {
const buf = cachedTextEncoder.encode(arg);
view.set(buf);
return {
read: arg.length,
written: buf.length
};
});
function passStringToWasm0(arg, malloc, realloc) {
if (realloc === undefined) {
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length);
getUint8Memory0().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}
let len = arg.length;
let ptr = malloc(len);
const mem = getUint8Memory0();
let offset = 0;
for (; offset < len; offset++) {
const code = arg.charCodeAt(offset);
if (code > 0x7F) break;
mem[ptr + offset] = code;
}
if (offset !== len) {
if (offset !== 0) {
arg = arg.slice(offset);
}
ptr = realloc(ptr, len, len = offset + arg.length * 3);
const view = getUint8Memory0().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
offset += ret.written;
}
WASM_VECTOR_LEN = offset;
return ptr;
}
/**
* @param {string} a
*/
export function foo(a) {
var ptr0 = passStringToWasm0(a, wasm.__wbindgen_malloc, wasm.__wbindgen_realloc);
var len0 = WASM_VECTOR_LEN;
wasm.foo(ptr0, len0);
}
export const __wbindgen_throw = function(arg0, arg1) {
throw new Error(getStringFromWasm0(arg0, arg1));
};

6
crates/cli/tests/reference/string-arg.rs Normal file
View File

@ -0,0 +1,6 @@
use wasm_bindgen::prelude::*;
#[wasm_bindgen]
pub fn foo(a: &str) {
drop(a);
}

12
crates/cli/tests/reference/string-arg.wat Normal file
View File

@ -0,0 +1,12 @@
(module
(type (;0;) (func (param i32) (result i32)))
(type (;1;) (func (param i32 i32)))
(type (;2;) (func (param i32 i32 i32) (result i32)))
(func $__wbindgen_realloc (type 2) (param i32 i32 i32) (result i32))
(func $__wbindgen_malloc (type 0) (param i32) (result i32))
(func $foo (type 1) (param i32 i32))
(memory (;0;) 17)
(export "memory" (memory 0))
(export "foo" (func $foo))
(export "__wbindgen_malloc" (func $__wbindgen_malloc))
(export "__wbindgen_realloc" (func $__wbindgen_realloc)))

40
crates/wasm-conventions/src/lib.rs
View File

@ -29,11 +29,11 @@ pub fn get_memory(module: &Module) -> Result<MemoryId, Error> {
}) })
} }
/// Discover the shadow stack pointer and add it to the module's exports as /// Get the `__shadow_stack_pointer`.
/// `__shadow_stack_pointer`.
/// ///
/// Adding it to the exports is useful for making sure it doesn't get GC'd. /// It must have been previously added to the module's exports via
pub fn export_shadow_stack_pointer(module: &mut Module) -> Result<(), Error> { /// `export_shadow_stack_pointer`.
pub fn get_shadow_stack_pointer(module: &Module) -> Result<GlobalId, Error> {
let candidates = module let candidates = module
.globals .globals
.iter() .iter()
@ -57,35 +57,5 @@ pub fn export_shadow_stack_pointer(module: &mut Module) -> Result<(), Error> {
_ => bail!("too many mutable globals to infer which is the shadow stack pointer"), _ => bail!("too many mutable globals to infer which is the shadow stack pointer"),
}; };
module.exports.add("__shadow_stack_pointer", ssp); Ok(ssp)
Ok(())
}
/// Unexport the shadow stack pointer that was previously added to the module's
/// exports as `__shadow_stack_pointer`.
pub fn unexport_shadow_stack_pointer(module: &mut Module) -> Result<(), Error> {
let e = module
.exports
.iter()
.find(|e| e.name == "__shadow_stack_pointer")
.map(|e| e.id())
.ok_or_else(|| anyhow!("did not find the `__shadow_stack_pointer` export in the module"))?;
module.exports.delete(e);
Ok(())
}
/// Get the `__shadow_stack_pointer`.
///
/// It must have been previously added to the module's exports via
/// `export_shadow_stack_pointer`.
pub fn get_shadow_stack_pointer(module: &Module) -> Result<GlobalId, Error> {
module
.exports
.iter()
.find(|e| e.name == "__shadow_stack_pointer")
.ok_or_else(|| anyhow!("did not find the `__shadow_stack_pointer` export in the module"))
.and_then(|e| match e.item {
walrus::ExportItem::Global(g) => Ok(g),
_ => bail!("`__shadow_stack_pointer` export is wrong kind"),
})
} }

10
src/anyref.rs
View File

@ -126,7 +126,7 @@ fn internal_error(msg: &str) -> ! {
std::thread_local!(pub static HEAP_SLAB: Cell<Slab> = Cell::new(Slab::new())); std::thread_local!(pub static HEAP_SLAB: Cell<Slab> = Cell::new(Slab::new()));
#[no_mangle] #[no_mangle]
pub extern "C" fn __wbindgen_anyref_table_alloc() -> usize { pub extern "C" fn __anyref_table_alloc() -> usize {
HEAP_SLAB HEAP_SLAB
.try_with(|slot| { .try_with(|slot| {
let mut slab = slot.replace(Slab::new()); let mut slab = slot.replace(Slab::new());
@ -138,7 +138,7 @@ pub extern "C" fn __wbindgen_anyref_table_alloc() -> usize {
} }
#[no_mangle] #[no_mangle]
pub extern "C" fn __wbindgen_anyref_table_dealloc(idx: usize) { pub extern "C" fn __anyref_table_dealloc(idx: usize) {
if idx < super::JSIDX_RESERVED as usize { if idx < super::JSIDX_RESERVED as usize {
return; return;
} }
@ -157,16 +157,16 @@ pub extern "C" fn __wbindgen_anyref_table_dealloc(idx: usize) {
} }
#[no_mangle] #[no_mangle]
pub unsafe extern "C" fn __wbindgen_drop_anyref_slice(ptr: *mut JsValue, len: usize) { pub unsafe extern "C" fn __anyref_drop_slice(ptr: *mut JsValue, len: usize) {
for slot in slice::from_raw_parts_mut(ptr, len) { for slot in slice::from_raw_parts_mut(ptr, len) {
__wbindgen_anyref_table_dealloc(slot.idx as usize); __anyref_table_dealloc(slot.idx as usize);
} }
} }
// Implementation of `__wbindgen_anyref_heap_live_count` for when we are using // Implementation of `__wbindgen_anyref_heap_live_count` for when we are using
// `anyref` instead of the JS `heap`. // `anyref` instead of the JS `heap`.
#[no_mangle] #[no_mangle]
pub unsafe extern "C" fn __wbindgen_anyref_heap_live_count_impl() -> u32 { pub unsafe extern "C" fn __anyref_heap_live_count() -> u32 {
HEAP_SLAB HEAP_SLAB
.try_with(|slot| { .try_with(|slot| {
let slab = slot.replace(Slab::new()); let slab = slot.replace(Slab::new());