* Add reference output tests for JS operations
This commit starts adding a test suite which checks in, to the
repository, test assertions for both the JS and wasm file outputs of a
Rust crate compiled with `#[wasm_bindgen]`. These aren't intended to be
exhaustive or large scale tests, but rather micro-tests to help observe
the changes in `wasm-bindgen`'s output over time.
The motivation for this commit is basically overhauling how all the GC
passes work in `wasm-bindgen` today. The reorganization is also included
in this commit as well.
Previously `wasm-bindgen` would, in an ad-hoc fashion, run the GC passes
of `walrus` in a bunch of places to ensure that less "garbage" was seen
by future passes. This not only was a source of slowdown but it also was
pretty brittle since `wasm-bindgen` kept breaking if extra iteams leaked
through.
The strategy taken in this commit is to have one precise location for a
GC pass, and everything goes through there. This is achieved by:
* All internal exports are removed immediately when generating the
nonstandard wasm interface types section. Internal exports,
intrinsics, and runtime support are all referenced by the various
instructions and/or sections that use them. This means that we now
have precise tracking of what an adapter uses.
* This in turn enables us to implement the `add_gc_roots` function for
`walrus` custom sections, which in turn allows walrus GC passes to do
what `unexport_unused_intrinsics` did before. That function is now no
longer necessary, but effectively works the same way. All intrinsics
are unexported at the beginning and then they're selectively
re-imported and re-exported through the JS glue generation pass as
necessary and defined by the bindings.
* Passes like the `anyref` pass are now much more precise about the
intrinsics that they work with. The `anyref` pass also deletes any
internal intrinsics found and also does some rewriting of the adapters
aftewards now to hook up calls to the heap count import to the heap
count intrinsic in the wasm module.
* Fix handling of __wbindgen_realloc
The final user of the `require_internal_export` function was
`__wbindgen_realloc`. This usage has now been removed by updating how we
handle usage of the `realloc` function.
The wasm interface types standard doesn't have a `realloc` function
slot, nor do I think it ever will. This means that as a polyfill for
wasm interface types we'll always have to support the lack of `realloc`.
For direct Rust to JS, however, we can still optionally handle
`realloc`. This is all handled with a few internal changes.
* Custom `StringToMemory` instructions now exist. These have an extra
`realloc` slot to store an intrinsic, if found.
* Our custom instructions are lowered to the standard instructions when
generating an interface types section.
* The `realloc` function, if present, is passed as an argument like the
malloc function when passing strings to wasm. If it's not present we
use a slower fallback, but if it's present we use the faster
implementation.
This should mean that there's little-to-no impact on existing users of
`wasm-bindgen`, but this should continue to still work for wasm
interface types polyfills and such. Additionally the GC passes now work
in that they don't delete `__wbindgen_realloc` which we later try to
reference.
* Add an empty test for the anyref pass
* Precisely track I32FromOptionAnyref's dependencies
This depends on the anyref table and a function to allocate an index if
the anyref pass is running, so be sure to track that in the instruction
itself for GC rooting.
* Trim extraneous exports from nop anyref module
Or if you're otherwise not using anyref slices, don't force some
intrinsics to exist.
* Remove globals from reference tests
Looks like these values adjust in slight but insignificant ways over
time
* Update the anyref xform tests
This commit is a pretty large scale rewrite of the internals of wasm-bindgen. No user-facing changes are expected as a result of this PR, but due to the scale of changes here it's likely inevitable that at least something will break. I'm hoping to get more testing in though before landing!
The purpose of this PR is to update wasm-bindgen to the current state of the interface types proposal. The wasm-bindgen tool was last updated when it was still called "WebIDL bindings" so it's been awhile! All support is now based on https://github.com/bytecodealliance/wasm-interface-types which defines parsers/binary format/writers/etc for wasm-interface types.
This is a pretty massive PR and unfortunately can't really be split up any more afaik. I don't really expect realistic review of all the code here (or commits), but some high-level changes are:
* Interface types now consists of a set of "adapter functions". The IR in wasm-bindgen is modeled the same way not.
* Each adapter function has a list of instructions, and these instructions work at a higher level than wasm itself, for example with strings.
* The wasm-bindgen tool has a suite of instructions which are specific to it and not present in the standard. (like before with webidl bindings)
* The anyref/multi-value transformations are now greatly simplified. They're simply "optimization passes" over adapter functions, removing instructions that are otherwise present. This way we don't have to juggle so much all over the place, and instructions always have the same meaning.
This commit adds support to attach `#[wasm_bindgen]` on an `async fn`
which will change the return value into a `Promise` in JS. This in
theory has the exact same semantics as an `async` function in JS where
you call it with all the arguments, nothing happens and you get a
promise back, and then later the promise actually resolves.
This commit also adds a helper trait, `IntoJsResult`, to allow `async`
functions with multiple kinds of return values instead of requiring
everything to be `Result<JsValue, JsValue>`.
This commit migrates all non-mutable slices incoming into Rust to use
the standard `AllocCopy` binding instead of using a custom `Slice`
binding defined by `wasm-bindgen`. This is done by freeing the memory
from Rust rather than freeing the memory from JS. We can't do this for
mutable slices yet but otherwise this should be working well!
We don't actually need this since we can simply pass in a number like 8
for the return pointer all the time. There's no need to allocate more
space in static data for a return pointer tha may not even get used!
This was used oh-so-long ago but hasn't been used in a very long time
since then. It's served no purpose for a very long time now and I don't
think we have a plan for giving it a purpose any time soon, so let's
remove it.
Instead of allocating space on the stack and returning a pointer we
should be able to use a single global memory location to communicate
this error payload information. This shouldn't run into any reentrancy
issues since it's only stored just before returning to wasm and it's
always read just after returning from wasm.
This allows using WebIDL bindings types to describe both of them instead
of having a custom ABI, allowing for more direct and rich bindings
eventually!
* Catch all closures by walking all `Descriptor` values and looking for
either `Function` or `Closure`.
* Update the correct arguments for wasm by ensuring that the closure
modifications skip the first two arguments.
This commit starts the `wasm-bindgen` CLI tool down the road to being a
true polyfill for WebIDL bindings. This refactor is probably the first
of a few, but is hopefully the largest and most sprawling and everything
will be a bit more targeted from here on out.
The goal of this refactoring is to separate out the massive
`crates/cli-support/src/js/mod.rs` into a number of separate pieces of
functionality. It currently takes care of basically everything
including:
* Binding intrinsics
* Handling anyref transformations
* Generating all JS for imports/exports
* All the logic for how to import and how to name imports
* Execution and management of wasm-bindgen closures
Many of these are separable concerns and most overlap with WebIDL
bindings. The internal refactoring here is intended to make it more
clear who's responsible for what as well as making some existing
operations much more straightforward. At a high-level, the following
changes are done:
1. A `src/webidl.rs` module is introduced. The purpose of this module is
to take all of the raw wasm-bindgen custom sections from the module
and transform them into a WebIDL bindings section.
This module has a placeholder `WebidlCustomSection` which is nowhere
near the actual custom section but if you squint is in theory very
similar. It's hoped that this will eventually become the true WebIDL
custom section, currently being developed in an external crate.
Currently, however, the WebIDL bindings custom section only covers a
subset of the functionality we export to wasm-bindgen users. To avoid
leaving them high and dry this module also contains an auxiliary
custom section named `WasmBindgenAux`. This custom section isn't
intended to have a binary format, but is intended to represent a
theoretical custom section necessary to couple with WebIDL bindings to
achieve all our desired functionality in `wasm-bindgen`. It'll never
be standardized, but it'll also never be serialized :)
2. The `src/webidl.rs` module now takes over quite a bit of
functionality from `src/js/mod.rs`. Namely it handles synthesis of an
`export_map` and an `import_map` mapping export/import IDs to exactly
what's expected to be hooked up there. This does not include type
information (as that's in the bindings section) but rather includes
things like "this is the method of class A" or "this import is from
module `foo`" and things like that. These could arguably be subsumed
by future JS features as well, but that's for another time!
3. All handling of wasm-bindgen "descriptor functions" now happens in a
dedicated `src/descriptors.rs` module. The output of this module is
its own custom section (intended to be immediately consumed by the
WebIDL module) which is in theory what we want to ourselves emit one
day but rustc isn't capable of doing so right now.
4. Invocations and generations of imports are completely overhauled.
Using the `import_map` generated in the WebIDL step all imports are
now handled much more precisely in one location rather than
haphazardly throughout the module. This means we have precise
information about each import of the module and we only modify
exactly what we're looking at. This also vastly simplifies intrinsic
generation since it's all simply a codegen part of the `rust2js.rs`
module now.
5. Handling of direct imports which don't have a JS shim generated is
slightly different from before and is intended to be
future-compatible with WebIDL bindings in its full glory, but we'll
need to update it to handle cases for constructors and method calls
eventually as well.
6. Intrinsic definitions now live in their own file (`src/intrinsic.rs`)
and have a separated definition for their symbol name and signature.
The actual implementation of each intrinsic lives in `rust2js.rs`
There's a number of TODO items to finish before this merges. This
includes reimplementing the anyref pass and actually implementing import
maps for other targets. Those will come soon in follow-up commits, but
the entire `tests/wasm/main.rs` suite is currently passing and this
seems like a good checkpoint.
