fluencelabs/wasm-bindgen
1
0
Fork 0
mirror of https://github.com/fluencelabs/wasm-bindgen synced 2025-06-19 16:01:23 +00:00
Code Issues Projects Releases Wiki Activity

Add reference output tests for JS operations ()

Browse Source 
* Add reference output tests for JS operations

This commit starts adding a test suite which checks in, to the
repository, test assertions for both the JS and wasm file outputs of a
Rust crate compiled with `#[wasm_bindgen]`. These aren't intended to be
exhaustive or large scale tests, but rather micro-tests to help observe
the changes in `wasm-bindgen`'s output over time.

The motivation for this commit is basically overhauling how all the GC
passes work in `wasm-bindgen` today. The reorganization is also included
in this commit as well.

Previously `wasm-bindgen` would, in an ad-hoc fashion, run the GC passes
of `walrus` in a bunch of places to ensure that less "garbage" was seen
by future passes. This not only was a source of slowdown but it also was
pretty brittle since `wasm-bindgen` kept breaking if extra iteams leaked
through.

The strategy taken in this commit is to have one precise location for a
GC pass, and everything goes through there. This is achieved by:

* All internal exports are removed immediately when generating the
  nonstandard wasm interface types section. Internal exports,
  intrinsics, and runtime support are all referenced by the various
  instructions and/or sections that use them. This means that we now
  have precise tracking of what an adapter uses.

* This in turn enables us to implement the `add_gc_roots` function for
  `walrus` custom sections, which in turn allows walrus GC passes to do
  what `unexport_unused_intrinsics` did before. That function is now no
  longer necessary, but effectively works the same way. All intrinsics
  are unexported at the beginning and then they're selectively
  re-imported and re-exported through the JS glue generation pass as
  necessary and defined by the bindings.

* Passes like the `anyref` pass are now much more precise about the
  intrinsics that they work with. The `anyref` pass also deletes any
  internal intrinsics found and also does some rewriting of the adapters
  aftewards now to hook up calls to the heap count import to the heap
  count intrinsic in the wasm module.

* Fix handling of __wbindgen_realloc

The final user of the `require_internal_export` function was
`__wbindgen_realloc`. This usage has now been removed by updating how we
handle usage of the `realloc` function.

The wasm interface types standard doesn't have a `realloc` function
slot, nor do I think it ever will. This means that as a polyfill for
wasm interface types we'll always have to support the lack of `realloc`.
For direct Rust to JS, however, we can still optionally handle
`realloc`. This is all handled with a few internal changes.

* Custom `StringToMemory` instructions now exist. These have an extra
  `realloc` slot to store an intrinsic, if found.
* Our custom instructions are lowered to the standard instructions when
  generating an interface types section.
* The `realloc` function, if present, is passed as an argument like the
  malloc function when passing strings to wasm. If it's not present we
  use a slower fallback, but if it's present we use the faster
  implementation.

This should mean that there's little-to-no impact on existing users of
`wasm-bindgen`, but this should continue to still work for wasm
interface types polyfills and such. Additionally the GC passes now work
in that they don't delete `__wbindgen_realloc` which we later try to
reference.

* Add an empty test for the anyref pass

* Precisely track I32FromOptionAnyref's dependencies

This depends on the anyref table and a function to allocate an index if
the anyref pass is running, so be sure to track that in the instruction
itself for GC rooting.

* Trim extraneous exports from nop anyref module

Or if you're otherwise not using anyref slices, don't force some
intrinsics to exist.

* Remove globals from reference tests

Looks like these values adjust in slight but insignificant ways over
time

* Update the anyref xform tests
This commit is contained in:
Alex Crichton
2019-12-04 12:01:39 -06:00
committed by GitHub
parent 9469c1641b
commit d7a4a772cf
52 changed files with 1263 additions and 357 deletions

98
crates/cli-support/src/js/binding.rs

@ -107,7 +107,12 @@ impl<'a, 'b> Builder<'a, 'b> {
instructions: &[InstructionData],
explicit_arg_names: &Option<Vec<String>>,
) -> Result<String, Error> {
if self.cx.aux.imports_with_assert_no_shim.contains(&adapter.id) {
if self
.cx
.aux
.imports_with_assert_no_shim
.contains(&adapter.id)
{
bail!("generating a shim for something asserted to have no shim");
}
@ -414,6 +419,35 @@ impl<'a, 'b> JsBuilder<'a, 'b> {
arg,
));
}
fn string_to_memory(
&mut self,
mem: walrus::MemoryId,
malloc: walrus::FunctionId,
realloc: Option<walrus::FunctionId>,
) -> Result<(), Error> {
self.typescript_required("string");
let pass = self.cx.expose_pass_string_to_wasm(mem)?;
let val = self.pop();
let malloc = self.cx.export_name_of(malloc);
let i = self.tmp();
let realloc = match realloc {
Some(f) => format!(", wasm.{}", self.cx.export_name_of(f)),
None => String::new(),
};
self.prelude(&format!(
"var ptr{i} = {f}({0}, wasm.{malloc}{realloc});",
val,
i = i,
f = pass,
malloc = malloc,
realloc = realloc,
));
self.prelude(&format!("var len{} = WASM_VECTOR_LEN;", i));
self.push(format!("ptr{}", i));
self.push(format!("len{}", i));
Ok(())
}
}
fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) -> Result<(), Error> {
@ -512,21 +546,15 @@ fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) ->
}
Instruction::Standard(wit_walrus::Instruction::StringToMemory { mem, malloc }) => {
js.typescript_required("string");
let pass = js.cx.expose_pass_string_to_wasm(*mem)?;
let val = js.pop();
let malloc = js.cx.export_name_of(*malloc);
let i = js.tmp();
js.prelude(&format!(
"var ptr{i} = {f}({0}, wasm.{malloc});",
val,
i = i,
f = pass,
malloc = malloc,
));
js.prelude(&format!("var len{} = WASM_VECTOR_LEN;", i));
js.push(format!("ptr{}", i));
js.push(format!("len{}", i));
js.string_to_memory(*mem, *malloc, None)?;
}
Instruction::StringToMemory {
mem,
malloc,
realloc,
} => {
js.string_to_memory(*mem, *malloc, *realloc)?;
}
Instruction::Retptr => js.stack.push(retptr_val.to_string()),
@ -679,16 +707,16 @@ fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) ->
js.push(format!("high{}", i));
}
Instruction::I32FromOptionAnyref => {
Instruction::I32FromOptionAnyref { table_and_alloc } => {
js.typescript_optional("any");
let val = js.pop();
js.cx.expose_is_like_none();
match (js.cx.aux.anyref_table, js.cx.aux.anyref_alloc) {
(Some(table), Some(alloc)) => {
let alloc = js.cx.expose_add_to_anyref_table(table, alloc)?;
match table_and_alloc {
Some((table, alloc)) => {
let alloc = js.cx.expose_add_to_anyref_table(*table, *alloc)?;
js.push(format!("isLikeNone({0}) ? 0 : {1}({0})", val, alloc));
}
_ => {
None => {
js.cx.expose_add_heap_object();
js.push(format!("isLikeNone({0}) ? 0 : addHeapObject({0})", val));
}
@ -756,6 +784,34 @@ fn instruction(js: &mut JsBuilder, instr: &Instruction, log_error: &mut bool) ->
js.push(format!("len{}", i));
}
Instruction::OptionString {
mem,
malloc,
realloc,
} => {
js.typescript_optional("string");
let func = js.cx.expose_pass_string_to_wasm(*mem)?;
js.cx.expose_is_like_none();
let i = js.tmp();
let malloc = js.cx.export_name_of(*malloc);
let val = js.pop();
let realloc = match realloc {
Some(f) => format!(", wasm.{}", js.cx.export_name_of(*f)),
None => String::new(),
};
js.prelude(&format!(
"var ptr{i} = isLikeNone({0}) ? 0 : {f}({0}, wasm.{malloc}{realloc});",
val,
i = i,
f = func,
malloc = malloc,
realloc = realloc,
));
js.prelude(&format!("var len{} = WASM_VECTOR_LEN;", i));
js.push(format!("ptr{}", i));
js.push(format!("len{}", i));
}
Instruction::OptionVector { kind, mem, malloc } => {
js.typescript_optional(kind.js_ty());
let func = js.cx.pass_to_wasm_function(*kind, *mem)?;

171
crates/cli-support/src/js/mod.rs

@ -20,7 +20,6 @@ pub struct Context<'a> {
imports_post: String,
typescript: String,
exposed_globals: Option<HashSet<Cow<'static, str>>>,
required_internal_exports: HashSet<Cow<'static, str>>,
next_export_idx: usize,
config: &'a Bindgen,
pub module: &'a mut Module,
@ -87,7 +86,6 @@ impl<'a> Context<'a> {
imports_post: String::new(),
typescript: "/* tslint:disable */\n".to_string(),
exposed_globals: Some(Default::default()),
required_internal_exports: Default::default(),
imported_names: Default::default(),
js_imports: Default::default(),
defined_identifiers: Default::default(),
@ -169,33 +167,12 @@ impl<'a> Context<'a> {
Ok(())
}
fn require_internal_export(&mut self, name: &'static str) -> Result<(), Error> {
if !self.required_internal_exports.insert(name.into()) {
return Ok(());
}
if self.module.exports.iter().any(|e| e.name == name) {
return Ok(());
}
bail!(
"the exported function `{}` is required to generate bindings \
but it was not found in the wasm file, perhaps the `std` feature \
of the `wasm-bindgen` crate needs to be enabled?",
name
);
}
pub fn finalize(&mut self, module_name: &str) -> Result<(String, String), Error> {
// Finalize all bindings for JS classes. This is where we'll generate JS
// glue for all classes as well as finish up a few final imports like
// `__wrap` and such.
self.write_classes()?;
// We're almost done here, so we can delete any internal exports (like
// `__wbindgen_malloc`) if none of our JS glue actually needed it.
self.unexport_unused_internal_exports();
// Initialization is just flat out tricky and not something we
// understand super well. To try to handle various issues that have come
// up we always remove the `start` function if one is present. The JS
@ -203,22 +180,6 @@ impl<'a> Context<'a> {
// previously present).
let needs_manual_start = self.unstart_start_function();
// After all we've done, especially
// `unexport_unused_internal_exports()`, we probably have a bunch of
// garbage in the module that's no longer necessary, so delete
// everything that we don't actually need. Afterwards make sure we don't
// try to emit bindings for now-nonexistent imports by pruning our
// `wasm_import_definitions` set.
walrus::passes::gc::run(self.module);
let remaining_imports = self
.module
.imports
.iter()
.map(|i| i.id())
.collect::<HashSet<_>>();
self.wasm_import_definitions
.retain(|id, _| remaining_imports.contains(id));
// Cause any future calls to `should_write_global` to panic, making sure
// we don't ask for items which we can no longer emit.
drop(self.exposed_globals.take().unwrap());
@ -739,25 +700,6 @@ impl<'a> Context<'a> {
Ok(())
}
fn unexport_unused_internal_exports(&mut self) {
let mut to_remove = Vec::new();
for export in self.module.exports.iter() {
match export.name.as_str() {
// Otherwise only consider our special exports, which all start
// with the same prefix which hopefully only we're using.
n if n.starts_with("__wbindgen") => {
if !self.required_internal_exports.contains(n) {
to_remove.push(export.id());
}
}
_ => {}
}
}
for id in to_remove {
self.module.exports.delete(id);
}
}
fn expose_drop_ref(&mut self) {
if !self.should_write_global("drop_ref") {
return;
@ -962,8 +904,6 @@ impl<'a> Context<'a> {
}
}
self.require_internal_export("__wbindgen_realloc")?;
// A fast path that directly writes char codes into WASM memory as long
// as it finds only ASCII characters.
//
@ -975,10 +915,18 @@ impl<'a> Context<'a> {
// charCodeAt on ASCII strings is usually optimised to raw bytes.
let encode_as_ascii = format!(
"\
if (realloc === undefined) {{
const buf = cachedTextEncoder.encode(arg);
const ptr = malloc(buf.length);
{mem}().subarray(ptr, ptr + buf.length).set(buf);
WASM_VECTOR_LEN = buf.length;
return ptr;
}}
let len = arg.length;
let ptr = malloc(len);
const mem = {}();
const mem = {mem}();
let offset = 0;
@ -988,7 +936,7 @@ impl<'a> Context<'a> {
mem[ptr + offset] = code;
}}
",
mem
mem = mem,
);
// TODO:
@ -997,14 +945,14 @@ impl<'a> Context<'a> {
// looping over the string to calculate the precise size, or perhaps using
// `shrink_to_fit` on the Rust side.
self.global(&format!(
"function {name}(arg, malloc) {{
"function {name}(arg, malloc, realloc) {{
{debug}
{ascii}
if (offset !== len) {{
if (offset !== 0) {{
arg = arg.slice(offset);
}}
ptr = wasm.__wbindgen_realloc(ptr, len, len = offset + arg.length * 3);
ptr = realloc(ptr, len, len = offset + arg.length * 3);
const view = {mem}().subarray(ptr + offset, ptr + len);
const ret = encodeString(arg, view);
{debug_end}
@ -1585,7 +1533,11 @@ impl<'a> Context<'a> {
if !self.should_write_global("handle_error") {
return Ok(());
}
self.require_internal_export("__wbindgen_exn_store")?;
let store = self
.aux
.exn_store
.ok_or_else(|| anyhow!("failed to find `__wbindgen_exn_store` intrinsic"))?;
let store = self.export_name_of(store);
match (self.aux.anyref_table, self.aux.anyref_alloc) {
(Some(table), Some(alloc)) => {
let add = self.expose_add_to_anyref_table(table, alloc)?;
@ -1593,21 +1545,22 @@ impl<'a> Context<'a> {
"
function handleError(e) {{
const idx = {}(e);
wasm.__wbindgen_exn_store(idx);
wasm.{}(idx);
}}
",
add,
add, store,
));
}
_ => {
self.expose_add_heap_object();
self.global(
self.global(&format!(
"
function handleError(e) {
wasm.__wbindgen_exn_store(addHeapObject(e));
}
function handleError(e) {{
wasm.{}(addHeapObject(e));
}}
",
);
store,
));
}
}
Ok(())
@ -1943,9 +1896,9 @@ impl<'a> Context<'a> {
let kind = match self.aux.export_map.get(&id) {
Some(export) => Kind::Export(export),
None => {
let core = self.wit.implements.iter().find(|pair| pair.1 == id);
let core = self.wit.implements.iter().find(|pair| pair.2 == id);
match core {
Some((core, _)) => Kind::Import(*core),
Some((core, _, _)) => Kind::Import(*core),
None => Kind::Adapter,
}
}
@ -2708,35 +2661,22 @@ impl<'a> Context<'a> {
Intrinsic::AnyrefHeapLiveCount => {
assert_eq!(args.len(), 0);
if self.config.anyref {
// Eventually we should add support to the anyref-xform to
// re-write calls to the imported
// `__wbindgen_anyref_heap_live_count` function into calls to
// the exported `__wbindgen_anyref_heap_live_count_impl`
// function, and to un-export that function.
//
// But for now, we just bounce wasm -> js -> wasm because it is
// easy.
self.require_internal_export("__wbindgen_anyref_heap_live_count_impl")?;
"wasm.__wbindgen_anyref_heap_live_count_impl()".into()
} else {
self.expose_global_heap();
prelude.push_str(
"
let free_count = 0;
let next = heap_next;
while (next < heap.length) {
free_count += 1;
next = heap[next];
}
",
);
format!(
"heap.length - free_count - {} - {}",
INITIAL_HEAP_OFFSET,
INITIAL_HEAP_VALUES.len(),
)
}
self.expose_global_heap();
prelude.push_str(
"
let free_count = 0;
let next = heap_next;
while (next < heap.length) {
free_count += 1;
next = heap[next];
}
",
);
format!(
"heap.length - free_count - {} - {}",
INITIAL_HEAP_OFFSET,
INITIAL_HEAP_VALUES.len(),
)
}
Intrinsic::InitAnyrefTable => {
@ -2953,15 +2893,32 @@ impl<'a> Context<'a> {
}
});
if let Some(export) = export {
self.required_internal_exports
.insert(export.name.clone().into());
return export.name.clone();
}
let name = format!("__wbindgen_export_{}", self.next_export_idx);
let default_name = format!("__wbindgen_export_{}", self.next_export_idx);
self.next_export_idx += 1;
let name = match id {
walrus::ExportItem::Function(f) => match &self.module.funcs.get(f).name {
Some(s) => to_js_identifier(s),
None => default_name,
},
_ => default_name,
};
self.module.exports.add(&name, id);
self.required_internal_exports.insert(name.clone().into());
return name;
// Not really an exhaustive list, but works for our purposes.
fn to_js_identifier(name: &str) -> String {
name.chars()
.map(|c| {
if c.is_ascii() && (c.is_alphabetic() || c.is_numeric()) {
c
} else {
'_'
}
})
.collect()
}
}
fn adapter_name(&self, id: AdapterId) -> String {