Add experimental support for the anyref type

This commit adds experimental support to `wasm-bindgen` to emit and
leverage the `anyref` native wasm type. This native type is still in a
proposal status (the reference-types proposal). The intention of
`anyref` is to be able to directly hold JS values in wasm and pass the
to imported functions, namely to empower eventual host bindings (now
renamed WebIDL bindings) integration where we can skip JS shims
altogether for many imports.

This commit doesn't actually affect wasm-bindgen's behavior at all
as-is, but rather this support requires an opt-in env var to be
configured. Once the support is stable in browsers it's intended that
this will add a CLI switch for turning on this support, eventually
defaulting it to `true` in the far future.

The basic strategy here is to take the `stack` and `slab` globals in the
generated JS glue and move them into wasm using a table. This new table
in wasm is managed at the fringes via injected shims. At
`wasm-bindgen`-time the CLI will rewrite exports and imports with shims
that actually use `anyref` if needed, performing loads/stores inside the
wasm module instead of externally in the wasm module.

This should provide a boost over what we have today, but it's not a
fantastic strategy long term. We have a more grand vision for `anyref`
being a first-class type in the language, but that's on a much longer
horizon and this is currently thought to be the best we can do in terms
of integration in the near future.

The stack/heap JS tables are combined into one wasm table. The stack
starts at the end of the table and grows down with a stack pointer (also
injected). The heap starts at the end and grows up (state managed in
linear memory). The anyref transformation here will hook up various
intrinsics in wasm-bindgen to the runtime functionality if the anyref
supoprt is enabled.

The main tricky treatment here was applied to closures, where we need JS
to use a different function pointer than the one Rust gives it to use a
JS function pointer empowered with anyref. This works by switching up a
bit how descriptors work, embedding the shims to call inside descriptors
rather than communicated at runtime. This means that we're accessing
constant values in the generated JS and we can just update the constant
value accessed.

crates/cli-support/src/js/closures.rs

@@ -10,7 +10,7 @@
 //! this works can be found in the code below.
 
 use crate::descriptor::Descriptor;
-use crate::js::js2rust::Js2Rust;
+use crate::js::js2rust::{ExportedShim, Js2Rust};
 use crate::js::Context;
 use failure::Error;
 use std::collections::{BTreeMap, HashSet};
@@ -142,7 +142,7 @@ impl ClosureDescriptors {
         let table = input.module.tables.get_mut(table_id);
         let table = match &mut table.kind {
             walrus::TableKind::Function(f) => f,
-            walrus::TableKind::Anyref(_) => unreachable!(),
+            _ => unreachable!(),
         };
         for idx in self.element_removal_list.iter().cloned() {
             log::trace!("delete element {}", idx);
@@ -178,6 +178,7 @@ impl ClosureDescriptors {
 
             let closure = instr.descriptor.closure().unwrap();
 
+            let mut shim = closure.shim_idx;
             let (js, _ts, _js_doc) = {
                 let mut builder = Js2Rust::new("", input);
                 builder.prelude("this.cnt++;");
@@ -192,9 +193,12 @@ impl ClosureDescriptors {
                     builder.rust_argument("this.a").rust_argument("b");
                 }
                 builder.finally("if (this.cnt-- == 1) d(this.a, b);");
-                builder.process(&closure.function)?.finish("function", "f")
+                builder.process(&closure.function)?.finish(
+                    "function",
+                    "f",
+                    ExportedShim::TableElement(&mut shim),
+                )
             };
-            input.expose_add_heap_object();
             input.function_table_needed = true;
             let body = format!(
                 "function(a, b, _ignored) {{
@@ -205,15 +209,19 @@ impl ClosureDescriptors {
                     cb.cnt = 1;
                     let real = cb.bind(cb);
                     real.original = cb;
-                    return addHeapObject(real);
+                    return {};
                 }}",
-                closure.shim_idx, closure.dtor_idx, js,
+                shim,
+                closure.dtor_idx,
+                js,
+                input.add_heap_object("real"),
             );
             input.export(&import_name, &body, None);
 
-            let id = input
-                .module
-                .add_import_func("__wbindgen_placeholder__", &import_name, ty);
+            let module = "__wbindgen_placeholder__";
+            let id = input.module.add_import_func(module, &import_name, ty);
+            input.anyref.import_xform(module, &import_name, &[], true);
+            input.module.funcs.get_mut(id).name = Some(import_name);
 
             let local = match &mut input.module.funcs.get_mut(*func).kind {
                 walrus::FunctionKind::Local(l) => l,

crates/cli-support/src/js/js2rust.rs

@@ -44,6 +44,15 @@ pub struct Js2Rust<'a, 'b: 'a> {
     /// The string value here is the class that this should be a constructor
     /// for.
     constructor: Option<String>,
+
+    /// metadata for anyref transformations
+    anyref_args: Vec<(usize, bool)>,
+    ret_anyref: bool,
+}
+
+pub enum ExportedShim<'a> {
+    Named(&'a str),
+    TableElement(&'a mut u32),
 }
 
 impl<'a, 'b> Js2Rust<'a, 'b> {
@@ -59,6 +68,8 @@ impl<'a, 'b> Js2Rust<'a, 'b> {
             ret_ty: String::new(),
             ret_expr: String::new(),
             constructor: None,
+            anyref_args: Vec::new(),
+            ret_anyref: false,
         }
     }
 
@@ -193,13 +204,25 @@ impl<'a, 'b> Js2Rust<'a, 'b> {
 
         if arg.is_anyref() {
             self.js_arguments.push((name.clone(), "any".to_string()));
-            self.cx.expose_add_heap_object();
-            if optional {
-                self.cx.expose_is_like_none();
-                self.rust_arguments
-                    .push(format!("isLikeNone({0}) ? 0 : addHeapObject({0})", name,));
+            if self.cx.config.anyref {
+                if optional {
+                    self.cx.expose_add_to_anyref_table()?;
+                    self.cx.expose_is_like_none();
+                    self.rust_arguments
+                        .push(format!("isLikeNone({0}) ? 0 : addToAnyrefTable({0})", name));
+                } else {
+                    self.anyref_args.push((self.rust_arguments.len(), true));
+                    self.rust_arguments.push(name);
+                }
             } else {
-                self.rust_arguments.push(format!("addHeapObject({})", name));
+                self.cx.expose_add_heap_object();
+                if optional {
+                    self.cx.expose_is_like_none();
+                    self.rust_arguments
+                        .push(format!("isLikeNone({0}) ? 0 : addHeapObject({0})", name));
+                } else {
+                    self.rust_arguments.push(format!("addHeapObject({})", name));
+                }
             }
             return Ok(self);
         }
@@ -383,14 +406,19 @@ impl<'a, 'b> Js2Rust<'a, 'b> {
 
         if arg.is_ref_anyref() {
             self.js_arguments.push((name.clone(), "any".to_string()));
-            self.cx.expose_borrowed_objects();
-            self.cx.expose_global_stack_pointer();
-            // the "stack-ful" nature means that we're always popping from the
-            // stack, and make sure that we actually clear our reference to
-            // allow stale values to get GC'd
-            self.finally("heap[stack_pointer++] = undefined;");
-            self.rust_arguments
-                .push(format!("addBorrowedObject({})", name));
+            if self.cx.config.anyref {
+                self.anyref_args.push((self.rust_arguments.len(), false));
+                self.rust_arguments.push(name);
+            } else {
+                // the "stack-ful" nature means that we're always popping from the
+                // stack, and make sure that we actually clear our reference to
+                // allow stale values to get GC'd
+                self.cx.expose_borrowed_objects();
+                self.cx.expose_global_stack_pointer();
+                self.finally("heap[stack_pointer++] = undefined;");
+                self.rust_arguments
+                    .push(format!("addBorrowedObject({})", name));
+            }
             return Ok(self);
         }
 
@@ -462,7 +490,7 @@ impl<'a, 'b> Js2Rust<'a, 'b> {
 
         if let Some(ty) = ty.vector_kind() {
             self.ret_ty = ty.js_ty().to_string();
-            let f = self.cx.expose_get_vector_from_wasm(ty);
+            let f = self.cx.expose_get_vector_from_wasm(ty)?;
             self.cx.expose_global_argument_ptr()?;
             self.cx.expose_uint32_memory();
             self.cx.require_internal_export("__wbindgen_free")?;
@@ -494,8 +522,8 @@ impl<'a, 'b> Js2Rust<'a, 'b> {
         // that `takeObject` will naturally pluck out `undefined`.
         if ty.is_anyref() {
             self.ret_ty = "any".to_string();
-            self.cx.expose_take_object();
-            self.ret_expr = format!("return takeObject(RET);");
+            self.ret_expr = format!("return {};", self.cx.take_object("RET"));
+            self.ret_anyref = true;
             return Ok(self);
         }
 
@@ -708,7 +736,12 @@ impl<'a, 'b> Js2Rust<'a, 'b> {
     /// Returns two strings, the first of which is the JS expression for the
     /// generated function shim and the second is a TypeScript signature of the
     /// JS expression.
-    pub fn finish(&self, prefix: &str, invoc: &str) -> (String, String, String) {
+    pub fn finish(
+        &mut self,
+        prefix: &str,
+        invoc: &str,
+        exported_shim: ExportedShim,
+    ) -> (String, String, String) {
         let js_args = self
             .js_arguments
             .iter()
@@ -754,6 +787,24 @@ impl<'a, 'b> Js2Rust<'a, 'b> {
             ts.push_str(&self.ret_ty);
         }
         ts.push(';');
+
+        if self.ret_anyref || self.anyref_args.len() > 0 {
+            match exported_shim {
+                ExportedShim::Named(name) => {
+                    self.cx
+                        .anyref
+                        .export_xform(name, &self.anyref_args, self.ret_anyref);
+                }
+                ExportedShim::TableElement(idx) => {
+                    *idx = self.cx.anyref.table_element_xform(
+                        *idx,
+                        &self.anyref_args,
+                        self.ret_anyref,
+                    );
+                }
+            }
+        }
+
         (js, ts, self.js_doc_comments())
     }
 

crates/cli-support/src/js/rust2js.rs

@@ -1,4 +1,5 @@
 use crate::descriptor::{Descriptor, Function};
+use crate::js::js2rust::ExportedShim;
 use crate::js::{Context, ImportTarget, Js2Rust};
 use failure::{bail, Error};
 
@@ -39,6 +40,11 @@ pub struct Rust2Js<'a, 'b: 'a> {
 
     /// Whether or not the last argument is a slice representing variadic arguments.
     variadic: bool,
+
+    /// list of arguments that are anyref, and whether they're an owned anyref
+    /// or not.
+    pub anyref_args: Vec<(usize, bool)>,
+    pub ret_anyref: bool,
 }
 
 impl<'a, 'b> Rust2Js<'a, 'b> {
@@ -55,6 +61,8 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
             catch: false,
             catch_and_rethrow: false,
             variadic: false,
+            anyref_args: Vec::new(),
+            ret_anyref: false,
         }
     }
 
@@ -101,7 +109,7 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
 
         if let Some(ty) = arg.vector_kind() {
             let abi2 = self.shim_argument();
-            let f = self.cx.expose_get_vector_from_wasm(ty);
+            let f = self.cx.expose_get_vector_from_wasm(ty)?;
             self.prelude(&format!(
                 "let v{0} = {prefix}{func}({0}, {1});",
                 abi,
@@ -141,12 +149,14 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
         // No need to special case `optional` here because `takeObject` will
         // naturally work.
         if arg.is_anyref() {
-            self.cx.expose_take_object();
-            self.js_arguments.push(format!("takeObject({})", abi));
+            let arg = self.cx.take_object(&abi);
+            self.js_arguments.push(arg);
+            self.anyref_args.push((self.arg_idx - 1, true));
             return Ok(());
         } else if arg.is_ref_anyref() {
-            self.cx.expose_get_object();
-            self.js_arguments.push(format!("getObject({})", abi));
+            let arg = self.cx.get_object(&abi);
+            self.js_arguments.push(arg);
+            self.anyref_args.push((self.arg_idx - 1, false));
             return Ok(());
         }
 
@@ -263,6 +273,7 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
 
         if let Some((f, mutable)) = arg.stack_closure() {
             let arg2 = self.shim_argument();
+            let mut shim = f.shim_idx;
             let (js, _ts, _js_doc) = {
                 let mut builder = Js2Rust::new("", self.cx);
                 if mutable {
@@ -274,10 +285,11 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
                 } else {
                     builder.rust_argument("this.a");
                 }
-                builder
-                    .rust_argument("this.b")
-                    .process(f)?
-                    .finish("function", "this.f")
+                builder.rust_argument("this.b").process(f)?.finish(
+                    "function",
+                    "this.f",
+                    ExportedShim::TableElement(&mut shim),
+                )
             };
             self.cx.function_table_needed = true;
             self.global_idx();
@@ -291,7 +303,7 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
                 abi,
                 arg2,
                 js = js,
-                idx = f.shim_idx,
+                idx = shim,
             ));
             self.finally(&format!("cb{0}.a = cb{0}.b = 0;", abi));
             self.js_arguments.push(format!("cb{0}.bind(cb{0})", abi));
@@ -349,16 +361,31 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
             return Ok(());
         }
         if ty.is_anyref() {
-            self.cx.expose_add_heap_object();
-            if optional {
-                self.cx.expose_is_like_none();
-                self.ret_expr = "
-                    const val = JS;
-                    return isLikeNone(val) ? 0 : addHeapObject(val);
-                "
-                .to_string();
+            if self.cx.config.anyref {
+                if optional {
+                    self.cx.expose_add_to_anyref_table()?;
+                    self.cx.expose_is_like_none();
+                    self.ret_expr = "
+                        const val = JS;
+                        return isLikeNone(val) ? 0 : addToAnyrefTable(val);
+                    "
+                    .to_string();
+                } else {
+                    self.ret_anyref = true;
+                    self.ret_expr = "return JS;".to_string()
+                }
             } else {
-                self.ret_expr = "return addHeapObject(JS);".to_string()
+                self.cx.expose_add_heap_object();
+                if optional {
+                    self.cx.expose_is_like_none();
+                    self.ret_expr = "
+                        const val = JS;
+                        return isLikeNone(val) ? 0 : addHeapObject(val);
+                    "
+                    .to_string();
+                } else {
+                    self.ret_expr = "return addHeapObject(JS);".to_string()
+                }
             }
             return Ok(());
         }
@@ -565,6 +592,8 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
             arg_idx: _,
             cx: _,
             global_idx: _,
+            anyref_args: _,
+            ret_anyref: _,
         } = self;
 
         !catch &&
@@ -581,7 +610,7 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
             js_arguments == shim_arguments
     }
 
-    pub fn finish(&mut self, invoc: &ImportTarget) -> Result<String, Error> {
+    pub fn finish(&mut self, invoc: &ImportTarget, shim: &str) -> Result<String, Error> {
         let mut ret = String::new();
         ret.push_str("function(");
         ret.push_str(&self.shim_arguments.join(", "));
@@ -596,7 +625,6 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
 
         let variadic = self.variadic;
         let ret_expr = &self.ret_expr;
-        let js_arguments = &self.js_arguments;
         let handle_variadic = |invoc: &str, js_arguments: &[String]| {
             let ret = if variadic {
                 let (last_arg, args) = match js_arguments.split_last() {
@@ -617,6 +645,7 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
             Ok(ret)
         };
 
+        let js_arguments = &self.js_arguments;
         let fixed = |desc: &str, class: &Option<String>, amt: usize| {
             if variadic {
                 bail!("{} cannot be variadic", desc);
@@ -670,7 +699,7 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
         };
 
         if self.catch {
-            self.cx.expose_handle_error();
+            self.cx.expose_handle_error()?;
             invoc = format!(
                 "\
                 try {{\n\
@@ -712,6 +741,33 @@ impl<'a, 'b> Rust2Js<'a, 'b> {
         ret.push_str(&invoc);
 
         ret.push_str("\n}\n");
+
+        if self.ret_anyref || self.anyref_args.len() > 0 {
+            // Some return values go at the the beginning of the argument list
+            // (they force a return pointer). Handle that here by offsetting all
+            // our arg indices by one, but throw in some sanity checks for if
+            // this ever changes.
+            if let Some(start) = self.shim_arguments.get(0) {
+                if start == "ret" {
+                    assert!(!self.ret_anyref);
+                    if let Some(next) = self.shim_arguments.get(1) {
+                        assert_eq!(next, "arg0");
+                    }
+                    for (idx, _) in self.anyref_args.iter_mut() {
+                        *idx += 1;
+                    }
+                } else {
+                    assert_eq!(start, "arg0");
+                }
+            }
+            self.cx.anyref.import_xform(
+                "__wbindgen_placeholder__",
+                shim,
+                &self.anyref_args,
+                self.ret_anyref,
+            );
+        }
+
         Ok(ret)
     }
 

crates/cli-support/src/lib.rs

@@ -35,6 +35,7 @@ pub struct Bindgen {
     // Experimental support for the wasm threads proposal, transforms the wasm
     // module to be "ready to be instantiated on any thread"
     threads: Option<wasm_bindgen_threads_xform::Config>,
+    anyref: bool,
 }
 
 enum Input {
@@ -62,6 +63,7 @@ impl Bindgen {
             emit_start: true,
             weak_refs: env::var("WASM_BINDGEN_WEAKREF").is_ok(),
             threads: threads_config(),
+            anyref: env::var("WASM_BINDGEN_ANYREF").is_ok(),
         }
     }
 
@@ -176,6 +178,22 @@ impl Bindgen {
                 (module, stem)
             }
         };
+
+        // This isn't the hardest thing in the world too support but we
+        // basically don't know how to rationalize #[wasm_bindgen(start)] and
+        // the actual `start` function if present. Figure this out later if it
+        // comes up, but otherwise we should continue to be compatible with
+        // LLVM's output today.
+        //
+        // Note that start function handling in `js/mod.rs` will need to be
+        // updated as well, because `#[wasm_bindgen(start)]` is inserted *after*
+        // a module's start function, if any, because we assume start functions
+        // only show up when injected on behalf of wasm-bindgen's passes.
+        if module.start.is_some() {
+            bail!("wasm-bindgen is currently incompatible with modules that \
+                   already have a start function");
+        }
+
         let mut program_storage = Vec::new();
         let programs = extract_programs(&mut module, &mut program_storage)
             .with_context(|_| "failed to extract wasm-bindgen custom sections")?;
@@ -233,7 +251,10 @@ impl Bindgen {
                 imported_statics: Default::default(),
                 direct_imports: Default::default(),
                 start: None,
+                anyref: Default::default(),
             };
+            cx.anyref.enabled = self.anyref;
+            cx.anyref.prepare(cx.module)?;
             for program in programs.iter() {
                 js::SubContext {
                     program,