WebAssembly experiment for musl libc with dynamic linking

The goal of this prototype was to get a WebAssembly libc off the ground. Dynamic linking came out of it for free which is mighty convenient. We should:

Focus on the libc aspect.
As a secondary goal use dynamic linking to inform WebAssembly's design.

Note: This experimental WebAssembly C library with dynamic linking is a hack. Don't rely on it: it's meant to inform the design of WebAssembly. Things are changing rapidly, so mixing different parts of the toolchain may break from time to time, try to keep them all in sync.

In this experiment dynamic linking is entirely done through JavaScript, which is acting as the dynamic linker / loader. This merely uses the WebAssembly object's capabilities.

Quick how-to

Pre-built Linux x86-64 Linux binaries are available on the waterfall, so are musl.wast, musl.wasm, and wasm.js. The waterfall marks as green the builds which are known to work properly. Click on a green build to download the archived binaries. Check out its last known good revision. You can build everything yourself using the waterfall's build.py.

Compile your program using LLVM:

  clang -S -O2 --target=wasm32-unknown-unknown foo.c

Creates a .s assembly file. Link it:

  s2wasm foo.s -o foo.wast

Creates a .wast WebAssembly s-expression. Assemble it:

  sexpr-wasm foo.wast -o foo.wasm

You now have a WebAssembly binary file.

Run .wasm files which import libc functions:

  d8 --expose-wasm musl/arch/wasm32/wasm.js -- foo.wasm musl-out/musl.wasm

Or run it without musl, using only wasm.js to emulate libc:

  d8 --expose-wasm musl/arch/wasm32/wasm.js -- foo.wasm

This may work... or not. File bugs on what's broken, or send patches!

libc + dynamic linking: how does it work?

In the current V8 implementation of WebAssembly binaries, each .wasm module:

Declares it imports and its exports.
Takes in a dictionary mapping Foreign Function Interface (FFI) names to corresponding functions.
Takes in its heap, an ArrayBuffer.

The wasm.js file:

Initializes the heap.
Implements a rudimentary C library in JavaScript, and adds these functions to the FFI object.
Loads .wasm files provided on the command-line, from last to first.
Adds each exported function to the FFI object, sometimes shadowing the JavaScript fallback.
Loads the first .wasm file provided and calls its main function.

Each loaded .wasm file is initialized with the same heap. They all share the same address space.

Calls from one WebAssembly module to another trampoline through JavaScript, but they should optimize well. We should figure out what we suggest developers use, so that the default pattern doesn't require gymnastics on the compiler's part.

A WebAssembly module with un-met imports will throw. This can be handled, add the missing function as a stub to FFI, and then load again (loop until success) but it's silly. If WebAssembly modules were loadable, imports inspectable, and FFI object provided later then we'd be better off. We could implement very fancy lazy-loading, where the developer can handle load failures. We could also easily implement dlopen / dlsym / dlclose.

It would also be good to be able to specify compilation / execution separately.

libc implementation details

The current libc implementation builds a subset of musl using the hacked-up libc.py script. It excludes files which triggered bugs throughout the toolchain, not that the files being built are bug free either.

The implementation is based on Emscripten's musl port, but is based on a much more recent musl and has no modifications to musl's code: all changes are in the arch/wasm32 directory. It aims to only communicate to the embedder using a syscall API, modeled after Linux' own syscall API. This may have shortcomings, but it's a good thing to try out since we can revisit later. Note the musl_hack functions in wasm.js: they fill in for functionality that's currently been hacked out and which musl expects to import. It should be exporting these instead of importing them. Maybe more functionality should be implemented in JavaScript, but experience with NaCl and Emscripten leads us to believe the syscall API is a good boundary.

The eventual goal is for the WebAssembly libc to be upstreamed to musl, and that'll require doing it right according to the musl community. We also want Emscripten to be able to use the same libc implementation. The approach in this repository may not be the right one.

Miscellaneous

Dynamic linking isn't in WebAssembly's current MVP because we thought it would be hard. This repository shows that it's possible, we therefore may as well design it right from the start, or make it entirely impossible for the MVP.

That'll including figuring out calling convention and ABI. Exports currently don't declare their signature in a WebAssembly module, even though they are in the binary format, and don't cause any failure when the APIs don't match. That should be fixed.

We'll also need to figure out how to make memory segments relocatable, and the AST references to the segments position independent. Do we even want to allow non-relocatable segments?

It seems like user code should be managing all of the heap, the first module that's loaded (even before libc) could therefore be a basic memory manager. The dynamic loading mechanism (implemented in JavaScript) would then query this heap manager to figure out where to locate segments, as well as to position user stacks. libc's malloc would then use this basic memory manager to implement runtime memory management, the same would be true for stack positioning, thread stacks, and thread-local storage allocation.

Why do dynamic linking now?

These basic experiments are finding bugs in the toolchain, if anything they're useful in making it more robust. It's also an unexpected usage of the APIs! It's better that we find it now and figure out what it means.

Having a standalone musl.wasm is much simpler for code deployment and allows caching.

Developers are in control: they can do the equivalent of -ffunction-sections and -fdata-sections but emit one .wasm file per section. This allows them to lazy-load and lazy-compile each function as needed, and even unload them when the program doesn't need them anymore.

6.7 KiB Raw Blame History