Create the web-sys crate mechanically from WebIDL (#409)

* Create a new `web-sys` crate

This will eventually contain all the WebIDL-generated bindings to Web APIs.

* ci: Test the new `web-sys` crate in CI

* web-sys: Add a small README

* web-sys: Vendor all the WebIDL files from mozilla-central

* backend: Add a pass to remove AST items that use undefined imports

This is necessary for the WebIDL frontend, which can't translate many WebIDL
constructs into equivalent wasm-bindgen AST things yet. It lets us make
incremental progress: we can generate bindings to methods we can support right
now even though there might be methods on the same interface that we can't
support yet.

* webidl: Add a bunch of missing semicolons

* webidl: Make parsing private

It was only `pub` so that we could test it, but we ended up moving towards
integration tests rather than unit tests that assert particular ASTs are parsed
from WebIDL files.

* webidl: Remove uses of undefined import types

* test-project-builder: Build projects in "very verbose" mode

This helps for debugging failing WebIDL-related tests.

* test-project-builder: Add more profiling timers

* test-project-builder: Detect when webpack-dev-server fails

Instead of going into an infinite loop, detect when webpack-dev-server fails to
start up and early exit the test.

* webidl: Specify version for dev-dependency on wasm-bindgen-backend

Instead of only a relative path.

* guide: Add section about contributing to `web-sys`

* WIP enable Event.webidl

Still need to fix and finish the test.

* Update expected webidl output

* Start out a test's status as incomplete

That way if we don't fill it in the error message doesn't look quite so bizarre

* Fix onerror function in headless mode

Otherwise we don't see any output!

* Fix package.json/node_modules handling in project generation

Make sure these are looked up in the git project root rather than the crate root

* Avoid logging body text

This was meant for debugging and is otherwise pretty noisy

* Fix a relative path

* More expected test fixes

* Fix a typo

* test-project-builder: Allow asynchronous tests

* webidl: Convert [Unforgeable] attributes into `#[wasm_bindgen(structural)]`

Fixes #432

* test-project-builder: Print generated WebIDL bindings for debugging purposes

Helps debug bad WebIDL bindings generation inside tests.

* When we can't find a descriptor, say which one can't be found

This helps when debugging things that need to become structural.

* web-sys: Test bindings for Event

* ci: Use `--manifest-path dir` instead of `cd dir && ...`

* web-sys: Just move .webidl files isntead of symlinking to enable them

* tests: Polyfill Array.prototype.values for older browsers in CI

* test-project-builder: Don't panic on poisoned headless test mutex

We only use it to serialize headless tests so that we don't try to bind the port
concurrently. Its OK to run another headless test if an earlier one panicked.

* JsValue: Add {is,as}_{object,function} methods

Allows dynamically casting values to `js::Object` and `js::Function`.

* tidy: Fix whitespace and missing semicolons

* Allow for dynamic feature detection of methods

If we create bindings to a method that doesn't exist in this implementation,
then it shouldn't fail until if/when we actually try and invoke that missing
method.

* tests: Do feature detection in Array.prototype.values test

* Add JsValue::{is_string, as_js_string} methods

And document all the cast/convert/check methods for js value.

* eslint: allow backtick string literals

* Only generate a fallback import function for non-structural imports

guide/src/SUMMARY.md

@@ -25,4 +25,9 @@
     - [Customizing imports](./design/import-customization.md)
     - [Rust Type conversions](./design/rust-type-conversions.md)
     - [Types in `wasm-bindgen`](./design/describe.md)
+  - [`web-sys`](./web-sys.md)
+    - [Overview](./web-sys/overview.md)
+    - [Testing](./web-sys/testing.md)
+    - [Logging](./web-sys/logging.md)
+    - [Supporting More Web APIs](./web-sys/supporting-more-web-apis.md)
   - [Team](./team.md)

guide/src/web-sys.md

@@ -0,0 +1,8 @@
+# `web-sys`
+
+The `web-sys` crate provides raw bindings to all of the Web's APIs, and its
+source lives at `wasm-bindgen/crates/web-sys`.
+
+The `web-sys` crate is **entirely** mechanically generated inside `build.rs`
+using `wasm-bindgen`'s WebIDL frontend and the WebIDL interface definitions for
+Web APIs.

guide/src/web-sys/logging.md

@@ -0,0 +1,23 @@
+# Logging
+
+The `wasm_bindgen_webidl` crate (used by `web-sys`'s `build.rs`) uses
+[`env_logger`][env_logger] for logging, which can be enabled by setting the
+`RUST_LOG=wasm_bindgen_webidl` environment variable while building the `web-sys`
+crate.
+
+Make sure to enable "very verbose" output during `cargo build` to see these logs
+within `web-sys`'s build script output.
+
+```sh
+cd crates/web-sys
+RUST_LOG=wasm_bindgen_webidl cargo build -vv
+```
+
+If `wasm_bindgen_webidl` encounters WebIDL constructs that it doesn't know how
+to translate into `wasm-bindgen` AST items, it will emit warn-level logs.
+
+```
+WARN 2018-07-06T18:21:49Z: wasm_bindgen_webidl: Unsupported WebIDL interface: ...
+```
+
+[env_logger]: https://crates.io/crates/env_logger

guide/src/web-sys/overview.md

@@ -0,0 +1,42 @@
+# `web-sys` Overview
+
+The `web-sys` crate has this file and directory layout:
+
+```text
+.
+├── build.rs
+├── Cargo.toml
+├── README.md
+├── src
+│   └── lib.rs
+└── webidls
+    ├── available
+    │   └── ...
+    └── enabled
+        └── ...
+```
+
+### `webidls/available/*.webidl`
+
+These are all the different WebIDL definitions we intend to support, but don't
+yet. At the time of writing, these are the majority of `.webidl`s.
+
+### `webidls/enabled/*.webidl`
+
+These are the WebIDL interfaces that we will actually generate bindings for (or
+at least bindings for *some* of the things defined in these files).
+
+### `build.rs`
+
+The `build.rs` invokes `wasm-bindgen`'s WebIDL frontend on all the WebIDL files
+in `webidls/enabled`. It writes the resulting bindings into the cargo build's
+out directory.
+
+### `src/lib.rs`
+
+The only thing `src/lib.rs` does is include the bindings generated at compile
+time in `build.rs`. Here is the whole `src/lib.rs` file:
+
+```rust
+{{#include ../../../crates/web-sys/src/lib.rs}}
+```

guide/src/web-sys/supporting-more-web-apis.md

@@ -0,0 +1,59 @@
+# Supporting More Web APIs in `web-sys`
+
+1. <input type="checkbox"/> Ensure that the `.webidl` file describing the
+   interface exists somewhere within the `crates/web-sys/webidls/enabled`
+   directory.
+
+   First, check to see whether we have the WebIDL definition file for
+   your API:
+
+   ```sh
+   grep -rn MyWebApi crates/web-sys/webidls
+   ```
+
+   * If your interface is defined in a `.webidl` file that is inside the
+     `crates/web-sys/webidls/enabled` directory, skip to step (3).
+
+   * If your interface isn't defined in any file yet, find the WebIDL definition
+     in the relevant standard and add it as a new `.webidl` file in
+     `crates/web-sys/webidls/enabled`. Make sure that it is a standard Web API!
+     We don't want to add non-standard APIs to this crate.
+
+   * If your interface is defined in a `.webidl` file within the
+     `crates/web-sys/webidls/available` directory, you need to move it into
+     `crates/web-sys/webidls/enabled`:
+
+     ```sh
+     cd crates/web-sys
+     git mv webidls/available/MyWebApi.webidl webidls/enabled/MyWebApi.webidl
+     ```
+
+2. <input type="checkbox"/> Verify that the `web-sys` crate still builds and
+   that its tests still pass with the new `.webidl` file enabled:
+
+   ```sh
+   cd crates/web-sys
+   cargo build
+   cargo test
+   ```
+
+3. <input type="checkbox"/> Verify that bindings are being generated for your new
+   API by generating the documentation and searching for the new API in it:
+
+   ```sh
+   cd crates/web-sys
+   cargo doc --open
+   # search for the new API in the opened docs
+   ```
+
+   * <input type="checkbox"/> If the new API is **not** showing up in the docs,
+     rebuild the `web-sys` crate [with logging enabled](/web-sys/logging.html)
+     and look for warning messages that mention your new API. Figure out why
+     bindings weren't generated and then add support to `wasm_bindgen_webidl` for
+     whatever is needed to generate your API's bindings.
+
+4. <input type="checkbox"/> Add a simple test for your new API to
+   `crates/web-sys/tests/all/`. See the [`web-sys` testing
+   documentation](/web-sys/testing.html) for details.
+
+5. <input type="checkbox"/> Send a pull request! 😊

guide/src/web-sys/testing.md

@@ -0,0 +1,21 @@
+# Testing
+
+You can test the `web-sys` crate by running `cargo test` within the
+`crates/web-sys` directory in the `wasm-bindgen` repository:
+
+```sh
+cd wasm-bindgen/crates/web-sys
+cargo test
+```
+
+These tests all use a headless Firefox browser. See the [*Headless Browser
+Tests* section for details on setup and
+configuration.](../contributing.html#headless-browser-tests)
+
+## Grouping Tests
+
+Because headless tests can have significant setup and tear down overheads, try
+and group tests together. Instead of having a different `#[test]` for every
+method on some interface, have a single `#[test]` for the interface and all of
+its methods. This will keep the test suite running fast, resulting in better
+developer ergonomics and CI turn around times. Thanks!