提交 6741e416 编写于 作者: B bors

Auto merge of #47102 - Diggsey:wasm-syscall, r=alexcrichton

Implement extensible syscall interface for wasm

Currently it's possible to run tests with the native wasm target, but it's not possible to tell whether they pass or to capture the output, because libstd throws away stdout, stderr and the exit code. While advanced libstd features should probably require more specific targets (eg. wasm-unknown-web) I think even the unknown target should at least support basic I/O.

Any solution is constrained by these factors:
- It must not be javascript specific
- There must not be too strong coupling between libstd and the host environment (because it's an "unknown" target)
- WebAssembly does not allow "optional" imports - all imports *must* be resolved.
- WebAssembly does not support calling the host environment through any channel *other* than imports.

The best solution I could find to these constraints was to give libstd a single required import, and implement a syscall-style interface through that import. Each syscall is designed such that a no-op implementation gives the most reasonable fallback behaviour. This means that the following import table would be perfectly valid:
```javascript
imports.env = { rust_wasm_syscall: function(index, data) {} }
```

Currently I have implemented these system calls:
- Read from stdin
- Write to stdout/stderr
- Set the exit code
- Get command line arguments
- Get environment variable
- Set environment variable
- Get time

It need not be extended beyond this set if being able to run tests for this target is the only goal.

edit:
As part of this PR I had to make a further change. Previously, the rust entry point would be automatically called when the webassembly module was instantiated. This was problematic because from the javascript side it was impossible to call exported functions, access program memory or get a reference to the instance.

To solve this, ~I changed the default behaviour to not automatically call the entry point, and added a crate-level attribute to regain the old behaviour. (`#![wasm_auto_run]`)~ I disabled this behaviour when building tests.
......@@ -312,6 +312,11 @@
# bootstrap)
#codegen-backends = ["llvm"]
# Flag indicating whether `libstd` calls an imported function to hande basic IO
# when targetting WebAssembly. Enable this to debug tests for the `wasm32-unknown-unknown`
# target, as without this option the test output will not be captured.
#wasm-syscall = false
# =============================================================================
# Options for specific targets
#
......
......@@ -107,6 +107,7 @@ pub struct Config {
pub debug_jemalloc: bool,
pub use_jemalloc: bool,
pub backtrace: bool, // support for RUST_BACKTRACE
pub wasm_syscall: bool,
// misc
pub low_priority: bool,
......@@ -282,6 +283,7 @@ struct Rust {
test_miri: Option<bool>,
save_toolstates: Option<String>,
codegen_backends: Option<Vec<String>>,
wasm_syscall: Option<bool>,
}
/// TOML representation of how each build target is configured.
......@@ -463,6 +465,7 @@ pub fn parse(args: &[String]) -> Config {
set(&mut config.rust_dist_src, rust.dist_src);
set(&mut config.quiet_tests, rust.quiet_tests);
set(&mut config.test_miri, rust.test_miri);
set(&mut config.wasm_syscall, rust.wasm_syscall);
config.rustc_parallel_queries = rust.experimental_parallel_queries.unwrap_or(false);
config.rustc_default_linker = rust.default_linker.clone();
config.musl_root = rust.musl_root.clone().map(PathBuf::from);
......
......@@ -423,6 +423,9 @@ fn std_features(&self) -> String {
if self.config.profiler {
features.push_str(" profiler");
}
if self.config.wasm_syscall {
features.push_str(" wasm_syscall");
}
features
}
......
......@@ -1286,6 +1286,14 @@ fn run(self, builder: &Builder) {
cargo.env(format!("CARGO_TARGET_{}_RUNNER", envify(&target)),
build.config.nodejs.as_ref().expect("nodejs not configured"));
} else if target.starts_with("wasm32") {
// Warn about running tests without the `wasm_syscall` feature enabled.
// The javascript shim implements the syscall interface so that test
// output can be correctly reported.
if !build.config.wasm_syscall {
println!("Libstd was built without `wasm_syscall` feature enabled: \
test output may not be visible.");
}
// On the wasm32-unknown-unknown target we're using LTO which is
// incompatible with `-C prefer-dynamic`, so disable that here
cargo.env("RUSTC_NO_PREFER_DYNAMIC", "1");
......
......@@ -28,14 +28,76 @@ let m = new WebAssembly.Module(buffer);
let memory = null;
function viewstruct(data, fields) {
return new Uint32Array(memory.buffer).subarray(data/4, data/4 + fields);
}
function copystr(a, b) {
if (memory === null) {
return null
}
let view = new Uint8Array(memory.buffer).slice(a, a + b);
let view = new Uint8Array(memory.buffer).subarray(a, a + b);
return String.fromCharCode.apply(null, view);
}
function syscall_write([fd, ptr, len]) {
let s = copystr(ptr, len);
switch (fd) {
case 1: process.stdout.write(s); break;
case 2: process.stderr.write(s); break;
}
}
function syscall_exit([code]) {
process.exit(code);
}
function syscall_args(params) {
let [ptr, len] = params;
// Calculate total required buffer size
let totalLen = -1;
for (let i = 2; i < process.argv.length; ++i) {
totalLen += Buffer.byteLength(process.argv[i]) + 1;
}
if (totalLen < 0) { totalLen = 0; }
params[2] = totalLen;
// If buffer is large enough, copy data
if (len >= totalLen) {
let view = new Uint8Array(memory.buffer);
for (let i = 2; i < process.argv.length; ++i) {
let value = process.argv[i];
Buffer.from(value).copy(view, ptr);
ptr += Buffer.byteLength(process.argv[i]) + 1;
}
}
}
function syscall_getenv(params) {
let [keyPtr, keyLen, valuePtr, valueLen] = params;
let key = copystr(keyPtr, keyLen);
let value = process.env[key];
if (value == null) {
params[4] = 0xFFFFFFFF;
} else {
let view = new Uint8Array(memory.buffer);
let totalLen = Buffer.byteLength(value);
params[4] = totalLen;
if (valueLen >= totalLen) {
Buffer.from(value).copy(view, valuePtr);
}
}
}
function syscall_time(params) {
let t = Date.now();
let secs = Math.floor(t / 1000);
let millis = t % 1000;
params[1] = Math.floor(secs / 0x100000000);
params[2] = secs % 0x100000000;
params[3] = Math.floor(millis * 1000000);
}
let imports = {};
imports.env = {
// These are generated by LLVM itself for various intrinsic calls. Hopefully
......@@ -48,68 +110,25 @@ imports.env = {
log10: Math.log10,
log10f: Math.log10,
// These are called in src/libstd/sys/wasm/stdio.rs and are used when
// debugging is enabled.
rust_wasm_write_stdout: function(a, b) {
let s = copystr(a, b);
if (s !== null) {
process.stdout.write(s);
}
},
rust_wasm_write_stderr: function(a, b) {
let s = copystr(a, b);
if (s !== null) {
process.stderr.write(s);
}
},
// These are called in src/libstd/sys/wasm/args.rs and are used when
// debugging is enabled.
rust_wasm_args_count: function() {
if (memory === null)
return 0;
return process.argv.length - 2;
},
rust_wasm_args_arg_size: function(i) {
return Buffer.byteLength(process.argv[i + 2]);
},
rust_wasm_args_arg_fill: function(idx, ptr) {
let arg = process.argv[idx + 2];
let view = new Uint8Array(memory.buffer);
Buffer.from(arg).copy(view, ptr);
},
// These are called in src/libstd/sys/wasm/os.rs and are used when
// debugging is enabled.
rust_wasm_getenv_len: function(a, b) {
let key = copystr(a, b);
if (key === null) {
return -1;
rust_wasm_syscall: function(index, data) {
switch (index) {
case 1: syscall_write(viewstruct(data, 3)); return true;
case 2: syscall_exit(viewstruct(data, 1)); return true;
case 3: syscall_args(viewstruct(data, 3)); return true;
case 4: syscall_getenv(viewstruct(data, 5)); return true;
case 6: syscall_time(viewstruct(data, 4)); return true;
default:
console.log("Unsupported syscall: " + index);
return false;
}
if (!(key in process.env)) {
return -1;
}
return Buffer.byteLength(process.env[key]);
},
rust_wasm_getenv_data: function(a, b, ptr) {
let key = copystr(a, b);
let value = process.env[key];
let view = new Uint8Array(memory.buffer);
Buffer.from(value).copy(view, ptr);
},
};
let module_imports = WebAssembly.Module.imports(m);
for (var i = 0; i < module_imports.length; i++) {
let imp = module_imports[i];
if (imp.module != 'env') {
continue
}
if (imp.name == 'memory' && imp.kind == 'memory') {
memory = new WebAssembly.Memory({initial: 20});
imports.env.memory = memory;
}
}
};
let instance = new WebAssembly.Instance(m, imports);
memory = instance.exports.memory;
try {
instance.exports.main();
} catch (e) {
console.error(e);
process.exit(101);
}
......@@ -824,9 +824,7 @@ fn binaryen_assemble(cgcx: &CodegenContext,
if cgcx.debuginfo != config::NoDebugInfo {
options.debuginfo(true);
}
if cgcx.crate_types.contains(&config::CrateTypeExecutable) {
options.start("main");
}
options.stack(1024 * 1024);
options.import_memory(cgcx.wasm_import_memory);
let assembled = input.and_then(|input| {
......@@ -1452,7 +1450,7 @@ fn start_executing_work(tcx: TyCtxt,
target_pointer_width: tcx.sess.target.target.target_pointer_width.clone(),
binaryen_linker: tcx.sess.linker_flavor() == LinkerFlavor::Binaryen,
debuginfo: tcx.sess.opts.debuginfo,
wasm_import_memory: wasm_import_memory,
wasm_import_memory,
assembler_cmd,
};
......
......@@ -48,3 +48,4 @@ jemalloc = ["alloc_jemalloc"]
force_alloc_system = []
panic-unwind = ["panic_unwind"]
profiler = ["profiler_builtins"]
wasm_syscall = []
......@@ -10,8 +10,8 @@
use ffi::OsString;
use marker::PhantomData;
use mem;
use vec;
use sys::ArgsSysCall;
pub unsafe fn init(_argc: isize, _argv: *const *const u8) {
// On wasm these should always be null, so there's nothing for us to do here
......@@ -21,38 +21,10 @@ pub unsafe fn cleanup() {
}
pub fn args() -> Args {
// When the runtime debugging is enabled we'll link to some extra runtime
// functions to actually implement this. These are for now just implemented
// in a node.js script but they're off by default as they're sort of weird
// in a web-wasm world.
if !super::DEBUG {
return Args {
iter: Vec::new().into_iter(),
_dont_send_or_sync_me: PhantomData,
}
}
// You'll find the definitions of these in `src/etc/wasm32-shim.js`. These
// are just meant for debugging and should not be relied on.
extern {
fn rust_wasm_args_count() -> usize;
fn rust_wasm_args_arg_size(a: usize) -> usize;
fn rust_wasm_args_arg_fill(a: usize, ptr: *mut u8);
}
unsafe {
let cnt = rust_wasm_args_count();
let mut v = Vec::with_capacity(cnt);
for i in 0..cnt {
let n = rust_wasm_args_arg_size(i);
let mut data = vec![0; n];
rust_wasm_args_arg_fill(i, data.as_mut_ptr());
v.push(mem::transmute::<Vec<u8>, OsString>(data));
}
Args {
iter: v.into_iter(),
_dont_send_or_sync_me: PhantomData,
}
let v = ArgsSysCall::perform();
Args {
iter: v.into_iter(),
_dont_send_or_sync_me: PhantomData,
}
}
......
......@@ -26,17 +26,11 @@
use io;
use os::raw::c_char;
// Right now the wasm backend doesn't even have the ability to print to the
// console by default. Wasm can't import anything from JS! (you have to
// explicitly provide it).
//
// Sometimes that's a real bummer, though, so this flag can be set to `true` to
// enable calling various shims defined in `src/etc/wasm32-shim.js` which should
// help receive debug output and see what's going on. In general this flag
// currently controls "will we call out to our own defined shims in node.js",
// and this flag should always be `false` for release builds.
const DEBUG: bool = false;
use ptr;
use sys::os_str::Buf;
use sys_common::{AsInner, FromInner};
use ffi::{OsString, OsStr};
use time::Duration;
pub mod args;
#[cfg(feature = "backtrace")]
......@@ -92,7 +86,7 @@ pub unsafe fn strlen(mut s: *const c_char) -> usize {
}
pub unsafe fn abort_internal() -> ! {
::intrinsics::abort();
ExitSysCall::perform(1)
}
// We don't have randomness yet, but I totally used a random number generator to
......@@ -103,3 +97,218 @@ pub unsafe fn abort_internal() -> ! {
pub fn hashmap_random_keys() -> (u64, u64) {
(1, 2)
}
// Implement a minimal set of system calls to enable basic IO
pub enum SysCallIndex {
Read = 0,
Write = 1,
Exit = 2,
Args = 3,
GetEnv = 4,
SetEnv = 5,
Time = 6,
}
#[repr(C)]
pub struct ReadSysCall {
fd: usize,
ptr: *mut u8,
len: usize,
result: usize,
}
impl ReadSysCall {
pub fn perform(fd: usize, buffer: &mut [u8]) -> usize {
let mut call_record = ReadSysCall {
fd,
len: buffer.len(),
ptr: buffer.as_mut_ptr(),
result: 0
};
if unsafe { syscall(SysCallIndex::Read, &mut call_record) } {
call_record.result
} else {
0
}
}
}
#[repr(C)]
pub struct WriteSysCall {
fd: usize,
ptr: *const u8,
len: usize,
}
impl WriteSysCall {
pub fn perform(fd: usize, buffer: &[u8]) {
let mut call_record = WriteSysCall {
fd,
len: buffer.len(),
ptr: buffer.as_ptr()
};
unsafe { syscall(SysCallIndex::Write, &mut call_record); }
}
}
#[repr(C)]
pub struct ExitSysCall {
code: usize,
}
impl ExitSysCall {
pub fn perform(code: usize) -> ! {
let mut call_record = ExitSysCall {
code
};
unsafe {
syscall(SysCallIndex::Exit, &mut call_record);
::intrinsics::abort();
}
}
}
fn receive_buffer<E, F: FnMut(&mut [u8]) -> Result<usize, E>>(estimate: usize, mut f: F)
-> Result<Vec<u8>, E>
{
let mut buffer = vec![0; estimate];
loop {
let result = f(&mut buffer)?;
if result <= buffer.len() {
buffer.truncate(result);
break;
}
buffer.resize(result, 0);
}
Ok(buffer)
}
#[repr(C)]
pub struct ArgsSysCall {
ptr: *mut u8,
len: usize,
result: usize
}
impl ArgsSysCall {
pub fn perform() -> Vec<OsString> {
receive_buffer(1024, |buffer| -> Result<usize, !> {
let mut call_record = ArgsSysCall {
len: buffer.len(),
ptr: buffer.as_mut_ptr(),
result: 0
};
if unsafe { syscall(SysCallIndex::Args, &mut call_record) } {
Ok(call_record.result)
} else {
Ok(0)
}
})
.unwrap()
.split(|b| *b == 0)
.map(|s| FromInner::from_inner(Buf { inner: s.to_owned() }))
.collect()
}
}
#[repr(C)]
pub struct GetEnvSysCall {
key_ptr: *const u8,
key_len: usize,
value_ptr: *mut u8,
value_len: usize,
result: usize
}
impl GetEnvSysCall {
pub fn perform(key: &OsStr) -> Option<OsString> {
let key_buf = &AsInner::as_inner(key).inner;
receive_buffer(64, |buffer| {
let mut call_record = GetEnvSysCall {
key_len: key_buf.len(),
key_ptr: key_buf.as_ptr(),
value_len: buffer.len(),
value_ptr: buffer.as_mut_ptr(),
result: !0usize
};
if unsafe { syscall(SysCallIndex::GetEnv, &mut call_record) } {
if call_record.result == !0usize {
Err(())
} else {
Ok(call_record.result)
}
} else {
Err(())
}
}).ok().map(|s| {
FromInner::from_inner(Buf { inner: s })
})
}
}
#[repr(C)]
pub struct SetEnvSysCall {
key_ptr: *const u8,
key_len: usize,
value_ptr: *const u8,
value_len: usize
}
impl SetEnvSysCall {
pub fn perform(key: &OsStr, value: Option<&OsStr>) {
let key_buf = &AsInner::as_inner(key).inner;
let value_buf = value.map(|v| &AsInner::as_inner(v).inner);
let mut call_record = SetEnvSysCall {
key_len: key_buf.len(),
key_ptr: key_buf.as_ptr(),
value_len: value_buf.map(|v| v.len()).unwrap_or(!0usize),
value_ptr: value_buf.map(|v| v.as_ptr()).unwrap_or(ptr::null())
};
unsafe { syscall(SysCallIndex::SetEnv, &mut call_record); }
}
}
pub enum TimeClock {
Monotonic = 0,
System = 1,
}
#[repr(C)]
pub struct TimeSysCall {
clock: usize,
secs_hi: usize,
secs_lo: usize,
nanos: usize
}
impl TimeSysCall {
pub fn perform(clock: TimeClock) -> Duration {
let mut call_record = TimeSysCall {
clock: clock as usize,
secs_hi: 0,
secs_lo: 0,
nanos: 0
};
if unsafe { syscall(SysCallIndex::Time, &mut call_record) } {
Duration::new(
((call_record.secs_hi as u64) << 32) | (call_record.secs_lo as u64),
call_record.nanos as u32
)
} else {
panic!("Time system call is not implemented by WebAssembly host");
}
}
}
unsafe fn syscall<T>(index: SysCallIndex, data: &mut T) -> bool {
#[cfg(feature = "wasm_syscall")]
extern {
#[no_mangle]
fn rust_wasm_syscall(index: usize, data: *mut Void) -> usize;
}
#[cfg(not(feature = "wasm_syscall"))]
unsafe fn rust_wasm_syscall(_index: usize, _data: *mut Void) -> usize { 0 }
rust_wasm_syscall(index as usize, data as *mut T as *mut Void) != 0
}
......@@ -8,16 +8,13 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.
use core::intrinsics;
use error::Error as StdError;
use ffi::{OsString, OsStr};
use fmt;
use io;
use mem;
use path::{self, PathBuf};
use str;
use sys::{unsupported, Void};
use sys::{unsupported, Void, ExitSysCall, GetEnvSysCall, SetEnvSysCall};
pub fn errno() -> i32 {
0
......@@ -87,36 +84,15 @@ pub fn env() -> Env {
}
pub fn getenv(k: &OsStr) -> io::Result<Option<OsString>> {
// If we're debugging the runtime then we actually probe node.js to ask for
// the value of environment variables to help provide inputs to programs.
// The `extern` shims here are defined in `src/etc/wasm32-shim.js` and are
// intended for debugging only, you should not rely on them.
if !super::DEBUG {
return Ok(None)
}
extern {
fn rust_wasm_getenv_len(k: *const u8, kl: usize) -> isize;
fn rust_wasm_getenv_data(k: *const u8, kl: usize, v: *mut u8);
}
unsafe {
let k: &[u8] = mem::transmute(k);
let n = rust_wasm_getenv_len(k.as_ptr(), k.len());
if n == -1 {
return Ok(None)
}
let mut data = vec![0; n as usize];
rust_wasm_getenv_data(k.as_ptr(), k.len(), data.as_mut_ptr());
Ok(Some(mem::transmute(data)))
}
Ok(GetEnvSysCall::perform(k))
}
pub fn setenv(_k: &OsStr, _v: &OsStr) -> io::Result<()> {
unsupported()
pub fn setenv(k: &OsStr, v: &OsStr) -> io::Result<()> {
Ok(SetEnvSysCall::perform(k, Some(v)))
}
pub fn unsetenv(_n: &OsStr) -> io::Result<()> {
unsupported()
pub fn unsetenv(k: &OsStr) -> io::Result<()> {
Ok(SetEnvSysCall::perform(k, None))
}
pub fn temp_dir() -> PathBuf {
......@@ -128,7 +104,7 @@ pub fn home_dir() -> Option<PathBuf> {
}
pub fn exit(_code: i32) -> ! {
unsafe { intrinsics::abort() }
ExitSysCall::perform(_code as isize as usize)
}
pub fn getpid() -> u32 {
......
......@@ -9,19 +9,19 @@
// except according to those terms.
use io;
use sys::{Void, unsupported};
use sys::{ReadSysCall, WriteSysCall};
pub struct Stdin(Void);
pub struct Stdin;
pub struct Stdout;
pub struct Stderr;
impl Stdin {
pub fn new() -> io::Result<Stdin> {
unsupported()
Ok(Stdin)
}
pub fn read(&self, _data: &mut [u8]) -> io::Result<usize> {
match self.0 {}
pub fn read(&self, data: &mut [u8]) -> io::Result<usize> {
Ok(ReadSysCall::perform(0, data))
}
}
......@@ -31,19 +31,7 @@ pub fn new() -> io::Result<Stdout> {
}
pub fn write(&self, data: &[u8]) -> io::Result<usize> {
// If runtime debugging is enabled at compile time we'll invoke some
// runtime functions that are defined in our src/etc/wasm32-shim.js
// debugging script. Note that this ffi function call is intended
// *purely* for debugging only and should not be relied upon.
if !super::DEBUG {
return unsupported()
}
extern {
fn rust_wasm_write_stdout(data: *const u8, len: usize);
}
unsafe {
rust_wasm_write_stdout(data.as_ptr(), data.len())
}
WriteSysCall::perform(1, data);
Ok(data.len())
}
......@@ -58,16 +46,7 @@ pub fn new() -> io::Result<Stderr> {
}
pub fn write(&self, data: &[u8]) -> io::Result<usize> {
// See comments in stdout for what's going on here.
if !super::DEBUG {
return unsupported()
}
extern {
fn rust_wasm_write_stderr(data: *const u8, len: usize);
}
unsafe {
rust_wasm_write_stderr(data.as_ptr(), data.len())
}
WriteSysCall::perform(2, data);
Ok(data.len())
}
......
......@@ -8,56 +8,50 @@
// option. This file may not be copied, modified, or distributed
// except according to those terms.
use fmt;
use time::Duration;
use sys::{TimeSysCall, TimeClock};
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug, Hash)]
pub struct Instant;
pub struct Instant(Duration);
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct SystemTime;
#[derive(Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Debug, Hash)]
pub struct SystemTime(Duration);
pub const UNIX_EPOCH: SystemTime = SystemTime;
pub const UNIX_EPOCH: SystemTime = SystemTime(Duration::from_secs(0));
impl Instant {
pub fn now() -> Instant {
panic!("not supported on web assembly");
Instant(TimeSysCall::perform(TimeClock::Monotonic))
}
pub fn sub_instant(&self, _other: &Instant) -> Duration {
panic!("can't sub yet");
pub fn sub_instant(&self, other: &Instant) -> Duration {
self.0 - other.0
}
pub fn add_duration(&self, _other: &Duration) -> Instant {
panic!("can't add yet");
pub fn add_duration(&self, other: &Duration) -> Instant {
Instant(self.0 + *other)
}
pub fn sub_duration(&self, _other: &Duration) -> Instant {
panic!("can't sub yet");
pub fn sub_duration(&self, other: &Duration) -> Instant {
Instant(self.0 - *other)
}
}
impl SystemTime {
pub fn now() -> SystemTime {
panic!("not supported on web assembly");
SystemTime(TimeSysCall::perform(TimeClock::System))
}
pub fn sub_time(&self, _other: &SystemTime)
pub fn sub_time(&self, other: &SystemTime)
-> Result<Duration, Duration> {
panic!()
}
pub fn add_duration(&self, _other: &Duration) -> SystemTime {
panic!()
self.0.checked_sub(other.0).ok_or_else(|| other.0 - self.0)
}
pub fn sub_duration(&self, _other: &Duration) -> SystemTime {
panic!()
pub fn add_duration(&self, other: &Duration) -> SystemTime {
SystemTime(self.0 + *other)
}
}
impl fmt::Debug for SystemTime {
fn fmt(&self, _f: &mut fmt::Formatter) -> fmt::Result {
panic!()
pub fn sub_duration(&self, other: &Duration) -> SystemTime {
SystemTime(self.0 - *other)
}
}
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册