profile
viewpoint

acfoltzer/cargo-fund 56

Discover funding links for your project's dependencies.

acfoltzer/cereal-vector 5

Serialize instances for Data.Vector types

acfoltzer/bit-vector 2

Simple bit vectors for Haskell

acfoltzer/accelerate 1

Embedded language for high-performance array computations

acfoltzer/cereal-derive 1

Derive instances of Serialize

acfoltzer/CloudHaskell 1

A distributed computing framework for Haskell

acfoltzer/buildroot 0

Buildroot, making embedded Linux easy. Note that this is not the official repository, but only a mirror. The official Git repository is at http://git.buildroot.net/buildroot/. Do not open issues or file pull requests here.

issue openedbytecodealliance/lucet

Reduce the reliance on strings in error types

We have, for example, the variant lucet_runtime::Error::LimitsExceeded(String). If a user of the runtime wants to know which limit they've exceeded, they have to look for keywords in the string. In this example, we should be able to enumerate which limits were exceeded, and possibly bundle that along with the Limits struct itself.

There are a few other variants worth considering, like InvalidArgument, NoLinearMemory, and UnsupportedFeature.

created time in 18 days

issue openedbytecodealliance/lucet

Add command-line option to `lucetc` to limit the globals size of produced modules.

We already have --max-reserved-size fulfilling a similar purpose. Being able to do the same for globals will help move errors from instantiation time to compile time when we know in advance the maximum globals size configured in the runtime.

created time in 18 days

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 380a750f78aca6cf98f2b92ee471c6c5484327c0

🈵 Add an option to terminate the runtime when `memory.grow` fails This new option determines whether to terminate the guest with a new `TerminationDetails::HeapOutOfMemory` variant when `memory.grow` fails, rather than returning `-1`. It is disabled by default, but can be set via `InstanceBuilder` or directly on the `Instance`. This behavior deviates from the WebAssembly spec, but is useful in practice for determining when guest programs fail due to an exhausted heap. Most languages will compile to code that includes an `unreachable` instruction if allocation fails, but this same instruction might also appear when other types of assertions fail, `panic!()` is called, etc. Terminating allows the error to be more directly identifiable.

view details

Adam C. Foltzer

commit sha 9661541fcd6f617c438bf528e3a75f58d092cbca

Merge branch 'main' into acf/terminate-on-heap-oom

view details

Adam C. Foltzer

commit sha 3af43827608c2439fb2016d1d3f934b3d1248b4c

Merge pull request #583 from bytecodealliance/acf/terminate-on-heap-oom 🈵 Add an option to terminate the runtime when `memory.grow` fails

view details

push time in 19 days

delete branch bytecodealliance/lucet

delete branch : acf/terminate-on-heap-oom

delete time in 19 days

PR merged bytecodealliance/lucet

🈵 Add an option to terminate the runtime when `memory.grow` fails

This new option determines whether to terminate the guest with a new TerminationDetails::HeapOutOfMemory variant when memory.grow fails, rather than returning -1. It is disabled by default, but can be set via InstanceBuilder or directly on the Instance.

This behavior deviates from the WebAssembly spec, but is useful in practice for determining when guest programs fail due to an exhausted heap.

Most languages will compile to code that includes an unreachable instruction if allocation fails, but this same instruction might also appear when other types of assertions fail, panic!() is called, etc. Terminating allows the error to be more directly identifiable.

+202 -41

1 comment

10 changed files

acfoltzer

pr closed time in 19 days

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha a9466e3d5ca6e5ab5ca613b011190c13da34bf85

☢️ Remove UB from `extern "C"` ABI on annotated hostcalls This was a programming error introduced in the transition between the `lucet_hostcalls!` macro and the `#[lucet_hostcall]` attribute. The expansion of the code in the attribute called the annotated function from within a `catch_unwind` without modifying its ABI. Since it is UB to unwind from a non-Rust ABI function, release mode optimizations would elide the code which would otherwise handle an unwind from the function. This caused the process to abort if the function ever _did_ unwind, since the system unwinding runtime would reach the end of the guest stack without finding any frames willing to handle the exception. I believe this behavior was very difficult, if not impossible, to observe in practice with the current state of the Lucet codebase. The only unwinding that these functions could directly induce are the assertions in `vmctx::instance_from_vmctx` that should never be violated except in the case of a Lucet programming error, or a dynamic linking problem. However, in #583 we are adding an option to instances that would cause `lucet_vmctx_grow_memory` to potentially terminate the instance via the `lucet_hostcall_terminate!()` mechanism that is implemented via unwinding. We discovered this problem during testing on that branch, and so we need to land this fix before it can proceed. This patch contains a fix to the immediate problem by removing the erroneous `extern "C"` specifiers. It also prevents the problem recurring in the future by generating a compile-time error from `#[lucet_hostcall]` if there is an ABI specifier present on the function it annotates.

view details

Adam C. Foltzer

commit sha 7666c3f08d0e57d766fdca476b5dd0e08b36ece0

remove some additional stray `extern "C"`s on hostcalls Thank you, macro error!

view details

Adam C. Foltzer

commit sha ca8a8fbf5cb6d28953470793a225677edc2572a7

one more straggler (I think)

view details

Adam C. Foltzer

commit sha a9a93553eae22dc03274b4039c4c835072771d8d

Merge pull request #584 from bytecodealliance/acf/fix-ub-hostcalls ☢️ Remove UB from `extern "C"` ABI on annotated hostcalls

view details

Adam C. Foltzer

commit sha 9661541fcd6f617c438bf528e3a75f58d092cbca

Merge branch 'main' into acf/terminate-on-heap-oom

view details

push time in 19 days

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha a9466e3d5ca6e5ab5ca613b011190c13da34bf85

☢️ Remove UB from `extern "C"` ABI on annotated hostcalls This was a programming error introduced in the transition between the `lucet_hostcalls!` macro and the `#[lucet_hostcall]` attribute. The expansion of the code in the attribute called the annotated function from within a `catch_unwind` without modifying its ABI. Since it is UB to unwind from a non-Rust ABI function, release mode optimizations would elide the code which would otherwise handle an unwind from the function. This caused the process to abort if the function ever _did_ unwind, since the system unwinding runtime would reach the end of the guest stack without finding any frames willing to handle the exception. I believe this behavior was very difficult, if not impossible, to observe in practice with the current state of the Lucet codebase. The only unwinding that these functions could directly induce are the assertions in `vmctx::instance_from_vmctx` that should never be violated except in the case of a Lucet programming error, or a dynamic linking problem. However, in #583 we are adding an option to instances that would cause `lucet_vmctx_grow_memory` to potentially terminate the instance via the `lucet_hostcall_terminate!()` mechanism that is implemented via unwinding. We discovered this problem during testing on that branch, and so we need to land this fix before it can proceed. This patch contains a fix to the immediate problem by removing the erroneous `extern "C"` specifiers. It also prevents the problem recurring in the future by generating a compile-time error from `#[lucet_hostcall]` if there is an ABI specifier present on the function it annotates.

view details

Adam C. Foltzer

commit sha 7666c3f08d0e57d766fdca476b5dd0e08b36ece0

remove some additional stray `extern "C"`s on hostcalls Thank you, macro error!

view details

Adam C. Foltzer

commit sha ca8a8fbf5cb6d28953470793a225677edc2572a7

one more straggler (I think)

view details

Adam C. Foltzer

commit sha a9a93553eae22dc03274b4039c4c835072771d8d

Merge pull request #584 from bytecodealliance/acf/fix-ub-hostcalls ☢️ Remove UB from `extern "C"` ABI on annotated hostcalls

view details

push time in 19 days

delete branch bytecodealliance/lucet

delete branch : acf/fix-ub-hostcalls

delete time in 19 days

PR merged bytecodealliance/lucet

Reviewers
☢️ Remove UB from `extern "C"` ABI on annotated hostcalls

This was a programming error introduced in the transition between the lucet_hostcalls! macro and the #[lucet_hostcall] attribute.

The expansion of the code in the attribute called the annotated function from within a catch_unwind without modifying its ABI. Since it is UB to unwind from a non-Rust ABI function, release mode optimizations would elide the code which would otherwise handle an unwind from the function. This caused the process to abort if the function ever did unwind, since the system unwinding runtime would reach the end of the guest stack without finding any frames willing to handle the exception.

I believe this behavior was very difficult, if not impossible, to observe in practice with the current state of the Lucet codebase. The only unwinding that these functions could directly induce are the assertions in vmctx::instance_from_vmctx that should never be violated except in the case of a Lucet programming error, or a dynamic linking problem.

However, in #583 we are adding an option to instances that would cause lucet_vmctx_grow_memory to potentially terminate the instance via the lucet_hostcall_terminate!() mechanism that is implemented via unwinding. We discovered this problem during testing on that branch, and so we need to land this fix before it can proceed.

This patch contains a fix to the immediate problem by removing the erroneous extern "C" specifiers. It also prevents the problem recurring in the future by generating a compile-time error from #[lucet_hostcall] if there is an ABI specifier present on the function it annotates.

+20 -18

0 comment

5 changed files

acfoltzer

pr closed time in 19 days

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha ca8a8fbf5cb6d28953470793a225677edc2572a7

one more straggler (I think)

view details

push time in 19 days

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 7666c3f08d0e57d766fdca476b5dd0e08b36ece0

remove some additional stray `extern "C"`s on hostcalls Thank you, macro error!

view details

push time in 19 days

pull request commentbytecodealliance/lucet

🈵 Add an option to terminate the runtime when `memory.grow` fails

Blocking this pending #584

acfoltzer

comment created time in 19 days

PR opened bytecodealliance/lucet

☢️ Remove UB from `extern "C"` ABI on annotated hostcalls

This was a programming error introduced in the transition between the lucet_hostcalls! macro and the #[lucet_hostcall] attribute.

The expansion of the code in the attribute called the annotated function from within a catch_unwind without modifying its ABI. Since it is UB to unwind from a non-Rust ABI function, release mode optimizations would elide the code which would otherwise handle an unwind from the function. This caused the process to abort if the function ever did unwind, since the system unwinding runtime would reach the end of the guest stack without finding any frames willing to handle the exception.

I believe this behavior was very difficult, if not impossible, to observe in practice with the current state of the Lucet codebase. The only unwinding that these functions could directly induce are the assertions in vmctx::instance_from_vmctx that should never be violated except in the case of a Lucet programming error, or a dynamic linking problem.

However, in #583 we are adding an option to instances that would cause lucet_vmctx_grow_memory to potentially terminate the instance via the lucet_hostcall_terminate!() mechanism that is implemented via unwinding. We discovered this problem during testing on that branch, and so we need to land this fix before it can proceed.

This patch contains a fix to the immediate problem by removing the erroneous extern "C" specifiers. It also prevents the problem recurring in the future by generating a compile-time error from #[lucet_hostcall] if there is an ABI specifier present on the function it annotates.

+16 -14

0 comment

2 changed files

pr created time in 19 days

create barnchbytecodealliance/lucet

branch : acf/fix-ub-hostcalls

created branch time in 19 days

PR opened bytecodealliance/lucet

Reviewers
🈵 Add an option to terminate the runtime when `memory.grow` fails

This new option determines whether to terminate the guest with a new TerminationDetails::HeapOutOfMemory variant when memory.grow fails, rather than returning -1. It is disabled by default, but can be set via InstanceBuilder or directly on the Instance.

This behavior deviates from the WebAssembly spec, but is useful in practice for determining when guest programs fail due to an exhausted heap.

Most languages will compile to code that includes an unreachable instruction if allocation fails, but this same instruction might also appear when other types of assertions fail, panic!() is called, etc. Terminating allows the error to be more directly identifiable.

+202 -41

0 comment

10 changed files

pr created time in 20 days

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 380a750f78aca6cf98f2b92ee471c6c5484327c0

🈵 Add an option to terminate the runtime when `memory.grow` fails This new option determines whether to terminate the guest with a new `TerminationDetails::HeapOutOfMemory` variant when `memory.grow` fails, rather than returning `-1`. It is disabled by default, but can be set via `InstanceBuilder` or directly on the `Instance`. This behavior deviates from the WebAssembly spec, but is useful in practice for determining when guest programs fail due to an exhausted heap. Most languages will compile to code that includes an `unreachable` instruction if allocation fails, but this same instruction might also appear when other types of assertions fail, `panic!()` is called, etc. Terminating allows the error to be more directly identifiable.

view details

push time in 20 days

create barnchbytecodealliance/lucet

branch : acf/terminate-on-heap-oom

created branch time in 20 days

PR closed servo/rust-url

url: add TryFrom<&[u8]> for Url and TryFrom tests

Per discussion on Matrix.

I didn't realize when first asking that TryFrom<&str> is already implemented, but hasn't been released. So, I ended up just adding the impl for byte slices, an error variant for invalid UTF-8 inputs, and tests for the impls.

+27 -1

5 comments

3 changed files

acfoltzer

pr closed time in a month

pull request commentservo/rust-url

url: add TryFrom<&[u8]> for Url and TryFrom tests

No worries, again it's just a convenience and not a dealbreaker for our adoption.

acfoltzer

comment created time in a month

pull request commentservo/rust-url

url: add TryFrom<&[u8]> for Url and TryFrom tests

@SimonSapin I definitely agree with you on these being separate domains of data, but I believe that's reflected by the use of TryFrom (rather than From) and the addition of the InvalidUtf8 error variant.

There wasn't much discussion on Matrix really: image

acfoltzer

comment created time in a month

pull request commentservo/rust-url

url: add TryFrom<&[u8]> for Url and TryFrom tests

I believe CI is failing because a credential is not available from my fork.

acfoltzer

comment created time in a month

PR opened servo/rust-url

url: add TryFrom<&[u8]> for Url and TryFrom tests

Per discussion on Matrix.

I didn't realize when first asking that TryFrom<&str> is already implemented, but hasn't been released. So, I ended up just adding the impl for byte slices, an error variant for invalid UTF-8 inputs, and tests for the impls.

+27 -1

0 comment

3 changed files

pr created time in a month

create barnchacfoltzer/rust-url

branch : try-from-bytes

created branch time in a month

PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent
PullRequestReviewEvent

pull request commenttokio-rs/tracing

subscriber: Impl TestWriter for correct capturing of logs by cargo test

Yay, thank you!

samrg472

comment created time in a month

push eventacfoltzer/http

Adam C. Foltzer

commit sha b00473c3ec942c03de55e47af1070b16a9dd23d8

Return an error rather than panicking when HeaderName is too long (#433) Fixes #432. This eliminates an undocumented panic from the `HeaderName` creation functions, returning instead an `InvalidHeaderName` when the name length exceeds `MAX_HEADER_NAME_LEN`. I considered making `InvalidHeaderName` a richer error type, but since it was already used for another type of length error (rejecting zero-length names) I figured it was okay to reuse. I also redefined `MAX_HEADER_NAME_LEN` slightly, so that it is equal to the largest allowed header length, rather than one past that value. This was motivated by discovering a bug in my comparison logic when I went to write the new test exercising the wrong-length error conditions.

view details

push time in 2 months

delete branch acfoltzer/http

delete branch : header-name-len-error

delete time in 2 months

PR opened hyperium/http

Return an error rather than panicking when HeaderName is too long

Fixes #432.

This eliminates an undocumented panic from the HeaderName creation functions, returning instead an InvalidHeaderName when the name length exceeds MAX_HEADER_NAME_LEN.

I considered making InvalidHeaderName a richer error type, but since it was already used for another type of length error (rejecting zero-length names) I figured it was okay to reuse.

I also redefined MAX_HEADER_NAME_LEN slightly, so that it is equal to the largest allowed header length, rather than one past that value. This was motivated by discovering a bug in my comparison logic when I went to write the new test exercising the wrong-length error conditions.

+27 -14

0 comment

2 changed files

pr created time in 2 months

create barnchacfoltzer/http

branch : header-name-len-error

created branch time in 2 months

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 69a187103f16600c95d05f982fa6551d280f3497

💢 Package unknown hostcall panics into termination details This prevents panics that aren't intentionally initiated by the Lucet runtime from attempting to unwind across Wasm stack frames. When #254 lands, this will no longer be necessary, but until then this allows us to safely transport the unknown panic payload across the FFI boundary so that panicking can resume on the other side.

view details

Adam C. Foltzer

commit sha d80e86cc0cd2cb5ec9e3254e4b99fdb6336e735f

Merge pull request #572 from bytecodealliance/acf/panic-ferry 💢 Package unknown hostcall panics into termination details

view details

push time in 2 months

delete branch bytecodealliance/lucet

delete branch : acf/panic-ferry

delete time in 2 months

PR merged bytecodealliance/lucet

💢 Package unknown hostcall panics into termination details

This prevents panics that aren't intentionally initiated by the Lucet runtime from attempting to unwind across Wasm stack frames. When #254 lands, this will no longer be necessary, but until then this allows us to safely transport the unknown panic payload across the FFI boundary so that panicking can resume on the other side.

+87 -10

0 comment

5 changed files

acfoltzer

pr closed time in 2 months

Pull request review commentbytecodealliance/lucet

💢 Package unknown hostcall panics into termination details

 pub mod lucet_result {                             },                             TerminationDetails::Provided(p) => lucet_terminated {                                 reason: lucet_terminated_reason::Provided,-                                provided: p+                                provided: *p                                     .downcast_ref()-                                    .map(|CTerminationDetails { details }| *details)-                                    .unwrap_or(ptr::null_mut()),+                                    .map(|CTerminationDetails { details }| details)+                                    .unwrap_or(&ptr::null_mut()),                             },                             TerminationDetails::Remote => lucet_terminated {                                 reason: lucet_terminated_reason::Remote,                                 provided: std::ptr::null_mut(),                             },+                            TerminationDetails::OtherPanic(p) => lucet_terminated {+                                reason: lucet_terminated_reason::OtherPanic,+                                // double box the panic payload so that the pointer passed to FFI+                                // land is thin

Exactly; Box::into_raw(x) where x is Box<dyn T> returns a *mut dyn T, which is two machine pointers wide for the vtable, so casting it to *mut c_void to store in the struct would be Bad. The extra box makes the returned raw pointer have the type *mut Box<dyn T>, which is fine because the vtable pointer is then stored in the box on the heap.

acfoltzer

comment created time in 2 months

PR opened bytecodealliance/lucet

💢 Package unknown hostcall panics into termination details

This prevents panics that aren't intentionally initiated by the Lucet runtime from attempting to unwind across Wasm stack frames. When #254 lands, this will no longer be necessary, but until then this allows us to safely transport the unknown panic payload across the FFI boundary so that panicking can resume on the other side.

+87 -10

0 comment

5 changed files

pr created time in 2 months

create barnchbytecodealliance/lucet

branch : acf/panic-ferry

created branch time in 2 months

issue openedhyperium/http

HeaderName constructors panic for names longer than 64KiB

Attempting to create a HeaderName longer than 64KiB fails, but with a panic rather than an InvalidHeaderName error: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f43aad9f01cc5c1bd05149129310c7e7

I think this should return Err(InvalidHeaderName) instead, but if there is good cause to keep it as a panic, that panic should be documented on the various constructors so it's less of a surprise.

If you agree about switching to the Err behavior, I'd be happy to put in a PR.

created time in 2 months

push eventacfoltzer/http

Adam C. Foltzer

commit sha 26681ddee15cb4885c6ef0b9820cf0627f3a6d88

Add `extensions_ref()` and `extensions_mut()` to req/resp builders (#403) A similar capability already exists for many of the other `Parts` fields in these builders, and is useful for the extensions map as well.

view details

Sean McArthur

commit sha 35d60da93fdcf95c1483c24e8157d6b900240209

Remove unnecessary parentheses (#404)

view details

Sean McArthur

commit sha 1af2eb7a124ced8055f03a5071255812b86627b4

Switch CI from travis to Github Actions (#405)

view details

Sean McArthur

commit sha 090badcf00426abe582332d19abf3ea0fd9db5fe

v0.2.1

view details

Steven Bosnick

commit sha f769e1e9074a4d9fa70e7137d640f0bbb03698b2

Add tests for valid and invalid Method's The tests for invalid methods test invalid byte sequences including invalid utf-8 and valid utf-8 that uses invalid characters for a Method. The tests for valid methods test both short and long extension methods. Also extract all of the unit tests into a "test" module.

view details

Steven Bosnick

commit sha 493b38fdc9f852a85c0f8cdefa4225047163bf05

Refactor Method internals Extract the inner types for ExtensionAllocated and ExtensionInline into a separate extension module that has the supporting functions as non-public elements. This refactoring moves the use of "unsafe" into this new "extension" module and provides a safe wrappers around the two uses of "unsafe".

view details

Steven Bosnick

commit sha dc865ff0fbb038c9ea3b78f6162f833a95d57e9d

Document the invariants that make Method safe The internal InlineExtension and AllocatedExtension types have invariants that ensure that the two uses of "unsafe" in the "extension" submodule of "method" are safe. This documents those invariants for future reference.

view details

Steven Bosnick

commit sha 25c56744cdfb9b0555777a889d7823785ec54560

Add tests for directly parsing a Scheme The tests include an [u8] that is invalid UTF-8.

view details

Steven Bosnick

commit sha 3ef1133c366a7a4779cf384e9536ba6ca891db08

Add comments to describe safety of Scheme The comments describe the postcondition on parse_exact() that makes the one use of "unsafe" in Scheme::try_from(&'a [u8]) sound.

view details

Steven Bosnick

commit sha 59733e146affb775eec7432a3e0ca3bd564b7dc2

Audit use of unsafe in uri/authority.rs (#414) * Add unit test for rejecting invalid UTF-8 * Add Authority::from_static() test * Refactor uri::Authority Extract the common code from three ways of creating an Authority into a private create_authority() function. * Add comments to explain the safety of Authority The comments describe the preconditions and postconditions that together ensure that the one use of 'unsafe' in uri/authority.rs is sound. * Fix typo

view details

Steven Bosnick

commit sha d8996d237f852f1b84cb0378943dbaa50a108287

Fix the "Build Status" indicator in README.md (#425) The "Build Status" markup badge had not been updated to refect the change of CI systems from TravisCI to GitHub Actions.

view details

Steven Bosnick

commit sha 4f11af5bd2c469b147b6303a4e5d95b5b5ecf6b4

Fix the build status badge in README.md (#426) This change makes the alt tag for the badge image closer to what is shown in the immage itself (both are will show the name of the workflow: CI) and makes the badge a link to the summary page for the CI workflow in the GitHub Actions web interface.

view details

Patrick Lühne

commit sha 4115fc1ea607273a1df0ca496dc5c68bd96af351

Clarify meaning of sensitive header values (#427) While the documentation mentions that header values can be marked as sensitive in order to inform outside components that these header values may require special treatment, it was unclear to me whether doing that affects the behavior of this crate. This adds a short note to the documentation to clarify that the main purpose of the sensitivity flag is to make components building on this crate to be aware of sensitive data, such that it can be treated with special care for security purposes.

view details

Steven Bosnick

commit sha 975dbdd70a882fbf74a77a5b90ad190602bb98ac

Audit the use of unsafe in byte_str.rs (#408) Add comments describing the invariant that makes the one use of unsafe sound.

view details

push time in 2 months

push eventacfoltzer/hyper

Sean McArthur

commit sha cb3f39c2dc6340060f6b17f354f04c872a947574

feat(lib): update Tokio, bytes, http, h2, and http-body

view details

Sean McArthur

commit sha 6ae5889f8378b6454d4dc620f33bd1678d0e00e4

feat(rt): introduce `rt::Executor` trait The `hyper::rt::Executor` trait allows defining custom executors to be used with hyper's `Client` and `Server`. Closes #1944 BREAKING CHANGE: Any type passed to the `executor` builder methods must now implement `hyper::rt::Executor`. `hyper::rt::spawn` usage should be replaced with `tokio::task::spawn`. `hyper::rt::run` usage should be replaced with `#[tokio::main]` or managing a `tokio::runtime::Runtime` manually.

view details

Adam C. Foltzer

commit sha 3cc93e796aad59b3996fc26b8839a783e0307925

feat(server): give `Server::local_addr` a more general type Allows `local_addr` to work for any executor type, rather than just the default `Exec`. The underlying `SpawnAll::local_addr()` is already similarly general, so no other changes are needed other than adding the extra type parameter to the `impl`.

view details

Sean McArthur

commit sha c63728eb38182ad2f93edd729dbf50f3d5c40479

feat(body): replace the `Payload` trait with `HttpBody` The `hyper::body::HttpBody` trait is a re-export from the `http-body` crate. This allows libraries to accept "HTTP bodies" without needing to depend on hyper. BREAKING CHANGE: All usage of `hyper::body::Payload` should be replaced with `hyper::body::HttpBody`.

view details

David Barsky

commit sha 19a7aab51f4b70ef1561ecd912c988d9ddfc7532

chore(dependencies): update tower-service to 0.3

view details

Markus Westerlind

commit sha 30ac01c1806236aef443a5ff9e119955941a188c

refactor(client): use async/await in HttpConnector (#2019) Closes #1984

view details

Sean McArthur

commit sha 9645a125e6be71bd09b53b1d8fd276f83e3c37a9

refactor(client): return HttpConnecting newtype

view details

Sean McArthur

commit sha 319e8aee1571d8d3639b3259e7a1edb964e6a26c

feat(client): remove `Destination` for `http::Uri` in connectors BREAKING CHANGE: All usage of `hyper::client::connect::Destination` should be replaced with `http::Uri`.

view details

Sean McArthur

commit sha 4d7a2266b88b2c5c92231bcd2bd75d5842198add

feat(client): change connectors to return an `impl Connection` Instead of returning a tuple `(impl AsyncRead + AsyncWrite, Connected)`, this adds a new trait, `hyper::client::connect::Connection`, which allows querying the connection type for a `Connected`. BREAKING CHANGE: Connectors no longer return a tuple of `(T, Connected)`, but a single `T: Connection`.

view details

Sean McArthur

commit sha a738d03fd3fa4bc57360014914105be72ed01069

chore(dependencies): update to http-body 0.3

view details

Sean McArthur

commit sha aa66de4f27748cad84efe8113744f607e7b6b79a

refactor(h1): un-split record_header_indicies for ARM

view details

Sean McArthur

commit sha 0b03b730531654b1b5f632099386ab27c94eb9f4

feat(lib): rename `unstable-stream` feature to `stream` and enable by default Closes #2034

view details

David Barsky

commit sha edbd10ac96c5cc6dbeca80ada80f143dbd13d118

feat(client): impl tower_service::Service for Client

view details

Sean McArthur

commit sha b0060f277e9851b4d0bcda4eb91f5bc4e36ef0b7

chore(CI): enable GitHub Actions

view details

Sean McArthur

commit sha 0dc89680cd386e17e581efb8608c07774eeee67d

style(lib): run rustfmt and enforce in CI

view details

Sean McArthur

commit sha a1e87c7fe6bf7d8f5aa17e5458621321744c0680

chore(README): update CI badge

view details

Sean McArthur

commit sha c56ccfb03366036dcc48d037e4c212d1f0fc7eb9

chore(ci): remove .appveyor.yml file

view details

Sean McArthur

commit sha 5a59875742500672f253719c1e1a16b4eddfacc7

feat(body): replace `Chunk` type with `Bytes` Closes #1931 BREAKING CHANGE: All usage of `hyper::Chunk` should be replaced with `bytes::Bytes` (or `hyper::body::Bytes`).

view details

Sean McArthur

commit sha 8ba9a8d2c4bab0f44b3f94a326b3b91c82d7877e

feat(body): add `body::aggregate` and `body::to_bytes` functions Adds utility functions to `hyper::body` to help asynchronously collecting all the buffers of some `HttpBody` into one. - `aggregate` will collect all into an `impl Buf` without copying the contents. This is ideal if you don't need a contiguous buffer. - `to_bytes` will copy all the data into a single contiguous `Bytes` buffer.

view details

Sean McArthur

commit sha 71101e701f65d5fc01fb80309c8a453aea1f4416

docs(client): show how to implement a Connector

view details

push time in 2 months

push eventacfoltzer/rust

Oliver Scherer

commit sha ee8dd4e3cce3bf158a31e50205ee3ebd76c40370

Fix const prop ICE we used to erase the local just before we tried to read it for diagnostics

view details

Yuki Okushi

commit sha b93ecc1dacd3d534b0fa24e4b14cd3b07bae9889

Address code reviews

view details

Nadrieril

commit sha 3cb31b6699558737b1a4650537f0facdc8cb7852

Fix #71977

view details

Eric Huss

commit sha 310c97b6ba7e6b8d4e3e7f337db0cff97f45f5c0

Fix caching issue when building tools.

view details

Ivan Tham

commit sha bc0d619325ccca85f804ab7f24752736748482d9

Fix spacing in Iterator fold doc

view details

Tomasz Miąsko

commit sha bcef848a6971217d4b8df50c17e047174018d316

Explain effects of debugging options from config.toml Co-authored-by: Teymour Aldridge <42674621+teymour-aldridge@users.noreply.github.com>

view details

Bastian Kauschke

commit sha 016e9f81573859a4601b645fd1178fdda95b2e73

expected found `&T` -> `T`

view details

Ralf Jung

commit sha 54d95ed25aa45f94b2a3d0a0e3a3323852878ecd

catch InvalidUninitBytes during validation

view details

Ralf Jung

commit sha c3fc4f0420b642a0e0041bb13492df2d4a68ae80

catch errors more locally around read_discriminant

view details

Ralf Jung

commit sha 751b594cc843a28c660a662db1a822a759af9603

const validation: add test for uninit bool

view details

Mark Rousskov

commit sha aae1215f7faa3aac5ada7e82585b86f1282cd89e

Shrink ParamEnv to 16 bytes

view details

Mark Rousskov

commit sha 8512d2efdef2c41aa529f44941f4c1f0e5fdd7de

Avoid deconstructing pointer for hashing

view details

Mark Rousskov

commit sha 3503247c11d80f0f17a36083878e6df114004d10

Shrink ParamEnv to 16 bytes

view details

Ralf Jung

commit sha 319c7f77dedf671b01194adb257dbe662c73df32

fmt

view details

Ralf Jung

commit sha 9cb1ffdd4fc372b18258b1bb12c55fd3c53d33f4

variant_count: avoid incorrect dummy implementation

view details

Ralf Jung

commit sha c478b5473d6623622d318d058477f5f09e2eeb52

add as_ptr method to raw slices

view details

Ralf Jung

commit sha 3b1d5e6d792fb47c9a95c4ea210ce88174f18b13

call the mut version as_mut_ptr and also add an as_ptr-like method to NonNull slices

view details

Ralf Jung

commit sha 90580c7b0e51050c892613664db79493df7a2af5

make unchecked slice indexing helper methods use raw pointers

view details

Ralf Jung

commit sha 5f5c98bd8a27b5313d0b1a5830076139c562b6b6

add (unchecked) indexing methods to raw pointers and NonNull

view details

Thom Chiovoloni

commit sha 980d8e1a0b18e89129cce23bb5a46c6a498dc5d2

Optimize is_ascii for &str and &[u8]

view details

push time in 2 months

startedsloria/dotfiles

started time in 2 months

issue openedtime-rs/time

OffsetDateTime does not correctly roundtrip through serde

It appears the offset gets dropped when serializing OffsetDateTime values. Given the following program:

fn main() {
    let now = time::OffsetDateTime::now_local();
    println!("now:\t{}", now);
    let serialized = serde_json::to_vec(&now).unwrap();
    let now_rt: time::OffsetDateTime = serde_json::from_slice(&serialized).unwrap();
    println!("now_rt:\t{}", now_rt);
}

I observe the output:

now:    2020-07-18 18:42:07.751662941 -7
now_rt: 2020-07-19 1:42:07.751662941 +0

I'm able to reproduce this both with the 0.2.16 release as well as a checkout of c49cca20.

I hope this is helpful for y'all! Thank you.

created time in 2 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

+# Hostcalls++Hostcalls are how Lucet guests interact with the world outside the WebAssembly+VM. For example, all functions in [WASI](https://github.com/bytecodealliance/wasmtime/blob/main/docs/WASI-intro.md) are implemented in terms of+hostcalls that can be exposed to a WebAssembly guest. This chapter discusses+implementation details of hostcalls as Lucet implements them.++Lucet implements hostcalls as imports of symbols specified by the+`bindings.json` provided to `lucetc`. This maps namespaced functions provided+to the WebAssembly module to symbol names the module should import when loaded.+Functionally, `lucet-runtime` currently relies on the dynamic linker to be able+to locate and fix up refereneces to these imported functions, and will fail to+load a module if the dynamic linker can't resolve all imports.++Hostcalls have an important intersection with safety properties Lucet seeks to+uphold: if a fault occurs in a WebAssembly guest, it should be isolated to that+guest, and the host should, generally, be able to continue execution. However,+a fault in a hostcall is a fault _outside_ the WebAssembly guest, back in+whatever code the host running `lucet-runtime` has provided. A fault here is+well outside any guarantees WebAssembly can offer, and the only sound option+Lucet has is to raise that issue in the host, if it has the option, and hope+the host knows what to do with it.++## Stack overflows++Generally, "kick the problem to the host and hope they know what to do" works.+For a general memory fault in host code, that gets handled no differently.+Language-specific features, like Rust panics, work too; if a `ud2` is present+in the hostcall's body, the `SIGILL` still causes the same kind of panic - not+a recoverable issue, but Lucet doesn't cause new problems here.++Unfortunately, memory faults like `SIGBUS` and `SIGSEGV` are typically fatal to+host applications. Memory faults in unpredictable locations even moreso. A+naive hostcall implementation scheme by calling import functions raises a real+risk here: if a WebAssembly guest consumes most, but not all, of the Lucet+guest's stack, _then_ makes a hostcall, the hostcall may consume the rest of+the guest's stack space and experience a stack overflow.++To mitigate the risk of an unknown WebAssembly guest being able to cause host+faults essentially on-demand, Lucet guards hostcalls by a trampoline function+that performs safety checks before actually making the call into host code.+Currently, there is one check: is there enough guest stack space remaining to+uphold some guaranteed amount available for hostcalls?++By guaranteeing some minimum available space, the problem of hostcall stack use+becomes the same as not overflowing stacks generally; if a host expects to+handle stack overflows in some manner, it probably still can, and if it allows+the system to do what it will on stack overflows, a hostcall overflowing the+guaranteed space will still observe a normal stack overflow. Hostcalls must+conform to the same requirements code would have without Lucet inolved, except+that the Lucet hostcall stack reservation may be more or less than the system's+configured thread size.++The good news is that while the hostcall stack reservation is a fixed size, it+is customizable: the field+[hostcall_reservation](https://docs.rs/lucet-runtime/0.7.0/lucet_runtime/struct.Limits.html#structfield.heap_memory_size)+in `Limits` specifies the space Lucet will require to be available, with a+default of 32KiB. Lucet requires that `hostcall_reservation` is between zero+and the guest's entire stack. Finally, a `hostcall_reservation` equal to the+entire guest stack size is allowed, and a de facto denial of hostcalls to the+guest - a Lucet guest will always have some stack space reserved for the+runtime-required backstop, so the availability check would always fail.++In terms of implementation, hostcall checks are done in entirely synthetic+functions generated by lucetc, prefixed with `trampoline_`. The trampoline+functions themselves are very simple, and have a shape like the following+Cranelift IR:+```+; A trampoline has the same signature as its hostcall - args plus a vmctx.+function %trampoline_$HOSTCALL($HOSTCALL_ARGS.., i64 vmctx) -> $HOSTCALL_RESULT {+    gv_vmctx = vmctx+    heap0 = static gv0++block0($HOSTCALL_ARGS.., vmctx: i64):+  ; The stack limit is recorded as part of the instance, just before the start of the guest heap.+  stack_limit_addr = heap_addr.i64 heap0, gv_vmctx, -$STACK_LIMIT_OFFSET+  stack_limit = load.i64 stack_limit_addr+  ; Compare the current stack pointer to stack_limit.+  ; The stack pointer is the LHS of this comparison, `stack_limit` the RHS.+  stack_cmp = ifcmp_sp stack_limit+  ; If the limit is greater than or equal to the stack pointer, there is+  ; insufficient space for the hostcall. Branch to the fail block and trap.+  ;+  ; "greater than or equal" may be surprising - it might be more natural to+  ; consider this comparison with arguments reversed; if the stack pointer is+  ; less than the limit, there is insufficient space.+  ;+  ; Even phrased like this, "less than" may be surprising, but is correct since+  ; the stack grows downward. Given an example layout:+  ; 0x0000: start of stack - start of the stack's allocation+  ; 0x2000: hostcall limit - reserve 0x2000 bytes+  ; 0x8000: base of stack - initial guest stack pointer+  ;+  ; and knowledge that the stack grows downward, the space from 0x2000 to+  ; 0x0000 is the reserved space, and a stack pointer below 0x2000 is in the+  ; reserved area, and thus voids the guarantee of reserved space by Lucet.+  brif ugte stack_cmp, stack_check_fail+  jump do_hostcall($HOSTCALL_ARGS.., vmctx)++do_hostcall($HOSTCALL_ARGS.., vmctx: i64):+  $HOSTCALL_RESULT.. = call $HOSTCALL($HOSTCALL_ARGS.., vmctx)+  return $HOSTCALL_RESULT++stack_check_fail():+  ; If the stack check fails, it's raised as a stack overflow in guest code.+  ; This is reasonably close to the actual occurrance, and allows the guest to be+  ; unwound.+  trap stk_ovf+```++### Lucet implementation considerations++Why do this trampoline and stack usage test, instead of observing failures in+guest stack guards? The issue here is twofold: first, we can't robustly+distinguish a stack overflow from an unlucky errant memory access for other+reasons. If hosts are built using stack probes, stack overflows will probably+be observed in places we expect, with patterns we can expect. But this is by no+means a guarantee, and stack accesses might not be through the stack pointer+directly (perhaps the address is loaded into an alternate register, and a fault+occurs without referencing the stack pointer at all). Second, if a hostcall+fault were to be recoverd by Lucet, the runtime may have to unwind a guest that
fault were to be recovered by Lucet, the runtime may have to unwind a guest that
iximeow

comment created time in 3 months

delete branch bytecodealliance/lucet

delete branch : acf/smaller-caches

delete time in 3 months

PR merged bytecodealliance/lucet

📉 Reduce CircleCI cache sizes by keying on `rust-toolchain`

This should prevent the cache of the /target directory from accruing complete sets of artifacts from multiple rustc versions over time.

This patch also fixes the use of {{ .Environment.CACHE_VERSION }} so that it applies to Linux, and so that it forms proper prefixes in the macOS cache keys.

+16 -10

1 comment

2 changed files

acfoltzer

pr closed time in 3 months

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 388f47d8c073233c076c609de61024d1fb53ee94

📉 Reduce CircleCI cache sizes by keying on `rust-toolchain` This should prevent the cache of the `/target` directory from accruing complete sets of artifacts from multiple `rustc` versions over time. This patch also fixes the use of `{{ .Environment.CACHE_VERSION }}` so that it applies to Linux, and so that it forms proper prefixes in the macOS cache keys.

view details

Adam C. Foltzer

commit sha 08dc00df0f43d01096174b04bc057900f425220b

build instead of check extra packages to speed up CI `cargo check` requires going through all the dependencies again, but `cargo build` should already have most of these built.

view details

Adam C. Foltzer

commit sha 3b47c388dd736e9d032850c9bb59c593ce92e313

Merge pull request #568 from bytecodealliance/acf/smaller-caches 📉 Reduce CircleCI cache sizes by keying on `rust-toolchain`

view details

push time in 3 months

pull request commentbytecodealliance/lucet

📉 Reduce CircleCI cache sizes by keying on `rust-toolchain`

We still will have some issues with cruft accruing as Cargo.lock dependencies evolve while we are pinned to a particular rustc version. Hopefully we'll be able to continue updating the rustc version regularly enough that it won't get too bad before we wipe the slate clean for the new version.

acfoltzer

comment created time in 3 months

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 08dc00df0f43d01096174b04bc057900f425220b

build instead of check extra packages to speed up CI `cargo check` requires going through all the dependencies again, but `cargo build` should already have most of these built.

view details

push time in 3 months

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 2cf91c484a0111ed7bf5ec12954535e1302efe5f

build instead of check extra packages to speed up CI `cargo check` requires going through all the dependencies again, but `cargo build` should already have most of these built.

view details

push time in 3 months

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha 388f47d8c073233c076c609de61024d1fb53ee94

📉 Reduce CircleCI cache sizes by keying on `rust-toolchain` This should prevent the cache of the `/target` directory from accruing complete sets of artifacts from multiple `rustc` versions over time. This patch also fixes the use of `{{ .Environment.CACHE_VERSION }}` so that it applies to Linux, and so that it forms proper prefixes in the macOS cache keys.

view details

push time in 3 months

PR opened bytecodealliance/lucet

📉 Reduce CircleCI cache sizes by keying on `rust-toolchain`

This should prevent the cache of the /target directory from accruing complete sets of artifacts from multiple rustc versions over time.

This patch also fixes the use of {{ .Environment.CACHE_VERSION }} so that it applies to Linux, and so that it forms proper prefixes in the macOS cache keys.

+14 -8

0 comment

1 changed file

pr created time in 3 months

create barnchbytecodealliance/lucet

branch : acf/smaller-caches

created branch time in 3 months

pull request commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

Also, if you're up for it, adding a page describing this to /docs/src would be greatly appreciated. There is hopefully material that exists already about the high-level design that you can adapt, but also a description of the low-level implementation via Cranelift IR would be :100:

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl<'a> FuncEnvironment for FuncInfo<'a> {             .expect("function indices are valid");         let func_decl = self.module_decls.get_func(unique_index).unwrap();         let signature = func.import_signature(func_decl.signature.clone());++        // if we're setting up a function ref for a call to an imported function, we'll need a+        // trampoline to check the stack first. in that case, return the trampoline function+        // instead.         let colocated = !func_decl.imported();+        let name = if colocated {

!colocated here means the FunctionDecl has an import name, so I think our insertion of the trampoline is correct. What might not be correct is that we're assuming everything without an import name is colocated per Cranelift's definition. Based on your description, that might not be true if the module becomes too large?

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl<'a> FuncInfo<'a> {     } } +/// Get the local trampoline function to do safety checks before calling an imported hostcall.+fn get_trampoline_func(+    trampolines: &mut HashMap<String, (FuncId, UniqueFuncIndex)>,+    clif_module: &mut ClifModule<ObjectBackend>,+    hostcall_index: UniqueFuncIndex,+    func_decl: &FunctionDecl,+    signature: &ir::Signature,+) -> Result<ir::ExternalName, ClifModuleError> {+    // decls.declare_new_function(clif_module, name, linkage, 0+    use std::collections::hash_map::Entry;+    let funcid = match trampolines.entry(func_decl.name.symbol().to_string()) {+        Entry::Occupied(o) => o.get().0,+        Entry::Vacant(v) => {+            let trampoline_name = format!("trampoline_{}", func_decl.name.symbol());+            println!("declaring trampoline {}", trampoline_name);++            // decls.declare_function(clif_module, trampoline_name, Linkage::Local)

Likewise

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl<'a> FuncInfo<'a> {     } } +/// Get the local trampoline function to do safety checks before calling an imported hostcall.+fn get_trampoline_func(+    trampolines: &mut HashMap<String, (FuncId, UniqueFuncIndex)>,+    clif_module: &mut ClifModule<ObjectBackend>,+    hostcall_index: UniqueFuncIndex,+    func_decl: &FunctionDecl,+    signature: &ir::Signature,+) -> Result<ir::ExternalName, ClifModuleError> {+    // decls.declare_new_function(clif_module, name, linkage, 0

Outdated?

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl<'a> Compiler<'a> {     } } +// Hostcall trampolines have the general shape of:+//+// ```+// fn trampoline_$hostcall(&vmctx, $hostcall_args) -> $hostcall_result {+//     if context.rsp < vmctx.instance_implicits.stack_limit {+//         // insufficient stack space to make the call+//         terminate_with_stack_overflow();+//     }+//+//     $hostcall(vmctx, $hostcall_args..)+// }+// ```+//+// but are specified here as Cranelift IR for lack of source to generate them from.+fn synthesize_trampoline(+    decls: &mut ModuleDecls,+    clif_module: &mut ClifModule<ObjectBackend>,+    function_map: &mut HashMap<FuncId, (u32, DataId, usize)>,+    hostcall_name: &str,+    trampoline_id: FuncId,+    hostcall_func_index: UniqueFuncIndex,+) -> Result<(), Error> {+    let mut trampoline_context = ClifContext::new();+    trampoline_context.func.name = ir::ExternalName::from(trampoline_id);+    // the trampoline's signature is the same as the hostcall it calls' signature+    let hostcall_sig = decls.info.signature_for_function(hostcall_func_index);+    trampoline_context.func.signature = hostcall_sig.clone();++    // We're going to load the stack limit later, create the global value to load while we+    // can.+    let vmctx = trampoline_context+        .func+        .create_global_value(ir::GlobalValueData::VMContext);+    let stack_limit_gv = trampoline_context+        .func+        .create_global_value(ir::GlobalValueData::Load {+            base: vmctx,+            offset: (-(std::mem::size_of::<InstanceRuntimeData>() as i32)+                + (offset_of!(InstanceRuntimeData, stack_limit) as i32))+                .into(),+            global_type: ir::types::I64,+            readonly: false,+        });++    let mut builder_ctx = FunctionBuilderContext::new();+    let mut builder = FunctionBuilder::new(&mut trampoline_context.func, &mut builder_ctx);++    let entry = builder.create_block();+    let hostcall_block = builder.create_block();+    builder.append_block_params_for_function_params(entry);+    // The hostcall block will end up having all the same arguments as the trampoline,+    // which itself matches the signature of the hostcall to be called.+    builder.append_block_params_for_function_params(hostcall_block);+    let trampoline_args = builder.block_params(entry).to_vec();++    let hostcall_decl = decls+        .get_func(hostcall_func_index)+        .expect("hostcall has been declared");+    let hostcall_sig_ref = builder.import_signature(hostcall_decl.signature.clone());+    let hostcall_ref = builder.import_function(ir::ExtFuncData {+        name: hostcall_decl.name.into(),+        signature: hostcall_sig_ref,+        colocated: false,+    });++    // Reserve a block for handling a stack check fail.+    let stack_check_fail = builder.create_block();++    // And start building the trampoline from entry.+    builder.switch_to_block(entry);++    let stack_limit = builder.ins().global_value(ir::types::I64, stack_limit_gv);+    let sp_cmp = builder.ins().ifcmp_sp(stack_limit);++    builder.ins().brif(+        ir::condcodes::IntCC::UnsignedGreaterThanOrEqual,+        sp_cmp,+        stack_check_fail,+        &[],+    );+    builder.ins().fallthrough(hostcall_block, &trampoline_args);++    builder.switch_to_block(hostcall_block);+    let hostcall_args = builder.block_params(hostcall_block).to_vec();+    let call_inst = builder.ins().call(hostcall_ref, &hostcall_args);+    let results = builder.inst_results(call_inst).to_vec();+    builder.ins().return_(&results);

to_vec()

Dangit, borrow checker!

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 fn main() {         .value_of("timeout")         .map(|t| Duration::from_millis(t.parse::<u64>().unwrap())); -    let limits = Limits {-        heap_memory_size,-        heap_address_space_size,-        stack_size,-        globals_size: 0, // calculated from module-        ..Limits::default()-    };+    let limits = Limits::default()+        .with_heap_memory_size(heap_memory_size)+        .with_heap_address_space_size(heap_address_space_size)+        .with_stack_size(stack_size)

I agree that's fine, but we should add a note to that effect in the help text for --stack-size

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl<'a> Compiler<'a> {     } } +// Hostcall trampolines have the general shape of:+//+// ```+// fn trampoline_$hostcall(&vmctx, $hostcall_args) -> $hostcall_result {+//     if context.rsp < vmctx.instance_implicits.stack_limit {+//         // insufficient stack space to make the call+//         terminate_with_stack_overflow();+//     }+//+//     $hostcall(vmctx, $hostcall_args..)+// }+// ```+//+// but are specified here as Cranelift IR for lack of source to generate them from.+fn synthesize_trampoline(+    decls: &mut ModuleDecls,+    clif_module: &mut ClifModule<ObjectBackend>,+    function_map: &mut HashMap<FuncId, (u32, DataId, usize)>,+    hostcall_name: &str,+    trampoline_id: FuncId,+    hostcall_func_index: UniqueFuncIndex,+) -> Result<(), Error> {+    let mut trampoline_context = ClifContext::new();+    trampoline_context.func.name = ir::ExternalName::from(trampoline_id);+    // the trampoline's signature is the same as the hostcall it calls' signature+    let hostcall_sig = decls.info.signature_for_function(hostcall_func_index);+    trampoline_context.func.signature = hostcall_sig.clone();++    // We're going to load the stack limit later, create the global value to load while we+    // can.+    let vmctx = trampoline_context+        .func+        .create_global_value(ir::GlobalValueData::VMContext);+    let stack_limit_gv = trampoline_context+        .func+        .create_global_value(ir::GlobalValueData::Load {+            base: vmctx,+            offset: (-(std::mem::size_of::<InstanceRuntimeData>() as i32)+                + (offset_of!(InstanceRuntimeData, stack_limit) as i32))+                .into(),+            global_type: ir::types::I64,+            readonly: false,

Why false here? Are we ever writing to the stack limit?

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl From<lucet_alloc_limits> for Limits {  impl From<&lucet_alloc_limits> for Limits {     fn from(limits: &lucet_alloc_limits) -> Limits {-        Limits {-            heap_memory_size: limits.heap_memory_size as usize,-            heap_address_space_size: limits.heap_address_space_size as usize,-            stack_size: limits.stack_size as usize,-            globals_size: limits.globals_size as usize,-            signal_stack_size: limits.signal_stack_size as usize,-        }+        Limits::default()+            .with_heap_memory_size(limits.heap_memory_size as usize)+            .with_heap_address_space_size(limits.heap_address_space_size as usize)+            .with_stack_size(limits.stack_size as usize)+            .expect("lucet_alloc_limts has a valid stack size")

This should be resolved by moving the check to validate

iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl Limits {             heap_memory_size: 16 * 64 * 1024,             heap_address_space_size: 0x0002_0000_0000,             stack_size: 128 * 1024,+            hostcall_reservation: 32 * 1024,             globals_size: 4096,             signal_stack_size: DEFAULT_SIGNAL_STACK_SIZE,+            _hidden: (),         }     }+    pub fn stack_size(&self) -> usize {+        self.stack_size+    }+    pub fn hostcall_reservation(&self) -> usize {+        self.hostcall_reservation+    }+    pub fn with_heap_memory_size(mut self, heap_memory_size: usize) -> Self {+        self.heap_memory_size = heap_memory_size;+        self+    }+    pub fn with_heap_address_space_size(mut self, heap_address_space_size: usize) -> Self {+        self.heap_address_space_size = heap_address_space_size;+        self+    }+    pub fn with_stack_size(mut self, stack_size: usize) -> Result<Self, Error> {+        if stack_size <= self.hostcall_reservation {+            return Err(Error::InvalidArgument(+                "stack size must be greater than hostcall reserved space",+            ));+        }++        self.stack_size = stack_size;+        Ok(self)+    }+    pub fn with_hostcall_reservation(mut self, hostcall_reservation: usize) -> Result<Self, Error> {+        // We allow `hostcall_reservation == self.stack_size`, a circumstance that guarantees+        // any hostcalls will fail with a StackOverflow.+        if hostcall_reservation > self.stack_size {+            return Err(Error::InvalidArgument(+                "hostcall reserved space must be less than stack size",
                "hostcall reserved space must be less than or equal to stack size",
iximeow

comment created time in 3 months

Pull request review commentbytecodealliance/lucet

Ensure a requested amount of stack is avaiable before hostcalls

 impl Limits {             heap_memory_size: 16 * 64 * 1024,             heap_address_space_size: 0x0002_0000_0000,             stack_size: 128 * 1024,+            hostcall_reservation: 32 * 1024,             globals_size: 4096,             signal_stack_size: DEFAULT_SIGNAL_STACK_SIZE,+            _hidden: (),         }     }+    pub fn stack_size(&self) -> usize {+        self.stack_size+    }+    pub fn hostcall_reservation(&self) -> usize {+        self.hostcall_reservation+    }+    pub fn with_heap_memory_size(mut self, heap_memory_size: usize) -> Self {+        self.heap_memory_size = heap_memory_size;+        self+    }+    pub fn with_heap_address_space_size(mut self, heap_address_space_size: usize) -> Self {+        self.heap_address_space_size = heap_address_space_size;+        self+    }+    pub fn with_stack_size(mut self, stack_size: usize) -> Result<Self, Error> {+        if stack_size <= self.hostcall_reservation {+            return Err(Error::InvalidArgument(+                "stack size must be greater than hostcall reserved space",+            ));

As discussed in Slack, we should be able to do these checks in Limits::validate() rather than when constructing limits. However, I really like having these interfaces!

Would you mind moving the check, and then removing _hidden and making the fields public once more?

As a bonus, with the checks gone these builder methods should all be const fn-able.

iximeow

comment created time in 3 months

push eventbytecodealliance/wasmtime

Adam C. Foltzer

commit sha 5a96b0deaac09a6d1dce3391a4de689d75147c71

🕳 Add virtual pipes to wasi-common This introduces `Handle` implementations for readable and writable pipes, backed by arbitrary `Read` and `Write` types, respectively. In particular, this allows for easily providing, capturing, or redirecting WASI stdio without having to resort to OS-provided file descriptors. The implementation is based heavily on `wasi_common::virtfs::InMemoryFile`, but without inapplicable operations like `seek` or `allocate`. Note that these types are not 1:1 replacements for real pipes, because they do not support `poll_oneoff`.

view details

Adam C. Foltzer

commit sha fddd94d23f5a3712ae6afbb9176fe8b32bcc5793

address review comments

view details

Adam C. Foltzer

commit sha 4f16f0dc322ce456a752e42ba8e6297b9013ef5b

Merge pull request #1949 from bytecodealliance/acf/virtual-pipes 🕳 Add virtual pipes to wasi-common

view details

push time in 3 months

delete branch bytecodealliance/wasmtime

delete branch : acf/virtual-pipes

delete time in 3 months

PR merged bytecodealliance/wasmtime

🕳 Add virtual pipes to wasi-common wasi wasi:impl

This introduces Handle implementations for readable and writable pipes, backed by arbitrary Read and Write types, respectively. In particular, this allows for easily providing, capturing, or redirecting WASI stdio without having to resort to OS-provided file descriptors.

The implementation is based heavily on wasi_common::virtfs::InMemoryFile, but without inapplicable operations like seek or allocate.

Note that these types are not 1:1 replacements for real pipes, because they do not support poll_oneoff.

+410 -1

2 comments

3 changed files

acfoltzer

pr closed time in 3 months

Pull request review commentbytecodealliance/wasmtime

🕳 Add virtual pipes to wasi-common

+//! Virtual pipes.+//!+//! These types provide easy implementations of `Handle` that mimic much of the behavior of Unix+//! pipes. These are particularly helpful for redirecting WASI stdio handles to destinations other+//! than OS files.+//!+//! Some convenience constructors are included for common backing types like `Vec<u8>` and `String`,+//! but the virtual pipes can be instantiated with any `Read` or `Write` type.+//!+//! Note that `poll_oneoff` is not supported for these types, so they do not match the behavior of+//! real pipes exactly.+use crate::handle::{Handle, HandleRights};+use crate::wasi::{types, Errno, Result};+use log::trace;+use std::any::Any;+use std::cell::{Cell, Ref, RefCell};+use std::io::{self, Read, Write};+use std::rc::Rc;++/// A virtual pipe read end.+///+/// A variety of `From` impls are provided so that common pipe types are easy to create. For example:+///+/// ```+/// # use wasi_common::WasiCtxBuilder;+/// # use wasi_common::virtfs::pipe::ReadPipe;+/// let mut ctx = WasiCtxBuilder::new();+/// let stdin = ReadPipe::from("hello from stdin!");+/// ctx.stdin(stdin);+/// ```+#[derive(Clone, Debug)]+pub struct ReadPipe<R: Read + Any> {+    rights: Cell<HandleRights>,+    reader: Rc<RefCell<R>>,+}++impl<R: Read + Any> ReadPipe<R> {+    /// Create a new pipe from a `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn new(r: R) -> Self {+        Self::from_shared(Rc::new(RefCell::new(r)))+    }++    /// Create a new pipe from a shareable `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn from_shared(reader: Rc<RefCell<R>>) -> Self {+        use types::Rights;+        Self {+            rights: Cell::new(HandleRights::new(+                Rights::FD_DATASYNC+                    | Rights::FD_FDSTAT_SET_FLAGS+                    | Rights::FD_READ+                    | Rights::FD_SYNC+                    | Rights::FD_FILESTAT_GET+                    | Rights::POLL_FD_READWRITE,+                Rights::empty(),+            )),+            reader,+        }+    }++    /// Try to convert this `ReadPipe<R>` back to the underlying `R` type.+    ///+    /// This will fail with `Err(self)` if multiple references to the underlying `R` exist.+    pub fn try_into_inner(mut self) -> std::result::Result<R, Self> {+        match Rc::try_unwrap(self.reader) {+            Ok(rc) => Ok(RefCell::into_inner(rc)),+            Err(reader) => {+                self.reader = reader;+                Err(self)+            }+        }+    }+}++impl From<Vec<u8>> for ReadPipe<io::Cursor<Vec<u8>>> {+    fn from(r: Vec<u8>) -> Self {+        Self::new(io::Cursor::new(r))+    }+}++impl From<&[u8]> for ReadPipe<io::Cursor<Vec<u8>>> {+    fn from(r: &[u8]) -> Self {+        Self::from(r.to_vec())+    }+}++impl From<String> for ReadPipe<io::Cursor<String>> {+    fn from(r: String) -> Self {+        Self::new(io::Cursor::new(r))+    }+}++impl From<&str> for ReadPipe<io::Cursor<String>> {+    fn from(r: &str) -> Self {+        Self::from(r.to_string())+    }+}++impl<R: Read + Any> Handle for ReadPipe<R> {+    fn as_any(&self) -> &dyn Any {+        self+    }++    fn try_clone(&self) -> io::Result<Box<dyn Handle>> {+        Ok(Box::new(Self {+            rights: self.rights.clone(),+            reader: self.reader.clone(),+        }))+    }++    fn get_file_type(&self) -> types::Filetype {+        types::Filetype::Unknown+    }++    fn get_rights(&self) -> HandleRights {+        self.rights.get()+    }++    fn set_rights(&self, rights: HandleRights) {+        self.rights.set(rights)+    }++    fn advise(+        &self,+        _advice: types::Advice,+        _offset: types::Filesize,+        _len: types::Filesize,+    ) -> Result<()> {+        Err(Errno::Spipe)+    }++    fn allocate(&self, _offset: types::Filesize, _len: types::Filesize) -> Result<()> {+        Err(Errno::Spipe)+    }++    fn fdstat_set_flags(&self, _fdflags: types::Fdflags) -> Result<()> {+        // do nothing for now

Sorry @sunfishcode, Github wasn't showing your comment in this thread when I left my reply. In my testing, O_APPEND seems to be ignored.

acfoltzer

comment created time in 3 months

push eventbytecodealliance/lucet

Adam C. Foltzer

commit sha c55fee3fe6d67d4606bb4f879191ea54a229477c

fix our cargo config so it doesn't mess up wasi-common's tests

view details

Adam C. Foltzer

commit sha 9a757581904681fcb5e0c2783aec5cf7dc3edcd9

use virtual pipes in lucet-wasi-fuzz

view details

push time in 3 months

push eventbytecodealliance/wasmtime

Adam C. Foltzer

commit sha fddd94d23f5a3712ae6afbb9176fe8b32bcc5793

address review comments

view details

push time in 3 months

Pull request review commentbytecodealliance/wasmtime

🕳 Add virtual pipes to wasi-common

+//! Virtual pipes.+//!+//! These types provide easy implementations of `Handle` that mimic much of the behavior of Unix+//! pipes. These are particularly helpful for redirecting WASI stdio handles to destinations other+//! than OS files.+//!+//! Some convenience constructors are included for common backing types like `Vec<u8>` and `String`,+//! but the virtual pipes can be instantiated with any `Read` or `Write` type.+//!+//! Note that `poll_oneoff` is not supported for these types, so they do not match the behavior of+//! real pipes exactly.+use crate::handle::{Handle, HandleRights};+use crate::wasi::{types, Errno, Result};+use log::trace;+use std::any::Any;+use std::cell::{Cell, Ref, RefCell};+use std::io::{self, Read, Write};+use std::rc::Rc;++/// A virtual pipe read end.+///+/// A variety of `From` impls are provided so that common pipe types are easy to create. For example:+///+/// ```+/// # use wasi_common::WasiCtxBuilder;+/// # use wasi_common::virtfs::pipe::ReadPipe;+/// let mut ctx = WasiCtxBuilder::new();+/// let stdin = ReadPipe::from("hello from stdin!");+/// ctx.stdin(stdin);+/// ```+#[derive(Clone, Debug)]+pub struct ReadPipe<R: Read + Any> {+    rights: Cell<HandleRights>,+    reader: Rc<RefCell<R>>,+}++impl<R: Read + Any> ReadPipe<R> {+    /// Create a new pipe from a `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn new(r: R) -> Self {+        Self::from_shared(Rc::new(RefCell::new(r)))+    }++    /// Create a new pipe from a shareable `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn from_shared(reader: Rc<RefCell<R>>) -> Self {+        use types::Rights;+        Self {+            rights: Cell::new(HandleRights::new(+                Rights::FD_DATASYNC+                    | Rights::FD_FDSTAT_SET_FLAGS+                    | Rights::FD_READ+                    | Rights::FD_SYNC+                    | Rights::FD_FILESTAT_GET+                    | Rights::POLL_FD_READWRITE,+                Rights::empty(),+            )),+            reader,+        }+    }++    /// Try to convert this `ReadPipe<R>` back to the underlying `R` type.+    ///+    /// This will fail with `Err(self)` if multiple references to the underlying `R` exist.+    pub fn try_into_inner(mut self) -> std::result::Result<R, Self> {+        match Rc::try_unwrap(self.reader) {+            Ok(rc) => Ok(RefCell::into_inner(rc)),+            Err(reader) => {+                self.reader = reader;+                Err(self)+            }+        }+    }+}++impl From<Vec<u8>> for ReadPipe<io::Cursor<Vec<u8>>> {+    fn from(r: Vec<u8>) -> Self {+        Self::new(io::Cursor::new(r))+    }+}++impl From<&[u8]> for ReadPipe<io::Cursor<Vec<u8>>> {+    fn from(r: &[u8]) -> Self {+        Self::from(r.to_vec())+    }+}++impl From<String> for ReadPipe<io::Cursor<String>> {+    fn from(r: String) -> Self {+        Self::new(io::Cursor::new(r))+    }+}++impl From<&str> for ReadPipe<io::Cursor<String>> {+    fn from(r: &str) -> Self {+        Self::from(r.to_string())+    }+}++impl<R: Read + Any> Handle for ReadPipe<R> {+    fn as_any(&self) -> &dyn Any {+        self+    }++    fn try_clone(&self) -> io::Result<Box<dyn Handle>> {+        Ok(Box::new(Self {+            rights: self.rights.clone(),+            reader: self.reader.clone(),+        }))+    }++    fn get_file_type(&self) -> types::Filetype {+        types::Filetype::Unknown+    }++    fn get_rights(&self) -> HandleRights {+        self.rights.get()+    }++    fn set_rights(&self, rights: HandleRights) {+        self.rights.set(rights)+    }++    fn advise(+        &self,+        _advice: types::Advice,+        _offset: types::Filesize,+        _len: types::Filesize,+    ) -> Result<()> {+        Err(Errno::Spipe)+    }++    fn allocate(&self, _offset: types::Filesize, _len: types::Filesize) -> Result<()> {+        Err(Errno::Spipe)+    }++    fn fdstat_set_flags(&self, _fdflags: types::Fdflags) -> Result<()> {+        // do nothing for now

Agreed that there should be more implementation guidance, but since I'm new to this corner of the codebase I'm not sure what to provide. Would you be okay with merging this for now with the expectation of doing that work later?

acfoltzer

comment created time in 3 months

Pull request review commentbytecodealliance/wasmtime

🕳 Add virtual pipes to wasi-common

+//! Virtual pipes.+//!+//! These types provide easy implementations of `Handle` that mimic much of the behavior of Unix+//! pipes. These are particularly helpful for redirecting WASI stdio handles to destinations other+//! than OS files.+//!+//! Some convenience constructors are included for common backing types like `Vec<u8>` and `String`,+//! but the virtual pipes can be instantiated with any `Read` or `Write` type.+//!+//! Note that `poll_oneoff` is not supported for these types, so they do not match the behavior of+//! real pipes exactly.+use crate::handle::{Handle, HandleRights};+use crate::wasi::{types, Errno, Result};+use log::trace;+use std::any::Any;+use std::cell::{Cell, Ref, RefCell};+use std::io::{self, Read, Write};+use std::rc::Rc;++/// A virtual pipe read end.+///+/// A variety of `From` impls are provided so that common pipe types are easy to create. For example:+///+/// ```+/// # use wasi_common::WasiCtxBuilder;+/// # use wasi_common::virtfs::pipe::ReadPipe;+/// let mut ctx = WasiCtxBuilder::new();+/// let stdin = ReadPipe::from("hello from stdin!");+/// ctx.stdin(stdin);+/// ```+#[derive(Clone, Debug)]+pub struct ReadPipe<R: Read + Any> {+    rights: Cell<HandleRights>,+    reader: Rc<RefCell<R>>,+}++impl<R: Read + Any> ReadPipe<R> {+    /// Create a new pipe from a `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn new(r: R) -> Self {+        Self::from_shared(Rc::new(RefCell::new(r)))+    }++    /// Create a new pipe from a shareable `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn from_shared(reader: Rc<RefCell<R>>) -> Self {+        use types::Rights;+        Self {+            rights: Cell::new(HandleRights::new(+                Rights::FD_DATASYNC+                    | Rights::FD_FDSTAT_SET_FLAGS+                    | Rights::FD_READ+                    | Rights::FD_SYNC+                    | Rights::FD_FILESTAT_GET+                    | Rights::POLL_FD_READWRITE,+                Rights::empty(),+            )),+            reader,+        }+    }++    /// Try to convert this `ReadPipe<R>` back to the underlying `R` type.+    ///+    /// This will fail with `Err(self)` if multiple references to the underlying `R` exist.+    pub fn try_into_inner(mut self) -> std::result::Result<R, Self> {+        match Rc::try_unwrap(self.reader) {+            Ok(rc) => Ok(RefCell::into_inner(rc)),+            Err(reader) => {+                self.reader = reader;+                Err(self)+            }+        }+    }+}++impl From<Vec<u8>> for ReadPipe<io::Cursor<Vec<u8>>> {+    fn from(r: Vec<u8>) -> Self {+        Self::new(io::Cursor::new(r))+    }+}++impl From<&[u8]> for ReadPipe<io::Cursor<Vec<u8>>> {+    fn from(r: &[u8]) -> Self {+        Self::from(r.to_vec())+    }+}++impl From<String> for ReadPipe<io::Cursor<String>> {+    fn from(r: String) -> Self {+        Self::new(io::Cursor::new(r))+    }+}++impl From<&str> for ReadPipe<io::Cursor<String>> {+    fn from(r: &str) -> Self {+        Self::from(r.to_string())+    }+}++impl<R: Read + Any> Handle for ReadPipe<R> {+    fn as_any(&self) -> &dyn Any {+        self+    }++    fn try_clone(&self) -> io::Result<Box<dyn Handle>> {+        Ok(Box::new(Self {+            rights: self.rights.clone(),+            reader: self.reader.clone(),+        }))+    }++    fn get_file_type(&self) -> types::Filetype {+        types::Filetype::Unknown+    }++    fn get_rights(&self) -> HandleRights {+        self.rights.get()+    }++    fn set_rights(&self, rights: HandleRights) {+        self.rights.set(rights)+    }++    fn advise(+        &self,+        _advice: types::Advice,+        _offset: types::Filesize,+        _len: types::Filesize,+    ) -> Result<()> {+        Err(Errno::Spipe)+    }++    fn allocate(&self, _offset: types::Filesize, _len: types::Filesize) -> Result<()> {+        Err(Errno::Spipe)+    }++    fn fdstat_set_flags(&self, _fdflags: types::Fdflags) -> Result<()> {+        // do nothing for now+        Ok(())+    }++    fn filestat_get(&self) -> Result<types::Filestat> {+        let stat = types::Filestat {+            dev: 0,+            ino: 0,+            nlink: 0,+            size: 0,+            atim: 0,+            ctim: 0,+            mtim: 0,+            filetype: self.get_file_type(),+        };+        Ok(stat)+    }++    fn filestat_set_size(&self, _st_size: types::Filesize) -> Result<()> {+        Err(Errno::Spipe)+    }++    fn preadv(&self, buf: &mut [io::IoSliceMut], offset: types::Filesize) -> Result<usize> {+        if offset != 0 {+            return Err(Errno::Spipe);+        }+        Ok(self.reader.borrow_mut().read_vectored(buf)?)+    }++    fn seek(&self, _offset: io::SeekFrom) -> Result<types::Filesize> {+        Err(Errno::Spipe)+    }++    fn read_vectored(&self, iovs: &mut [io::IoSliceMut]) -> Result<usize> {+        trace!("read_vectored(iovs={:?})", iovs);+        Ok(self.reader.borrow_mut().read_vectored(iovs)?)+    }++    fn create_directory(&self, _path: &str) -> Result<()> {+        Err(Errno::Notdir)+    }++    fn openat(+        &self,+        _path: &str,+        _read: bool,+        _write: bool,+        _oflags: types::Oflags,+        _fd_flags: types::Fdflags,+    ) -> Result<Box<dyn Handle>> {+        Err(Errno::Notdir)+    }++    fn link(+        &self,+        _old_path: &str,+        _new_handle: Box<dyn Handle>,+        _new_path: &str,+        _follow: bool,+    ) -> Result<()> {+        Err(Errno::Notdir)+    }++    fn readlink(&self, _path: &str, _buf: &mut [u8]) -> Result<usize> {+        Err(Errno::Notdir)+    }++    fn readlinkat(&self, _path: &str) -> Result<String> {+        Err(Errno::Notdir)+    }++    fn rename(&self, _old_path: &str, _new_handle: Box<dyn Handle>, _new_path: &str) -> Result<()> {+        Err(Errno::Notdir)+    }++    fn remove_directory(&self, _path: &str) -> Result<()> {+        Err(Errno::Notdir)+    }++    fn symlink(&self, _old_path: &str, _new_path: &str) -> Result<()> {+        Err(Errno::Notdir)+    }++    fn unlink_file(&self, _path: &str) -> Result<()> {+        Err(Errno::Notdir)+    }

Yeah, I was actually thinking about breaking up Handle, but since this part of the code is new to me I didn't want to stir things up too much. Are you alright with saving this for a future PR?

acfoltzer

comment created time in 3 months

Pull request review commentbytecodealliance/wasmtime

🕳 Add virtual pipes to wasi-common

+//! Virtual pipes.+//!+//! These types provide easy implementations of `Handle` that mimic much of the behavior of Unix+//! pipes. These are particularly helpful for redirecting WASI stdio handles to destinations other+//! than OS files.+//!+//! Some convenience constructors are included for common backing types like `Vec<u8>` and `String`,+//! but the virtual pipes can be instantiated with any `Read` or `Write` type.+//!+//! Note that `poll_oneoff` is not supported for these types, so they do not match the behavior of+//! real pipes exactly.+use crate::handle::{Handle, HandleRights};+use crate::wasi::{types, Errno, Result};+use log::trace;+use std::any::Any;+use std::cell::{Cell, Ref, RefCell};+use std::io::{self, Read, Write};+use std::rc::Rc;++/// A virtual pipe read end.+///+/// A variety of `From` impls are provided so that common pipe types are easy to create. For example:+///+/// ```+/// # use wasi_common::WasiCtxBuilder;+/// # use wasi_common::virtfs::pipe::ReadPipe;+/// let mut ctx = WasiCtxBuilder::new();+/// let stdin = ReadPipe::from("hello from stdin!");+/// ctx.stdin(stdin);+/// ```+#[derive(Clone, Debug)]+pub struct ReadPipe<R: Read + Any> {+    rights: Cell<HandleRights>,+    reader: Rc<RefCell<R>>,+}++impl<R: Read + Any> ReadPipe<R> {+    /// Create a new pipe from a `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn new(r: R) -> Self {+        Self::from_shared(Rc::new(RefCell::new(r)))+    }++    /// Create a new pipe from a shareable `Read` type.+    ///+    /// All `Handle` read operations delegate to reading from this underlying reader.+    pub fn from_shared(reader: Rc<RefCell<R>>) -> Self {+        use types::Rights;+        Self {+            rights: Cell::new(HandleRights::new(+                Rights::FD_DATASYNC+                    | Rights::FD_FDSTAT_SET_FLAGS+                    | Rights::FD_READ+                    | Rights::FD_SYNC+                    | Rights::FD_FILESTAT_GET+                    | Rights::POLL_FD_READWRITE,+                Rights::empty(),+            )),+            reader,+        }+    }++    /// Try to convert this `ReadPipe<R>` back to the underlying `R` type.+    ///+    /// This will fail with `Err(self)` if multiple references to the underlying `R` exist.+    pub fn try_into_inner(mut self) -> std::result::Result<R, Self> {+        match Rc::try_unwrap(self.reader) {+            Ok(rc) => Ok(RefCell::into_inner(rc)),+            Err(reader) => {+                self.reader = reader;+                Err(self)+            }+        }+    }+}++impl From<Vec<u8>> for ReadPipe<io::Cursor<Vec<u8>>> {+    fn from(r: Vec<u8>) -> Self {+        Self::new(io::Cursor::new(r))+    }+}++impl From<&[u8]> for ReadPipe<io::Cursor<Vec<u8>>> {

Sadly &'a [u8] is not Any due to the non-static lifetime, but Any is required to implement Handle :sob:

acfoltzer

comment created time in 3 months

PR opened bytecodealliance/wasmtime

🕳 Add virtual pipes to wasi-common

This introduces Handle implementations for readable and writable pipes, backed by arbitrary Read and Write types, respectively. In particular, this allows for easily providing, capturing, or redirecting WASI stdio without having to resort to OS-provided file descriptors.

The implementation is based heavily on wasi_common::virtfs::InMemoryFile, but without inapplicable operations like seek or allocate.

Note that these types are not 1:1 replacements for real pipes, because they do not support poll_oneoff.

+415 -1

0 comment

3 changed files

pr created time in 3 months

push eventbytecodealliance/wasmtime

Adam C. Foltzer

commit sha 5a96b0deaac09a6d1dce3391a4de689d75147c71

🕳 Add virtual pipes to wasi-common This introduces `Handle` implementations for readable and writable pipes, backed by arbitrary `Read` and `Write` types, respectively. In particular, this allows for easily providing, capturing, or redirecting WASI stdio without having to resort to OS-provided file descriptors. The implementation is based heavily on `wasi_common::virtfs::InMemoryFile`, but without inapplicable operations like `seek` or `allocate`. Note that these types are not 1:1 replacements for real pipes, because they do not support `poll_oneoff`.

view details

push time in 3 months

push eventbytecodealliance/wasmtime

Adam C. Foltzer

commit sha d6f070f230cb213d75b689e98cd861926453d8f4

🕳 Add virtual pipes to wasi-common This introduces `Handle` implementations for readable and writable pipes, backed by arbitrary `Read` and `Write` types, respectively. In particular, this allows for easily providing, capturing, or redirecting WASI stdio without having to resort to OS-provided file descriptors. The implementation is based heavily on `wasi_common::virtfs::InMemoryFile`, but without inapplicable operations like `seek` or `allocate`.

view details

push time in 3 months

PR opened bytecodealliance/lucet

Use draft new wasi-common virtual pipe interface

This uses a branch on wasmtime that's based on the wasmtime revision in this branch. I will be doing a PR with the same contents into wasmtime main once I write some docs for it.

+14 -24

0 comment

3 changed files

pr created time in 3 months

create barnchbytecodealliance/lucet

branch : acf/virtual-pipes

created branch time in 3 months

create barnchbytecodealliance/wasmtime

branch : acf/virtual-pipes-backport

created branch time in 3 months

create barnchbytecodealliance/wasmtime

branch : acf/virtual-pipes

created branch time in 3 months

more