profile
viewpoint
Josh Triplett joshtriplett https://joshtriplett.org Free and Open Source Software developer. @rust-lang developer and language team lead. he/him or they/them.

joshtriplett/async-pidfd 33

Rust crate to use process file descriptors (pidfd) for Linux

google/channel-id-enclave 28

Stores Chromium Channel ID private keys in an Intel SGX enclave.

joshtriplett/colorparse 1

Rust library to parse color configuration strings (in Git syntax) into ansi_term::Style objects

joshtriplett/anyhow 0

A better Box<dyn Error>

joshtriplett/augmented-ui 0

Cyberpunk-inspired web UI made easy. Get augmented.

joshtriplett/blog.rust-lang.org 0

The Rust Programming Language Blog

joshtriplett/cargo 0

The Rust package manager

joshtriplett/cargo-edit 0

A utility for managing cargo dependencies from the command line.

joshtriplett/cbor 0

CBOR support for serde.

joshtriplett/cbor_event 0

cbor library in rust

issue openedasync-rs/parallel-stream

Please document how to handle errors

With standard iterators, it's possible to call map with a function that returns a Result<T, E>, then collect into a Result<Vec<T>, E>. I'd like to do something similar with a ParallelStream, but it looks like the collect method only supports collecting into a Vec, not a Result.

Would you consider providing some documentation on how to handle errors that occur in one of the parallel function calls?

created time in 18 hours

issue openedsystemd/systemd

boot time (as reported by systemd-analyze) charges systemd startup time to the kernel

systemd version the issue has been seen with

systemd 246 (246.6-1)

Used distribution

Latest Debian sid

The boot time statistics, as reported by systemd-analyze, treat the time that systemd is up-and-running as the time where the kernel is done booting. However, this can vary substantially, if systemd takes a while to initialize, such as if reading it and its dependencies from disk takes time. For instance:

...
[    0.060286] Run /sbin/init as init process
[    0.060286]   with arguments:
[    0.060287]     /sbin/init
[    0.060287]   with environment:
[    0.060287]     HOME=/
[    0.060288]     TERM=linux
[    0.060288]     BOOT_IMAGE=/vmlinuz
[    1.059489] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2408f24b9db, max_idle_ns: 440795321801 ns
[    1.059509] clocksource: Switched to clocksource tsc
[    3.068643] systemd[1]: systemd 246.6-1 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +ZSTD +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=hybrid)
[    3.068658] systemd[1]: Detected virtualization kvm.
...

In this case, the kernel booted in 60ms and started running /sbin/init. But systemd-analyze reports:

Startup finished in 3.057s (kernel) + 1.109s (userspace) = 4.167s 

The time shown as "kernel" here is primarily counting the time taken for systemd to load.

created time in 20 hours

pull request commentrust-lang/rust

Add Linux-specific pidfd process extensions

I'd love to have this! Thanks for implementing it.

Aaron1011

comment created time in 2 days

Pull request review commentrust-lang/rust

Add Linux-specific pidfd process extensions

 impl Command { pub struct Process {     pid: pid_t,     status: Option<ExitStatus>,+    // On Linux, stores the pidfd created for this child.+    // This is -1 if the user did not request pidfd creation,+    // or if the pidfd could not be created for some reason+    // (e.g. the `clone3` syscall was not available).+    #[cfg(target_os = "linux")]+    pidfd: libc::c_int,

Consider using RawFd for this.

Aaron1011

comment created time in 2 days

PullRequestReviewEvent

delete branch joshtriplett/rust

delete branch : remove-duplicate-link-libraries

delete time in 2 days

pull request commentnrc/rustaceans.org

Add Patreon support

@nrc Is this not something you'd be open to adding, or was there something wrong with this PR?

joshtriplett

comment created time in 2 days

issue commentrust-lang/rust

When linking statically, native libraries should go inside --start-group/--end-group for robustness

@mati865 Thanks for getting numbers on this. That sounds like a pretty compelling argument that there's no longer any measurable performance loss.

I think it'd be a good idea to make this change, to simplify static linking.

joshtriplett

comment created time in 3 days

Pull request review commentmarshallpierce/rust-base64

Add EncoderStringWriter

+use crate::Config;+use std::io;+use std::io::Write;+use super::encoder::EncoderWriter;++/// A `Write` implementation that base64-encodes data using the provided config and accumulates the+/// resulting base64 in memory, which is then exposed as a String via `into_inner()`.+///+/// # Examples+///+/// ```+/// use std::io::Write;+///+/// let mut enc = base64::write::EncoderStringWriter::new(base64::STANDARD);+///+/// enc.write_all(b"asdf").unwrap();+///+/// // get the resulting String+/// let b64_string = enc.into_inner();+///+/// assert_eq!("YXNkZg==", &b64_string);+/// ```+///+/// # Panics+///+/// Calling `write()` (or related methods) or `finish()` after `finish()` has completed without+/// error is invalid and will panic.+///+/// # Performance+///+/// B64-encoded data is buffered in the heap since the point is to collect it in a String.+pub struct EncoderStringWriter {+    encoder: EncoderWriter<Vec<u8>>,+}++impl EncoderStringWriter {+    /// Create a new EncoderStringWriter that will encode with the provided config.+    pub fn new(config: Config) -> EncoderStringWriter {+        EncoderStringWriter::from(String::new(), config)+    }++    /// Create a new EncoderStringWriter that will append to the provided string.+    pub fn from(s: String, config: Config) -> EncoderStringWriter {+        EncoderStringWriter { encoder: EncoderWriter::new(s.into_bytes(), config) }+    }++    /// Encode all remaining buffered data, including any trailing incomplete input triples and+    /// associated padding.+    ///+    /// Once this succeeds, no further writes or calls to this method are allowed.+    ///+    /// Returns the base64-encoded form of the accumulated written data.+    pub fn into_inner(mut self) -> String {+        let buf = self.encoder.finish()+            .expect("Writing to a Vec<u8> should never fail");++        String::from_utf8(buf).expect("Base64 should always be valid UTF-8")

Ah, I misread and didn't catch that this was taking ownership of the Vec and not copying the memory. Nevermind, then.

I'm not concerned about paying the cost for the UTF-8 validity checking; it'd be nice to avoid that, but not worth adding unsafe code for.

marshallpierce

comment created time in 3 days

PullRequestReviewEvent

delete branch joshtriplett/highfive

delete branch : josh

delete time in 3 days

pull request commentrust-lang/highfive

Remove myself from the set of "libs" reviewers

On Thu, Sep 24, 2020 at 04:48:29PM -0700, Mark Rousskov wrote:

Yes, that's true. Even API surface expansion usually has implementation to review; usually the policy I follow is to perform that review, and then say something like "r=me on implementation; r? @some-libs-team-member on API"

It would be really nice to have a concrete way to write r? @rust-lang/libs or similar and have a random libs team member assigned. That would help for such situations.

joshtriplett

comment created time in 3 days

pull request commentrust-lang/highfive

Remove myself from the set of "libs" reviewers

Hm, so current policy is that library implementation review is under compiler team, whereas library API is under library team.

Well, I'm also not on the compiler team. :)

I believe that anyone who feels comfortable approving code should feel free to do so

So, effectively, anything that doesn't add API surface area is reasonable to review? If that's the case, then feel free to close this, and I'll review with that premise in mind.

joshtriplett

comment created time in 3 days

issue commentByron/gitoxide

Ensure robustness when killed in the middle

For example, cloning into the same location will have one git process fail early as it races to 'creation of .git directory' only, instead of racing for moving the cloned repository into place. Interestingly, when kill -9ed, git probably would be unable to recover such a partial clone as it couldn't differentiate a race from a dead process.

You could potentially handle that with file locks, which don't outlive the process holding them.

joshtriplett

comment created time in 3 days

issue commentmarshallpierce/rust-base64

Please provide a Writer that encodes to String

@marshallpierce Yes, that looks great! I left one comment there, about avoiding a copy of the data, but otherwise that would work perfectly for my use case.

joshtriplett

comment created time in 3 days

Pull request review commentmarshallpierce/rust-base64

Add EncoderStringWriter

+use crate::Config;+use std::io;+use std::io::Write;+use super::encoder::EncoderWriter;++/// A `Write` implementation that base64-encodes data using the provided config and accumulates the+/// resulting base64 in memory, which is then exposed as a String via `into_inner()`.+///+/// # Examples+///+/// ```+/// use std::io::Write;+///+/// let mut enc = base64::write::EncoderStringWriter::new(base64::STANDARD);+///+/// enc.write_all(b"asdf").unwrap();+///+/// // get the resulting String+/// let b64_string = enc.into_inner();+///+/// assert_eq!("YXNkZg==", &b64_string);+/// ```+///+/// # Panics+///+/// Calling `write()` (or related methods) or `finish()` after `finish()` has completed without+/// error is invalid and will panic.+///+/// # Performance+///+/// B64-encoded data is buffered in the heap since the point is to collect it in a String.+pub struct EncoderStringWriter {+    encoder: EncoderWriter<Vec<u8>>,+}++impl EncoderStringWriter {+    /// Create a new EncoderStringWriter that will encode with the provided config.+    pub fn new(config: Config) -> EncoderStringWriter {+        EncoderStringWriter::from(String::new(), config)+    }++    /// Create a new EncoderStringWriter that will append to the provided string.+    pub fn from(s: String, config: Config) -> EncoderStringWriter {+        EncoderStringWriter { encoder: EncoderWriter::new(s.into_bytes(), config) }+    }++    /// Encode all remaining buffered data, including any trailing incomplete input triples and+    /// associated padding.+    ///+    /// Once this succeeds, no further writes or calls to this method are allowed.+    ///+    /// Returns the base64-encoded form of the accumulated written data.+    pub fn into_inner(mut self) -> String {+        let buf = self.encoder.finish()+            .expect("Writing to a Vec<u8> should never fail");++        String::from_utf8(buf).expect("Base64 should always be valid UTF-8")

It would be nice to accumulate directly into a String, to avoid first writing to a Vec<u8> and then copying into a String.

marshallpierce

comment created time in 3 days

PullRequestReviewEvent

pull request commentrust-lang/rust

Include libunwind in the rust-src component.

I don't think there's any issue here. libunwind, as pointed out in an earlier comment, has a more permissive license than the rest of LLVM.

ehuss

comment created time in 3 days

pull request commentrust-lang/highfive

Remove myself from the set of "libs" reviewers

@Mark-Simulacrum I feel comfortable reviewing such code, but in many cases, I've felt like I'd be overstepping to approve something rather than getting someone on the libs team to review. In other words, I feel like I'm getting assigned to things for which I could r+ but for which I feel like I don't have the necessary authority to r+.

joshtriplett

comment created time in 3 days

create barnchjoshtriplett/highfive

branch : josh

created branch time in 3 days

PR opened rust-lang/highfive

Remove myself from the set of "libs" reviewers

I'm not on the libs team.

+3 -3

0 comment

1 changed file

pr created time in 3 days

pull request commentrust-lang/rust

Move `slice::check_range` to `RangeBounds`

@dylni Interesting, thanks for pointing me at that. :)

dylni

comment created time in 3 days

pull request commentrust-lang/rust

Std/thread: deny unsafe op in unsafe fn

@bors r+

poliorcetics

comment created time in 3 days

Pull request review commentrust-lang/rust

Improve documentation for `std::{f32,f64}::mul_add`

 impl f32 {     /// Fused multiply-add. Computes `(self * a) + b` with only one rounding     /// error, yielding a more accurate result than an unfused multiply-add.     ///-    /// Using `mul_add` can be more performant than an unfused multiply-add if-    /// the target architecture has a dedicated `fma` CPU instruction.+    /// Using `mul_add` *can* be more performant than an unfused multiply-add if+    /// the target architecture has a dedicated `fma` CPU instruction. However,+    /// this is not always true, and care must be taken not to overload the+    /// architecture's available FMA units when using many FMA instructions+    /// in a row, which can cause a stall and performance degradation.

This seems a little too specific, and may lead people to overgeneralize. This is highly likely to be CPU-dependent and model-dependent, and I wouldn't want to see someone thinking they should manually attempt to limit the number of FMA operations they do. (This is also something better left to a compiler's instruction scheduler, if anything.)

Having a warning here seems reasonable, but could it be generalized a little to just suggest that it "may be more performant", and that "the performance of FMA depends on the architecture and CPU, and may depend on the number of FMA operations versus non-FMA operations performed in a row"?

termhn

comment created time in 3 days

PullRequestReviewEvent

pull request commentrust-lang/rust

Revert "Function to convert OpenOptions to c_int"

Interesting; I'd agree with the assessment elsewhere that OpenOptionsExt should really have been sealed, but since it can't be, this is indeed a stability break. I don't think we can do much better than a default implementation that calls unimplemented!() with an appropriate message, here.

jyn514

comment created time in 3 days

issue commentByron/gitoxide

Ensure robustness when killed in the middle

@Byron Catching interrupts won't handle kill -9, or a power failure, or similar.

You don't have to make entire operations atomic when that isn't possible. It's OK if an interrupted fetch leaves some refs created and others not; another fetch will clear that up. (Clones can be made atomic by doing them into a temporary directory and renaming that directory into place.) What must not happen is a ref getting created but the object it references not existing, or an incomplete pack existing where git looks for packs, or other cases where the repository is in an inconsistent state. There should always be an order you can perform operations in such that the repository remains consistent.

joshtriplett

comment created time in 4 days

issue openedByron/gitoxide

Ensure robustness when killed in the middle

git is somewhat robust when killed in the middle. libgit2 seems somewhat less so. I'd love to see gitoxide be rock-solid on that front.

When doing things like clone, fetch, and similar, it's important to deal with network failure, ctrl-c, systems being powered off, and other sources of abrupt interruption. It's important that this never leave the repository in an inconsistent state (e.g. references to objects that don't exist, corrupted packs or indexes that git will choke on, and similar). This includes things like putting things in temporary locations and atomic-renaming them in place, or ensuring that objects are put in place before references to them.

created time in 4 days

issue commentmicrosoft/WSL2-Linux-Kernel

Please enable CONFIG_BLK_DEV_NBD

Filed at https://github.com/microsoft/WSL/issues/5968 .

joshtriplett

comment created time in 5 days

issue openedmicrosoft/WSL

Please support mounting NBD disks

The Linux kernel supports Network Block Device (NBD) disks. I'd like to be able to mount such disks, containing any filesystem Linux supports, and access them from both Linux and Windows.

This is a capability easily available on a native Linux system, and it would be helpful to have the same capability on WSL2.

created time in 5 days

pull request commentrust-lang/rfcs

Safer Transmute

@jswrenn Can you clarify something, please? Is the expectation that user code will need to write Here!(), or is the expectation that it would be an implicit default and not something people need to write explicitly?

jswrenn

comment created time in 6 days

pull request commentrust-lang/rust

Add Atomic*::from_mut.

As mentioned on the PR for it, adding this new feature-gated cfg looks like a good solution to me from a language perspective.

m-ou-se

comment created time in 6 days

pull request commentrust-lang/rust

Add cfg(target_has_atomic_equal_alignment) and use it for Atomic::from_mut.

Adding this new feature-gated cfg for atomic alignment seems perfectly reasonable to me from a language perspective.

m-ou-se

comment created time in 6 days

issue commentrust-lang/rust

Statically link against glibc

I have this successfully working; just waiting on a release of the libc crate containing https://github.com/rust-lang/libc/pull/1899 .

NilsIrl

comment created time in 7 days

create barnchjoshtriplett/rust

branch : remove-duplicate-link-libraries

created branch time in 7 days

PR opened rust-lang/rust

Remove duplicated library links between std and libc

The libc crate is already responsible for linking in the appropriate libraries, and std doing the same thing results in duplicated library names on the linker command line. Removing this duplication slightly reduces linker time, and makes it simpler to adjust the set or order of linked libraries in one place (such as to add static linking support).

+0 -4

0 comment

1 changed file

pr created time in 7 days

push eventjoshtriplett/rust-libc

Josh Triplett

commit sha ac0a783953832205372cb6b6982f2071f9170964

Add support for building with static glibc This will need corresponding changes in rust-lang/rust to activate, but this will make it possible to make those changes. Note that despite the apparent redundancy in config directives, the link directives cannot be simplified any further. Attempting to factor out the checks for `target_feature = "crt-static"` does not work.

view details

Josh Triplett

commit sha e005f4cad598c7278dfcef52ac7cbe1203c963f2

Consolidate handling of libutil, and handle crt-static The two library blocks that specify `#[link(name = "util")]` do not actually reference any functions in `libutil`; the functions that do use `libutil` don't have any reference to it. And having two library blocks specify it results in two separate inclusions of `-lutil` on the linker command line. Move the link lines up to `src/unix/mod.rs`, making it easier to see all the libraries `libc` links to. This also makes `libutil` respect `target-feature=+crt-static`.

view details

Josh Triplett

commit sha 5bf6ee5d8e06aea64a2b20c47afe5062da1a7848

Consolidate handling of libdl, and handle crt-static Move the link line for `libdl` up to `src/unix/mod.rs`, making it easier to see all the libraries `libc` links to. This also makes `libdl` respect `target-feature=+crt-static`.

view details

push time in 7 days

pull request commentrust-lang/rust

Move `slice::check_range` to `RangeBounds`

@dylni I'm not a libs member.

dylni

comment created time in 7 days

pull request commentrust-lang/rust

Format Duration microseconds with "us" suffix, without Unicode

On Sun, Sep 20, 2020 at 05:26:32AM -0700, Andrew Gallant wrote:

Misrendering of Unicode characters is not a theoretical concern, it's a practical one, and one I've personally experienced. The scenarios I mentioned were not hypotheses, they were real systems I've worked with in the past; please take this as testimonial evidence that such failures are not hypothetical, in case that was not previously clear. This was not me saying "this might break", this was me saying "I've seen exactly this kind of breakage on real systems".

I believe you that it has happened and can happen. That's never been in doubt. But that doesn't mean I don't want to understand the failure modes for myself. The best way to do that I think is to reproduce the failure mode. This often leads to learning things. For example, in the process of doing that, I did discover a failure mode, but it was not as bad as you might imagine. While the output is mojibake, it still renders the SI unit. To me, that's "good enough" and does not make it worth changing this. I never would have learned this if I hadn't asked for how to reproduce the failure mode and tried it myself.

Part of the problem is that I don't have a recipe by which you can take a modern Linux system and create an environment where Unicode will break that badly. I've had such experiences over the course of years encountering various random lab systems, bits of crufty old infrastructure, weird pieces of serial hardware and magic boxes implementing SSH or telnet in firmware, web-based terminal emulators, BIOS VGA capture devices, and many many other such environments.

joshtriplett

comment created time in 7 days

pull request commentrust-lang/rust

Refactor memchr to allow optimization

I'm not the right reviewer for this. Someone from libs should review.

bugadani

comment created time in 7 days

issue commentrust-lang/rust

When linking statically, native libraries should go inside --start-group/--end-group for robustness

@petrochenkov The issue is that most crates don't account for static linking at all, and while the specification of #[link does emit multiple library links when specified multiple times, that's not intuitive behavior at all. People are used to the behavior of dynamically linked libraries, which handle reverse/circular dependencies automatically. (And as far as I can tell, ldd also handles circular dependencies automatically.)

I've tested it, and as far as I can tell, "significant performance cost" is not accurate. I haven't measured any performance difference.

Based on that, I think it'd be reasonable to do this by default, to make static linking more user-friendly and less fiddly. And to be clear, I'm only suggesting emitting this when linking statically.

joshtriplett

comment created time in 7 days

startednikhilm/ninja-rs

started time in 7 days

pull request commentrust-lang/libc

Add SYS_pidfd_open and SYS_clone3

I'd really like to have these constants available as well.

Aaron1011

comment created time in 7 days

issue openedrust-lang/rust

When linking statically, native libraries should go inside --start-group/--end-group for robustness

When rustc links rlib libraries, it puts all the libraries inside a --start-group/--end-group pair, causing the linker to resolve backreferences from a library later in the command line to one earlier in the command line.

However, when rustc links native libraries, it puts libraries on the command line in whatever order it encountered them in code.

With shared linking, that works fine. But with static linking, the order has a crucial semantic significance: symbols in a library will get thrown away unless a library listed previously on the command line has a corresponding unresolved symbol.

Rust #[link] directives don't account for this, and crates in general don't worry about link order, precisely because dynamic linking is the default. When linking statically to multiple libraries, where one depends on another, this is very likely to result in symbol resolution failures at link time. In addition, if libraries have circular references (such as between glibc and libgcc), there is no order that will allow the libraries to link without duplicating at least one library, which seems like something #[link] directives should not need to account for.

I would propose that when linking statically, Rust should always put all libraries, both rlibs and native libraries, inside one large --start-group/--end-group pair, which will allow the linker to handle symbol references both backwards and forwards, as well as circular symbol dependencies.

created time in 7 days

pull request commentrust-lang/rust

Make RawFd implement the RawFd traits

It would be nice if it were an opaque type rather than an alias, but I don't think that could be changed backwards-compatibly.

withoutboats

comment created time in 7 days

pull request commentrust-lang/rust

Std/thread: deny unsafe op in unsafe fn

@bors r+ rollup

poliorcetics

comment created time in 7 days

issue commentrust-lang/rust

Tracking issue for RFC 2582, `&raw [mut | const] $place` (raw_ref_op)

@RalfJung Re-reading the RFC, I do think it makes sense to switch to just plain getelementptr when raw refs are involved, and we can always switch to a new nowrap variant if LLVM adds one.

Centril

comment created time in 8 days

pull request commentrust-lang/rust

Format Duration microseconds with "us" suffix, without Unicode

us is the correct SI unit for microseconds, rendered in ASCII. Many other tools render it as such, and I don't think there's any chance of confusion there.

Misrendering of Unicode characters is not a theoretical concern, it's a practical one, and one I've personally experienced. The scenarios I mentioned were not hypotheses, they were real systems I've worked with in the past; please take this as testimonial evidence that such failures are not hypothetical, in case that was not previously clear. This was not me saying "this might break", this was me saying "I've seen exactly this kind of breakage on real systems".

This is a sufficiently plausible failure mode that if Duration continues to produce such output, I'm going to have to use some other duplicate Duration-printing code just to avoid it. I would really like to avoid that.

I'm genuinely baffled as to the weighing of factors in the balance here.

joshtriplett

comment created time in 8 days

pull request commentrust-lang/libc

Add support for building with static glibc

Also, as far as I can tell, this change (together with some corresponding changes in rust-lang/rust) will noticeably speed up linking of all Rust binaries on linux-gnu targets, by reducing the amount of work the linker needs to do, since it won't need to process some libraries twice. I've tested this by manually re-invoking the linker and timing it, and overall, eliminating repeated libraries seems to provide a ~5% improvement in link time for a "hello world" Rust program.

joshtriplett

comment created time in 8 days

pull request commentrust-lang/libc

Add support for building with static glibc

Both of the failures seem entirely unrelated to this PR.

joshtriplett

comment created time in 8 days

pull request commentrust-lang/libc

Add support for building with static glibc

Update: had to reverse the order of the libraries so that symbol resolution worked correctly when linking statically.

joshtriplett

comment created time in 8 days

push eventjoshtriplett/rust-libc

Josh Triplett

commit sha f03aca06eb32808a4c57ac370af94a88ccc95f61

Add support for building with static glibc This will need corresponding changes in rust-lang/rust to activate, but this will make it possible to make those changes. Note that despite the apparent redundancy in config directives, the link directives cannot be simplified any further. Attempting to factor out the checks for `target_feature = "crt-static"` does not work.

view details

Josh Triplett

commit sha 618b55b6695886a25a0daa82db3642d08ef1dfaa

Consolidate handling of libutil, and handle crt-static The two library blocks that specify `#[link(name = "util")]` do not actually reference any functions in `libutil`; the functions that do use `libutil` don't have any reference to it. And having two library blocks specify it results in two separate inclusions of `-lutil` on the linker command line. Move the link lines up to `src/unix/mod.rs`, making it easier to see all the libraries `libc` links to. This also makes `libutil` respect `target-feature=+crt-static`.

view details

Josh Triplett

commit sha 8e65bdb4750c56e058d6b95cb10834c70e09f21a

Consolidate handling of libdl, and handle crt-static Move the link line for `libdl` up to `src/unix/mod.rs`, making it easier to see all the libraries `libc` links to. This also makes `libdl` respect `target-feature=+crt-static`.

view details

push time in 8 days

pull request commentrust-lang/libc

Add support for building with static glibc

OK, this should work now.

joshtriplett

comment created time in 8 days

push eventjoshtriplett/rust-libc

Josh Triplett

commit sha e929f273f7c68008b28bbc01a344a110258c72ec

Consolidate handling of libutil, and handle crt-static The two library blocks that specify `#[link(name = "util")]` do not actually reference any functions in `libutil`; the functions that do use `libutil` don't have any reference to it. And having two library blocks specify it results in two separate inclusions of `-lutil` on the linker command line. Move the link lines up to `src/unix/mod.rs`, making it easier to see all the libraries `libc` links to. This also makes `libutil` respect `target-feature=+crt-static`.

view details

Josh Triplett

commit sha 74cfbfd734a6e12a6394010d8da8242141088056

Consolidate handling of libdl, and handle crt-static Move the link line for `libdl` up to `src/unix/mod.rs`, making it easier to see all the libraries `libc` links to. This also makes `libdl` respect `target-feature=+crt-static`.

view details

push time in 8 days

pull request commentrust-lang/libc

Add support for building with static glibc

Still WIP, need to do something similar for the link lines for dl and util buried inside Linux-specific code.

joshtriplett

comment created time in 8 days

push eventjoshtriplett/rust-libc

Josh Triplett

commit sha d76f0d0f218f899d558c7cb18fa5f1cc41f26053

Add support for building with static glibc This will need corresponding changes in rust-lang/rust to activate, but this will make it possible to make those changes. Note that despite the apparent redundancy in config directives, the link directives cannot be simplified any further. Attempting to factor out the checks for `target_feature = "crt-static"` does not work.

view details

push time in 8 days

pull request commentrust-lang/libc

Add support for building with static glibc

Currently working on the corresponding changes in rust-lang/rust. It would really help to have a libc release with this PR included.

joshtriplett

comment created time in 8 days

create barnchjoshtriplett/rust-libc

branch : support-static-glibc

created branch time in 8 days

PR opened rust-lang/libc

Add support for building with static glibc

This will need corresponding changes in rust-lang/rust to activate, but this will make it possible to make those changes.

Note that despite the apparent redundancy in config directives, the link directives cannot be simplified any further. Attempting to factor out the checks for target_feature = "crt-static" does not work.

+11 -1

0 comment

2 changed files

pr created time in 8 days

pull request commentrust-lang/rust

Format Duration microseconds with "us" suffix, without Unicode

It seems like the discussion on this died out.

I feel like there's a degree of "can we reproduce the problems here" bar being set here that doesn't seem appropriate. Almost by definition, the failure modes here are not common cases; rather, they're cases that nobody wants to be in but people sometimes find themselves in. "Serial terminal via weird network serial concentrator box", or "broken old SSH implementation", or "terrible web-based terminal", or "system using old bitmap font", or "user supplied non-UTF-8 input data and expects the output to be in the same charset but one UTF-8 character makes a charset detector assume UTF-8", or any number of other things. When you're using such a setup, you're probably already having a bad day, and mangled Unicode just makes it worse. I've seen comments that there's "no evidence

I'd like to bring the topic back to the specific issue at hand. We're talking about the Debug implementation of Duration, and whether it should print μs or us for microseconds. This seems like an incredibly minor, purely cosmetic usage of Unicode, and one where if we didn't do so, nobody would be surprised or confused by the output.

This is not an indictment of Unicode, nor is it a lack of support for Unicode in Rust. If you use Unicode, Rust will deal with it perfectly, and pass it through to your system, which may or may not deal with it perfectly. The question at hand is whether Rust's standard library will force a program that otherwise isn't processing Unicode data to generate Unicode output, for the sake of a cosmetic SI prefix.

I would like Unicode to work everywhere. I would like all systems that don't support Unicode to be fixable.

But Rust's philosophy has never been "let's try to trigger breakage in the user's setup to force them to fix it". Outputting Unicode characters when the user didn't provide any as input data, when there's no way to know if the output file/device/etc supports Unicode, is doing precisely that. It doesn't seem substantively different to me than saying "oh, you have non-UTF-8 in a filename, we're just going to break so that you have to fix that".

Everything should use Unicode. But the point of the robustness principle is that we should handle Unicode, but not forcibly introduce Unicode into non-Unicode data, for maximum compatibility. If the user is processing Unicode data, Rust passes it through correctly, and their system can't handle Unicode, that's not Rust's fault. If the user is processing ASCII data on a system that can't handle Unicode, but Rust introduces Unicode, that's Rust's fault.

joshtriplett

comment created time in 8 days

issue commentrust-lang/libc

libc expose SIGRTMIN and SIGRTMAX

I think it would be reasonable to expose a function libc_sigrtmin() that calls __libc_current_sigrtmin().

Gelox

comment created time in 8 days

pull request commentrust-lang/rust

black_box: silence unused_mut warning when building with cfg(miri)

@bors r+

RalfJung

comment created time in 10 days

issue commentrust-lang/rust

Tracking issue for RFC 2523, `#[cfg(accessible(::path::to:thing))]`

Reporting an error on paths that can potentially point to associated items shouldn't be much harder than reporting an error on multi-segment paths, so the single-segment restriction would mostly be artificial. It can be done as an intermediate step though, if the second item in #64797 (comment) (which is the current blocker) is going to be (partially) implemented by someone unfamiliar with resolve.

I read that message, and what I was wondering was whether that second item would be substantially simpler to implement and stabilize if it doesn't have to take trait methods or any other type-dependent resolution into account.

Centril

comment created time in 10 days

pull request commentRust-for-Linux/linux

Initial Rust support

For crate dependencies, we should vendor those into the kernel tree under a specific path, and then configure that path as a directory registry, along with passing the appropriate cargo options to never touch the network.

For bindgen/cbindgen, I'm wondering if we can get distributions to package it, and then just expect people to have it installed like cargo and rustc.

ojeda

comment created time in 10 days

pull request commentrust-lang/rust

black_box: silence unused_mut warning when building with cfg(miri)

Shouldn't this use cfg_attr so that it only applies on miri?

RalfJung

comment created time in 11 days

issue commentrust-lang/rust

Tracking issue for RFC 2523, `#[cfg(accessible(::path::to:thing))]`

This came up in the @rust-lang/cargo meeting today. If you've specified conditionals on a dependency, such as with [target.'cfg(some_platform)'.dependencies], you might not want to duplicate that information into corresponding cfg directives when using that dependency. It'd be nice to, instead, write #[cfg(accessible(dep_name))], which would be true whenever that dep was available.

Would it be potentially reasonable, and avoid the trait-path corner cases, to only allow #[cfg(accessible(single_token))] for now? You could still write use std::foo::bar; #[cfg(accessible(bar))] item;, but you couldn't access methods that way (trait method or otherwise), so type-specific resolution shouldn't be an issue. Would that work?

Centril

comment created time in 11 days

issue commentrust-lang/rustup

Add support for `.zst` for component tarballs in channels

Does decompression time not correlate with compression time much then? (Beyond "disk reading is slower with more data")?

No, it doesn't. Compression can take anywhere from "faster than gzip" to "several minutes", with corresponding improvements in data size. Taking several minutes would be worth it for stable release tarballs, while nightly/CI versions could scale that back a little and aim for taking the same amount of time in CI that we currently do.

kinnison

comment created time in 12 days

issue commentrust-lang/rustup

Add support for `.zst` for component tarballs in channels

@Mark-Simulacrum Depends heavily on compression level and what tradeoff we want to make. How much time does the current xz compression use?

kinnison

comment created time in 12 days

delete branch joshtriplett/cargo-edit

delete branch : keep-sorted-dependencies-sorted

delete time in 13 days

issue commentlibgit2/libgit2

git_blob_create_from_stream readback of partial buffer?

@ethomson Yes, exactly.

You could call git_blob_create_from_stream to start writing out a blob incrementally, and then at some point during the writing of that blob, you could call git_blob_create_from_stream_partial_len to get the number of bytes written so far, then call git_blob_create_from_stream_partial_read to read them out. (Or possibly that should be git_blob_create_from_stream_partial_pread with an offset.)

joshtriplett

comment created time in 13 days

issue commentrust-lang/rust

Don't discourage people from using `std::fmt::Write`

@camelid I think there should be a more nuanced explanation there. It seems fine to impl std::fmt::Write if you only want to accept Unicode. If you can accept either raw bytes or Unicode, you should always prefer to implement std::io::Write, which works fine for UTF-8 bytes too.

camelid

comment created time in 13 days

issue commentrust-lang/rustup

Add support for `.zst` for component tarballs in channels

For reference, on my laptop:

$ du -hs rust-1.46.0-x86_64-unknown-linux-gnu.tar*
842M	rust-1.46.0-x86_64-unknown-linux-gnu.tar
211M	rust-1.46.0-x86_64-unknown-linux-gnu.tar.gz
122M	rust-1.46.0-x86_64-unknown-linux-gnu.tar.xz
99M	rust-1.46.0-x86_64-unknown-linux-gnu.tar.zst
$ time gzip -dc rust-1.46.0-x86_64-unknown-linux-gnu.tar.gz >/dev/null

real	0m4.890s
user	0m4.853s
sys	0m0.036s
$ time xz -dc rust-1.46.0-x86_64-unknown-linux-gnu.tar.xz >/dev/null

real	0m9.218s
user	0m9.160s
sys	0m0.056s
$ time zstd -qdc rust-1.46.0-x86_64-unknown-linux-gnu.tar.zst >/dev/null

real	0m0.839s
user	0m0.811s
sys	0m0.028s
kinnison

comment created time in 13 days

push eventjoshtriplett/cargo-edit

Josh Triplett

commit sha f8efc5ae6ec71e253b25ecbb754b4e62f5cb1bb9

Keep dependencies in sorted order if they were already sorted This should avoid the need to pass the `-s` option, except to sort dependencies that are currently unsorted. Adjust the documentation of that option accordingly.

view details

push time in 13 days

issue commentrust-lang/rust

std::process::Command doesn't follow Unix signal safety in forked child

We also can't rely on any particular safety property of execvp that happens to be true in a particular implementation, since it isn't guaranteed by the standard.

ericyoungblut

comment created time in 13 days

pull request commentrust-lang/rust

Write manifest for MAJOR.MINOR channel to enable rustup convenience

On Sun, Sep 13, 2020 at 02:40:08PM -0700, Mark Rousskov wrote:

Hm, so I think for now I'd be fine with rustup behaving for these 1.45 toolchain files just like it does for stable/beta/nightly and the 1.x.y releases/beta-YYYY-MM-DD/nightly-YYYY-MM-DD releases. That is, we would expect each 1.x channel to only represent itself, and explicit installs of parts of that channel would not "see" that channel's prior installation. So this sequence:

Let's set the stage as stable=1.45.1.

rustup install 1.45.1 # downloads and installs
rustup install 1.45 # downloads and installs 1.45.1, again, but calls it 1.45
rustup install stable # downloads and installs 1.45.1, again, but calls it stable
# We release stable 1.45.2
rustup update # updates 1.45 to 1.45.2, stable to 1.45.2. Does not touch 1.45.1 installation.

If this behavior is something we can all agree is reasonable to start with, then I think this change to the manifest builder will probably do what we want, so we can merge this PR.

This seems like reasonable behavior to start with. It'd be nice to avoid re-downloads of identical files in the future, but that doesn't seem critical.

carols10cents

comment created time in 13 days

issue commentkillercup/cargo-edit

Please support configuration to turn -s on by default

Implemented in #421.

joshtriplett

comment created time in 13 days

PR opened killercup/cargo-edit

Keep dependencies in sorted order if they were already sorted

This should avoid the need to pass the -s option, except to sort dependencies that are currently unsorted. Adjust the documentation of that option accordingly.

+73 -4

0 comment

4 changed files

pr created time in 13 days

create barnchjoshtriplett/cargo-edit

branch : keep-sorted-dependencies-sorted

created branch time in 13 days

issue commentrust-lang/rust

Deadlocks on NetBSD around fork()

man 7 signal-safety does not list execvp, so it is not async signal safe

To the best of my knowledge, execvp is only considered "not async signal safe" because all functions that read the environment aren't safe against concurrent modifications: if you change PATH concurrently from another thread, you have no guarantees whether it'll use the old PATH, the new, or some combination thereof. execvp is listed as MT-Safe env, meaning "it's multi-thread safe as long as you don't expect environment variables to do any synchronization for you".

he32

comment created time in 13 days

fork joshtriplett/cargo-edit

A utility for managing cargo dependencies from the command line.

http://killercup.github.io/cargo-edit/

fork in 13 days

issue commentservo/bincode

Documentation says varint is the default, but it isn't unless you use Options

On Fri, Sep 11, 2020 at 10:01:40AM -0700, Zoey Riordan wrote:

@pythonesque 1.3 introduced a new options interface, this was not a breaking change since the old interface was left in tact and still works the same way. For a similar reason the basic bincode::serialize methods were left with the old configuration. Changing their behavior would be a breaking change and I am committed not to break the crate without a very good reason. The documentation should be clearer though so I will work on updating that.

Updating the documentation would suffice. The top-level functions should state what configuration they use and that it isn't the default, and make it clear that you should create a bincode::DefaultOptions::new() and use its methods if you want the defaults.

joshtriplett

comment created time in 13 days

issue commentmicrosoft/WSL2-Linux-Kernel

Please enable CONFIG_BLK_DEV_NBD

With this enabled, you wouldn't need kernel command-line arguments to use it; you could just set it up with the normal NBD tools.

joshtriplett

comment created time in 13 days

issue commentkillercup/cargo-edit

Please support configuration to turn -s on by default

@kevincox "Insert in sorted order if already sorted, otherwise insert at end" sounds great to me. Then -s is only needed if you have an unsorted list and want it sorted, and we can change its help to "sort currently unsorted dependency list".

joshtriplett

comment created time in 13 days

pull request commentrust-lang/rfcs

Destructuring assignment

@varkor Thank you!

With that changed:

@rfcbot fcp merge

varkor

comment created time in 13 days

pull request commentrust-lang/cargo

Mark temporary directories as excluded from macOS Spotlight

On Mon, Sep 14, 2020 at 07:35:39PM -0700, Kornel wrote:

Thank you for the input. Here's the plan:

  • I'll make existing target folders keep everything as-is (real directories) to improve back compat and ease migration.

When you upgrade Rust, you're already going to rebuild everything and you can't do an incremental build, so replacing the build and incremental directories seems perfectly fine.

  • New target/ directories will have symlinks to a subdirectory in system's temp if it's on the same device. Otherwise keeps existing behavior.

Only for build and incremental, right?

Also, if the system's temp directory is not on the same device, how about using adjacent build.noindex and incremental.noindex directories as the symlink targets?

  • To keep this change small, the the subdirectory in temp will be unique per workspace to avoid adding new form of sharing of build artifacts (these dirs could be unified later)

You mentioned using a hash, but please also prefix it with a crate name or similar, for future troubleshooting, so that it's possible to figure out roughly where it came from.

kornelski

comment created time in 13 days

issue openedlibgit2/libgit2

git_blob_create_from_buffer readback of partial buffer?

For some use cases, when streaming into a blob, I'd like to be able to read back the streamed data before I'm done writing. Would it be possible to have functions git_blob_crate_from_buffer_partial_len and git_blob_crate_from_buffer_partial_read? This would avoid the need to stream into a separate temporary file.

created time in 14 days

issue commentalgesten/ureq

Reads HTTP response byte-by-byte, one recvfrom syscall per byte

Thanks for the fix, much appreciated! Would you consider a point release of ureq with this fix included?

joshtriplett

comment created time in 15 days

push eventjoshtriplett/msgpack-rust

Josh Triplett

commit sha 605ecd52f1e3066da4376310934a9e46af9880dc

Add config to choose binary vs human-readable, for Serializer and Deserializer Add a defaulted type parameter to `Deserializer` and all its auxiliary types, threading the config ZSTs through. Delegate `Serializer::is_human_readable` and `Deserializer::is_human_readable` to the config. Test round-tripping for the new config variants this introduces.

view details

push time in 15 days

Pull request review comment3Hren/msgpack-rust

Add config to choose binary vs human-readable, for Serializer and Deserializer

 impl<R: Read> Deserializer<ReadReader<R>> {     } } +impl<R: Read, C: SerializerConfig> Deserializer<R, C> {+    /// Consumes this deserializer and returns a new one, which will deserialize types with+    /// human-readable representations (`Deserializer::is_human_readable` will return `true`).+    ///+    /// This is primarily useful if you need to interoperate with serializations produced by older+    /// versions of `rmp-serde`.+    pub fn with_human_readable(self) -> Deserializer<R, HumanReadableConfig<C>> {+        let Deserializer { rd, config, marker, depth } = self;+        Deserializer {+            rd,+            config: HumanReadableConfig::new(config),+            marker,+            depth,+        }+    }++    /// Consumes this deserializer and returns a new one, which will deserialize types with+    /// binary representations (`Deserializer::is_human_readable` will return `false`).+    ///+    /// This is the default MessagePack deserialization mechanism, consuming the most compact+    /// representation.+    pub fn with_binary(self) -> Deserializer<R, BinaryConfig<C>> {

I can bump the version to 0.15.0 if you prefer.

joshtriplett

comment created time in 15 days

PullRequestReviewEvent

pull request commentrust-lang/rust

unions: test move behavior of non-Copy fields

LGTM.

RalfJung

comment created time in 15 days

issue comment3Hren/msgpack-rust

msgpack doesn't work with OsString

@ajeetdsouza @daboross https://github.com/serde-rs/serde/pull/1888 should fix this.

ajeetdsouza

comment created time in 16 days

issue comment3Hren/msgpack-rust

cargo test fails on recent serde

This should be fixed now.

FauxFaux

comment created time in 16 days

issue comment3Hren/msgpack-rust

Implement serde's `is_human_readable` to return `false`

https://github.com/3Hren/msgpack-rust/pull/257 (together with the serde fix in https://github.com/serde-rs/serde/pull/1888 ) should fix this.

aszkid

comment created time in 16 days

issue comment3Hren/msgpack-rust

best way to round trip `Result`

@thedavidmeister I believe https://github.com/serde-rs/serde/pull/1888 (released in serde 1.0.116) should fix this.

thedavidmeister

comment created time in 16 days

issue comment3Hren/msgpack-rust

std::net::IpAddr fails to roundtrip using 0.14.1

And https://github.com/3Hren/msgpack-rust/pull/257 adds config support for binary vs human-readable.

ghost

comment created time in 16 days

create barnchjoshtriplett/msgpack-rust

branch : inhuman

created branch time in 16 days

PR opened 3Hren/msgpack-rust

Add config to choose binary vs human-readable, for Serializer and Deserializer

Add a defaulted type parameter to Deserializer and all its auxiliary types, threading the config ZSTs through. Delegate Serializer::is_human_readable and Deserializer::is_human_readable to the config. Test round-tripping for the new config variants this introduces.

+296 -46

0 comment

5 changed files

pr created time in 16 days

delete branch joshtriplett/msgpack-rust

delete branch : update-for-serde-fix

delete time in 16 days

more