profile
viewpoint
Jonathan Reem reem @GauntletNetworks San Francisco, CA https://medium.com/@jreem/

chris-morgan/rust-http 391

Completely OBSOLETE Rust HTTP library (server and client)

josephg/node-browserchannel 288

An implementation of a google browserchannel server in node.js

chris-morgan/anymap 162

A safe and convenient store for one value of each type

Gankra/collect-rs 65

Miscellaneous Collections

reem/adamantium 48

Functional, Persistent Data Structures in Rust

iron/iron-test 28

Utilities for testing Iron and Iron Middleware

reem/awesome-rust 2

A curated list of awesome Rust code and resources.

reem/beatdots 1

A fast, greedy algorithm for solving the popular game of dots.

reem/bincode 1

A binary encoder / decoder implementation in Rust.

pull request commentrust-lang/rust

Add 'for more info' link to 'irrefutable *-let pattern' warning

My preference would be to not add one-off links like this. Ideally, every warning should (automatically) have some way to get more information. The issue is that it is not clear exactly what that should look like. Some options that were considered:

  • Add an HTTP link similar to this PR. This has several drawbacks:
    • Links are release-specific. It could always link to nightly, but that would cause problems with renamed lints.
    • Doesn't work offline.
    • Is very long and wordy, adding quite a bit of characters to the error message.
  • Add it to the --explain flag. This has several concerns:
    • It's not really clear if anyone is actually using --explain.
    • Explain has historically been only for Exxxx codes, would it be confusing if it was also for warnings?
    • Not really clear how --explain will work in relation with -Zteach in the future.

There is some history in #48041 and #48337 about trying to reduce the verbosity with --explain. I'm not sure how that would work with warnings, it would need some rewording. In some situations, there can be a lot of warnings, and I'd be concerned that it would spew out way too much text.

camelid

comment created time in 5 minutes

pull request commentrust-lang/rust

A slightly clearer diagnostic when misusing const

:hourglass: Testing commit 19003517cea93d337df071e191ecd57259543cd4 with merge bb0d481b5a9e78145f5644ec46015065fa83b4cc... <!-- homu: {"type":"BuildStarted","head_sha":"19003517cea93d337df071e191ecd57259543cd4","merge_sha":"bb0d481b5a9e78145f5644ec46015065fa83b4cc"} -->

rylev

comment created time in 6 minutes

PR merged rust-lang/rust

Fix src/test/ui/env-vars.rs on 128-core machines on Windows S-waiting-on-bors merged-by-bors

On Windows, the environment variable NUMBER_OF_PROCESSORS has special meaning. Unfortunately, you can get different answers, depending on whether you are enumerating all environment variables or querying a specific variable. This was causing the src/test/ui/env-vars.rs test to fail on machines with more than 64 processors when run on Windows.

+8 -0

5 comments

1 changed file

sivadeilra

pr closed time in 7 minutes

push eventrust-lang/rust

Arlie Davis

commit sha 957061bb977751767e2b543b29d65581283a94fe

Fix src/test/ui/env-vars.rs on 128-core machines on Windows On Windows, the environment variable NUMBER_OF_PROCESSORS has special meaning. Unfortunately, you can get different answers, depending on whether you are enumerating all environment variables or querying a specific variable. This was causing the src/test/ui/env-vars.rs test to fail on machines with more than 64 processors when run on Windows.

view details

bors

commit sha e792288df31636ca28108516c63a00ce4267063a

Auto merge of #79685 - sivadeilra:fix_env_vars_win, r=davidtwco Fix src/test/ui/env-vars.rs on 128-core machines on Windows On Windows, the environment variable NUMBER_OF_PROCESSORS has special meaning. Unfortunately, you can get different answers, depending on whether you are enumerating all environment variables or querying a specific variable. This was causing the src/test/ui/env-vars.rs test to fail on machines with more than 64 processors when run on Windows.

view details

push time in 7 minutes

pull request commentrust-lang/rust

Fix src/test/ui/env-vars.rs on 128-core machines on Windows

:sunny: Test successful - checks-actions Approved by: davidtwco Pushing e792288df31636ca28108516c63a00ce4267063a to master... <!-- homu: {"type":"BuildCompleted","approved_by":"davidtwco","base_ref":"master","builders":{"checks-actions":"https://github.com/rust-lang-ci/rust/runs/1504733820"},"merge_sha":"e792288df31636ca28108516c63a00ce4267063a"} -->

sivadeilra

comment created time in 7 minutes

Pull request review commentrust-lang/rust

Rustdoc: Use correct def_id for doctree::Import

 impl FormatRenderer for JsonRenderer {             } else if let types::ItemEnum::EnumItem(ref mut e) = new_item.inner {                 e.impls = self.get_impls(id, cache)             }-            self.index.borrow_mut().insert(id.into(), new_item);+            let removed = self.index.borrow_mut().insert(id.into(), new_item.clone());+            // FIXME(adotinthevoid): Currently, the index is duplicated. This is a sanity check+            // to make sure the items are unique.+            if let Some(old_idem) = removed {+                assert_eq!(old_idem, new_item);
            if let Some(old_item) = removed {
                assert_eq!(old_item, new_item);
aDotInTheVoid

comment created time in 18 minutes

pull request commentrust-lang/rust

Fix trimming of lint docs

Thanks for taking a look at this. The second trim was a mistake in #79522. I would kinda prefer something like the following change:

diff --git a/src/tools/lint-docs/src/lib.rs b/src/tools/lint-docs/src/lib.rs
index 326b7948098..ea54a351e03 100644
--- a/src/tools/lint-docs/src/lib.rs
+++ b/src/tools/lint-docs/src/lib.rs
@@ -143,8 +143,8 @@ fn lints_from_file(&self, path: &Path) -> Result<Vec<Lint>, Box<dyn Error>> {
                     Some((lineno, line)) => {
                         let line = line.trim();
                         if let Some(text) = line.strip_prefix("/// ") {
-                            doc_lines.push(text.trim().to_string());
-                        } else if line.starts_with("///") {
+                            doc_lines.push(text.to_string());
+                        } else if line == "///" {
                             doc_lines.push("".to_string());
                         } else if line.starts_with("// ") {
                             // Ignore comments.
camelid

comment created time in 18 minutes

PR opened rust-lang/rust

Rustdoc: Use correct def_id for doctree::Import

The default overwrites the crate root, which crashes rustdoc-json.

While investigating this, It turns out somehow, some items are being documented twice. I'm not sure how this is happening but for now, we just make sure they were the same if they have the same id.

Zulip descussion

Bless script (Once this is more pollished I'll submit it)

r? @jyn514

+249 -41

0 comment

5 changed files

pr created time in 21 minutes

PR closed rust-lang/rust

[Experiment] Add `never` as a type alias for `!` S-experimental S-waiting-on-review T-lang

This effectively renames the never-type ! to never, but should be fully backwards compatible. This is an idea I had and I just tried it out. It was surprisingly simple to implement. Example:

fn foo() -> never {
    let x: never = panic!();
}

Motivation

The never-type ! is an essential feature in Rust to make type inference in eg. match work:

let x = match some_enum {
    Variant1 => 42u32,
    Variant2 => return true,
    _ => panic!(),
}

Here x can be inferred to have the type u32, because the other two match arms have the type !, meaning that they never yield a value that would need to be stored in x. So in some sense, ! is a primitive type that marks control flow.

Currently though, you never need to write the !, besides the return type of a diverging function. This makes it an "niche" or "advanced" language feature. With the stabilization of the never-type this might change though.

The syntax of using the ! symbol is confusing though, when you first stumble on it as someone learning Rust. Any intuition for what the ! symbol might mean is misleading, since it's already used with macros, top-level attributes and doc comments in Rust. This is made worse by the fact that you can't even google it. The ! symbol is just filtered out of any search query, and I even get amusingly unrelated result like hotel bookings when searching for "Rust !".

Solution Idea

To fix that I had the idea introduce a primitive type whose name is what we have been calling this feature since 2016: never. Actually, it is just a type alias pub type never = ! in core::primitive that is exported in the prelude. This should make it fully backwards compatible, since you can use never and ! interchangeably, and the new prelude export is shadowed if somebody made their own type called never.

I changed some return types of core and std functions, and some never-type tests to verify that this proof-of-concept works. To my own surprise, bootstrapping works and all UI tests pass. There was one weird interaction with a stability attribute and one case where I had to use the full path crate::primitive::never, indicating that there are issues in some corner cases, but I haven't investigated further.

Downsides

The biggest downside is that existing stable Rust code should be migrated to never, since there shouldn't be two ways to express exactly the same. An edition migration would be a good fit: Warn on 2018 and below, error on 2021. But even if the change is straightforward and probably easy to implement in rustfix, the ecosystem impact should be carefully considered.

I'd argue it's a relatively small cost, but also only a small gain. Apparently the Syntax for diverging functions has always been low-priority, so it was overlooked in the rush to Rust 1.0, and again ignored in 2018. Even in the long discussion of the never type RFC the syntax was barely mentioned. Though it's said where it comes from ("Yes, it was a bottom type, that's what bot and the bang sign refer to (! is supposed to look like )") and it was decided to call the feature never.

Another issue is that it becomes bad practice to name a variable never. You can do it, just like you can name a variable u32 or char, but it will look ugly when syntax highlighters color it like the other primitives. This will impact existing code that uses never as a name. I guess it won't be a problem for types, other than someone defining their own enum never {} type.

Also, diagnostics would need changes. They mention ! in type errors when never is used.

What now?

First of all, I'm curious what other people think. Does this make sense? Is it to late to make such a change to the never-type? Is it worth the downsides?

If there is interest, should I write a more detailed proposal?

+81 -35

3 comments

19 changed files

Julian-Wollersberger

pr closed time in 24 minutes

pull request commentrust-lang/rust

[Don't merge] Perf test for using closure precise capture

:hourglass: Trying commit 20692cb032c425db94717786f9ca35978e65addf with merge d5c47437deb5a77afed53e0bba2a050fbeaf974c... <!-- homu: {"type":"TryBuildStarted","head_sha":"20692cb032c425db94717786f9ca35978e65addf","merge_sha":"d5c47437deb5a77afed53e0bba2a050fbeaf974c"} -->

arora-aman

comment created time in 26 minutes

pull request commentrust-lang/rust

[Don't merge] Perf test for using closure precise capture

Awaiting bors try build completion

arora-aman

comment created time in 26 minutes

pull request commentrust-lang/rust

[Don't merge] Perf test for using closure precise capture

@bors try @rust-timer queue

arora-aman

comment created time in 26 minutes

Pull request review commentrust-lang/rust

Fix trimming of lint docs

 impl<'a> LintExtractor<'a> {             let (doc, name) = loop {                 match lines.next() {                     Some((lineno, line)) => {-                        let line = line.trim();-                        if let Some(text) = line.strip_prefix("/// ") {-                            doc_lines.push(text.trim().to_string());-                        } else if line.starts_with("///") {+                        if let Some(text) = line.trim_start().strip_prefix("/// ") {+                            doc_lines.push(text.to_string());+                        } else if line.trim_start().starts_with("///") {                             doc_lines.push("".to_string());

No, I think that will cause the same issue that this is trying to fix.

camelid

comment created time in 28 minutes

Pull request review commentrust-lang/rfcs

RFC: Serve crates-io registry over HTTP as static files

 Dependency files compress well. Currently the largest file of `rustc-ap-rustc_da  Even in the worst case of downloading the entire index file by file, it should still use significantly less bandwidth than git clone (individually compressed files currently add up to about 39MiB). +An "incremental changelog" file (described in a later section) can be used to avoid many conditional requests.+ ## Handling deleted crates  When a client checks freshness of a crate that has been deleted, it will make a request to the server and notice a 404/410/451 HTTP status. The client can then act accordingly, and clean up local data (even tarball and source checkout). -If the client is not interested in deleted crate, it won't check it, but chances are it never did, and didn't download it. If ability to proactively erase caches of deleted crates is important, then the "incremental changelog" feature can be extended to notify about deletions.+If the client is not interested in the deleted crate, it won't check it, but chances are it never did, and didn't download it. If ability to proactively erase caches of deleted crates is important, then the "incremental changelog" feature can be extended to notify about deletions.++## Dealing with inconsistent HTTP caches++The index does not require all files to form one cohesive snapshot. The index is updated one file at a time. Every file is updated in a separate commit, so for every file change there exists an index state that is valid with or without it. The index only needs to preserve a partial order of updates.++From Cargo's perspective dependencies are always allowed to update independently. If crate's dependencies' files are refreshed before the crate itself, it won't be different than if someone had used an older version of the crate.++The only case where stale caches can cause a problem is when a new version of a crate depends on the latest version of a newly-published dependency, and caches expired for the parent crate before expiring for the dependency. Cargo will prevent that from happening, at least for the datacenter it can see. Cargo requires dependencies with sufficient versions to be already visible in the index, and won't publish a "broken" crate.++Ideally, the server should ensure that a previous file change is visible everywhere before making the next change, i.e. make the CDN purge the changed file, and wait for the purge to be executed before updating files that may depend on it. This may be difficult to guarantee in a global CDNs, so Cargo needs a recovery mechanism:++If a crate <var>A</var> is found to depend on a crate <var>B</var> with a version that doesn't appear to exist in the index, Cargo should fetch the crate <var>B</var> again with a cache buster. The cache buster can be a query string appended to the URL with either the current timestamp, or timestamp parsed from the `last-modified` header of the crate <var>A</var>'s response: `?cachebust=12345678`.

Do CDNs not obey cache-control headers in requests? That's what browsers normally use when the user Ctrl-F5s.

kornelski

comment created time in 28 minutes

Pull request review commentrust-lang/rust

Fix trimming of lint docs

 impl<'a> LintExtractor<'a> {             let (doc, name) = loop {                 match lines.next() {                     Some((lineno, line)) => {-                        let line = line.trim();-                        if let Some(text) = line.strip_prefix("/// ") {-                            doc_lines.push(text.trim().to_string());-                        } else if line.starts_with("///") {+                        if let Some(text) = line.trim_start().strip_prefix("/// ") {+                            doc_lines.push(text.to_string());+                        } else if line.trim_start().starts_with("///") {                             doc_lines.push("".to_string());

Maybe something like

if let Some(text) = line.trim_start().strip_prefix("///") {
    doc_lines.push(text.trim_start().to_string());
}
camelid

comment created time in 33 minutes

Pull request review commentrust-lang/rust

Fix trimming of lint docs

 impl<'a> LintExtractor<'a> {             let (doc, name) = loop {                 match lines.next() {                     Some((lineno, line)) => {-                        let line = line.trim();-                        if let Some(text) = line.strip_prefix("/// ") {-                            doc_lines.push(text.trim().to_string());-                        } else if line.starts_with("///") {+                        if let Some(text) = line.trim_start().strip_prefix("/// ") {+                            doc_lines.push(text.to_string());+                        } else if line.trim_start().starts_with("///") {                             doc_lines.push("".to_string());

Yeah, it does seem weird. @ehuss: did you do this intentionally or was it a mistake?

camelid

comment created time in 35 minutes

issue commentdiesel-rs/diesel

Add support for `HAVING` clauses

I would like to work on this issue

weiznich

comment created time in an hour

Pull request review commentrust-lang/rfcs

RFC: Serve crates-io registry over HTTP as static files

 Dependency files compress well. Currently the largest file of `rustc-ap-rustc_da  Even in the worst case of downloading the entire index file by file, it should still use significantly less bandwidth than git clone (individually compressed files currently add up to about 39MiB). +An "incremental changelog" file (described in a later section) can be used to avoid many conditional requests.+ ## Handling deleted crates  When a client checks freshness of a crate that has been deleted, it will make a request to the server and notice a 404/410/451 HTTP status. The client can then act accordingly, and clean up local data (even tarball and source checkout). -If the client is not interested in deleted crate, it won't check it, but chances are it never did, and didn't download it. If ability to proactively erase caches of deleted crates is important, then the "incremental changelog" feature can be extended to notify about deletions.+If the client is not interested in the deleted crate, it won't check it, but chances are it never did, and didn't download it. If ability to proactively erase caches of deleted crates is important, then the "incremental changelog" feature can be extended to notify about deletions.++## Dealing with inconsistent HTTP caches++The index does not require all files to form one cohesive snapshot. The index is updated one file at a time. Every file is updated in a separate commit, so for every file change there exists an index state that is valid with or without it. The index only needs to preserve a partial order of updates.++From Cargo's perspective dependencies are always allowed to update independently. If crate's dependencies' files are refreshed before the crate itself, it won't be different than if someone had used an older version of the crate.++The only case where stale caches can cause a problem is when a new version of a crate depends on the latest version of a newly-published dependency, and caches expired for the parent crate before expiring for the dependency. Cargo will prevent that from happening, at least for the datacenter it can see. Cargo requires dependencies with sufficient versions to be already visible in the index, and won't publish a "broken" crate.++Ideally, the server should ensure that a previous file change is visible everywhere before making the next change, i.e. make the CDN purge the changed file, and wait for the purge to be executed before updating files that may depend on it. This may be difficult to guarantee in a global CDNs, so Cargo needs a recovery mechanism:++If a crate <var>A</var> is found to depend on a crate <var>B</var> with a version that doesn't appear to exist in the index, Cargo should fetch the crate <var>B</var> again with a cache buster. The cache buster can be a query string appended to the URL with either the current timestamp, or timestamp parsed from the `last-modified` header of the crate <var>A</var>'s response: `?cachebust=12345678`.

When it comes to protecting the origin server from being overwhelmed with requests from cache misses, there are two distinct scenarios:

  1. Traffic from well-meaning clients that end up requesting too much
  2. Intentional malicious traffic that tries to cause a Denial Of Service

For the case 1, use of timestamps rounded to a second limits the traffic to one cache bust per second per affected crate. The need for cache bust can arise only in rare circumstances and only for a limited time after the crate has been published, so it's basically just 1 request per second on average. It won't take down even most feeble servers. And even that can be further reduced to one extra upstream request in total by using parent crate's last-modified time, so that all clients will request the same URL deterministically, regardless of their clocks.

As for the case 2, I haven't seen any CDN that would try to fight it by not contacting the origin on cache misses. CDNs deal with attacks at earlier stages by identifying abnormal traffic patterns. Layer 7 attacks are relatively expensive for attackers, and easy to block by CDNs. Even if someone did try this, there are way better URLs to attack: search (requires a DB lookup, and you can't give a strict allowlist to the CDN) or POST requests to signup, etc. that a CDN always has to forward.

kornelski

comment created time in an hour

Pull request review commentrust-lang/rust

Fix trimming of lint docs

 impl<'a> LintExtractor<'a> {             let (doc, name) = loop {                 match lines.next() {                     Some((lineno, line)) => {-                        let line = line.trim();-                        if let Some(text) = line.strip_prefix("/// ") {-                            doc_lines.push(text.trim().to_string());-                        } else if line.starts_with("///") {+                        if let Some(text) = line.trim_start().strip_prefix("/// ") {+                            doc_lines.push(text.to_string());+                        } else if line.trim_start().starts_with("///") {                             doc_lines.push("".to_string());

I don't think ///a should push an empty line.

camelid

comment created time in an hour

issue commentrust-lang/rust

Add syntax extension to add warning blocks

So I discussed this a bit with @jyn514 on the side, we agree that the effect on the JSON backend isn't major since HTML is already a subset of markdown and docs use HTML already (especially for images and tables).

My position is as follows:

  • I strictly prefer HTML over some other scoped syntax with nuances people have to learn.
  • I don't think there's as much value in unscoped syntax

which makes my position "HTML or nothing", basically. I'm not wholly against unscoped single-line syntax. We NOTE: and EXAMPLE: syntax in W3C specs and you invariably need to add more stuff to it (there's an HTML class in these specs for this reason).

Furthermore, Rust has a strong culture of wrapping text, so this means we need WARNING: to work well while wrapped, and that means we're again introducing some form of scoping mechanism (e.g. indenting the text), which editors will have to learn, as well.

GuillaumeGomez

comment created time in an hour

pull request commentrust-lang/rust

Add 'for more info' link to 'irrefutable *-let pattern' warning

Ugh, x.py is rebuilding LLVM for some reason 😩

camelid

comment created time in an hour

Pull request review commentrust-lang/rust

Fix trimming of lint docs

 impl<'a> LintExtractor<'a> {             let (doc, name) = loop {                 match lines.next() {                     Some((lineno, line)) => {-                        let line = line.trim();-                        if let Some(text) = line.strip_prefix("/// ") {-                            doc_lines.push(text.trim().to_string());

I'm not sure why this was triming the docs, but this is what was causing the lack of indentation.

camelid

comment created time in an hour

PR opened rust-lang/rust

Fix trimming of lint docs

Fixes #79748.

It was removing all the indentation before.

r? @Mark-Simulacrum

+6 -7

0 comment

1 changed file

pr created time in an hour

pull request commentrust-lang/rfcs

Infallible promotion

Team member @scottmcm has proposed to merge this. The next step is review by the rest of the tagged team members:

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

RalfJung

comment created time in an hour

pull request commentrust-lang/rfcs

Infallible promotion

Thanks for the write-up, Ralf! I knew some of the various pieces here, but between this and its earlier MCP version still learned a bunch -- the details of how the checked operators are promoted was an interesting nuance, for example, and not something I'd seen documented elsewhere. It was great having it all in order to bring everything together.

Since it LGTM (and we asked you to write it), @rfcbot fcp merge

RalfJung

comment created time in an hour

pull request commentrust-lang/rust

Close all tags when rendering rustdoc summaries

(This needs a test, so the PR is draft.)

camelid

comment created time in an hour

PR opened rust-lang/rust

Close all tags when rendering rustdoc summaries

Otherwise we may generate HTML like <em>hello<b> if the text after the <b> tag is too long. Now it should correctly generate <em>hello<b></b></em>. It's perhaps not "ideal" that we generate an empty <b> tag, but it's at least valid HTML.

It isn't a huge deal that we didn't close all the tags because the browser is probably smart enough to figure out where the tags should end, but it's still good to fix :)

r? @GuillaumeGomez cc @jyn514

+6 -3

0 comment

1 changed file

pr created time in an hour

issue commentrust-lang/rust

Padding to ensure a value is well-aligned is not used for niche value optimization

This is more-or-less a duplicate of #70230

jhpratt

comment created time in an hour

issue commentrust-lang/rust

rustc lint listing code snippets don't have any indentation

cc @ehuss since you're the only one who's touched lint-docs since September.

camelid

comment created time in an hour

more