profile
viewpoint
Eduard-Mihai Burtescu eddyb @LykenSol Bucharest, Romania Senior Compiler/PL Engineer

alexcrichton/rustc-demangle 90

Rust symbol demangling

Centril/rfc-effects 8

Preparing an RFC on effect polymorphism in Rust with focus on 'const'

eddyb/cprep 5

C/C++ preprocessor.

eddyb/aoc 2

Advent of Code solutions in Rust

eddyb/bootstrap 2

HTML, CSS, and JS toolkit from Twitter

eddyb/hematite 2

A simple Minecraft written in Rust with the Piston game engine

eddyb/1ml 0

1ML prototype interpreter

eddyb/cargo 0

The Rust package manager

eddyb/clap-rs 0

A full featured, fast Command Line Argument Parser for Rust

pull request commentrust-lang/rust

Allow the last field of sized tuples to participate in field reordering (if it's a ZST)

Although, hmm, we've been trying to guarantee certain things, including offsets of ZST fields, even if it might not be necessary.

With this PR, given a x: &(T, [(); N]), these would be different: x.1.as_ptr() and (x as &(T, [()])).1.as_ptr(). (same thing for dyn Trait, not sure which is more likely to cause issues in practice)

Do we want to let that happen? I recall @RalfJung bringing up before other cases in which we've been too careless with details that unsafe code might want to rely on. That's part of why I've avoided simpler approaches such as just ignoring ZST fields and always e.g. using offset 0 for them, unless repr(C) is used (or worse, using 0x1 as *const () for all &() values).

Anyway, the implementation is fine, as long as we're fine with unsafe code potentially observing a ZST field in a non-repr(C) type having different offsets after going through an unsizing coercion.

r? @nikomatsakis

erikdesjardins

comment created time in an hour

issue commentrust-lang/rust

Tracking issue for RFC 2360, `hint::bench_black_box`

You need to avoid the case when T may have destructors, as transmute_copy will unsafely duplicate the value, but other than that, seems like it could work great, thanks! (we could presumably also have cases for types smaller than usize, or types the size of two usizes, like &[T], as long as we don't introduce extra work beyond forcing the values into registers)

Centril

comment created time in 3 hours

pull request commentrust-lang/rust

Allow the last field of sized tuples to participate in field reordering (if it's a ZST)

Ugh I'm really sorry, I only now got a triage PM about this, I don't recall seeing this PR before (I guess I haven't gotten around to checking the PRs assigned to me in the last 18 days - in general, feel free to PM me on Zulip).

This looks fine, I guess, I'm only worried about the perf loss from sorting twice (which I'm not sure can be avoided).

@bors try @rust-timer queue

erikdesjardins

comment created time in 4 hours

issue commentrust-lang/rust

Tracking issue for RFC 2360, `hint::bench_black_box`

I was seeing some weird effects from using black_box in conjunction with counting instructions, and then I remembered that black_box's current incarnation has to place the value on the stack.

So I tried making my own, and ended up with this, which is sadly limited to values that fit in a register (godbolt):

pub fn better_black_box(mut x: u64) -> u64 {
    unsafe { asm!("/* {x} */", x = inout(reg) x); }
    x
}

From my brief testing, it seems to result in 3 less instructions than std::hint::black_box, at this time, as counted by the CPU (what perf stat calls instructions:u, although I'm using rdpmc directly to get my data).

<hr/>

Might be interesting to see if we could specialize the version in libstd (based on the type) so that it's cheaper, for at least integers, but maybe there's not really any point to that. cc @Amanieu

Centril

comment created time in 13 hours

pull request commentrust-lang/rust

Correctly parse `{} && false` in tail expression

@bors r+

estebank

comment created time in a day

pull request commentrust-lang/rust

Fix ICE #75307 in `format`

@bors r+

estebank

comment created time in a day

Pull request review commentrust-lang/rust

Rewrite the `Visitor` for `non_ssa_locals`

 impl<'mir, 'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>> Visitor<'tcx>         self.super_terminator(terminator, location);     } -    fn visit_place(&mut self, place: &mir::Place<'tcx>, context: PlaceContext, location: Location) {+    fn visit_place(+        &mut self,+        place: &mir::Place<'tcx>,+        mut context: PlaceContext,+        location: Location,+    ) {         debug!("visit_place(place={:?}, context={:?})", place, context);-        self.process_place(&place.as_ref(), context, location);++        // Non-uses do not force locals onto the stack.+        if !context.is_use() {+            return;+        }++        let mir::Place { local, projection } = *place;++        // Reads from ZSTs do not require memory accesses and do not count when determining what+        // needs to live on the stack.+        if is_consume(context) {+            let ty = place.ty(self.fx.mir, self.fx.cx.tcx()).ty;+            let ty = self.fx.monomorphize(&ty);+            let span = self.fx.mir.local_decls[local].source_info.span;+            if self.fx.cx.spanned_layout_of(ty, span).is_zst() {+                return;+            }+        }++        let mut has_disqualifying_projection = false;++        let mut projection: &[_] = projection.as_ref();+        while let [ref proj_base @ .., elem] = *projection {+            projection = proj_base;+            self.super_projection_elem(local, proj_base, elem, context, location);++            // Projections like `(*x)[12]` are allowed but not `*(x[12])`, since a `Deref` of a+            // local acts like a `Copy` of that local.+            if let mir::PlaceElem::Deref = elem {+                has_disqualifying_projection = false;+                context = PlaceContext::NonMutatingUse(NonMutatingUseContext::Copy);+                continue;+            }++            // Ignoring `Deref`s, the only allowed projections are reads of scalar fields.+            if is_consume(context) && matches!(elem, mir::ProjectionElem::Field(..)) {+                let base_ty = mir::Place::ty_from(local, proj_base, self.fx.mir, self.fx.cx.tcx());+                let base_ty = self.fx.monomorphize(&base_ty);+                let span = self.fx.mir.local_decls[local].source_info.span;+                let layout = self.fx.cx.spanned_layout_of(base_ty.ty, span);++                if !ty_requires_alloca(self.fx, layout) {

Hmm, I feel like this doesn't capture the important aspect here: only for some layouts we can extract their fields as SSA values. So maybe instead of !ty_requires_alloca it could be layout_allows_ssa (i.e. with flipped meaning).

ecstatic-morse

comment created time in 2 days

Pull request review commentrust-lang/rust

Rewrite the `Visitor` for `non_ssa_locals`

 impl<'mir, 'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>> Visitor<'tcx>         self.super_terminator(terminator, location);     } -    fn visit_place(&mut self, place: &mir::Place<'tcx>, context: PlaceContext, location: Location) {+    fn visit_place(+        &mut self,+        place: &mir::Place<'tcx>,+        mut context: PlaceContext,+        location: Location,+    ) {         debug!("visit_place(place={:?}, context={:?})", place, context);-        self.process_place(&place.as_ref(), context, location);++        // Non-uses do not force locals onto the stack.+        if !context.is_use() {+            return;+        }++        let mir::Place { local, projection } = *place;++        // Reads from ZSTs do not require memory accesses and do not count when determining what+        // needs to live on the stack.+        if is_consume(context) {+            let ty = place.ty(self.fx.mir, self.fx.cx.tcx()).ty;+            let ty = self.fx.monomorphize(&ty);+            let span = self.fx.mir.local_decls[local].source_info.span;+            if self.fx.cx.spanned_layout_of(ty, span).is_zst() {+                return;+            }+        }++        let mut has_disqualifying_projection = false;++        let mut projection: &[_] = projection.as_ref();+        while let [ref proj_base @ .., elem] = *projection {+            projection = proj_base;+            self.super_projection_elem(local, proj_base, elem, context, location);++            // Projections like `(*x)[12]` are allowed but not `*(x[12])`, since a `Deref` of a+            // local acts like a `Copy` of that local.+            if let mir::PlaceElem::Deref = elem {+                has_disqualifying_projection = false;+                context = PlaceContext::NonMutatingUse(NonMutatingUseContext::Copy);+                continue;+            }++            // Ignoring `Deref`s, the only allowed projections are reads of scalar fields.+            if is_consume(context) && matches!(elem, mir::ProjectionElem::Field(..)) {+                let base_ty = mir::Place::ty_from(local, proj_base, self.fx.mir, self.fx.cx.tcx());+                let base_ty = self.fx.monomorphize(&base_ty);+                let span = self.fx.mir.local_decls[local].source_info.span;+                let layout = self.fx.cx.spanned_layout_of(base_ty.ty, span);++                if !ty_requires_alloca(self.fx, layout) {+                    continue;+                }+            }++            has_disqualifying_projection = true;+        }

Looking more at this loop, I would split it into several parts:

  • a loop to just call super_projection_elem uniformly on all the elems, to get it out of the way (it only serves to visit the index local of an Index projection, with a `Copy context)
  • find the first Deref, if there is one, e.g. with .position(|elem| matches!(elem, mir::PlaceElem::Deref))
  • loop over the "direct projections" (i.e. before any Deref, if any) and do self.not_ssa(local); break; at the first sign of trouble

I mostly want this for clarity, but that last part has some promise for making this more efficient, so here's a few more ideas:

  • context wouldn't change across it, meaning if is_consume(context) can be placed around the whole loop
  • there could be a quick check that all the elements are fields, to avoid time even looking at the layouts
  • we probably should check all the field projections (like today / this PR) but if we go forward, from the local to the innermost field, we should almost always hit the non-SSA layout first, making the rejection cheaper
  • we could call visit_local before the loop, and skip the loop if non_ssa_locals already contains the local (see also the comment at the top of this file, which would make this more useful)
  • if we keep the monomorphized TyAndLayout of the local (which we always have to compute anyway) around, we could start from it and just go layout = layout.field(self.fx, i) at every step instead of computing the whole type and monomorphizing every time
ecstatic-morse

comment created time in 2 days

Pull request review commentrust-lang/rust

Rewrite the `Visitor` for `non_ssa_locals`

 use super::FunctionCx; use crate::traits::*; use rustc_data_structures::graph::dominators::Dominators; use rustc_index::bit_set::BitSet;-use rustc_index::vec::{Idx, IndexVec};+use rustc_index::vec::IndexVec; use rustc_middle::mir::traversal;-use rustc_middle::mir::visit::{-    MutatingUseContext, NonMutatingUseContext, NonUseContext, PlaceContext, Visitor,-};+use rustc_middle::mir::visit::{MutatingUseContext, NonMutatingUseContext, PlaceContext, Visitor}; use rustc_middle::mir::{self, Location, TerminatorKind}; use rustc_middle::ty;-use rustc_middle::ty::layout::HasTyCtxt;+use rustc_middle::ty::layout::{HasTyCtxt, TyAndLayout}; use rustc_target::abi::LayoutOf;  pub fn non_ssa_locals<'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>>(     fx: &FunctionCx<'a, 'tcx, Bx>, ) -> BitSet<mir::Local> {+    trace!("non_ssa_locals({:?})", fx.instance.def_id());+     let mir = fx.mir;     let mut analyzer = LocalAnalyzer::new(fx); -    analyzer.visit_body(&mir);+    for (block, data) in traversal::reverse_postorder(mir) {+        analyzer.visit_basic_block_data(block, data);+    }      for (local, decl) in mir.local_decls.iter_enumerated() {         let ty = fx.monomorphize(&decl.ty);         debug!("local {:?} has type `{}`", local, ty);         let layout = fx.cx.spanned_layout_of(ty, decl.source_info.span);-        if fx.cx.is_backend_immediate(layout) {-            // These sorts of types are immediates that we can store-            // in an Value without an alloca.-        } else if fx.cx.is_backend_scalar_pair(layout) {-            // We allow pairs and uses of any of their 2 fields.-        } else {-            // These sorts of types require an alloca. Note that-            // is_llvm_immediate() may *still* be true, particularly-            // for newtypes, but we currently force some types-            // (e.g., structs) into an alloca unconditionally, just so-            // that we don't have to deal with having two pathways-            // (gep vs extractvalue etc).++        if ty_requires_alloca(&analyzer.fx, layout) {             analyzer.not_ssa(local);         }

Actually, huh, we have access to FunctionCx, I'm tempted to say we should create the FunctionCx with all of the local TyAndLayouts already computed (although we couldn't have placeholder LocalRefs easily I don't think?) - but maybe that's too much for this PR?

ecstatic-morse

comment created time in 2 days

Pull request review commentrust-lang/rust

Rewrite the `Visitor` for `non_ssa_locals`

 use super::FunctionCx; use crate::traits::*; use rustc_data_structures::graph::dominators::Dominators; use rustc_index::bit_set::BitSet;-use rustc_index::vec::{Idx, IndexVec};+use rustc_index::vec::IndexVec; use rustc_middle::mir::traversal;-use rustc_middle::mir::visit::{-    MutatingUseContext, NonMutatingUseContext, NonUseContext, PlaceContext, Visitor,-};+use rustc_middle::mir::visit::{MutatingUseContext, NonMutatingUseContext, PlaceContext, Visitor}; use rustc_middle::mir::{self, Location, TerminatorKind}; use rustc_middle::ty;-use rustc_middle::ty::layout::HasTyCtxt;+use rustc_middle::ty::layout::{HasTyCtxt, TyAndLayout}; use rustc_target::abi::LayoutOf;  pub fn non_ssa_locals<'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>>(     fx: &FunctionCx<'a, 'tcx, Bx>, ) -> BitSet<mir::Local> {+    trace!("non_ssa_locals({:?})", fx.instance.def_id());+     let mir = fx.mir;     let mut analyzer = LocalAnalyzer::new(fx); -    analyzer.visit_body(&mir);+    for (block, data) in traversal::reverse_postorder(mir) {+        analyzer.visit_basic_block_data(block, data);+    }      for (local, decl) in mir.local_decls.iter_enumerated() {         let ty = fx.monomorphize(&decl.ty);         debug!("local {:?} has type `{}`", local, ty);         let layout = fx.cx.spanned_layout_of(ty, decl.source_info.span);-        if fx.cx.is_backend_immediate(layout) {-            // These sorts of types are immediates that we can store-            // in an Value without an alloca.-        } else if fx.cx.is_backend_scalar_pair(layout) {-            // We allow pairs and uses of any of their 2 fields.-        } else {-            // These sorts of types require an alloca. Note that-            // is_llvm_immediate() may *still* be true, particularly-            // for newtypes, but we currently force some types-            // (e.g., structs) into an alloca unconditionally, just so-            // that we don't have to deal with having two pathways-            // (gep vs extractvalue etc).++        if ty_requires_alloca(&analyzer.fx, layout) {             analyzer.not_ssa(local);         }

Maybe we should do this first, and even keep the monomorphized TyAndLayouts around, to make it cheaper to check if "consume" field projections can be handled in an SSA way.

ecstatic-morse

comment created time in 2 days

Pull request review commentrust-lang/rust

Rewrite the `Visitor` for `non_ssa_locals`

 impl<'mir, 'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>> Visitor<'tcx>         self.super_terminator(terminator, location);     } -    fn visit_place(&mut self, place: &mir::Place<'tcx>, context: PlaceContext, location: Location) {+    fn visit_place(+        &mut self,+        place: &mir::Place<'tcx>,+        mut context: PlaceContext,+        location: Location,+    ) {         debug!("visit_place(place={:?}, context={:?})", place, context);-        self.process_place(&place.as_ref(), context, location);++        // Non-uses do not force locals onto the stack.+        if !context.is_use() {+            return;+        }++        let mir::Place { local, projection } = *place;++        // Reads from ZSTs do not require memory accesses and do not count when determining what+        // needs to live on the stack.+        if is_consume(context) {+            let ty = place.ty(self.fx.mir, self.fx.cx.tcx()).ty;+            let ty = self.fx.monomorphize(&ty);+            let span = self.fx.mir.local_decls[local].source_info.span;+            if self.fx.cx.spanned_layout_of(ty, span).is_zst() {+                return;+            }+        }++        let mut has_disqualifying_projection = false;++        let mut projection: &[_] = projection.as_ref();+        while let [ref proj_base @ .., elem] = *projection {+            projection = proj_base;+            self.super_projection_elem(local, proj_base, elem, context, location);++            // Projections like `(*x)[12]` are allowed but not `*(x[12])`, since a `Deref` of a+            // local acts like a `Copy` of that local.+            if let mir::PlaceElem::Deref = elem {+                has_disqualifying_projection = false;+                context = PlaceContext::NonMutatingUse(NonMutatingUseContext::Copy);+                continue;+            }++            // Ignoring `Deref`s, the only allowed projections are reads of scalar fields.

s/allowed/SSA-capable (or similar phrasing) would help IMO.

ecstatic-morse

comment created time in 2 days

Pull request review commentrust-lang/rust

Rewrite the `Visitor` for `non_ssa_locals`

 impl<'mir, 'a, 'tcx, Bx: BuilderMethods<'a, 'tcx>> Visitor<'tcx>         self.super_terminator(terminator, location);     } -    fn visit_place(&mut self, place: &mir::Place<'tcx>, context: PlaceContext, location: Location) {+    fn visit_place(+        &mut self,+        place: &mir::Place<'tcx>,+        mut context: PlaceContext,+        location: Location,+    ) {         debug!("visit_place(place={:?}, context={:?})", place, context);-        self.process_place(&place.as_ref(), context, location);++        // Non-uses do not force locals onto the stack.+        if !context.is_use() {+            return;+        }++        let mir::Place { local, projection } = *place;++        // Reads from ZSTs do not require memory accesses and do not count when determining what+        // needs to live on the stack.+        if is_consume(context) {+            let ty = place.ty(self.fx.mir, self.fx.cx.tcx()).ty;+            let ty = self.fx.monomorphize(&ty);+            let span = self.fx.mir.local_decls[local].source_info.span;+            if self.fx.cx.spanned_layout_of(ty, span).is_zst() {+                return;+            }+        }++        let mut has_disqualifying_projection = false;++        let mut projection: &[_] = projection.as_ref();+        while let [ref proj_base @ .., elem] = *projection {+            projection = proj_base;+            self.super_projection_elem(local, proj_base, elem, context, location);++            // Projections like `(*x)[12]` are allowed but not `*(x[12])`, since a `Deref` of a+            // local acts like a `Copy` of that local.

This is a bit iffy in that it doesn't mention pointers at all, which is what is being "copied", and the arbitrary example of indexing had me confused for a while, although I see what it's saying now.

I would frankly just say that a Deref is a pointer/reference Copy and be done with it.

ecstatic-morse

comment created time in 2 days

pull request commentrust-lang/rust

use type folder + normalization for MIR assignment type-checking

As per https://github.com/rust-lang/rust/issues/75313#issuecomment-672216146, we likely shouldn't need to land this, once we fix normalization of ty::Opaque.

RalfJung

comment created time in 3 days

issue commentrust-lang/rust

Panic when compiling gluon on 1.46

@RalfJung for the record, I specifically meant after removing all the bound lifetimes. miri already normalizes, so it would have to be stronger than that.

<hr/>

However, I believe I may have found the bug, and it's actually in normalization itself: https://github.com/rust-lang/rust/blob/cbe7c5ce705896d4e22bf6096590bc1f17993b78/src/librustc_trait_selection/traits/project.rs#L327

That if !substs.has_escaping_bound_vars() condition shouldn't be there - IIRC I copied it from the ty::Projection arm.

While for associated type projections it makes sense, due to having to look up the impl (which may have different results depending on how the lifetimes are bound, or be unsound, I guess), it does not for ty::Opaque, which behaves more like a type alias, and which should be completely sound to normalize under a binder.

cc @nikomatsakis

nukeop

comment created time in 3 days

issue commentrust-lang/rust

Panic when compiling gluon on 1.46

Are you normalizing? The reveal_all aspect is not going to be used unless you actually normalize. You might need to do the hacky excessive erasure of lifetimes before even attempting to normalize as the bound lifetimes may block normalization.

(so the bottom-up folder could erase lifetimes then normalize, at every single step)

nukeop

comment created time in 3 days

pull request commentrust-lang/rust

shim: monomorphic `FnPtrShim`s during construction

We've made this work (I hope), but given my involvement I'd like someone else to take a look as well. We could also use a test where polymorphization breaks today without this change.

r? @nikomatsakis

davidtwco

comment created time in 5 days

issue commentrust-lang/rust

Symbol ambiguity with polymorphization and new mangling scheme

Polymorphisation correctly determines that I and E are unused and that T is used in the closure (T is used by the argument).

Incredible, I wish I thought of this when the polymorphization work was ongoing, I guess I was overfocusing on shims.

So the problem is T is an alias for I::Item, not an "independent variable", the only reason it appears to work in other situations is the trait system is feeding you the value of T. Probably the simplest hackaround is to mangle trait impl "parameters".

But it seems more prudent to consider any parameter not named in the TraitRef as depending on all parameters that do appear in the TraitRef (or you can try to do something more clever by looking at the where clauses).

davidtwco

comment created time in 5 days

pull request commentrust-lang/rust

instance: only polymorphize upvar substs

@bors r+

davidtwco

comment created time in 5 days

Pull request review commentrust-lang/rust

instance: only polymorphize upvar substs

 fn polymorphize<'tcx>(     let unused = tcx.unused_generic_params(def_id);     debug!("polymorphize: unused={:?}", unused); +    // If this is a closure or generator then we need to handle the case where another closure+    // from the function is captured as an upvar and hasn't been polymorphized. In this case,+    // the unpolymorphized upvar closure would result in a polymorphized closure producing+    // multiple mono items (and eventually symbol clashes).

Wondering if we should move back to not tracking that information in the ty::Closure at all. cc @nikomatsakis

davidtwco

comment created time in 5 days

issue commentrust-lang/rust

Panic when compiling gluon on nightly

It looks like the bound regions are preventing us from normalizing an opaque type, leading us to conclude that the LHS and RHS types are not actually equal.

@RalfJung You're missing the impl Iterator from the original. Here's a short repro:

fn iter_slice<'a, T>(xs: &'a [T]) -> impl Iterator<Item = &'a T> {
    xs.iter()
}

fn main() {
    iter_slice::<()> as fn(_) -> _;
}

<hr/>

For the record, the thing of note about primitive!(2, std::regex::prim::captures) is that it casts (twice, even, presumably accounting for the two errors), the function to primitive_cast!(2) (which is just fn(_, _) -> _)

The function is gluon's std::regex::prim::captures, with the signature:

fn captures<'a>(
    re: &Regex,
    text: &'a str,
) -> Option<Collect<impl Iterator<Item = Option<Match<'a>>>>>
nukeop

comment created time in 5 days

Pull request review commentrust-lang/rust

Emit == null instead of <= null for niche check

 impl<'a, 'tcx, V: CodegenObject> PlaceRef<'tcx, V> {                 };                 let relative_max = niche_variants.end().as_u32() - niche_variants.start().as_u32();                 let is_niche = {-                    let relative_max = if relative_max == 0 {+                    if relative_max == 0 {

You can remove the extra block, it only existed for the large let relative_max = ...;.

nikic

comment created time in 7 days

pull request commentrust-lang/rust

rustc_metadata: track the simplified Self type for every trait impl.

That's somewhat expected, it's hard to tell what the actual wins are from relative measurements, because the delta is absolute across the board (not exactly a startup cost, but very similar for any one given trait).

I mostly went on the basis of @Mark-Simulacrum finding out that a lot of the cost was in trait_impls_of -> type_of. So this was potentially more than half of the regression (in absolute terms), but it's far from making extra dependencies free.

eddyb

comment created time in 8 days

delete branch eddyb/rust

delete branch : rmeta-indexed-trait-impls

delete time in 9 days

Pull request review commentrust-lang/rust

rustc_metadata: track the simplified Self type for every trait impl.

 pub(super) fn all_local_trait_impls<'tcx>( pub(super) fn trait_impls_of_provider(tcx: TyCtxt<'_>, trait_id: DefId) -> TraitImpls {     let mut impls = TraitImpls::default(); -    {-        let mut add_impl = |impl_def_id: DefId| {-            let impl_self_ty = tcx.type_of(impl_def_id);

Right, I think all the cost is in decoding the Ty from type_of (simplification should be relatively cheap).

And based on the detailed query profiling info, I'm guessing it's ty::Adt containing a &'tcx ty::AdtDef instead of just a DefId, and somehow item_attrs getting called for every struct/enum/union implementing the trait.

I'm suspecting this is similar to what @nnethercote was talking about a few months ago, where the deserialization of doc comment attributes was a significant cost (I don't recall what happened with that line of investigation though).

eddyb

comment created time in 11 days

pull request commentrust-lang/rust

rustc_metadata: track the simplified Self type for every trait impl.

I'm very suspicious of the decrease in item_attrs getting decoded. I wonder if we could avoid that and maybe get some of the same wins? Although the biggest delta seems to be in metadata_decode_entry and I don't really know what that measures.

eddyb

comment created time in 13 days

pull request commentrust-lang/rust

rustc_metadata: track the simplified Self type for every trait impl.

@bors try @rust-timer queue

eddyb

comment created time in 13 days

PR opened rust-lang/rust

rustc_metadata: track the simplified Self type for every trait impl.

For the traits_impls_of query, we index the impls by fast_reject::SimplifiedType (a "shallow type"), which allows some simple cases like impl Trait<..> for Foo<..> to be efficiently iterated over, by e.g. for_each_relevant_impl.

This PR encodes the fast_reject::SimplifiedType cross-crate to avoid needing to deserialize the Self type of every impl in order to simplify it - the simplification itself should be cheap, but the deserialization is less so.

We could go further from here and make loading the list of impls lazy, for a given simplified Self type, but that would have more complicated implications for performance, and this PR doesn't do anything in that regard.

r? @nikomatsakis cc @Mark-Simulacrum

+54 -37

0 comment

6 changed files

pr created time in 13 days

create barncheddyb/rust

branch : rmeta-indexed-trait-impls

created branch time in 13 days

pull request commentrust-lang/rust

Rename HAIR to THIR (Typed HIR).

@bors r=nikomatsakis rollup=never p=1 (somewhat prone to conflict)

Lezzz

comment created time in 13 days

pull request commentrust-lang/rust

symbol mangling: use ty::print::Print for consts

@bors r+

lcnr

comment created time in 16 days

pull request commentrust-lang/rust

Fix #[track_caller] shims for trait objects.

@bors r+ rollup=never (assuming we want fixes like these to land separately, for ease of bisecting etc.)

anp

comment created time in 19 days

fork eddyb/fish-shell

The user-friendly command line shell.

https://fishshell.com

fork in 22 days

pull request commentrust-lang/rust

polymorphize GlobalAlloc::Function

@bors r+ rollup=never (I know it's small, but we want to be able to bisect anything polymorphization-related)

lcnr

comment created time in 22 days

issue commentrust-lang/rustup

Error on update leading to rollback

Nightlies are built once a day - well, night :).

More accurately, around midnight GMT/UTC (not exactly sure what timezone) the last PR that landed is promoted to "nightly". (Effectively every time we merge a PR we build a "nightly" from it - rustup-toolchain-install-master lets you install any PR build, although I believe we delete old ones after a number of days. you could use it to test that the bug is fixed, maybe?)

So you have to wait another 13h-14h or so before a new nightly is published.

conikeec

comment created time in 23 days

issue commentrust-lang/rust

Regression in nightly-2020-07-22: collection encountered polymorphic constant

Huh the code that emits the ICE is kind of broken in terms of user expectations: https://github.com/rust-lang/rust/blob/4825e12fc9c79954aa0fe18f5521efa6c19c7539/src/librustc_mir/monomorphize/collector.rs#L625-L630

It found a constant vtable::<T, S> (or maybe vtable::<T, S>::promoted0) somewhere but the Span it uses isn't that of the use site (self.body.source_info(location).span), but rather the definition site of the constant (fn vtable).

IMO it should print the constant (as part of the message), but use the use site as the ICE site. Printing the ty::Const and not the DefId will ensure that the generic parameters are shown, which should help with debugging bugs like this.

dtolnay

comment created time in 23 days

pull request commentrust-lang/rustc-dev-guide

[WIP] Fix and clean up the terminology for stages

Name rustbuild instead of saying 'the bootstrap binary'

sigh We should rename src/bootstrap, I remember discussing it with @Mark-Simulacrum but I forget what we thought might work well.

Use 'stage0 rustc' and 'stage0 compiler' to mean stage0-rustc

I think "stageN rustc", and even more so "stageN compiler" is ambiguous and ideally we would never use it in documentation. It could mean one of two things:

  • stageN/bin/rustc input i.e. "compiler used to compile stageN-*"
  • stageN-rustc output i.e. "compiler built at stage N"

Use 'bootstrap' to mean build/$target/stage0 (i.e. the beta compiler)

This probably is fine, i.e. referring to "bootstrap compiler" and explaining it's one of these:

  • latest beta for building nightly
  • latest stable for building beta
  • previous stable for building stable

Mention that 'ignore-stage1' means 'ignore programs linked to stage1', not built by it.

Isn't it simpler to say "tests ran at stage 1" , meaning as part of x.py test --stage 1?

<hr/>

We never use "built by stage N" to mean "compiled by stageN-rustc/release/rustc (aka stageN+1/bin/rustc)" and I'd much rather prefer disambiguating than jumping from one side of an ambiguity to the other (in this case, to the side that's even less commonly used/understood).

jyn514

comment created time in 24 days

pull request commentrust-lang/rust

Revert "Compare tagged/niche-filling layout and pick the best one"

Oh wow that's definitely much larger than just computing both layouts was (then again I didn't use black_box so maybe LLVM made it seem cheaper than it really is).

@bors r+

Mark-Simulacrum

comment created time in 24 days

pull request commentrust-lang/rust

remove some const arg in ty dep path boilerplate

@bors r+

lcnr

comment created time in 24 days

pull request commentrust-lang/rust

remove some const arg in ty dep path boilerplate

@bors rollup=never (just in case there's a perf impact, even if small)

lcnr

comment created time in 24 days

Pull request review commentrust-lang/rust

remove some const arg in ty dep path boilerplate

 pub struct WithOptConstParam<T> {  impl<T> WithOptConstParam<T> {     /// Creates a new `WithOptConstParam` setting `const_param_did` to `None`.+    #[inline(always)]     pub fn unknown(did: T) -> WithOptConstParam<T> {         WithOptConstParam { did, const_param_did: None }     } }  impl WithOptConstParam<LocalDefId> {+    /// Returns `Some((did, param_did))` if `def_id` is a const argument,+    /// `None` otherwise.+    #[inline(always)]+    pub fn try_fetch(did: LocalDefId, tcx: TyCtxt<'_>) -> Option<(LocalDefId, DefId)> {+        tcx.opt_const_param_of(did).map(|param_did| (did, param_did))+    }++    /// In case `self` is unknown but `self.did` is a const argument, this returns+    /// a `WithOptConstParam` with the correct `const_param_did`.+    #[inline(always)]+    pub fn try_upgrade(self, tcx: TyCtxt<'_>) -> Option<WithOptConstParam<LocalDefId>> {

Hmm, so if we wanted this name to be more explicit, it could be try_upgrade_to_const_arg, right? Then the other one could be try_lookup_as_const_arg.

Alternatively, try_upgrade_with_const_param and try_lookup_const_param_for.

lcnr

comment created time in 24 days

Pull request review commentrust-lang/rust

std: Don't emit debuginfo for backtrace deps

 overflow-checks = false # per-crate configuration isn't specifiable in the environment. codegen-units = 10000 +# These dependencies of the standard library implement symbolication for+# backtraces on most platforms. Their debuginfo causes both linking to be slower+# (more data to chew through) and binaries to be larger without really all that+# much benefit. This section turns them all to down to have no debuginfo which+# helps to improve link times a little bit.+[profile.release.package.miniz_oxide]+debug = 0+[profile.release.package.gimli]+debug = 0+[profile.release.package.addr2line]+debug = 0+[profile.release.package.object]+debug = 0+[profile.release.package.adler]+debug = 0

Just out of curiosity, does TOML have the Nix thing of letting you write gimli = { debug = 0 } as gimli.debug = 0? IMO using the dot notation would feel more natural, but then again maybe that's personal bias.

alexcrichton

comment created time in 24 days

pull request commentrust-lang/rust

Rollup of 18 pull requests

Oh and because of the revert was tested, we can measure the impact of this rollup without the big linker regression: https://perf.rust-lang.org/compare.html?start=39d5a61f2e4e237123837f5162cc275c2fd7e625&end=556b0eb41b4273f0dcde2cbc2bc9f53e2c09d688

Manishearth

comment created time in 24 days

pull request commentrust-lang/rust

Rollup of 18 pull requests

Update on the perf: the regression was (mostly?) in the linker, from #74416, which #74478 reverted.

Manishearth

comment created time in 25 days

pull request commentrust-lang/rust

Use LocalDefId instead of HirId for reachable_set elements.

Somehow this ended up without an assignee?

r? @nikomatsakis

eddyb

comment created time in 25 days

issue closedmagiclen/enum-ordinalize

Consider not transmuting mem::Discriminant at all.

The current version of this crate relies on transmuting mem::Discriminant<T> values: https://github.com/magiclen/enum-ordinalize/blob/80b7469d113212be67fb0616639ddea0b9cadfc9/src/lib.rs#L423-L445

But it's clear from the code that this pathway was kept despite it having broken at least once. Why was it kept, for performance?

IIRC (from conversations with @nox, I believe), LLVM can optimize identity switches, you just have to get the values right (we added the ability to specify explicit discriminants in enums with data, to allow "lining up" two enums for that optimization).

Please consider not using unsafe code to get around standard library stability, or at least putting it behind a nightly-only feature (which could be made to only compile on nightly by use of #![cfg_attr(feature = "nightly", feature(...))]).

cc @hoverbear (who stumbled over an older version of this crate being broken, we assume)

closed time in 25 days

eddyb

issue commentmagiclen/enum-ordinalize

Consider not transmuting mem::Discriminant at all.

Thanks! I'm closing this issue, since this seems resolved now.

eddyb

comment created time in 25 days

pull request commentrust-lang/rust

std: Test not emitting debuginfo for backtrace deps

@rust-timer queue

alexcrichton

comment created time in 25 days

issue commentrust-lang/rust

Consider merging `values` and `targets` fields on MIR `switchInt`?

@oli-obk The problem is that they're returned, so you'd need something like a many-cased enum of different iterators. It could work, it was just cleaner to use Cow<[_]> I think.

wesleywiser

comment created time in 25 days

pull request commentrust-lang/rust

std: Switch from libbacktrace to gimli

@oli-obk Ignore the colors (although execution counts may matter in some cases), look at the "Time" column, most of what I looked at had run_linker at the top, with a large factor between it and the next thing.

LLVM would not be affected, yeah, since all of the code is already compiled into libstd. We should be able to see a change in the size of the rust-std component, in the first nightly that included this PR.

alexcrichton

comment created time in 25 days

pull request commentrust-lang/rust

std: Switch from libbacktrace to gimli

Could that be from linking all that code into the final executable? Seems like the main thing that would affect incremental runs too.

alexcrichton

comment created time in 25 days

pull request commentrust-lang/rust

test caching opt_const_param_of on disc

@bors r+ rollup

lcnr

comment created time in a month

pull request commentrust-lang/rust

small coherence cleanup

r? @nikomatsakis (sorry, not sure what the plans are in this area)

lcnr

comment created time in a month

issue commentmagiclen/enum-ordinalize

Consider not transmuting mem::Discriminant at all.

Just confirmed that this example:

pub enum Foo {
    A,
    B(i32),
    C(String),
    D([u8; 200])
}

pub fn discr(foo: &Foo) -> isize {
    match foo {
        Foo::A => 0,
        Foo::B(_) => 1,
        Foo::C(_) => 2,
        Foo::D(_) => 3,
    }
}

compiles to:

  %0 = getelementptr inbounds %Foo, %Foo* %foo, i64 0, i32 0, i64 0
  %1 = load i8, i8* %0, align 8, !range !2
  %_2 = zext i8 %1 to i64
  ret i64 %_2

Even if I change the 2 to a 5, LLVM ends up using a [i64 0, i64 1, i64 5, i64 3] "switch table" instead of a switch.

Maybe when the transmute was originally written this optimization wasn't available?

eddyb

comment created time in a month

issue openedmagiclen/enum-ordinalize

Consider not transmuting mem::Discriminant at all.

The current version of this crate relies on transmuting mem::Discriminant<T> values: https://github.com/magiclen/enum-ordinalize/blob/80b7469d113212be67fb0616639ddea0b9cadfc9/src/lib.rs#L423-L445

But it's clear from the code that this pathway was kept despite it having broken at least once. Why was it kept, for performance?

IIRC (from conversations with @nox, I believe), LLVM can optimize identity switches, you just have to get the values right (we added the ability to specify explicit discriminants in enums with data, to allow "lining up" two enums for that optimization).

Please consider not using unsafe code to get around standard library stability, or at least putting it behind a nightly-only feature (which could be made to only compile on nightly by use of #![cfg_attr(feature = "nightly", feature(...))]).

cc @hoverbear (who stumbled over an older version of this crate being broken, we assume)

created time in a month

pull request commentrust-lang/rust

Support `#[track_caller]` on closures.

Sounds like you need to make some of the Fn-trait-related shims pretend to have #[track_caller] depending on whether the underlying function/closure does. I didn't think about that when I suggested closures "should just work" but:

You might be able to get away with testing a case where a closure doesn't implement more than FnOnce (e.g. capture something like a String that is entirely consumed by the closure body), because that shouldn't get any shims.

Then again, you mention coercing to a fn pointer being the only case that doesn't work, I'm not sure what the best solution for that is, maybe checking that we're in a reify shim and doing Instance::resolve on the DefId/Substs pair of the reify shim? That way, we may get back an InstanceDef::Item referring to the closure, or a shim that makes it clear what the closure DefId is?

anp

comment created time in a month

pull request commentrust-lang/rust

Compute `query::Providers` almost entirely at compile-time.

@nikomatsakis Would be interesting if we had a quick way of measuring the performance impact of that (I guess we can do the grunt work of writing Box::new(...) around each call site - or maybe the box operator works?).

I'm not as worried about anything happening before TyCtxt is constructed, even if it can affect startup costs. But I expect vtable calls to be more expensive (and would be pleasantly surprised to learn that I am wrong).

Also, we kind of want to keep providers as pure as possible, but using Fn() wouldn't necessarily change much.

eddyb

comment created time in a month

Pull request review commentrust-lang/rust

Don't keep {Closure,Generator}Substs synthetics in an Instance.

 pub(super) fn type_of(tcx: TyCtxt<'_>, def_id: DefId) -> Ty<'_> {          Node::Field(field) => icx.to_ty(&field.ty), -        Node::Expr(&Expr { kind: ExprKind::Closure(.., gen), .. }) => {-            let substs = InternalSubsts::identity_for_item(tcx, def_id);-            if let Some(movability) = gen {-                tcx.mk_generator(def_id, substs, movability)-            } else {-                tcx.mk_closure(def_id, substs)-            }+        Node::Expr(&Expr { kind: ExprKind::Closure(..), .. }) => {+            tcx.typeck_tables_of(def_id.expect_local()).node_type(hir_id)

Ah, ok, I see, the point is that you want Instance to be able to substitute only the base_params and to still get a valid type... hmm. Interesting. I'm not sure what I think about it.

Me neither, I think I'd prefer if we could implement e.g. fn_sig on closures instead, and use that from Instance. I could look into that approach once #74314 lands.

I'm surprised this gives much of a win, can you explain why it does?

All the wins are incremental, so it probably only makes sense if a closure changes its capture set/types, although that would still imply codegen is needed?

Or maybe non-generic closures are involved, which would now result in an Instance with empty substs, which may be handled differently by CGUs.

eddyb

comment created time in a month

pull request commentrust-lang/rust

rustc_typeck: construct {Closure,Generator}Substs more directly.

@nikomatsakis Yeah, let's land polymorphization first, it's slightly cleaner than this PR anyway.

eddyb

comment created time in a month

issue commentrust-lang/rustc-perf

Update `script-servo` benchmark

The script-servo benchmark is about 3 years old, and recently a change took place that reduced the amount of generated LLVM IR from 10.2M lines to 7.2M lines, by avoiding bloat for one crucial function: servo/servo#26585

With that in mind it might be a good time to retire script-servo and introduce script-servo-2. It's not one of the "stable" benchmarks so there should be no problem with this.

Why would we change the benchmark? That means we'll no longer be measuring anything that reduces that bloat, such as MIR optimizations or (as it turned out to be the case), polymorphization.

I thought we kept benchmarks as unchanged as we could on perf.rust-lang.org so the data has meaning across time.

nnethercote

comment created time in a month

delete branch eddyb/rust

delete branch : zlib-on-nixos

delete time in a month

Pull request review commentrust-lang/rust

Add lazy initialization primitives to std

+//! Lazy values and one-time initialization of static data.++use crate::cell::{Cell, UnsafeCell};+use crate::fmt;+use crate::mem;+use crate::ops::Deref;++/// A cell which can be written to only once.+///+/// Unlike `RefCell`, a `OnceCell` only provides shared `&T` references to its value.+/// Unlike `Cell`, a `OnceCell` doesn't require copying or replacing the value to access it.+///+/// # Examples+///+/// ```+/// #![feature(once_cell)]+///+/// use std::lazy::OnceCell;+///+/// let cell = OnceCell::new();+/// assert!(cell.get().is_none());+///+/// let value: &String = cell.get_or_init(|| {+///     "Hello, World!".to_string()+/// });+/// assert_eq!(value, "Hello, World!");+/// assert!(cell.get().is_some());+/// ```+#[unstable(feature = "once_cell", issue = "74465")]+pub struct OnceCell<T> {+    // Invariant: written to at most once.+    inner: UnsafeCell<Option<T>>,+}

It didn't help that GitHub hides the std::sync version by default, it was clearer once I found it.

KodrAus

comment created time in a month

pull request commentrust-lang/rust

Rollup of 11 pull requests

Wait, that's quickcheck, so it's possible it's a buggy test. cc @BurntSushi

@bors retry

Manishearth

comment created time in a month

Pull request review commentrust-lang/rust

Add lazy initialization primitives to std

+//! Lazy values and one-time initialization of static data.++use crate::cell::{Cell, UnsafeCell};+use crate::fmt;+use crate::mem;+use crate::ops::Deref;++/// A cell which can be written to only once.+///+/// Unlike `RefCell`, a `OnceCell` only provides shared `&T` references to its value.+/// Unlike `Cell`, a `OnceCell` doesn't require copying or replacing the value to access it.+///+/// # Examples+///+/// ```+/// #![feature(once_cell)]+///+/// use std::lazy::OnceCell;+///+/// let cell = OnceCell::new();+/// assert!(cell.get().is_none());+///+/// let value: &String = cell.get_or_init(|| {+///     "Hello, World!".to_string()+/// });+/// assert_eq!(value, "Hello, World!");+/// assert!(cell.get().is_some());+/// ```+#[unstable(feature = "once_cell", issue = "74465")]+pub struct OnceCell<T> {+    // Invariant: written to at most once.+    inner: UnsafeCell<Option<T>>,+}

Do we rely on implicit !Sync from UnsafeCell, or do we try to be explicit? cc @RalfJung

(Sorry if this was asked already, I just saw this PR and thought it was neat, and wasn't sure if this OnceCell was meant to be thread-safe or not)

KodrAus

comment created time in a month

pull request commentrust-lang/rust

Compare tagged/niche-filling layout and pick the best one

Thanks a lot! I've closed my original unfinished PR.

erikdesjardins

comment created time in a month

PR closed rust-lang/rust

[experiment] ty/layout: compute both niche-filling and tagged layouts for enums. S-waiting-on-review

This is something I've wanted to do for a while, since if we can get away with it perf-wise, it should allow us to always pick the better of the two, even when the conditions would be otherwise subtle.

+8 -4

14 comments

1 changed file

eddyb

pr closed time in a month

pull request commentrust-lang/rust

[experiment] ty/layout: compute both niche-filling and tagged layouts for enums.

Thanks to @erikdesjardins, this was finished as #74069.

eddyb

comment created time in a month

pull request commentrust-lang/rust

Rollup of 11 pull requests

thread 'test_frequency::prop_frequency_indexed' panicked at '[quickcheck] TEST FAILED (runtime error). Arguments: (CsvData { data: [[[]], [[]], [[239, 187, 191]], [[]], [[]]] })
Error: "assertion failed: `(left == right)`\n  left: `[(\"(NULL)\", 4)]`,\n right: `[(\"(NULL)\", 3), (\"\\u{feff}\", 1)]`"', D:\a\rust\rust\src\libstd\macros.rs:13:23


failures:
    test_frequency::prop_frequency_indexed

This doesn't look spurious.

Manishearth

comment created time in a month

pull request commentrust-lang/rust

Rollup of 11 pull requests


Run src/ci/scripts/install-awscli.sh
Reading package lists...
Building dependency tree...
Reading state information...
Package python3-setuptools is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

@bors retry

Manishearth

comment created time in a month

Pull request review commentrust-lang/rust

Fully destructure constants into patterns

 impl<'a, 'tcx> ConstToPat<'a, 'tcx> {                 PatKind::Wild             }             // keep old code until future-compat upgraded to errors.-            ty::Adt(adt_def, _) if !self.type_marked_structural(cv.ty) => {-                debug!("adt_def {:?} has !type_marked_structural for cv.ty: {:?}", adt_def, cv.ty);-                let path = tcx.def_path_str(adt_def.did);-                let msg = format!(-                    "to use a constant of type `{}` in a pattern, \-                     `{}` must be annotated with `#[derive(PartialEq, Eq)]`",-                    path, path,-                );-                self.saw_const_match_error.set(true);-                tcx.sess.span_err(span, &msg);-                PatKind::Wild-            }-            // keep old code until future-compat upgraded to errors.-            ty::Ref(_, adt_ty @ ty::TyS { kind: ty::Adt(_, _), .. }, _)-                if !self.type_marked_structural(adt_ty) =>+            ty::Adt(adt_def, _)+                if !self.type_marked_structural(cv.ty) && !self.backcompat_hack.get() =>             {-                let adt_def =-                    if let ty::Adt(adt_def, _) = adt_ty.kind { adt_def } else { unreachable!() };--                debug!(-                    "adt_def {:?} has !type_marked_structural for adt_ty: {:?}",-                    adt_def, adt_ty-                );--                // HACK(estebank): Side-step ICE #53708, but anything other than erroring here-                // would be wrong. Returnging `PatKind::Wild` is not technically correct.+                debug!("adt_def {:?} has !type_marked_structural for cv.ty: {:?}", adt_def, cv.ty);

The diff here looks very weird. I'm guessing you removed the entire ty::Ref(_, adt_ty @ ty::TyS { kind: ty::Adt(_, _), .. }, _) arm but git has a hard time spotting that?

Can you keep the old one-line ty::Adt(adt_def, _) if !self.type_marked_structural(cv.ty) pattern and have a if self.backcompat_hack.get() { return ...; } inside?

That feels like it should fix the diff, and be more cohesive.

oli-obk

comment created time in a month

Pull request review commentrust-lang/rust

Fully destructure constants into patterns

 struct ConstToPat<'a, 'tcx> {     // value.     saw_const_match_error: Cell<bool>, +    // For backcompat we need to keep allowing non-structurally-eq types behind references.+    // See also all the `cant-hide-behind` tests.+    backcompat_hack: Cell<bool>,

This needs a much more descriptive name, perhaps including behind_reference?

oli-obk

comment created time in a month

Pull request review commentrust-lang/rust

Fully destructure constants into patterns

 impl<'a, 'tcx> Builder<'a, 'tcx> {          let deref_ty = match ty.kind {             ty::Ref(_, deref_ty, _) => deref_ty,-            _ => bug!("non_scalar_compare called on non-reference type: {}", ty),

Isn't this expecting a reference because the caller of non_scalar_compare borrowed the values being compared?

cc @nikomatsakis @matthewjasper I'm unclear on what the intended use of non_scalar_compare is.

oli-obk

comment created time in a month

Pull request review commentrust-lang/rust

Fully destructure constants into patterns

 pub(crate) fn destructure_const<'tcx>(      mir::DestructuredConst { variant, fields } }++pub(crate) fn deref_const<'tcx>(+    tcx: TyCtxt<'tcx>,+    param_env: ty::ParamEnv<'tcx>,+    val: &'tcx ty::Const<'tcx>,+) -> &'tcx ty::Const<'tcx> {+    trace!("deref_const: {:?}", val);+    let ecx = mk_eval_cx(tcx, DUMMY_SP, param_env, false);+    let op = ecx.eval_const_to_op(val, None).unwrap();+    let mplace = ecx.deref_operand(op).unwrap();++    let ty = match mplace.meta {+        MemPlaceMeta::None => mplace.layout.ty,+        MemPlaceMeta::Poison => bug!("poison metadata in `deref_const`: {:#?}", mplace),+        // In case of unsized types, figure out the real type behind.+        MemPlaceMeta::Meta(scalar) => match mplace.layout.ty.kind {+            ty::Dynamic(..) => ecx.read_drop_type_from_vtable(scalar).unwrap().1,+            ty::Str => bug!("there's no sized equivalent of a `str`"),+            ty::Slice(elem_ty) => tcx.mk_array(elem_ty, scalar.to_machine_usize(&tcx).unwrap()),+            _ => bug!(+                "type {} should not have metadata, but had {:?}",+                mplace.layout.ty,+                mplace.meta+            ),

I don't see this using struct_tail, so it presumably will fail on Newtype<[T]>?

oli-obk

comment created time in a month

Pull request review commentrust-lang/rust

Fully destructure constants into patterns

 pub(crate) fn destructure_const<'tcx>(      mir::DestructuredConst { variant, fields } }++pub(crate) fn deref_const<'tcx>(+    tcx: TyCtxt<'tcx>,+    param_env: ty::ParamEnv<'tcx>,+    val: &'tcx ty::Const<'tcx>,+) -> &'tcx ty::Const<'tcx> {+    trace!("deref_const: {:?}", val);+    let ecx = mk_eval_cx(tcx, DUMMY_SP, param_env, false);+    let op = ecx.eval_const_to_op(val, None).unwrap();+    let mplace = ecx.deref_operand(op).unwrap();++    let ty = match mplace.meta {+        MemPlaceMeta::None => mplace.layout.ty,+        MemPlaceMeta::Poison => bug!("poison metadata in `deref_const`: {:#?}", mplace),+        // In case of unsized types, figure out the real type behind.+        MemPlaceMeta::Meta(scalar) => match mplace.layout.ty.kind {+            ty::Dynamic(..) => ecx.read_drop_type_from_vtable(scalar).unwrap().1,

I really don't want us to be doing this, it's magically extracting a lifetime-erased version of the original type from a vtable, something that is otherwise impossible.

Note that the lifetime-erased aspect might interact poorly with borrow-checking, but I'm not sure how much of a problem that is in practice. (And I guess destructure_const has the same problem, ugh - maybe we need to get the field types from original type itself, and not rely on the TyAndLayout?)

IMO deref_const should produce an unsized ty::Const instead of trying to divine a sized type.

If we don't want ty::Const to be able to represent unsized values, we can offer a different interface (at least for slices).

oli-obk

comment created time in a month

pull request commentrust-lang/rust

Rollup of 11 pull requests

@bors retry

Manishearth

comment created time in a month

pull request commentrust-lang/rust

Use pathdiff crate

I've tested it with:

./x.py build --stage 1 --keep-stage 1 src/librustc_codegen_ssa

For the record, if you just want to make sure it compiles, ./x.py check src/librustc_codegen_ssa (maybe even without the path, I don't know if it makes much of a difference), should be much faster.

But you can also replace --stage 1 --keep-stage 1 with --stage 0, I am not aware of any benefits to building the compiler at stage 1 by using --keep-stage, only pain and suffering additional complexity.

FedericoPonzi

comment created time in a month

Pull request review commentrust-lang/rust

diagnostics: shorten paths of unique symbols

 mod y {      #[rustc_clean(label="typeck_tables_of", cfg="cfail2")]     pub fn y() {-        //[cfail2]~^ ERROR `typeck_tables_of(y::y)` should be clean but is not+        //[cfail2]~^ ERROR `typeck_tables_of(y)` should be clean but is not         x::x();     } }  mod z {     #[rustc_dirty(label="typeck_tables_of", cfg="cfail2")]     pub fn z() {-        //[cfail2]~^ ERROR `typeck_tables_of(z::z)` should be dirty but is not+        //[cfail2]~^ ERROR `typeck_tables_of(z)` should be dirty but is not

These don't seem unambiguous, how does this happen? Maybe we should ignore namespaces, to be conservative? (Although presumably this has been discussed already)

da-x

comment created time in a month

Pull request review commentrust-lang/rust

diagnostics: shorten paths of unique symbols

 impl<'a, 'tcx> FnCtxt<'a, 'tcx> {                    on the left and may require reallocation. This \                    requires ownership of the string on the left"; -        let is_std_string = |ty| &format!("{:?}", ty) == "std::string::String";+        let is_std_string =+            |ty| with_crate_prefix(|| &format!("{:?}", ty) == "std::string::String");

Should be diagnostic item.

da-x

comment created time in a month

pull request commentjimblandy/perf-event

Add support for sampling

@jimblandy To expand a bit: I plan to use rdpmc to avoid incurring the cost of a syscall, and the remaining costs I want to golf have to do with the few memory reads from the perf mmap page, around the rdpmc instruction (search for "rdpmc" in the perf_event_open manpage for an example).

Oh and I guess because it's inline assembly, this low-overhead approach would be nightly-only for a while.

bobbobbio

comment created time in a month

issue commentrust-random/rand

"Excessive TLS usage", but take this title with a grain of salt

Maybe that's not a concern if THREAD_RNG_KEY doesn't actually have a destructor, but it's still scary and not documented.

AFAIK accessing TLS storage during TLS dtor runs is not a problem, so when there is no destructor this should be fine.

But I guess you are saying that if this got boxed, there would be a destructor, and thus a problem?

Specifically I'm thinking of accessing TLS storage on which drop_in_place had executed (this gets even worse if e.g. thread_local! adds an Option around the data, and turns it into None).

If it were boxed, the accesses would refer an already-freed heap allocation instead, which is clearly UAF.

fasterthanlime

comment created time in a month

issue commentrust-lang/rust

#[track_caller] on closures?

Although... the example is a bit suspect, it doesn't specify what the expected behavior is, but the call of the closure in the with method is what you'll get as the assert! failure location, if #[track_caller] works correctly on closures.

If you want to propagate all the way to main, that's hard. You want to defer errors until you can panic outside the closure, so that you're not inside the with call.

We could also try to make a way to override the "caller location" of a call (and pass in an outer one), but the ergonomics of that are worse for this example (where assert!(DONE.with(|b| b)) "just works").

anp

comment created time in a month

issue commentrust-lang/rust

#[track_caller] on closures?

I would expect this to "just work", if the attribute syntax parses.

anp

comment created time in a month

issue commentrust-lang/rust

Rustdoc link with module@ does not link to the module

"Modules count as all three" sounds wrong? Modules live in the type namespace last I checked.

dtolnay

comment created time in a month

Pull request review commentrust-lang/rust

lint: use `transparent_newtype_field` to avoid ICE

 impl<'a, 'tcx> ImproperCTypesVisitor<'a, 'tcx> {         match ty.kind {             ty::FnPtr(_) => true,             ty::Ref(..) => true,-            ty::Adt(field_def, substs) if field_def.repr.transparent() && !field_def.is_union() => {-                for field in field_def.all_fields() {-                    let field_ty = self.cx.tcx.normalize_erasing_regions(-                        self.cx.param_env,-                        field.ty(self.cx.tcx, substs),-                    );-                    if field_ty.is_zst(self.cx.tcx, field.did) {

I don't think Ty should have methods that interact with layout, feels like an abstraction violation. If something relies on layout that should be clear (i.e. is_zst callers should use layout_of themselves - not to mention that e.g. LateLintContext should have better layout_of integration than doing it on Ty)

davidtwco

comment created time in a month

Pull request review commentrust-lang/rust

bootstrap.py: patch RPATH on NixOS to handle the new zlib dependency.

 def fix_executable(fname):         nix_os_msg = "info: you seem to be running NixOS. Attempting to patch"         print(nix_os_msg, fname) -        try:-            interpreter = subprocess.check_output(-                ["patchelf", "--print-interpreter", fname])-            interpreter = interpreter.strip().decode(default_encoding)-        except subprocess.CalledProcessError as reason:-            print("warning: failed to call patchelf:", reason)-            return--        loader = interpreter.split("/")[-1]--        try:-            ldd_output = subprocess.check_output(-                ['ldd', '/run/current-system/sw/bin/sh'])-            ldd_output = ldd_output.strip().decode(default_encoding)-        except subprocess.CalledProcessError as reason:-            print("warning: unable to call ldd:", reason)-            return--        for line in ldd_output.splitlines():-            libname = line.split()[0]-            if libname.endswith(loader):-                loader_path = libname[:len(libname) - len(loader)]-                break+        # Only build `stage0/.nix-deps` once.+        nix_deps_dir = self.nix_deps_dir+        if not nix_deps_dir:+            nix_deps_dir = "{}/.nix-deps".format(self.bin_root())+            if not os.path.exists(nix_deps_dir):+                os.makedirs(nix_deps_dir)++            nix_deps = [+                # Needed for the path of `ld-linux.so` (via `nix-support/dynamic-linker`).+                "stdenv.cc.bintools",++                # Needed as a system dependency of `libLLVM-*.so`.+                "zlib",++                # Needed for patching ELF binaries (see doc comment above).+                "patchelf",+            ]++            # Run `nix-build` to "build" each dependency (which will likely reuse+            # the existing `/nix/store` copy, or at most download a pre-built copy).+            # Importantly, we don't rely on `nix-build` printing the `/nix/store`+            # path on stdout, but use `-o` to symlink it into `stage0/.nix-deps/$dep`,+            # ensuring garbage collection will never remove the `/nix/store` path+            # (which would break our patched binaries that hardcode those paths).+            for dep in nix_deps:+                try:+                    subprocess.check_output([+                        "nix-build", "<nixpkgs>",+                        "-A", dep,+                        "-o", "{}/{}".format(nix_deps_dir, dep),+                    ])+                except subprocess.CalledProcessError as reason:+                    print("warning: failed to call nix-build:", reason)+                    return++            self.nix_deps_dir = nix_deps_dir++        patchelf = "{}/patchelf/bin/patchelf".format(nix_deps_dir)++        if fname.endswith(".so"):+            # Dynamic library, patch RPATH to point to system dependencies.+            dylib_deps = ["zlib"]+            rpath_entries = [

For the record, NixOS rustup also hardcodes $ORIGIN/../lib:, so I think we're fine.

eddyb

comment created time in a month

Pull request review commentrust-lang/rust

remove some const arg in ty dep path boilerplate

 pub struct WithOptConstParam<T> {  impl<T> WithOptConstParam<T> {     /// Creates a new `WithOptConstParam` setting `const_param_did` to `None`.+    #[inline(always)]     pub fn unknown(did: T) -> WithOptConstParam<T> {         WithOptConstParam { did, const_param_did: None }     } }  impl WithOptConstParam<LocalDefId> {+    /// Returns `Some((did, param_did))` if `def_id` is a const argument,+    /// `None` otherwise.+    #[inline(always)]+    pub fn try_fetch(did: LocalDefId, tcx: TyCtxt<'_>) -> Option<(LocalDefId, DefId)> {

I think it's clearer if you start with unknown and then use the other method. Bonus: the other case likely takes the unknown anyway, so it would only be created once.

lcnr

comment created time in a month

Pull request review commentrust-lang/rust

remove some const arg in ty dep path boilerplate

 pub struct WithOptConstParam<T> {  impl<T> WithOptConstParam<T> {     /// Creates a new `WithOptConstParam` setting `const_param_did` to `None`.+    #[inline(always)]     pub fn unknown(did: T) -> WithOptConstParam<T> {         WithOptConstParam { did, const_param_did: None }     } }  impl WithOptConstParam<LocalDefId> {+    /// Returns `Some((did, param_did))` if `def_id` is a const argument,+    /// `None` otherwise.+    #[inline(always)]+    pub fn try_fetch(did: LocalDefId, tcx: TyCtxt<'_>) -> Option<(LocalDefId, DefId)> {+        tcx.opt_const_param_of(did).map(|param_did| (did, param_did))+    }++    /// In case `self` is unknown but `self.did` is a const argument, this returns+    /// a `WithOptConstParam` with the correct `const_param_did`.+    #[inline(always)]+    pub fn try_update(self, tcx: TyCtxt<'_>) -> Option<WithOptConstParam<LocalDefId>> {

Note that my suggestion used "upgrade", in the sense of "promote", not "update".

lcnr

comment created time in a month

pull request commentrust-lang/rust

Rename TypeckTables to TypeckResults.

Looks like a network error?

@bors retry

Lezzz

comment created time in a month

Pull request review commentrust-lang/rust

bootstrap.py: patch RPATH on NixOS to handle the new zlib dependency.

 def fix_executable(fname):         nix_os_msg = "info: you seem to be running NixOS. Attempting to patch"         print(nix_os_msg, fname) -        try:-            interpreter = subprocess.check_output(-                ["patchelf", "--print-interpreter", fname])-            interpreter = interpreter.strip().decode(default_encoding)-        except subprocess.CalledProcessError as reason:-            print("warning: failed to call patchelf:", reason)-            return--        loader = interpreter.split("/")[-1]--        try:-            ldd_output = subprocess.check_output(-                ['ldd', '/run/current-system/sw/bin/sh'])-            ldd_output = ldd_output.strip().decode(default_encoding)-        except subprocess.CalledProcessError as reason:-            print("warning: unable to call ldd:", reason)-            return--        for line in ldd_output.splitlines():-            libname = line.split()[0]-            if libname.endswith(loader):-                loader_path = libname[:len(libname) - len(loader)]-                break+        # Only build `stage0/.nix-deps` once.+        nix_deps_dir = self.nix_deps_dir+        if not nix_deps_dir:+            nix_deps_dir = "{}/.nix-deps".format(self.bin_root())+            if not os.path.exists(nix_deps_dir):+                os.makedirs(nix_deps_dir)++            nix_deps = [+                # Needed for the path of `ld-linux.so` (via `nix-support/dynamic-linker`).+                "stdenv.cc.bintools",++                # Needed as a system dependency of `libLLVM-*.so`.+                "zlib",++                # Needed for patching ELF binaries (see doc comment above).+                "patchelf",+            ]++            # Run `nix-build` to "build" each dependency (which will likely reuse+            # the existing `/nix/store` copy, or at most download a pre-built copy).+            # Importantly, we don't rely on `nix-build` printing the `/nix/store`+            # path on stdout, but use `-o` to symlink it into `stage0/.nix-deps/$dep`,+            # ensuring garbage collection will never remove the `/nix/store` path+            # (which would break our patched binaries that hardcode those paths).+            for dep in nix_deps:+                try:+                    subprocess.check_output([+                        "nix-build", "<nixpkgs>",+                        "-A", dep,

Not sure how to do that and keep the symlinks.

eddyb

comment created time in a month

Pull request review commentrust-lang/rust

bootstrap.py: patch RPATH on NixOS to handle the new zlib dependency.

 def fix_executable(fname):         nix_os_msg = "info: you seem to be running NixOS. Attempting to patch"         print(nix_os_msg, fname) -        try:-            interpreter = subprocess.check_output(-                ["patchelf", "--print-interpreter", fname])-            interpreter = interpreter.strip().decode(default_encoding)-        except subprocess.CalledProcessError as reason:-            print("warning: failed to call patchelf:", reason)-            return--        loader = interpreter.split("/")[-1]--        try:-            ldd_output = subprocess.check_output(-                ['ldd', '/run/current-system/sw/bin/sh'])-            ldd_output = ldd_output.strip().decode(default_encoding)-        except subprocess.CalledProcessError as reason:-            print("warning: unable to call ldd:", reason)-            return--        for line in ldd_output.splitlines():-            libname = line.split()[0]-            if libname.endswith(loader):-                loader_path = libname[:len(libname) - len(loader)]-                break+        # Only build `stage0/.nix-deps` once.+        nix_deps_dir = self.nix_deps_dir+        if not nix_deps_dir:+            nix_deps_dir = "{}/.nix-deps".format(self.bin_root())+            if not os.path.exists(nix_deps_dir):+                os.makedirs(nix_deps_dir)++            nix_deps = [+                # Needed for the path of `ld-linux.so` (via `nix-support/dynamic-linker`).+                "stdenv.cc.bintools",++                # Needed as a system dependency of `libLLVM-*.so`.+                "zlib",++                # Needed for patching ELF binaries (see doc comment above).+                "patchelf",+            ]++            # Run `nix-build` to "build" each dependency (which will likely reuse+            # the existing `/nix/store` copy, or at most download a pre-built copy).+            # Importantly, we don't rely on `nix-build` printing the `/nix/store`+            # path on stdout, but use `-o` to symlink it into `stage0/.nix-deps/$dep`,+            # ensuring garbage collection will never remove the `/nix/store` path+            # (which would break our patched binaries that hardcode those paths).+            for dep in nix_deps:+                try:+                    subprocess.check_output([+                        "nix-build", "<nixpkgs>",+                        "-A", dep,+                        "-o", "{}/{}".format(nix_deps_dir, dep),+                    ])+                except subprocess.CalledProcessError as reason:+                    print("warning: failed to call nix-build:", reason)+                    return++            self.nix_deps_dir = nix_deps_dir++        patchelf = "{}/patchelf/bin/patchelf".format(nix_deps_dir)++        if fname.endswith(".so"):+            # Dynamic library, patch RPATH to point to system dependencies.+            dylib_deps = ["zlib"]+            rpath_entries = [

That would require a command to read it, which... I can do it if you want, it's just uglier (and I've gotten used to hardcoding the $ORIGIN/../lib part in my patchelf commands).

eddyb

comment created time in a month

pull request commentrust-lang/rust

Polymorphization

@bors r+ rollup=never

davidtwco

comment created time in a month

Pull request review commentrust-lang/rust

Polymorphization

 fn word_index_and_mask<T: Idx>(elem: T) -> (usize, Word) {     let mask = 1 << (elem % WORD_BITS);     (word_index, mask) }++/// Integral type used to represent the bit set.+pub trait FiniteBitSetTy:+    BitAnd<Output = Self>+    + BitAndAssign+    + BitOrAssign+    + Clone+    + Copy+    + Shl+    + Not<Output = Self>+    + PartialEq+    + Sized+{+    /// Size of the domain representable by this type, e.g. 64 for `u64`.+    const DOMAIN_SIZE: u32;++    /// Value which represents the `FiniteBitSet` having every bit set.+    const FILLED: Self;+    /// Value which represents the `FiniteBitSet` having no bits set.+    const EMPTY: Self;++    /// Value for one as the integral type.+    const ONE: Self;+    /// Value for zero as the integral type.+    const ZERO: Self;++    /// Perform a checked left shift on the integral type.+    fn checked_shl(self, rhs: u32) -> Option<Self>;+    /// Perform a checked right shift on the integral type.+    fn checked_shr(self, rhs: u32) -> Option<Self>;+}++impl FiniteBitSetTy for u64 {+    const DOMAIN_SIZE: u32 = 64;++    const FILLED: Self = Self::MAX;+    const EMPTY: Self = Self::MIN;++    const ONE: Self = 1u64;+    const ZERO: Self = 0u64;++    fn checked_shl(self, rhs: u32) -> Option<Self> {+        self.checked_shl(rhs)+    }++    fn checked_shr(self, rhs: u32) -> Option<Self> {+        self.checked_shr(rhs)+    }+}++impl std::fmt::Debug for FiniteBitSet<u64> {+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {+        write!(f, "{:064b}", self.0)+    }+}++impl FiniteBitSetTy for u128 {+    const DOMAIN_SIZE: u32 = 128;++    const FILLED: Self = Self::MAX;+    const EMPTY: Self = Self::MIN;++    const ONE: Self = 1u128;+    const ZERO: Self = 0u128;++    fn checked_shl(self, rhs: u32) -> Option<Self> {+        self.checked_shl(rhs)+    }++    fn checked_shr(self, rhs: u32) -> Option<Self> {+        self.checked_shr(rhs)+    }+}++impl std::fmt::Debug for FiniteBitSet<u128> {+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {+        write!(f, "{:0128b}", self.0)+    }+}++/// A fixed-sized bitset type represented by an integer type. Indices outwith than the range+/// representable by `T` are considered set.+#[derive(Copy, Clone, Eq, PartialEq, RustcDecodable, RustcEncodable)]+pub struct FiniteBitSet<T: FiniteBitSetTy>(pub T);++impl<T: FiniteBitSetTy> FiniteBitSet<T> {+    /// Creates a new, empty bitset.+    pub fn new_empty() -> Self {+        Self(T::EMPTY)+    }++    /// Sets the `index`th bit.+    pub fn set(&mut self, index: u32) {+        self.0 |= T::ONE.checked_shl(index).unwrap_or(T::ZERO);+    }++    /// Unsets the `index`th bit.+    pub fn clear(&mut self, index: u32) {+        self.0 &= !T::ONE.checked_shl(index).unwrap_or(T::ZERO);+    }++    /// Sets the `i`th to `j`th bits.+    pub fn set_range(&mut self, range: Range<u32>) {+        assert!(range.end <= T::DOMAIN_SIZE);+        let bits = T::FILLED+            .checked_shl(range.end - range.start)

Oh I missed the .not(). Disregard.

davidtwco

comment created time in a month

Pull request review commentrust-lang/rust

Polymorphization

 fn word_index_and_mask<T: Idx>(elem: T) -> (usize, Word) {     let mask = 1 << (elem % WORD_BITS);     (word_index, mask) }++/// Integral type used to represent the bit set.+pub trait FiniteBitSetTy:+    BitAnd<Output = Self>+    + BitAndAssign+    + BitOrAssign+    + Clone+    + Copy+    + Shl+    + Not<Output = Self>+    + PartialEq+    + Sized+{+    /// Size of the domain representable by this type, e.g. 64 for `u64`.+    const DOMAIN_SIZE: u32;++    /// Value which represents the `FiniteBitSet` having every bit set.+    const FILLED: Self;+    /// Value which represents the `FiniteBitSet` having no bits set.+    const EMPTY: Self;++    /// Value for one as the integral type.+    const ONE: Self;+    /// Value for zero as the integral type.+    const ZERO: Self;++    /// Perform a checked left shift on the integral type.+    fn checked_shl(self, rhs: u32) -> Option<Self>;+    /// Perform a checked right shift on the integral type.+    fn checked_shr(self, rhs: u32) -> Option<Self>;+}++impl FiniteBitSetTy for u64 {+    const DOMAIN_SIZE: u32 = 64;++    const FILLED: Self = Self::MAX;+    const EMPTY: Self = Self::MIN;++    const ONE: Self = 1u64;+    const ZERO: Self = 0u64;++    fn checked_shl(self, rhs: u32) -> Option<Self> {+        self.checked_shl(rhs)+    }++    fn checked_shr(self, rhs: u32) -> Option<Self> {+        self.checked_shr(rhs)+    }+}++impl std::fmt::Debug for FiniteBitSet<u64> {+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {+        write!(f, "{:064b}", self.0)+    }+}++impl FiniteBitSetTy for u128 {+    const DOMAIN_SIZE: u32 = 128;++    const FILLED: Self = Self::MAX;+    const EMPTY: Self = Self::MIN;++    const ONE: Self = 1u128;+    const ZERO: Self = 0u128;++    fn checked_shl(self, rhs: u32) -> Option<Self> {+        self.checked_shl(rhs)+    }++    fn checked_shr(self, rhs: u32) -> Option<Self> {+        self.checked_shr(rhs)+    }+}++impl std::fmt::Debug for FiniteBitSet<u128> {+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {+        write!(f, "{:0128b}", self.0)+    }+}++/// A fixed-sized bitset type represented by an integer type. Indices outwith than the range+/// representable by `T` are considered set.+#[derive(Copy, Clone, Eq, PartialEq, RustcDecodable, RustcEncodable)]+pub struct FiniteBitSet<T: FiniteBitSetTy>(pub T);++impl<T: FiniteBitSetTy> FiniteBitSet<T> {+    /// Creates a new, empty bitset.+    pub fn new_empty() -> Self {+        Self(T::EMPTY)+    }++    /// Sets the `index`th bit.+    pub fn set(&mut self, index: u32) {+        self.0 |= T::ONE.checked_shl(index).unwrap_or(T::ZERO);+    }++    /// Unsets the `index`th bit.+    pub fn clear(&mut self, index: u32) {+        self.0 &= !T::ONE.checked_shl(index).unwrap_or(T::ZERO);+    }++    /// Sets the `i`th to `j`th bits.+    pub fn set_range(&mut self, range: Range<u32>) {+        assert!(range.end <= T::DOMAIN_SIZE);+        let bits = T::FILLED+            .checked_shl(range.end - range.start)

Doesn't this result in range.end - range.start trailing 0s? I don't think that's the intent.

davidtwco

comment created time in a month

Pull request review commentrust-lang/rust

Polymorphization

 define_tables! {     super_predicates: Table<DefIndex, Lazy!(ty::GenericPredicates<'tcx>)>,     mir: Table<DefIndex, Lazy!(mir::Body<'tcx>)>,     promoted_mir: Table<DefIndex, Lazy!(IndexVec<mir::Promoted, mir::Body<'tcx>>)>,+    unused_generic_params: Table<DefIndex, Lazy<FiniteBitSet<u64>>>,

For posterity: image

Resolving this as I think it's a clientside rendering bug on my side.

davidtwco

comment created time in a month

PR opened rust-lang/rust

bootstrap.py: patch RPATH on NixOS to handle the new zlib dependency.

This is a stop-gap until #74420 is resolved (assuming we'll patch beta to statically link zlib).

However, I've been meaning to rewrite the NixOS support we have in bootstrap.py for a while now, and had to in order to cleanly add zlib as a dependency (the second commit is a relatively small delta in functionality, compared to the first).

Previously, we would extract the ld-linux.so path from the output of ldd /run/current-system/sw/bin/sh, which assumes a lot. On top of that we didn't use any symlinks, which meant if the user ran GC (nix-collect-garbage), e.g. after updating their system, their stage0 binaries would suddenly be broken (i.e. referring to files that no longer exist). We were also using patchelf directly, assuming it can be found in $PATH (which is not necessarily true).

My new approach relies on using nix-build to get the following "derivations" (packages, more or less):

  • stdenv.cc.bintools, which has a nix-support/dynamic-linker file containing the path to ld-linux.so
  • patchelf (so that the user doesn't need to have it installed)
  • zlib, for the libz.so dependency of libLLVM-*.so (until #74420 is resolved, presumably)

This is closer to how software is built on Nix, but I've tried to keep it as simple as possible (and not add e.g. a stage0.nix file). Symlinks to each of those dependencies are kept in stage0/.nix-deps, which prevents GC from invalidating stage0 binaries.

r? @nagisa cc @Mark-Simulacrum @oli-obk @davidtwco

+66 -38

0 comment

1 changed file

pr created time in a month

create barncheddyb/rust

branch : zlib-on-nixos

created branch time in a month

pull request commentrust-lang/rust

compiletest: Rewrite extract_*_version functions

r? @Mark-Simulacrum

lzutao

comment created time in a month

Pull request review commentrust-lang/rust

Polymorphization

+//! Polymorphization Analysis+//! =========================+//!+//! This module implements an analysis of functions, methods and closures to determine which+//! generic parameters are unused (and eventually, in what ways generic parameters are used - only+//! for their size, offset of a field, etc.).++use rustc_hir::{def::DefKind, def_id::DefId};+use rustc_index::bit_set::FiniteBitSet;+use rustc_middle::mir::{+    visit::{TyContext, Visitor},+    Local, LocalDecl, Location,+};+use rustc_middle::ty::{+    self,+    fold::{TypeFoldable, TypeVisitor},+    query::Providers,+    Const, Ty, TyCtxt,+};+use rustc_span::symbol::sym;+use std::convert::TryInto;++/// Provide implementations of queries relating to polymorphization analysis.+pub fn provide(providers: &mut Providers) {+    providers.unused_generic_params = unused_generic_params;+}++/// Determine which generic parameters are used by the function/method/closure represented by+/// `def_id`. Returns a bitset where bits representing unused parameters are set (`is_empty`+/// indicates all parameters are used).+fn unused_generic_params(tcx: TyCtxt<'_>, def_id: DefId) -> FiniteBitSet<u64> {+    debug!("unused_generic_params({:?})", def_id);++    if !tcx.sess.opts.debugging_opts.polymorphize {+        // If polymorphization disabled, then all parameters are used.+        return FiniteBitSet::new_empty();+    }++    let generics = tcx.generics_of(def_id);+    debug!("unused_generic_params: generics={:?}", generics);++    // Exit early when there are no parameters to be unused.+    if generics.count() == 0 {+        return FiniteBitSet::new_empty();+    }++    // Exit early when there is no MIR available.+    if !tcx.is_mir_available(def_id) {+        debug!("unused_generic_params: (no mir available) def_id={:?}", def_id);+        return FiniteBitSet::new_empty();+    }++    // Create a bitset with N rightmost zeros for each parameter.+    let generics_count: u32 =+        generics.count().try_into().expect("more generic parameters than can fit into a `u32`");+    let mut used_parameters = FiniteBitSet::<u64>::new_filled().shl(generics_count);

This is the only use of new_filled and shl. A better way to handle this would be:

  • start empty
  • have a set_range method and pass 0..generics_count to that
    • this results in the "unused parameters" set
  • flip it to get "used parameters"

You might be able to avoid flipping by keeping track of "unused parameters" instead (with 0..generics_count set initially) and clearing the appropriate bit when encountering a use of the parameter.

I believe this will also remove the need for a flip operation.

davidtwco

comment created time in a month

Pull request review commentrust-lang/rust

Polymorphization

+//! Polymorphization Analysis+//! =========================+//!+//! This module implements an analysis of functions, methods and closures to determine which+//! generic parameters are unused (and eventually, in what ways generic parameters are used - only+//! for their size, offset of a field, etc.).++use rustc_hir::{def::DefKind, def_id::DefId};+use rustc_index::bit_set::FiniteBitSet;+use rustc_middle::mir::{+    visit::{TyContext, Visitor},+    Local, LocalDecl, Location,+};+use rustc_middle::ty::{+    self,+    fold::{TypeFoldable, TypeVisitor},+    query::Providers,+    Const, Ty, TyCtxt,+};+use rustc_span::symbol::sym;+use std::convert::TryInto;++/// Provide implementations of queries relating to polymorphization analysis.+pub fn provide(providers: &mut Providers) {+    providers.unused_generic_params = unused_generic_params;+}++/// Determine which generic parameters are used by the function/method/closure represented by+/// `def_id`. Returns a `u64` where a bit is set if the parameter with that index is unused (ie.+/// a value of zero indicates that all parameters are used).

I guess this comment as well.

davidtwco

comment created time in a month

Pull request review commentrust-lang/rust

Polymorphization

+//! Polymorphization Analysis+//! =========================+//!+//! This module implements an analysis of functions, methods and closures to determine which+//! generic parameters are unused (and eventually, in what ways generic parameters are used - only+//! for their size, offset of a field, etc.).++use rustc_hir::{def::DefKind, def_id::DefId};+use rustc_index::bit_set::FiniteBitSet;+use rustc_middle::mir::{+    visit::{TyContext, Visitor},+    Local, LocalDecl, Location,+};+use rustc_middle::ty::{+    self,+    fold::{TypeFoldable, TypeVisitor},+    query::Providers,+    Const, Ty, TyCtxt,+};+use rustc_span::symbol::sym;+use std::convert::TryInto;++/// Provide implementations of queries relating to polymorphization analysis.+pub fn provide(providers: &mut Providers) {+    providers.unused_generic_params = unused_generic_params;+}++/// Determine which generic parameters are used by the function/method/closure represented by+/// `def_id`. Returns a `u64` where a bit is set if the parameter with that index is unused (ie.+/// a value of zero indicates that all parameters are used).+fn unused_generic_params(tcx: TyCtxt<'_>, def_id: DefId) -> FiniteBitSet<u64> {+    debug!("unused_generic_params({:?})", def_id);++    if !tcx.sess.opts.debugging_opts.polymorphize {+        // If polymorphization disabled, then all parameters are used.+        return FiniteBitSet::new_empty();+    }++    let generics = tcx.generics_of(def_id);+    debug!("unused_generic_params: generics={:?}", generics);++    // Exit early when there are no parameters to be unused.+    if generics.count() == 0 {+        return FiniteBitSet::new_empty();+    }++    // Exit early when there is no MIR available.+    if !tcx.is_mir_available(def_id) {+        debug!("unused_generic_params: (no mir available) def_id={:?}", def_id);+        return FiniteBitSet::new_empty();+    }++    // Use a `u64` as a bitset. Starting with all ones, shift left by the number of parameters,+    // leaving N zeros for each parameter. When a parameter is marked as used, the bit (from the+    // left) corresponding to the parameter index will be flipped. This is the opposite of what+    // will be returned.+    let generics_count: u32 =+        generics.count().try_into().expect("more generic parameters than can fit into a `u32`");+    let mut used_parameters = FiniteBitSet::<u64>::new_filled().shl(generics_count);+    debug!("unused_generic_params: (start) used_parameters={:?}", used_parameters);+    mark_used_by_default_parameters(tcx, def_id, generics, &mut used_parameters);+    debug!("unused_generic_params: (after default) used_parameters={:?}", used_parameters);++    // Visit MIR and accumululate used generic parameters.+    let body = tcx.optimized_mir(def_id);+    let mut vis =+        UsedGenericParametersVisitor { tcx, def_id, used_parameters: &mut used_parameters };+    vis.visit_body(body);+    debug!("unused_generic_params: (after visitor) used_parameters={:?}", used_parameters);++    mark_used_by_predicates(tcx, def_id, &mut used_parameters);+    debug!("unused_generic_params: (after predicates) used_parameters={:?}", used_parameters);++    // Invert the u64 so that used is 0 and unused is 1. This makes checking if all parameters are+    // used easy - just compare with zero.

Same for this comment.

davidtwco

comment created time in a month

more