profile
viewpoint
Michael Woerister michaelwoerister Berlin, Germany michaelwoerister.github.io

michaelwoerister/hamt-rs 149

A Persistent Map Implementation based on Hash Array Mapped Tries

aardvark-platform/aardvark.base 111

Aardvark is an open-source platform for visual computing, real-time graphics and visualization. This repository is the basis for most platform libraries and provides basic functionality such as data-structures, math and much more.

aardvark-platform/aardvark.docs 90

Simple examples combining multiple packages provided by the aardvark platform. Each platform repository comes with separate examples -- here we collect overarching examples using for example aardvark.rendering and aardvark.media.

aardvark-platform/aardvark.rendering 70

The dependency-aware, high-performance aardvark rendering engine. This repo is part of aardvark - an open-source platform for visual computing, real-time graphics and visualization.

aardvark-platform/aardvark.media 33

Serverside, functional (ELM style) front-end and UI for aardvark, an open-source platform for visual computing, real-time graphics and visualization.

aardvark-platform/aardvark.algodat 14

Advanced data-structures (e.g. spatial acceleration data-structures such as octree, kdTree), part of aardvark, an open-source platform for visual computing, real-time graphics and visualization.

aardvark-platform/template 6

project template for aardvark projects with build script for bootstrapping new aardvark projects (including all necessary dependencies).

aardvark-platform/fablish 4

Elm style applications in .NET for composable user interfaces

aardvark-community/aardvark.semantictextonforests 3

An implementation of semantic texton forests (including a .NET wrapper for libsvm written in C++/CLI).

aardvark-platform/aardvark.fake 1

Script extensions for FAKE build scripts such as native dependency injection and cabal style add-source functionality

PullRequestReviewEvent
PullRequestReviewEvent

Pull request review commentrust-lang/measureme

Add a way to create an `EventId` with multiple arguments

 impl<'p> EventIdBuilder<'p> {             StringComponent::Ref(arg),         ]))     }++    pub fn from_label_and_args(&self, label: StringId, args: &[StringId]) -> EventId {+        // The capacity is the number of args + the same number for arg separators + the label.+        let mut parts = Vec::with_capacity(args.len() * 2 + 1);

:+1:

wesleywiser

comment created time in 20 days

pull request commentrust-lang/measureme

Add a way to create an `EventId` with multiple arguments

Looks good!

Do we have any unit tests for this area?

wesleywiser

comment created time in 21 days

Pull request review commentrust-lang/measureme

Add a way to create an `EventId` with multiple arguments

 impl<'p> EventIdBuilder<'p> {             StringComponent::Ref(arg),         ]))     }++    pub fn from_label_and_args(&self, label: StringId, args: &[StringId]) -> EventId {+        // The capacity is the number of args + the same number for arg separators + the label.+        let mut parts = Vec::with_capacity(args.len() * 2 + 1);

Looks like a good use case for a SmallVec<[StringComponent; 8]>. I'd be OK with adding a dependency on https://crates.io/crates/smallvec.

wesleywiser

comment created time in 21 days

PullRequestReviewEvent

Pull request review commentrust-lang/measureme

Allow StringTable to use the full 32-bit address space

 impl<'s> StringComponent<'s> {                 &mut bytes[s.len()..]             }             StringComponent::Ref(string_id) => {-                assert!(string_id.0 == string_id.0 & STRING_ID_MASK);-                let tagged = string_id.0 | (1u32 << 31);--                &mut bytes[0..4].copy_from_slice(&tagged.to_be_bytes());-                &mut bytes[4..]+                assert!(STRING_REF_ENCODED_SIZE == 5);

I added a comment but ended up not replacing the literal 5 with STRING_REF_ENCODED_SIZE. I don't know about you but writing STRING_REF_ENCODED_SIZE did not actually seem to make the code more readable.

What might make it more readable would be to have a bunch of constants like STRING_REF_TAG_OFFSET, STRING_REF_VALUE_START_OFFSET, and STRING_REF_VALUE_END_OFFSET, so that the encoding is kind of explained in one place. I don't think that's quite worth the trouble though.

michaelwoerister

comment created time in 22 days

PullRequestReviewEvent

push eventmichaelwoerister/measureme

Michael Woerister

commit sha f80765ff8307db8d091190010ea21dc21a0fccdb

Address PR feedback for 5-byte encoding of string refs.

view details

push time in 22 days

Pull request review commentrust-lang/measureme

Allow StringTable to use the full 32-bit address space

 impl<'s> StringComponent<'s> {                 &mut bytes[s.len()..]             }             StringComponent::Ref(string_id) => {-                assert!(string_id.0 == string_id.0 & STRING_ID_MASK);-                let tagged = string_id.0 | (1u32 << 31);--                &mut bytes[0..4].copy_from_slice(&tagged.to_be_bytes());-                &mut bytes[4..]+                assert!(STRING_REF_ENCODED_SIZE == 5);

Yeah, the assert! is supposed to be a self-testing explanation of the magic numbers below. I think the best thing to do is (1) leaving the assertion in, (2) adding a comment that explains its purpose, and (3) applying the changes you suggest below.

For this kind of low-level byte shuffling stuff I like to add these kinds of tripwires that decrease the possibility of overlooking necessary adaptations when the encoding changes.

michaelwoerister

comment created time in 22 days

PullRequestReviewEvent

PR opened rust-lang/measureme

Allow StringTable to use the full 32-bit address space

This PR changes the encoding for string references within the StringTable to use 5 bytes instead of 4 bytes. This way we can address 4 GB of data instead of 1 GB. The code becomes quite a bit simpler because we can remove a lot of manual bit-fiddling operations.

The only downside of the new approach is that string reference heavy data (e.g. from recording query keys) becomes about 5% larger. Between the larger address space support and the maintainability improvements this seems like an acceptable tradeoff to me. (I also have a half-done implementation of the same functionality without the additional space requirements -- but that turned out to make for such messy de-/serialization code that I abandoned it)

The size of profiling data that is collected without query keys should not be affected by this change.

r? @wesleywiser

+58 -124

0 comment

3 changed files

pr created time in a month

create barnchmichaelwoerister/measureme

branch : more-address-space2

created branch time in a month

pull request commentrust-lang/measureme

Implement paged data serialization

Yeah, I expect performance to be the same as the proof of concept implementation.

michaelwoerister

comment created time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 4f176926393abee946d7d26bbd5a805f84db9cd4

Address nits in paged serialization implementation.

view details

push time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha eec358ef201627a69f734e25e5c26504091da365

Add documentation to measureme::serialization module.

view details

push time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 410bcafbbc75336b3c4f3d7a4e03990cb1c51725

Update file header handling for paged data format.

view details

push time in a month

PR opened rust-lang/measureme

[wip] Implement paged data serialization

This PR implements the new paging-based data format, roughly as described in #128. However, the format has been tweaked to allow variable sized pages.

This is a work in progress:

  • [x] Make SerializationSink paging-based
  • [x] Write unit tests for SerializationSink
  • [ ] Add documentation about the data format
  • [ ] Add more explanatory comments throughout the serialization module
  • [ ] Add a proper file header to the monolithic file in addition to the three stream headers
+344 -131

0 comment

7 changed files

pr created time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 1825a06d998175d63139005b0f2b95268a1de9bc

Implement paged serialization approach. This allows to store all data in a single file instead of three separate ones.

view details

push time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha f8d7314e3f15ba53b10dc30cbebe40d277fc5636

Implement paged serialization approach. This allows to store all data in a single file instead of three separate ones.

view details

push time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 6572fabd64e364400aa155fab437419fd78065f2

[wip] paged serialization

view details

push time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha b37692612d33fb52aff75d0592b7bd0a52fa9831

[wip] paged serialization

view details

push time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 35dde743b0b334a8014ce283bd2bcd8929d075c2

[wip] paged serialization

view details

push time in a month

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 66b57d10d40b48a62a88791e5c674a108a3d78d0

[wip] paged serialization

view details

push time in a month

create barnchmichaelwoerister/measureme

branch : paged-sink-2

created branch time in a month

Pull request review commentrust-lang/measureme

Remove the SerializationSink trait

 impl Addr {     } } -pub trait SerializationSink: Sized + Send + Sync + 'static {-    fn from_path(path: &Path) -> Result<Self, Box<dyn Error + Send + Sync>>;--    /// Atomically write `num_bytes` to the sink. The implementation must ensure-    /// that concurrent invocations of `write_atomic` do not conflict with each-    /// other.-    ///-    /// The `write` argument is a function that must fill the output buffer-    /// passed to it. The output buffer is guaranteed to be exactly `num_bytes`-    /// large.-    fn write_atomic<W>(&self, num_bytes: usize, write: W) -> Addr-    where-        W: FnOnce(&mut [u8]);+#[derive(Debug)]+pub struct SerializationSink {+    data: Mutex<Inner>,+}++/// The `BackingStorage` is what the data gets written to.+trait BackingStorage: Write + Send + Debug {+    fn drain_bytes(&mut self) -> Vec<u8>;

Done.

michaelwoerister

comment created time in a month

PullRequestReviewEvent

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 4303a6d52d247f015f37ea655775ef4b2c2ce759

Add clarifying comment to BackingStorage::drain_bytes().

view details

push time in a month

Pull request review commentrust-lang/measureme

Remove the SerializationSink trait

 impl Addr {     } } -pub trait SerializationSink: Sized + Send + Sync + 'static {-    fn from_path(path: &Path) -> Result<Self, Box<dyn Error + Send + Sync>>;--    /// Atomically write `num_bytes` to the sink. The implementation must ensure-    /// that concurrent invocations of `write_atomic` do not conflict with each-    /// other.-    ///-    /// The `write` argument is a function that must fill the output buffer-    /// passed to it. The output buffer is guaranteed to be exactly `num_bytes`-    /// large.-    fn write_atomic<W>(&self, num_bytes: usize, write: W) -> Addr-    where-        W: FnOnce(&mut [u8]);+#[derive(Debug)]+pub struct SerializationSink {+    data: Mutex<Inner>,+}++/// The `BackingStorage` is what the data gets written to.+trait BackingStorage: Write + Send + Debug {+    fn drain_bytes(&mut self) -> Vec<u8>;

I just tried it but it's complicated because analyzeme uses these things in its tests too -- but Cargo does not compile the dependencies of crate X with --test or --cfg test when it compiles crate X with --test. I suspect one can somehow use Cargo features to get this to work (although I couldn't find an example of this in the Cargo book).

michaelwoerister

comment created time in 2 months

PullRequestReviewEvent

PR opened rust-lang/measureme

Remove the SerializationSink trait

This PR removes the SerializationSink trait and thus the ability to plug in different serialization strategies at the client level. The FileSerializationSink is now the only implementation and is renamed to just SerializationSink.

This simplifies the public interface of the measureme crate.

r? @wesleywiser

+217 -376

0 comment

14 changed files

pr created time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 603f0c9db01e28733ebe941065f7669543d9eb26

Remove the generic SerializationSink parameter from Profiler.

view details

Michael Woerister

commit sha b4097fc1eaa57c7fe568cd0a4aea3b1f97a0bc31

Remove the generic SerializationSink parameter from StringTableBuilder.

view details

Michael Woerister

commit sha b74043c2b2394abc9dd3bdcf9e97f0fbaa27bb6d

Remove the SerializationSink trait.

view details

Michael Woerister

commit sha e5add84857479a472b82b3de36add29662a15eac

Rename FileSerializationSink to just SerializationSink, now that there is a single kind.

view details

push time in 2 months

create barnchmichaelwoerister/measureme

branch : remove_serialization_sink_trait

created branch time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 86617c7dfc79d29731f9ee5844113a449fc8ba17

Make regex version_checker more robust.

view details

push time in 2 months

PR opened rust-lang/measureme

Add a CI check that all crates in the workspace have a consistent version

This should catch crate version inconsistencies early.

+85 -2

0 comment

7 changed files

pr created time in 2 months

create barnchmichaelwoerister/measureme

branch : better-version-checks

created branch time in 2 months

PR closed rust-lang/rust

[Experimental] Performance test new paging-based measureme data sink. S-waiting-on-author

This PR will try to make sure that the new measureme file format (as described in https://github.com/rust-lang/measureme/issues/128) does not regress performance while doing self-profiling.

The perf.rlo run is actually expected to fail (because the tools on the perf server cannot handle the new format yet) but hopefully we'll still get the top-level timings that should show potential regressions.

The implementation of the new format as referenced by this PR is not production ready yet, but it should give a good indication of final performance. Once we know if this method of performance testing actually yields results, we can also test an alternative implementation that is slightly simpler.

cc @wesleywiser @Mark-Simulacrum

+24 -11

36 comments

6 changed files

michaelwoerister

pr closed time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

I'm going to close this PR, we got the numbers we need for deciding on how to forward.

michaelwoerister

comment created time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

Yeah, this looks actually better than the more complicated version. That's good to know. These things so often are so counterintuitive :)

Thanks a lot for getting those numbers. I think it's really great that perf.rlo is using an SQL database now, btw.

michaelwoerister

comment created time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

The collector runs rustc 2-3 times: once for self-profile, and then 1-2 times without it. By default, the perf.rust-lang.org frontend shows the minimum of each stat across those runs. Since self-profile does impose some overhead, that's usually not the self-profile run. The page I posted switched min to max (manually editing the SQL query).

Huh, that basically means we are not measuring anything useful with these perf runs, right? :) That is, I'm trying to find out the overhead of the new serialization strategy (and especially the overhead of the two new approaches relative to each other), but the "min" numbers actually (probably) don't contain self-profiled runs. I guess the solution is to ask you to manually run the modified query again for the current run :)

I guess it might create a recursive "self feeding" problem, but in theory if we record events for the serialization as well, then crox should be able to show us when those are being serialized and to what extent we're waiting on them. I agree that it seems likely we're waiting at the end.

There's definitely some things that we can record. For things that are application specific (e.g. the handling of query keys before they make it into measureme) this is easy. For things that happen inside of measureme, like flushing to disk once the in-memory buffers are full, we could have some kind of built-in recording in measureme itself. That might not even be that hard.

However, the final flushing of the file would need some kind of special handling. Right now we do that in SerializationSink::drop(). But that should actually be doable too. You always lose that last bit of information for how long it takes to write that very last measurement to disk but we could make sure that that is just a single event and thus does not account for much data.

michaelwoerister

comment created time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

@Mark-Simulacrum I just pushed a version that writes data to disk in a blocking way. That data format is not affected by this so the tooling on the server should still be compatible.

michaelwoerister

comment created time in 2 months

push eventmichaelwoerister/rust

Michael Woerister

commit sha 2ad078be926af62e9b40bc813f67df0268d44743

Try out blocking paged data sink.

view details

push time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

Interesting, yes. There seem to be a few minor regressions. Maybe there is a bit of a pattern here: Most regressions seem to be scenarios that are quick to execute and where there's little time spent in LLVM and the linker. Maybe the background thread becomes the bottleneck towards the end, when it is still writing stuff to disk and the rest of the program is only waiting on it to finish?

How did you generate those numbers? Is there a special mode in the collector?

michaelwoerister

comment created time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

Alright, those numbers look encouraging. I'll put together a version that doesn't use a background thread tomorrow.

michaelwoerister

comment created time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

Thanks @Mark-Simulacrum, I was able to do a full rustc-perf run with this version locally. Let's see if something else breaks.

One way or the other, I'll need to rewrite the whole thing anyway before it can be merged. The current version has quite a bit of room for improvement in terms of code quality.

michaelwoerister

comment created time in 2 months

push eventmichaelwoerister/rust

Michael Woerister

commit sha e5d80ea18ecb513fca6f0c33a99dbe3bf476bcde

Use experimental paging-based data sink for measureme.

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 26d150475c8d83613beffe970d1c18dd2494f607

Proof of concept impl for paged serialization.

view details

push time in 2 months

create barnchmichaelwoerister/measureme

branch : paging-hacks

created branch time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 0ab861b92000927a9e99c363987ee4561d5d6c67

Proof of concept impl for paged serialization.

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha a71b8ca1e5f6f2ec4e9b2186287bef92b45d7c5f

Proof of concept impl for paged serialization.

view details

push time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

Looks like this currently fails in rustc, when running on helloworld

Indeed it does. SelfProfiler::bulk_map_query_invocation_id_to_single_string() seems to write quite large chunks of data at once, more than the page size in some cases. I'll make the code a bit more robust against this kind of thing.

michaelwoerister

comment created time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha e841427b91245485cfe833785abbd87c9dadd433

Proof of concept impl for paged serialization.

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 46f5a5a16ec12b9d1036d9688aff6b6a58718c36

Proof of concept impl for paged serialization.

view details

push time in 2 months

pull request commentrust-lang/rust

[Experimental] Performance test new paging-based measureme data sink.

OK, this seems to pass all checks now. @Xanewok, would you be so kind and start another perf run for me?

michaelwoerister

comment created time in 2 months

push eventmichaelwoerister/rust

Michael Woerister

commit sha ca1bb18264167719f514e66b94870f465036810f

Use experimental paging-based data sink for measureme.

view details

push time in 2 months

push eventmichaelwoerister/rust

Michael Woerister

commit sha 8fd2459b4fb2eab5798387e88b792ba1fd51e786

Use experimental paging-based data sink for measureme.

view details

push time in 2 months

push eventmichaelwoerister/rust

Michael Woerister

commit sha 236b3e5088ed7b1d06235045371e4b565c8408ff

Use experimental paging-based data sink for measureme.

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha c00409471a2d4ba17012303037fdf82952fea92b

Proof of concept impl for paged serialization.

view details

push time in 2 months

create barnchmichaelwoerister/rust

branch : perftest-paged-measureme-background-thread

created branch time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha da4d0a6e75d9bf5289383fe73be1ab444ea7de9f

Proof of concept impl for paged serialization.

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha d989144703a935097b120ff89447679a3409bf39

Proof of concept impl for paged serialization.

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha 32d841fc02387f16ae94ef57dda71fc5b312998a

Proof of concept impl for paged serialization.

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha e04db1648d1e39fecb86465f6410d5207d86ab4f

wip

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha b41cfaa6b07e9b1cae13b7a23e738d1dd99fbeda

wip

view details

push time in 2 months

push eventmichaelwoerister/measureme

Michael Woerister

commit sha dba3ec03ba9fe7214eadd19d04e5ff77537141bc

wip

view details

push time in 2 months

create barnchmichaelwoerister/measureme

branch : paged_sinks

created branch time in 2 months

issue commentrust-lang/measureme

Single file on disk format

Well, it turns out that the rustc_middle is too large already for the current encoding scheme, so the common-grammar approach described above (which would restrict the address space even more) is off the table, I guess.

@wesleywiser, do you mind if I take a stab at implementing the paging scheme? Are there any other efforts that might conflict with that?

wesleywiser

comment created time in 2 months

create barnchmichaelwoerister/measureme

branch : single-file-common-grammar

created branch time in 2 months

issue commentrust-lang/measureme

Single file on disk format

OK, I spent a little time thinking about this. Splitting things into pages is a good idea, I had not thought of that. But it actually seems like a generalization of a simpler single-file approach that was proposed a few times in the past, namely keeping everything in memory until the end and then just concatenating the three "files" into a single file on disk. Splitting things into pages is similar -- i.e. it also concatenates the data, just in an interleaved fashion -- with the advantage that there's only ever a limited amount of stuff that needs to reside in memory.

This also makes me think that it's actually a bit redundant to have one kind of page that handles event and string data and another kind of page that handles string index data. We could instead just have three kinds of pages, one for events, one for string data, and one for string indices. I think that would make things relatively straightforward to implement.


An alternative approach to having just a single file would be to have the binary format for string data, string indices, and events all follow the same grammar, so that one can unambiguously parse the file. #96 is a first step towards that and would unify the format for string data and string indices. It looks like we could easily take a few bits from the RawEvent's thread id field and use those as an unambiguous tag for events. In essence, one would always be able to tell from the first byte if a given record is an event, a string data list, or a string index entry.

There are a couple of downsides to that approach though:

  • In order to build the string index table one has to parse the entire file once. Not sure if that is an actual performance issue, as I suppose that parsing can be done basically as quickly as loading the data from disk, and the paging approach also has to touch most of the file in order to find all string index pages.
  • In order to iterate over events in reverse order (which many of the tools do), one also has to parse the entire file and build an in-memory table that records the location of each event in the stream.
  • Events and string data all share the same address space now. Since currently string ids are often actual 30-bit addresses, things could get quite crowded for larger profiles, since 30 bits is only 1 gigabyte. The paging approach, in contrast, would allow us to have a separate address space for each kind of page.

I'm not quite sure which approach I like more. The paging approach might be more efficient but it also adds a whole new concept to the file format. The common grammar approach just extends an existing concept, but requires more up-front work in the post-processing tools.

wesleywiser

comment created time in 2 months

issue commentrust-lang/measureme

Single file on disk format

I haven't thought about all the implications here in detail but I think that implementing https://github.com/rust-lang/measureme/issues/96 first might simplify this approach because it makes string index data just a special case of regular string data.

Depending on whether with have a few disambiguation bits to spare at the start of each event record, we might even be able to freely mix and match string and event data.

I'll think a bit more about this. I need to read up on the data format again.

wesleywiser

comment created time in 2 months

pull request commentrust-lang/measureme

Remove byteorder dependency

Nice! Fewer dependencies is always good!

workingjubilee

comment created time in 2 months

pull request commentrust-lang/rust

self-profile: Cache more query key strings when doing self-profiling.

Thanks for the review!

michaelwoerister

comment created time in 2 months

push eventmichaelwoerister/rust

Alexis Bourget

commit sha 430f19a82eb95aac1d6ac3dbd9688288cfcbe602

Document unsafety in library/core/src/slice/mod.rs

view details

Alexis Bourget

commit sha 92b1975eaaae0891cb8d5ed50fb7a17f37ce0b38

Added the missing SAFETY: comments

view details

Yuto Kawamura

commit sha 165a6e597e6991f18f9684b5aa7667ef4b6e3955

Fix wasi::fs::OpenOptions to imply write when append is on

view details

Temirkhan Myrzamadi

commit sha 8a4fecb554353a89f854726c02058e764863e64a

Fix some Clippy warnings in librustc_serialize

view details

Alexis Bourget

commit sha 3a709fe7021218b613aa7fdce7794df5a8ff0da5

Add precisions about ZSTs and fix nits raised in review

view details

Alexis Bourget

commit sha 5a0de2f828db7c06624f55f69877197f9b26c753

Improve safety comments for usize, fix some other unclear parts

view details

南浦月

commit sha d892a07c67dfa4f347b62ecf96de184c41dae5dd

add Ipv6Addr::to_ipv4_mapped

view details

Lukas Kalbertodt

commit sha 709d1056b890de640c3d8dabd70562950df98dcf

Fix minor things in the `f32` primitive docs All of these were review comments in #74621 that I first fixed in that PR, but later accidentally overwrote by a force push.

view details

Esteban Küber

commit sha 17ada052e7f8b785eaadb6fb49197357ad644d65

Detect tuple variants used as struct pattern and suggest correct pattern

view details

Esteban Küber

commit sha 0f7205f2020ff5acefac83e354acea4c69f2ea0e

Fix suggestion to use lifetime in type

view details

Esteban Küber

commit sha 7956b1cef71ef633fb0e5e55451c5bb9ee9c59e2

Assoc `const`s don't have generics Fix #74264.

view details

Esteban Küber

commit sha b9585fda7bc32db3b129aa7ec43322ae0f089635

When suggesting `for` lts, consider existing lifetime names Fix #72404.

view details

Esteban Küber

commit sha becd479482935a63633861595e276bae6533b25b

review comment: simplify code by using slice pat

view details

Alexis Bourget

commit sha 91ba92b6df4d1626c484108d4018f359f07b16ae

Change safety comment for usize with the one from LukasKalbertodt review

view details

Poliorcetics

commit sha a308e74e1310fb02bcc4b46c44da769dfe426812

Add some texts to make the tidy check for unsafe documentation pass

view details

Esteban Küber

commit sha 6a3deb0ae04c4cb6400b30fecd1cbbee0506348b

Suggest using `'static` in assoc consts and suggest when multiple lts are needed

view details

Mark Rousskov

commit sha a3c73ca88c58082123a217ddaea535639093a4d5

Remove SynchronizationScope::Other Added in b7615389978eae2ae9f3cba9a776fd8da3f743ca, it started out already unused.

view details

Eric Huss

commit sha 7291c6dccd8a490b4dd9993621c4144f9faf75da

Update cargo

view details

Eric Huss

commit sha 1902983f4520a798c50ee795322def8a60184aa9

Move forge platform-support to the rustc book.

view details

Eric Huss

commit sha 392116e021f786b81dd12faec189a7d199dcd999

Platform Support page updates. A few updates: - Some minor wording and formatting changes. - Remove the `cargo` column. - Explain the columns up-front. - Add no-wrap on the target-triple, which looks better to me. - Minor mention on how to install support for a built-in target via rustup.

view details

push time in 2 months

Pull request review commentrust-lang/rust

self-profile: Cache more query key strings when doing self-profiling.

 impl SpecIntoSelfProfilingString for DefIndex {     } } +impl SpecIntoSelfProfilingString for LocalDefId {+    fn spec_to_self_profile_string(+        &self,+        builder: &mut QueryKeyStringBuilder<'_, '_, '_>,+    ) -> StringId {+        builder.def_id_to_string_id(DefId { krate: LOCAL_CRATE, index: self.local_def_index })+    }+}++impl<T: SpecIntoSelfProfilingString> SpecIntoSelfProfilingString for WithOptConstParam<T> {+    fn spec_to_self_profile_string(+        &self,+        builder: &mut QueryKeyStringBuilder<'_, '_, '_>,+    ) -> StringId {+        // We print `WithOptConstParam` values as tuples to make them shorter+        // and more readable, without losing information:+        //+        // "WithOptConstParam { did: foo::bar, const_param_did: Some(foo::baz) }"+        // becomes "(foo::bar, foo::baz)" and+        // "WithOptConstParam { did: foo::bar, const_param_did: None }"+        // becomes "(foo::bar, _)".++        let mut components: SmallVec<[StringComponent<'_>; 8]> = SmallVec::new();

Ah yes, that's true. I'll change that.

michaelwoerister

comment created time in 2 months

Pull request review commentrust-lang/rust

self-profile: Cache more query key strings when doing self-profiling.

 impl SpecIntoSelfProfilingString for DefIndex {     } } +impl SpecIntoSelfProfilingString for LocalDefId {+    fn spec_to_self_profile_string(+        &self,+        builder: &mut QueryKeyStringBuilder<'_, '_, '_>,+    ) -> StringId {+        builder.def_id_to_string_id(DefId { krate: LOCAL_CRATE, index: self.local_def_index })+    }+}++impl<T: SpecIntoSelfProfilingString> SpecIntoSelfProfilingString for WithOptConstParam<T> {+    fn spec_to_self_profile_string(+        &self,+        builder: &mut QueryKeyStringBuilder<'_, '_, '_>,+    ) -> StringId {+        // We print `WithOptConstParam` values as tuples to make them shorter+        // and more readable, without losing information:+        //+        // "WithOptConstParam { did: foo::bar, const_param_did: Some(foo::baz) }"+        // becomes "(foo::bar, foo::baz)" and+        // "WithOptConstParam { did: foo::bar, const_param_did: None }"+        // becomes "(foo::bar, _)".

What does self profiling need the WithOptConstParam for?

Some queries (e.g. mir_build) take WithOptConstParam values as keys. Since self-profiling can record a string representation for each query key that gets used some of those keys will be stringified WithOptConstParam values. You can find the full list of queries (and their keys) in https://github.com/rust-lang/rust/blob/master/src/librustc_middle/query/mod.rs.

You probably want to uniquely identify query calls? Considering that self.const_param_did should be unique for each DefId (as we otherwise cause an ICE while building mir) you might also be able to store this as (did, bool) for some additional wins.

I'm not quite sure I follow. You are saying that there is an invariant around const_param_did that makes sure that it is always the same (or None) for a given did value?

In any case, I think it's a good idea to keep this as close to the original value as possible. The encoding format should make sure that there is no data redundancy (assuming that that DefId was already mentioned somewhere already).

michaelwoerister

comment created time in 2 months

PR opened rust-lang/rust

self-profile: Cache more query key strings when doing self-profiling.

This PR adds optimized SpecIntoSelfProfilingString implementations for two common query key types (LocalDefId and WithOptConstParam). This makes raw self-profiling data on disk 8-9% smaller for my two test cases (regex and cargo).

The on-disk format is not affected, so no tooling updates need to happen.

I also tried adding an impl for Ty<'tcx> (which should reduce size quite a bit) but the compiler did not allow me to add a specialized impl parameterized with 'tcx. I don't know if there is an actual problem with that or if the implementation of specialization just doesn't support it yet.

cc @wesleywiser @Mark-Simulacrum

+50 -1

0 comment

1 changed file

pr created time in 2 months

push eventmichaelwoerister/rust

ThibsG

commit sha 442c8ae23b90874485468b3becfc011f8c9d40bb

Fix FP for `suspicious_arithmetic_impl` from `suspicious_trait_impl` lint

view details

Andreas Jonson

commit sha 30659cd45ee01f5626aeef4bbe29c68536dc833d

older toolchains not valid any more with the change to llvm 10 the parameter LLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN do not do any thing as min and soft error is the same. see https://github.com/rust-lang/llvm-project/blob/86b120e6f302d39cd6973b6391fb299d7bc22122/llvm/cmake/modules/CheckCompilerVersion.cmake

view details

flip1995

commit sha 51f2a6f8b6eea9ebefddff39e87a1ca16c59827c

Add documentation for basic Clippy hacking

view details

flip1995

commit sha 17903f6d7107c6d31ee15f4c46af29d1f4aa363f

Mention lint naming guidelines earlier

view details

flip1995

commit sha 3a4cc9f7f085e73fbfe57e8c896b90a5fe61c4f4

Address review comments

view details

Dmitry Murzin

commit sha b375f1dd20bf07175ec06d13e1e9dc8b20287cd3

Add suggestion for `iter_skip_next` lint

view details

flip1995

commit sha d164ab65f706540f0132268c8ad2237d2e53e08a

Merge commit 'da5a6fb1b65ec6581a67e942a3850f6bc15a552c' into clippyup

view details

bors

commit sha f5d429cd762423901f946abd800dc2cd91366ccf

Auto merge of #5820 - ThibsG:FixSuspiciousArithmeticImpl, r=flip1995 Fix FP for `suspicious_arithmetic_impl` from `suspicious_trait_impl` … As discussed in #3215, the `suspicious_trait_impl` lint causes too many false positives, as it is complex to find out if binary operations are suspicious or not. This PR restricts the number of binary operations to at most one, otherwise we don't lint. This can be seen as very conservative, but at least FP can be reduced to bare minimum. Fixes: #3215 changelog: limit the `suspicious_arithmetic_impl` lint to one binop, to avoid many FPs

view details

Eduard-Mihai Burtescu

commit sha 8dc9d4d567dae3481a47ad677e263cb3bd57cdf5

[experiment] ty/layout: compute both niche-filling and tagged layouts for enums.

view details

Erik Desjardins

commit sha b34d3004914a1b6b2e7c91b2c41ac1af28745874

compare tagged/niche-filling layout and pick the best one

view details

Erik Desjardins

commit sha 144b1592e8186ab7c24212058bb2c897260897e3

document test changes

view details

Ryan1729

commit sha 5a644964fc05752a1283dab238b81de7583f7d03

run cargo dev new_lint specifically: cargo dev new_lint --name derive_ord_xor_partial_ord --category correctness --pass late

view details

Ryan1729

commit sha fc20ee63a105c0df78113126e8749f5958d7dc47

move derive_ord_xor_partial_ord into derive mod so we can reuse derive_hash_xor_partial_eq code later

view details

Ryan1729

commit sha 0722991b62fd6e4d7d7a51425274f3288bcc96bc

add test for derive_ord_xor_partial_ord based on test for derive_hash_xor_partial_eq

view details

Ryan1729

commit sha 068acbd27b19a4a7be3a9d00954ecfad8a0e6553

initial implementation based on code for `derive_hash_xor_partial_eq` which is showing one error when there should be four

view details

Ryan1729

commit sha a8d6eda93049f0077c1515bec35fe0359ea43f96

use get_trait_def_id to check for Ord trait

view details

Ryan1729

commit sha 6c3e4591b87e6c690b31166867484675dcb1e48c

update reference since we see the expected four errors

view details

Ryan1729

commit sha 7dc974815ec8736f026dc10a070137e0d4601d52

remove is_local check since getting the def_id directly makes it unnecessary

view details

Ryan1729

commit sha 431924ccf69bc4d4f0597f12749e8b1bcb285710

add description for derive_ord_xor_partial_ord

view details

Ryan1729

commit sha 668b7474b47791c8c9af10130356b681b3bf3a84

run cargo dev fmt and fix overly long line

view details

push time in 2 months

create barnchmichaelwoerister/rust

branch : sp-cache-more-query-keys

created branch time in 3 months

more