profile
viewpoint

glandium/boxext 18

[rust] Extensions to the Box type

glandium/about-startup 7

Firefox extension to display startup timings

glandium/adb4ff 1

Android Debug Bridge for Firefox

glandium/android-bootchart 1

Standalone bootchart program derived from the Android init code

glandium/b2g-manifest 1

Repo manifests for building B2G

glandium/bootchart 1

merge of bootchart-collector and pybootchartgui

glandium/busybox 1

Fork of git://android.git.linaro.org/platform/external/busybox.git

glandium/ActiveData-ETL 0

The ETL process responsible for filling ActiveData

glandium/allocator_api 0

[rust] A copy of the unstable allocator_api

issue commentrust-lang/wg-allocators

Support for stateless size classes

The rationale for the current definition of min is that a container using an allocator (e.g. Box) does not need to store the size returned by the allocator anywhere: it can just recompute the initial layout it passed to alloc when deallocating.

mahkoh

comment created time in a day

issue commentrust-lang/wg-allocators

Rename `AllocRef` to `Allocator` and `alloc` to `allocate`

I think I'm too close to the subject matter to not be confused by it, but I think @aaron1011's original point was very valid. So maybe they can speak from experience?

TimDiekmann

comment created time in a day

issue commentmozilla/sccache

sccache breaks -Wimplicit-fallthrough

It does seem like that option would be required. However I'm failing to reproduce now (possibly due to a new compiler, possibly that I changed something else in sccache configuration since then before I dropped using it)

anholt

comment created time in a day

issue commentmozilla/sccache

sccache breaks -Wimplicit-fallthrough

Hmm, maybe preprocessing was removing the /* fallthrough */ comment? I'm not sure if the preprocessor touches regular comments, I thought it didn't. Could you retest with rewrite_includes_only = true in your config's [dist] section?

anholt

comment created time in a day

issue closedrust-lang/wg-allocators

Rename `AllocRef` to `Allocator` and `alloc` to `allocate`

With rust-lang/rust#77187 we released AllocRef to the wild. It was pointed out, the names AllocRef and alloc are confusing sometimes. The main reason for this, is, that "alloc" is used as a verb and as a noun.

in https://github.com/rust-lang/rust/pull/77187#discussion_r511169783 @Aaron1011 noted, that:

[Box::alloc_ref] sounds like we're allocating a reference, not obtaining a reference to the allocator.

However, alloc_ref is the lowercased name of the Trait. @Amanieu also suggested, to rename it to Box::allocator, but the term allocator is never used anywhere, so I decided to stick with alloc_ref for now.

I think, it makes sense, to rename AllocRef to Allocator and (de)alloc to (de)allocate. The main reason for calling it AllocRef instead of Alloc was, that we wanted to express, that AllocRef should be a ZST or a reference to the actual allocator (not a reference to an "alloc" :wink: ). This however is also pointed out in the documentation and we don't expect many people to implement an allocator. A very minor downside is the longer name of Allocator and allocate, but it's way more clear, if you read those terms. The trait will also not be used often directly. In most cases, it will simply be passed to Box or Vec. So most people neither have to call allocate or deallocate, nor have to import Allocator into scope. They probably won't even care, how the allocator is implemented (ZST or reference) as long they can simply pass an allocator to those structs.

This proposal also includes renaming Box::alloc_ref to Box::allocator.

cc @Amanieu @Lokathor @Wodann @CAD97 @scottjmaddox @vertexclique

closed time in a day

TimDiekmann

issue openedrust-lang/wg-allocators

Use a `Storage` backend for collection rather than an allocator-based one

At internals.rlo a nice proposal was made. I like to track the proposal in this repository but keep the discussion at the linked topic to avoid splitting the context.

created time in a day

issue commentrust-lang/wg-allocators

Rename `AllocRef` to `Allocator` and `alloc` to `allocate`

Hmm, isn't AllocError both an allocation error and the error type for allocators? :smile: The more I play around with the Allocator trait, the more I think, that the error should be associative but I leave this question open (and #23 closed) until I finished some work with the storage-approach for collections but the concerns are very valid IMO.

I know this goes beyond the scope of the original proposal, but since we are asking ourselves the question; I'm wondering whether there are more instances that might be ambiguous?

Do you have something in mind specifically?

TimDiekmann

comment created time in a day

issue commentrust-lang/wg-allocators

Rename `AllocRef` to `Allocator` and `alloc` to `allocate`

When reviewing the PR, I couldn't help but wonder whether people had similar confusions about whether AllocError is an allocation error or an error type for allocators.

I know this goes beyond the scope of the original proposal, but since we are asking ourselves the question; I'm wondering whether there are more instances that might be ambiguous?

TimDiekmann

comment created time in 2 days

pull request commentmozilla/sccache

Add the ability to clear a cache.

Created as https://github.com/paritytech/sccache/pull/27

rthomas

comment created time in 2 days

PR opened mozilla/sccache

Pass -clang:-frewrite-includes to clang-cl.

Fixes #894.

+8 -1

0 comment

1 changed file

pr created time in 2 days

PR opened mozilla/sccache

Emit line numbers when preprocessing for distributed compilation.

Fixes #893.

+15 -3

0 comment

1 changed file

pr created time in 2 days

issue openedmozilla/sccache

Support rewrite_includes_only with clang-cl

To avoid large swathes of compiler warnings during distributed compilation, we try to limit client-side preprocessing to only resolve includes, and not substitute #defines etc. This mode is enabled when rewrite_includes_only = true is set in the dist config.

However, this mode is currently unimplemented on Windows: msvc.rs ignores the rewrite_includes_only parameter.

https://github.com/mozilla/sccache/blob/e66c9c15142a7e583d6ab80bd614bdffb2ebcc47/src/compiler/msvc.rs#L69

MSVC's cl.exe does not appear to support an argument to unlock this mode. However, clang-cl does, in an underhanded way, with the -clang:-frewrite-includes argument. So we can support rewrite_includes_only in msvc.rs if clang-cl is used.

With this change, I've been able to build Firefox on Windows with distributed sccache successfully. Without this change, the build was 1. failing because some directories had -Werror enabled and 2. extremely slow, because the console delayed everything by trying to keep up with the huge amounts of warning output.

created time in 2 days

issue openedmozilla/sccache

Incorrect line numbers in error messages when using distributed compilation from Windows

msvc.rs always passes "-EP" rather than "-E" in the preprocessing step:

https://github.com/mozilla/sccache/blob/e66c9c15142a7e583d6ab80bd614bdffb2ebcc47/src/compiler/msvc.rs#L689

"-EP" means "don't emit line number annotations".

Originally this was using "-E" (which does emit line number annotations), but it was changed to "-EP" in #723. This change mirrored a change to gcc.rs from #208. However, when support for distributed compilation was added in #249, gcc.rs was updated to only use "-P" when no distribution was happening:

https://github.com/mozilla/sccache/commit/4b04cb067c43edffc2e838358b5a597cf36cd283#diff-33ed0b50ade3d44507b82055ded2e81fc58756670c39943d2b1305354c649dd6R353-R359

#249 did not make the equivalent change to msvc.rs.

msvc.rs should be updated similarly, so that it uses "-E" if may_dist and otherwise "-EP".

created time in 2 days

pull request commentmozilla/sccache

Add the ability to clear a cache.

Sure thing I'll try to get to it this weekend - has this repo been abandoned?

rthomas

comment created time in 2 days

pull request commentmozilla/sccache

Add the ability to clear a cache.

@rthomas would you mind re-creating this against paritytech/sccache ?

rthomas

comment created time in 2 days

issue commentmozilla/sccache

-D XXX (with a space after -D) leads to non-cacheable compiler call

The MSDN docs for /D do indicate that it allows both -DFOO and -D FOO. sccache's MSVC argument parser already has support for this kind of argument, it just doesn't treat -D that way: https://github.com/mozilla/sccache/blob/6628e1f70db3d583cb5e79210603ad878de3d315/src/compiler/msvc.rs#L257

That would need to be changed from Concatenated to CanBeSeparated.

shaforostoff

comment created time in 2 days

issue commentmozilla/sccache

sccache v0.2.13 does not work when built with rustc 1.48

In case people are interested this looks like a breaking change in rust 1.48 (https://github.com/rust-lang/rust/pull/71274/) where mem::uninitialized now panics if the type can't hold the zero value:

$ SCCACHE_LOG=debug SCCACHE_START_SERVER=1 SCCACHE_NO_DAEMON=1 target/debug/sccache
thread 'main' panicked at 'attempted to leave type `linked_hash_map::Node<std::ffi::OsString, u64>` uninitialized, which is invalid', /home/andy/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/mem/mod.rs:658:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

(I assume OsString is a NonNull<u8> or similar).

Until (unless?) Mozilla cut a new release there is a pre-built binary alongside each release (e.g. at https://github.com/mozilla/sccache/releases/tag/0.2.13) that works, or you can use master or @drahnr's fork at https://github.com/paritytech/sccache.

SomeoneToIgnore

comment created time in 3 days

issue openedmozilla/sccache

-D XXX (with a space after -D) leads to non-cacheable compiler call

Having -D XXX in compiler call (with a space after -D) results in non-cacheable compiler call because it is treated as multiple input files.

sccache should either properly report such problem, or treat -D XXX as -DXXX, like cl.exe does.

Or at least, please add this info to caveats, in addition to /Zi -> /Z7 requirement.

My environment: Win8.1, latest sccache from scoop, Visual Studio 2017. Using Ninja from command-line.

created time in 3 days

issue openedrust-lang/wg-allocators

Support for stateless size classes

I've seen >60% resident set size reductions by bypassing the System allocator for large allocations and using mmap directly.

For this purpose I've implemented a global allocator that delegates method calls as follows:

  • alloc for large allocations: mmap
  • alloc for small allocations: System.alloc
  • dealloc for large allocations: munmap
  • dealloc for small allocations: System.dealloc
  • realloc for large->large reallocation: mremap
  • realloc for small->small reallocation: System.realloc
  • realloc for small->large reallocation: mmap + copy + System.dealloc
  • realloc for large->small reallocation: mremap down to the page size

The allocator recognizes if an existing allocation is large or small by inspecting the size stored in the layout. (That's the stateless part.)

Unfortunately, the permissive nature of memory fitting breaks the bold case:

The provided layout.size() must fall in the range min ..= max, where:

  • min is the size of the layout most recently used to allocate the block, and
  • max is the latest actual size returned from alloc, grow, or shrink.

In the bold case above, min would be a small allocation. Therefore, realloc has to return a small allocation. To do this, it has to System.alloc + copy + munmap.

For my usecase in particular (and possibly also the usecases of Vec and such), the following stricter requirement would work:

  • min is the latest actual size returned from alloc, grow, or shrink rounded down to the next multiple of layout.align()

This solution is not great if you want to delegate small allocations to an AllocRef instead of the System allocator. You would have to check if allocations returned by the AllocRef look like large allocations and, if so, replace them by mmap allocations.

created time in 4 days

issue commentmozilla/sccache

sccache does not work with docker build

We face exactly the same problem and in our case this was also a symptom of #887.

BenD0G

comment created time in 4 days

issue closedrust-lang/wg-allocators

Change parameters of `AllocRef` to take `NonNull<[u8]>, usize` instead of `NonNull<u8>, Layout`

Currently, AllocRef contains those signatures (leaving out _zeroed variants):

fn alloc(Layout) -> Result<NonNull<[u8]>, AllocErr>;
unsafe fn dealloc(&mut self, NonNull<u8>, Layout);
unsafe fn grow/shrink(&mut self, NonNull<u8>, Layout, usize) -> Result<NonNull<[u8]>, AllocErr>;

I like to propose to change those to take NonNull<[u8]> as they also return a slice. Also this would change the Layout parameter to take the alignment instead.

fn alloc(Layout) -> Result<NonNull<[u8]>, AllocErr>;
unsafe fn dealloc(&mut self, NonNull<[u8]>, align: usize);
unsafe fn grow/shrink(&mut self, NonNull<[u8]>, align: usize, new_size: usize) -> Result<NonNull<[u8]>, AllocErr>;

This would also change the Memory fitting safety section wording a bit:

  • The block must be allocated with the same alignment as align, and
  • The provided ptr.len() must fall in the range min ..= max, where:
    • min is the size of the layout most recently used to allocate the block, and
    • max is the latest actual size returned from alloc, grow, or shrink.

The first condition implies, that the alignment has to meet the Layout::from_size_align requirements.

The main advantage is, that one can simply pass a returned pointer back to the allocator, which will always fit the memory block. While this don't lift any safety conditions, it's more safe to use it. For structs storing only a NonNull<T> (like Box), a NonNull<[T]> can still be constructed with NonNull::slice_from_raw_parts(self.0, mem::align_of::<T>()), regardless of the returned ptr.len(), as it was requested with Layout::new<T>().

A minor downside are the parameters in grow and shrink as they take two usizes in a row. While the parameter order should be intuitive, this would be resolved, if we decide to allow reallocation to a different alignment (#5). Then, those methods would look like this:

unsafe fn grow/shrink(&mut self, NonNull<[u8]>, usize, Layout) -> Result<NonNull<[u8]>, AllocErr>;

closed time in 5 days

TimDiekmann

PR opened glandium/git-cinnabar

Fix helper version fallback.

Can't run .decode on a string.

+1 -1

0 comment

1 changed file

pr created time in 6 days

issue commentmozilla/sccache

sccache panic on linked_hash_map::Node being uninitialized

This is caused by lru-disk-cache depending on linked-hash-map v0.2.1. The issue was patched in v0.5.3. The issue to update the version is here: https://github.com/mozilla/sccache/issues/813 CVE: https://cve.mitre.org/cgi-bin/cvename.CGI?name=CVE-2020-25573 PR that fixes in linked-hash-map: https://github.com/containers/linked-hash-map/pull/100 Hope this info helps.

PeterGrace

comment created time in 6 days

issue commentglandium/git-cinnabar

`make install` does not work

In Fedora, I do:

install -d %{buildroot}%{gitexecdir}
install -p -m 0755 git-cinnabar %{buildroot}%{gitexecdir}
install -p -m 0755 git-cinnabar-helper %{buildroot}%{gitexecdir}
install -p -m 0755 git-remote-hg %{buildroot}%{gitexecdir}
install -d %{buildroot}%{gitexecdir}/cinnabar
install -p -m 0644 cinnabar/*.py %{buildroot}%{gitexecdir}/cinnabar
install -d %{buildroot}%{gitexecdir}/cinnabar/cmd
install -p -m 0644 cinnabar/cmd/*.py %{buildroot}%{gitexecdir}/cinnabar/cmd
install -d %{buildroot}%{gitexecdir}/cinnabar/hg
install -p -m 0644 cinnabar/hg/*.py %{buildroot}%{gitexecdir}/cinnabar/hg

Maybe something similar could work in the Makefile?

mjsir911

comment created time in 6 days

issue commentglandium/git-cinnabar

WARNING Mercurial libraries not found?

I had a similar issue but with git cinnabar python -c "import mercurial" raising ImportError: No module named mercurial. I fixed it via sudo pip install --target="/Library/Python/2.7/site-packages" mercurial

marcoscaceres

comment created time in 6 days

PR opened glandium/git-cinnabar

Add missing port number in test setup.

Otherwise it will fail to listen on port 88 when running as non-root.

+1 -0

0 comment

1 changed file

pr created time in 7 days

IssuesEvent

issue openedrust-lang/wg-allocators

Add a note to `AllocRef`, that implementation don't always has to be `'static`

In rust-lang/rust#79327 some 'static bounds were added to the Pin-API of Box. A note should be added somewhere (most likely to AllocRef, that this is not always required.

as we add allocator parameters, it can be easy to think that it'll always be 'static. Noting somewhere that this is not true explicitly I think would be good. Not directly related to Pin, though.

<sup>This is a reminder for me so that I don't forget to add it as soon as I have a chance.</sup>

created time in 7 days

issue openedmozilla/sccache

sccache does not work with docker build

sccache does not appear to work at all in build instructions of dockerfiles - depending on what I try I get errors about sending data to the server, connecting to the server or timeouts waiting for the sccache server to start.

I could have sworn this used to work, but I've tried many variations, and building on both WSL (Docker version 19.03.12, build 48a66213fe) and on Docker Desktop for Windows (Docker version 19.03.13, build 4484c46d9d), all with the same output as below.

sccache --version: sccache 0.2.13

Any help would be greatly appreciated!

Sample Dockerfile and Output

Dockerfile:

FROM rust
RUN cargo install sccache
ENV RUSTC_WRAPPER=/usr/local/cargo/bin/sccache
RUN USER=root cargo init hello_world
WORKDIR /hello_world
RUN cargo build

Output:

<snip>
#8 [5/5] RUN cargo build
#8 0.365 error: failed to run `rustc` to learn about target-specific information
#8 0.365
#8 0.365 Caused by:
#8 0.365   process didn't exit successfully: `/usr/local/cargo/bin/sccache rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit code: 2)
#8 0.365   --- stderr
#8 0.365   error: failed to execute compile
#8 0.365   caused by: Failed to send data to or receive data from server
#8 0.365   caused by: Failed to read response header
#8 0.365   caused by: failed to fill whole buffer
#8 ERROR: executor failed running [/bin/sh -c cargo build]: runc did not terminate sucessfully

created time in 7 days

issue commentmozilla/sccache

dist-client does not print directory path when cache dir creation fails.

Or particularly https://github.com/paritytech/sccache/pulls/10

Firstyear

comment created time in 12 days

more