profile
viewpoint
Xùdōng Yáng Wyverald Google Sydney, Australia

Wyverald/net9-auth 3

The authentication module for net9.

Wyverald/bazel 0

a fast, scalable, multi-language and extensible build system

Wyverald/bazel-website 0

Website for Bazel, a fast, scalable, multi-language and extensible build system

Wyverald/bazelisk 0

A user-friendly launcher for Bazel.

Wyverald/core 0

Cloud Robotics Core: Kubernetes, Federation, App Management

Wyverald/kythe 0

Kythe is a pluggable, (mostly) language-agnostic ecosystem for building tools that work with code.

push eventbazelbuild/bazel

cmita

commit sha bb9631eb7b73f027b8eaabffebf83360f2a26bcd

Expose textual and modular hdrs parameters in cc_common.create_compilation_context. These are undocumented parameters and are not meant to be used externally. They are guarded behind an internal allow list. PiperOrigin-RevId: 344538715

view details

push time in 23 minutes

push eventbazelbuild/bazel

cmita

commit sha e596689d36a5db8dbad8305362247a39fadd463e

Expose CcCompilationContext::getTransitiveCompilationPrerequisites to Starlark Usage is still undocumented and guarded by an internal allow list. PiperOrigin-RevId: 344538482

view details

push time in 26 minutes

issue openedbazelbuild/bazel

generate-xml.sh fails to execute

Description of the problem / feature request:

I'm running some test of TensorFlow using bazel but on our multi-core POWER9 system it fails with e.g.

ERROR: /dev/shm/s3248973-EasyBuild/TensorFlow/2.4.0/fosscuda-2019b-Python-3.7.4/TensorFlow/tensorflow-r2.4/tensorflow/core/platform/BUILD:1142:11: failed (Exit 1): generate-xml.sh failed: error executing command

I.e. there is no good error message, it simply failed to execute that script which comes from the Bazel installation. I verified that the executed command (bazel -s) runs correctly and the script hence also exists

I even modified that script in the Bazel sources to print something at the start but that doesn't show up. So it seems that script is not (yet?) created when Bazel tries to execute it. I hence expect a race condition or something but am unable to verify this.

Any hints, ideas, ...?

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Sorry, only thing I have is the command I use to test TF:

bazel --output_base=/dev/shm/s3248973-EasyBuild/TensorFlow/2.4.0/fosscuda-2019b-Python-3.7.4/tmptspeEg-bazel-tf/output_base --install_base=/dev/shm/s3248973-EasyBuild/TensorFlow/2.4.0/fosscuda-2019b-Python-3.7.4/tmptspeEg-bazel-tf/output_base/inst_base --output_user_root=/dev/shm/s3248973-EasyBuild/TensorFlow/2.4.0/fosscuda-2019b-Python-3.7.4/tmptspeEg-bazel-tf/output_user_root --host_jvm_args=-Xms512m --host_jvm_args=-Xmx4096m test --compilation_mode=opt --config=opt --subcommands --verbose_failures --config=noaws --jobs=64 --copt="-fPIC"  --distinct_host_configuration=false --test_output=errors --local_test_jobs=1 --build_tests_only --test_tag_filters='-no_gpu,-no_oss,-oss_serial,-benchmark-test,-no_oss_py37,-v1only' --build_tag_filters='-no_gpu,-no_oss,-oss_serial,-benchmark-test,-no_oss_py37,-v1only'  -- //tensorflow/core/... //tensorflow/cc/... //tensorflow/c/... -//tensorflow/core:example_java_proto -//tensorflow/core/example:example_protos_closure

What operating system are you running Bazel on?

RHEL 7.6

What's the output of bazel info release?

release 3.4.1- (@non-git)

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

EXTRA_BAZEL_ARGS="--jobs=176 --host_javabase=@local_jdk//:jdk" ./compile.sh

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

ERROR: /dev/shm/s3248973-EasyBuild/TensorFlow/2.4.0/fosscuda-2019b-Python-3.7.4/TensorFlow/tensorflow-r2.4/tensorflow/core/platform/BUILD:1142:11:  failed (Exit 1): generate-xml.sh failed: error executing command 
  (cd /dev/shm/s3248973-EasyBuild/TensorFlow/2.4.0/fosscuda-2019b-Python-3.7.4/tmptspeEg-bazel-tf/output_base/execroot/org_tensorflow && \
  exec env - \
    PATH=/usr/bin:/bin \
> Replace these lines with your answer.
>
> If the files are large, upload as attachment or provide link.

    TEST_BINARY=tensorflow/core/platform/platform_strings_test \
    TEST_NAME=//tensorflow/core/platform:platform_strings_test \
    TEST_SHARD_INDEX=0 \
    TEST_TOTAL_SHARDS=0 \
  external/bazel_tools/tools/test/generate-xml.sh bazel-out/ppc-opt/testlogs/tensorflow/core/platform/platform_strings_test/test.log bazel-out/ppc-opt/testlogs/tensorflow/core/platform/platform_strings_test/test.xml 0 0)
Execution platform: @local_execution_config_platform//:platform

created time in 29 minutes

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

It's probably not too challenging to set up a bazel-bench pipeline to benchmark the java_tools commits. We can talk more next week if you're interested.

joeleba

comment created time in 41 minutes

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

we might want to consider automatic releasing or at least benchmarking the commits that go into java_tools... i almost have a script for that by now

joeleba

comment created time in an hour

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

aha, the first commit brought back the System.gc that @larsrc-google recently removed and the second commit removed it again

joeleba

comment created time in an hour

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

Raw data:

RESULTS:
Bazel commit: /tmp/benchmark-4-febf4438cd_HEAD_1, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: None
  metric          mean                median          stddev      pval   
    wall:  163.424s             163.606s               0.718s            
     cpu:   54.550s              54.520s               1.046s            
  system:   16.302s              16.280s               0.094s            
  memory:   92.400s              92.000s               0.490s            

Bazel commit: /tmp/benchmark-4-febf4438cd_HEAD, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: None
  metric          mean                median          stddev      pval   
    wall:  163.633s ( +0.13%)   162.889s ( -0.44%)     1.184s    0.12698 
     cpu:   53.980s ( -1.04%)    53.710s ( -1.49%)     1.565s    0.12698 
  system:   16.528s ( +1.39%)    16.290s ( +0.06%)     0.549s    0.12698 
  memory:   92.800s ( +0.43%)    93.000s ( +1.09%)     0.400s    0.12698 
no regression

RESULTS:
Bazel commit: /tmp/benchmark-5-5c8e584e88_HEAD_1, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: None
  metric          mean                median          stddev      pval   
    wall:  162.196s             161.956s               1.264s            
     cpu:   53.400s              53.060s               0.952s            
  system:   16.470s              16.430s               0.294s            
  memory:   92.800s              93.000s               0.400s            

Bazel commit: /tmp/benchmark-5-5c8e584e88_HEAD, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: None
  metric          mean                median          stddev      pval   
    wall:  226.739s (+39.79%)   224.517s (+38.63%)     7.650s    0.99206 
     cpu:   53.824s ( +0.79%)    54.330s ( +2.39%)     1.454s    0.12698 
  system:   16.730s ( +1.58%)    16.730s ( +1.83%)     0.140s    0.64286 
  memory:   92.200s ( -0.65%)    92.000s ( -1.08%)     0.748s    0.12698 
anything else I can help you with?

RESULTS:
Bazel commit: /tmp/benchmark-8-a4251eab69_HEAD_1, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: https://github.com/bazelbuild/bazel.git
  metric          mean                median          stddev      pval   
    wall:  222.846s             221.188s              5.016s             
     cpu:   53.466s              54.300s              1.425s             
  system:   16.564s              16.500s              0.177s             
  memory:   92.200MB             92.000MB             0.400MB            

Bazel commit: /tmp/benchmark-8-a4251eab69_HEAD, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: https://github.com/bazelbuild/bazel.git
  metric          mean                median          stddev      pval   
    wall:  164.262s  (-26.29%)  162.785s  (-26.40%)   3.078s     0.99206 
     cpu:   56.008s  ( +4.75%)   55.070s  ( +1.42%)   2.373s     0.64286 
  system:   16.352s  ( -1.28%)   16.160s  ( -2.06%)   0.411s     0.64286 
  memory:   92.800MB ( +0.65%)   93.000MB ( +1.09%)   0.400MB    0.64286
joeleba

comment created time in an hour

issue closedbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

Description of the problem / feature request:

Our bazel-bench pipeline runs nightly benchmarking of a sample of the bazel commits throughout the day, by using the bazel binaries built at those commits to build the Bazel project itself (bazel build //src:bazel). We've identified the following:

  1. 5c92b11dc5e1decf1b2a30b72d9bdaaf6e3e4380 regressed wall time by ~48%.
Bazel commit: 5c92b11dc5e1decf1b2a30b72d9bdaaf6e3e4380, Project commit: c282526c071389cd6f88cb77565283b257316267, Project source: https://github.com/bazelbuild/bazel.git
  metric          mean                median          stddev      pval   
    wall:  116.968s ( -0.93%)   117.305s ( -0.48%)     2.287s    0.00000 
     cpu:   79.557s ( +0.29%)    80.480s ( +0.51%)     1.464s    0.40000 
  system:   19.167s ( -1.83%)    19.240s ( -2.58%)     0.148s    0.40000 
  memory:   94.667s ( +0.00%)    95.000s ( +0.00%)     0.471s    0.00000 

Bazel commit: c282526c071389cd6f88cb77565283b257316267, Project commit: c282526c071389cd6f88cb77565283b257316267, Project source: https://github.com/bazelbuild/bazel.git
  metric          mean                median          stddev      pval   
    wall:  173.391s (+48.24%)   173.878s (+48.23%)     1.361s    0.90000 
     cpu:   81.277s ( +2.16%)    81.930s ( +1.80%)     2.404s    0.40000 
  system:   18.187s ( -5.11%)    18.450s ( -4.11%)     0.794s    0.40000 
  memory:   94.000s ( -0.70%)    94.000s ( -1.05%)     0.000s    0.40000 

image

  1. cd3480e4d53160d20c634134d3917c4e53ac1c49 improved wall time by ~30%.
Bazel commit: 1517286783762647b4922ed1c656262002abedf1, Project commit: 1517286783762647b4922ed1c656262002abedf1, Project source: https://github.com/bazelbuild/bazel.git
  metric          mean                median          stddev      pval   
    wall:  184.560s             185.592s              1.563s             
     cpu:   82.817s              84.350s              2.996s             
  system:   19.877s              19.230s              1.111s             
  memory:   95.000MB             95.000MB             0.000MB            

Bazel commit: cd3480e4d53160d20c634134d3917c4e53ac1c49, Project commit: 1517286783762647b4922ed1c656262002abedf1, Project source: https://github.com/bazelbuild/bazel.git
  metric          mean                median          stddev      pval   
    wall:  128.454s  (-30.40%)  129.437s  (-30.26%)   1.443s     0.90000 
     cpu:   86.013s  ( +3.86%)   87.650s  ( +3.91%)   2.625s     0.40000 
  system:   20.917s  ( +5.23%)   20.820s  ( +8.27%)   0.227s     0.40000 
  memory:   94.000MB ( -1.05%)   94.000MB ( -1.05%)   0.000MB    0.90000 

image

Feature requests: what underlying problem are you trying to solve with this feature?

Maybe we can use this to figure out what in java_tools regressed the performance of Bazel and what corrected it (partially), and whether there's something more we can do.

What operating system are you running Bazel on?

Ubuntu18.04

Any other information, logs, or outputs that you want to share?

To reproduce, clone bazelbuild/bazel-bench and run:

# 1st benchmark.
bazel run benchmark --  \
--bazel_source=https://github.com/bazelbuild/bazel.git \
--project_source=https://github.com/bazelbuild/bazel.git \
--collect_memory \
--runs=3 \
--bazel_commits=5c92b11dc5e1decf1b2a30b72d9bdaaf6e3e4380,c282526c071389cd6f88cb77565283b257316267 \
--project_commits=c282526c071389cd6f88cb77565283b257316267 \
-- build //src:bazel

# 2nd benchmark
bazel run benchmark --  \
--bazel_source=https://github.com/bazelbuild/bazel.git \
--project_source=https://github.com/bazelbuild/bazel.git \
--collect_memory \
--data_directory=/tmp/.bazel-bench/out \
--runs=3 \
--bazel_commits=1517286783762647b4922ed1c656262002abedf1,cd3480e4d53160d20c634134d3917c4e53ac1c49 \
--project_commits=1517286783762647b4922ed1c656262002abedf1 \
-- build //src:bazel

closed time in an hour

joeleba

Pull request review commentrust-lang/rfcs

RFC: Serve crates-io registry over HTTP as static files

+- Feature Name: http_index+- Start Date: 2019-10-18+- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)+- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)++# Summary+[summary]: #summary++Selective download of the crates-io index over HTTP, similar to a solution used by Ruby's Bundler. Changes transport from an ahead-of-time Git clone to HTTP fetch as-needed, while keeping existing content and structure of the index. Most importantly, the proposed solution works with static files and doesn't require custom server-side APIs.++# Motivation+[motivation]: #motivation++The full crate index is relatively big and slow to download. It will keep growing as crates.io grows, making the problem worse. The requirement to download the full index slows down the first use of Cargo. It's especially slow and wasteful in stateless CI environments, which download the full index, use only a tiny fraction of it, and throw it away. Caching of the index in hosted CI environments is difficult (`.cargo` dir is large) and often not effective (e.g. upload and download of large caches in Travis CI is almost as slow as a fresh index download).++The kind of data stored in the index is not a good fit for the git protocol. The index content (as of eb037b4863) takes 176MiB as an uncompressed tarball, 16MiB with `gz -1`, and 10MiB compressed with `xz -6`. Git clone reports downloading 215MiB. That's more than just the uncompressed latest index content, and over **20 times more** than a compressed tarball.++A while ago, GitHub indicated they [don't want to support shallow clones of large repositories](http://blog.cocoapods.org/Master-Spec-Repo-Rate-Limiting-Post-Mortem/). libgit2 doesn't support shallow clones yet. Squashing of the index history adds complexity to management and consumption of the index (which is also used by tools other than Cargo), and still doesn't solve problems of the git protocol inefficiency and overall growth.++# Guide-level explanation+[guide-level-explanation]: #guide-level-explanation++Expose the index over HTTP as simple files, keeping the existing content and directory layout unchanged (the existing raw.githubusercontent.com view may even be enough for this). The current format is structured like this:++```+/config.json+/ac/ti+/ac/ti/action+/ac/ti/actiondb+/ac/ti/actions+/ac/ti/actions-toolkit-sys+/ac/ti/activation+/ac/ti/activeds-sys+…+```++To learn about crates and resolve dependencies, Cargo (or any other client) would make requests to known URLs for each dependency it needs to learn about, e.g. `https://index.example.com/se/rd/serde`. For each dependency the client would also have to request information about its dependencies, recursively, until all dependencies are fetched (and cached) locally.++It's possible to request dependency files in parallel, so the worst-case latency of such dependency resolution is limited to the maximum depth of the dependency tree. In practice it's less, because dependencies occur in multiple places in the tree, allowing earlier discovery and increasing parallelization. Additionally, if there's a lock file, all dependencies listed in it can be speculatively checked in parallel. Similarly, cached dependency files can be used to speculatively check known sub-dependencies sooner.++## Greedy fetch++To simplify the implementation, and parallelize fetches effectively, Cargo will have to fetch all dependency information before performing the actual dependency resolution algorithm. This means it'll have to pessimistically fetch information about all sub dependencies of all dependency versions that *may* match known version requirements. This won't add much overhead, because requests are per create, not per crate version. It causes additional fetches only for dependencies that were used before, but were later dropped. Fetching is still narrowed by required version ranges, so even worst cases can be avoided by bumping version requirements. For example:++* foo v1.0.1 depends on old-dep v1.0.0+* foo v1.0.2 depends on maybe-dep v1.0.2+* foo v1.0.3 depends on maybe-dep v1.0.3+* foo v1.0.4 has no dependencies++If a dependency requires `foo >=1.0.2`, then Cargo would need to fetch information about `maybe-dep` (once), even if `foo v1.0.4` ends up being selected later. However, it would not need to fetch `old-dep`. If the version requirement was upgraded to `foo >=v1.0.4` then there wouldn't be any extra fetches.++## Offline support++The proposed solution fully preserves Cargo's ability to work offline. Fetching of crates while online by necessity downloads enough of the index to use them, and all this data remains cached for use offline.++## Bandwidth reduction++Cargo supports HTTP/2, which handles many similar requests efficiently.++All fetched dependency files can be cached, and refreshed using conditional HTTP requests (with `Etag` or `If-Modified-Since` headers), to avoid redownloading of files that haven't changed.++Dependency files compress well. Currently the largest file of `rustc-ap-rustc_data_structures` compresses from 1MiB to 26KiB with Brotli. Many servers support transparently serving pre-compressed files (i.e. request for `/rustc-ap-rustc_data_structures` can be served from `rustc-ap-rustc_data_structures.gz` with an appropriate content encoding header), so the index can use high compression levels without increasing CPU cost of serving the files.++Even in the worst case of downloading the entire index file by file, it should still use significantly less bandwidth than git clone (individually compressed files add up to about 39MiB).++## Optionally, a rotated incremental changelog++To further reduce number requests needed to update the index, the index may maintain an append-only log of changes. For each change (crate version published or yanked), the log would append a line with: epoch number (explained below), last-modified timestamp, and the name of the changed crate, e.g.++```+1 2019-10-18 23:51:23 oxigen+1 2019-10-18 23:51:25 linda+1 2019-10-18 23:51:29 rv+1 2019-10-18 23:52:00 anyhow+1 2019-10-18 23:53:03 build_id+1 2019-10-18 23:56:16 canonical-form+1 2019-10-18 23:59:01 cotton+1 2019-10-19 00:01:44 kg-utils+1 2019-10-19 00:08:45 serde_traitobject
1 2019-10-18T23:51:23Z oxigen
1 2019-10-18T23:51:25Z linda
1 2019-10-18T23:51:29Z rv
1 2019-10-18T23:52:00Z anyhow
1 2019-10-18T23:53:03Z build_id
1 2019-10-18T23:56:16Z canonical-form
1 2019-10-18T23:59:01Z cotton
1 2019-10-19T00:01:44Z kg-utils
1 2019-10-19T00:08:45Z serde_traitobject

I guess it would probably be best to use ISO8601 dates in UTC here to simplify the parsing.

kornelski

comment created time in an hour

issue openedbazelbuild/bazel

bazel_bootstrap_distfile_test failing in postsubmit on Windows

The test //src/test/shell/bazel:bazel_bootstrap_distfile_test has started to fail in our postsubmit pipeline on Windows only.

Here's a list of broken jobs:

  • https://buildkite.com/bazel/bazel-bazel/builds/14736#745cad19-516a-479b-8084-9154bf14c893
  • https://buildkite.com/bazel/bazel-bazel/builds/14740#75c2bc28-cf58-4f5d-a152-a94ef3d5cde1
  • https://buildkite.com/bazel/bazel-bazel/builds/14742#65cec04d-80c2-4661-8f61-e1c4276ddde3
  • https://buildkite.com/bazel/bazel-bazel/builds/14743#a2981e51-fe4e-43da-bd32-efacfaf4fd5b

I have not seen it happen before job 14736, so this might be the culprit. 🤔 I'm not aware of any CI infrastructure or Windows image changes in the last days.

Here's an example log: https://storage.googleapis.com/bazel-untrusted-buildkite-artifacts/745cad19-516a-479b-8084-9154bf14c893/src%5Ctest%5Cshell%5Cbazel%5Cbazel_bootstrap_distfile_test%5Cattempt_1.log

The relevant error message seems to be:

ERROR: Analysis of target '//src:bazel_nojdk.exe' failed; build aborted: invalid registered execution platform '//:default_host_platform': no such target '//:default_host_platform': target 'default_host_platform' not declared in package '' defined by C:/b/iwgf237n/execroot/io_bazel/_tmp/a6b4c571d5cd258ae95c5906c6c25060/bazelbootstrap.iIshY86j/BUILD

When the test fails during a job, it will consistently fail during all attempts, always with the same error message. It is thus not a typically flaky test which fails with a random chance.

created time in an hour

push eventbazelbuild/bazel

Ivo List

commit sha 6202271b9b1bdf1b73b1e2ad55f30cf467899005

Use the java_tools release split into platform independent and prebuilt part. This builds on top of the commit "Separate java_tools into platform independent and prebuilt part.". - java_tools release 11.0 is used - java_tools platform independent repo is added - java toolchain definitions appropriate for new release are added to tools/jdk/BUILD - remote_java_tools are replaced with aliases - tests are reenabled Changelog: ``` $ git log --pretty=format:"%h %an %s" cd3480e4...5a91c25ca43547870bcf73cf4b427277c7f62d8d -- src/java_tools/ third_party/ijar/ tools/jdk/BUILD.java_tools d10013de94 Ivo List Separate java_tools into platform independent and prebuilt part. 3c9ad2dc20 philwo Separate ijar sources from deployment zip. (#12556) 2d990cf2fc philwo Fix #10127: Remove Python 2 dependency from tools/android. de2dab0723 Ivo List Separate ijar sources from deployment zip. b17e9e4dfb Googler Automatic code cleanup. ``` Benchmark: ``` Bazel commit: /tmp/bazel-head, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: None metric mean median stddev pval wall: 185.647s 185.445s 2.224s cpu: 55.616s 54.410s 1.792s system: 18.043s 18.060s 0.330s memory: 94.000s 94.000s 0.000s Bazel commit: /tmp/comius-bazel, Project commit: c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1, Project source: None metric mean median stddev pval wall: 184.544s ( -0.59%) 184.456s ( -0.53%) 1.626s 0.03730 cpu: 54.484s ( -2.03%) 54.990s ( +1.07%) 1.940s 0.42483 system: 17.364s ( -3.76%) 17.260s ( -4.43%) 0.392s 0.94697 memory: 94.000s ( +0.00%) 94.000s ( +0.00%) 0.000s 0.00000 ``` Closes #12552. Fixes #8406. PiperOrigin-RevId: 344530341

view details

push time in 2 hours

PR closed bazelbuild/bazel

Reviewers
Use the java_tools release split into platform independent and prebuilt part. cla: yes

This builds on top of the commit "Separate java_tools into platform independent and prebuilt part.".

  • java_tools release 11.0 is used
  • java_tools platform independent repo is added
  • java toolchain definitions appropriate for new release are added to tools/jdk/BUILD
  • remote_java_tools are replaced with aliases
  • tests are reenabled
+341 -241

2 comments

13 changed files

comius

pr closed time in 2 hours

issue closedbazelbuild/bazel

Add a platform agnostic release for java_tools.

Create a way to release a java_tools version (in addition to the linux, darwin and winodws versions) without including pre-built ijar and singlejar.

closed time in 2 hours

iirina

PR closed bazelbuild/bazel

Reduce platform dependant selects in //tools/jdk/BUILD cla: yes

Macro default_java_toolchain used to create a java_toolchain rule with each attribute set using a platform dependent select. For example ijar attribute was set to //tools/jdk:ijar, which was a macro remote_java_tools_filegroup. That macro contains a select choosing the right @remote_java_tools repository.

With Java toolchainisation this become a fragile mechanism (selects need to match the toolchain...), which would be hard to maintain.

In previous commits java_toolchain_default was implemented, which is part of each java_tools repository and does not use any selects.

For backward compatibility default_java_toolchain was reimplemented to invoke java_toolchain_default. Implementation is not perfect, because it load 3 java_tools repositories (linux,darwin,windows). Not to invoke this misfeature, some macros were moved into utils.bzl. The use of default_java_toolchain should be discouraged.

remote_java_tools_filegroup/import targets in //tools/jdk/BUILD that were previous used only by default_java_toolchain were removed.

default_java_toolchain targets in //tools/jdk/BUILD were replaced with select aliases. Those will be removed when Java toolchainsation is complete.

legacy_toolchain was removed as a fall-back - it was an awkard one - going from Java 11 to Java 8 language level.

+163 -269

1 comment

7 changed files

comius

pr closed time in 2 hours

issue commentbazelbuild/bazel

Release 4.0 - December 2020

Just to make you aware, #12574 is still under investigation but may make another cherry pick necessary.

philwo

comment created time in 2 hours

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

Releasing java_tools at:

commit 4 no regression febf4438cd cushon Restore GenClass's --temp_dir flag once more

commit 5 38% regression in wall 5c8e584e88 steinman Move WorkRequestHandler from build jar to Worker package. This is part of the cleanup associated with enabling JSON workers.

commit 8 TODO a4251eab69 steinman Refactor WorkRequestHandler to be an interface, of which Proto is one implementation.

joeleba

comment created time in 2 hours

push eventbazelbuild/bazel

djasper

commit sha 97104490bf67109bccf49b09e5900d2589f4b40b

Intern includes extracted during C++ include scanning. #includes are often write the same way in many files during a build so that this can save substantial amounts of memory in large builds. RELNOTES: None. PiperOrigin-RevId: 344527165

view details

push time in 3 hours

issue commentbazelbuild/bazel

Doc for @bazel_tools

Not sure is it important there, but i came across this error/warning when run ibazel:

iBazel [3:29PM]: Querying for files to watch...
Loading: 0 packages loaded
ERROR: /home/dzintars/.cache/bazel/_bazel_dzintars/73eb78004b7f7dbbf7672ded04d50930/external/bazel_tools/tools/osx/BUILD:80:19: every rule of type xcode_config_alias implicitly depends upon the target '@bazel_tools//tools/objc:host_xcodes', but this target could not be found because of: no such target '@bazel_tools//tools/objc:host_xcodes': target 'host_xcodes' not declared in package 'tools/objc' defined by /home/dzintars/.cache/bazel/_bazel_dzintars/73eb78004b7f7dbbf7672ded04d50930/external/bazel_tools/tools/objc/BUILD.bazel
ERROR: /home/dzintars/.cache/bazel/_bazel_dzintars/73eb78004b7f7dbbf7672ded04d50930/external/bazel_tools/tools/osx/BUILD:80:19: every rule of type xcode_config_alias implicitly depends upon the target '@bazel_tools//tools/objc:host_xcodes', but this target could not be found because of: no such target '@bazel_tools//tools/objc:host_xcodes': target 'host_xcodes' not declared in package 'tools/objc' defined by /home/dzintars/.cache/bazel/_bazel_dzintars/73eb78004b7f7dbbf7672ded04d50930/external/bazel_tools/tools/objc/BUILD.bazel
ERROR: Evaluation of subquery "deps(set(//platform/web/shell/server))" failed (did you want to use --keep_going?): errors were encountered while computing transitive closure
Loading: 1 packages loaded
Loading: 1 packages loaded
iBazel [3:29PM]: Bazel query failed: exit status 7

So i am looking forward to resolve this "warning".

olafure

comment created time in 4 hours

issue closedbazelbuild/bazel

BAZEL_VS environment variable does not work

ATTENTION! Please read and follow:

  • if this is a question about how to build / test / query / deploy using Bazel, or a discussion starter, send it to bazel-discuss@googlegroups.com
  • if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

I'm using bazel 2.0.0 with VS Code bazel plugin. I have set BAZEL_VS env variable to C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools globally. However, when I build a target in VS Code (clicking on build target from right-click menu in BAZEL BUILD TARGETS sidebar), it failed to probe compiler with following error logs:

ERROR: An error occurred during the fetch of repository 'local_config_cc':
   Traceback (most recent call last):
        File "C:/users/yangxi/_bazel_yangxi/y7qzzhw6/external/bazel_tools/tools/cpp/cc_configure.bzl", line 120                configure_windows_toolchain(<1 more arguments>)
        File "C:/users/yangxi/_bazel_yangxi/y7qzzhw6/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", 
line 690, in configure_windows_toolchain
                _get_msvc_vars(repository_ctx, <1 more arguments>)
        File "C:/users/yangxi/_bazel_yangxi/y7qzzhw6/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", 
line 535, in _get_msvc_vars
                setup_vc_env_vars(<2 more arguments>)
        File "C:/users/yangxi/_bazel_yangxi/y7qzzhw6/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", 
line 318, in setup_vc_env_vars
                _check_env_vars(env_map, cmd, <1 more arguments>)
        File "C:/users/yangxi/_bazel_yangxi/y7qzzhw6/external/bazel_tools/tools/cpp/windows_cc_configure.bzl", 
line 324, in _check_env_vars
                auto_configure_fail(<1 more arguments>)
        File "C:/users/yangxi/_bazel_yangxi/y7qzzhw6/external/bazel_tools/tools/cpp/lib_cc_configure.bzl", line 112, in auto_configure_fail
                fail(<1 more arguments>)

Auto-Configuration Error: Setting up VC environment variables failed, INCLUDE is not set by the following command:
    "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\\VCVARSALL.BAT" amd64

It seems the specified version of VS is not used. Instead it is still using VS 2015.

At current, I could only build targets inside Developer Command Prompt for VS 2017, which have set up many stuffs.

Feature requests: what underlying problem are you trying to solve with this feature?

Replace this line with your answer.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Replace this line with your answer.

What operating system are you running Bazel on?

Windows 10

What's the output of bazel info release?

release 2.0.0

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

Replace this line with your answer.

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

Replace this line with your answer.

Have you found anything relevant by searching the web?

Replace these lines with your answer.

Places to look:

  • StackOverflow: http://stackoverflow.com/questions/tagged/bazel
  • GitHub issues: https://github.com/bazelbuild/bazel/issues
  • email threads on https://groups.google.com/forum/#!forum/bazel-discuss

Any other information, logs, or outputs that you want to share?

Replace these lines with your answer.

If the files are large, upload as attachment or provide link.

closed time in 5 hours

jiandingzhe

issue commentbazelbuild/bazel

BAZEL_VS environment variable does not work

I tried it and I think it is fixed, I will close it.

jiandingzhe

comment created time in 5 hours

push eventbazelbuild/bazel

lberki

commit sha 253933f3adda134494a4f55838b3e16e54652f23

Remove the --incompatible_load_python_rules flag. Migrating Python rules to Starlark this way did not work. RELNOTES: The --incompatible_load_python_rules_from_bzl flag is now a no-op. PiperOrigin-RevId: 344505298

view details

push time in 7 hours

push eventbazelbuild/bazel

plf

commit sha 8bc85c2ad4be69b86a53b139729c2e3716cd1243

Fix coverage tests so that they work on CI on Ubuntu 20 Progress on #12164. Fixes remaining coverage test. The issue was caused by Ubuntu 20 being the first distribution that we have on CI to use gcc 9. From gcc 9 and above the intermediate format used by gcov is different and relies on json zipped files. I adapted the test to work with version 9 and higher and left the code that tests gcc versions 7 and 8 for now, but will be removed eventually. RELNOTES:none PAIR=cmita PiperOrigin-RevId: 344502710

view details

push time in 8 hours

push eventbazelbuild/bazel

cmita

commit sha b149c9979a3417b691a9afe79b4160391e93725e

Use the standard private allowlist check for CcDebugInfo The same list is used, so there isn't really any point in keeping the separate check function.. PiperOrigin-RevId: 344500205

view details

push time in 8 hours

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

Let me know if you need help performing the benchmarks.

Can you also run benchmark on #12552? It has release 11.0. I don't have a workstation only cloudtop.

Sure, gimme a moment.

joeleba

comment created time in 8 hours

PR closed bazelbuild/bazel

Accept proto_library as //tools/objc:protobuf_well_known_types cla: yes

Part of the migration of Protobuf rules/aspects to ProtoSource. A future change will make ProtoInfo mandatory (similar to what was done for proto_lang_toolchain).

Note that this is for Obj-C Protobuf rules which don't exist in Bazel.

Updates #10005

+14 -1

1 comment

2 changed files

Yannic

pr closed time in 8 hours

push eventbazelbuild/bazel

Yannic Bonenberger

commit sha c85ecbed46d4703c7d6bf6b52dab98972dbe9ea1

Accept proto_library as //tools/objc:protobuf_well_known_types Part of the migration of Protobuf rules/aspects to `ProtoSource`. A future change will make `ProtoInfo` mandatory (similar to what was done for `proto_lang_toolchain`). Note that this is for Obj-C Protobuf rules which don't exist in Bazel. Updates #10005 Closes #12486. PiperOrigin-RevId: 344498042

view details

push time in 8 hours

issue commentbazelbuild/bazel

Compilation modes dbg and opt have no effect with compiler mingw-gcc on Windows

/cc @mai93, maybe Mai could take a look?

johnplate

comment created time in 9 hours

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

Let me know if you need help performing the benchmarks.

Can you also run benchmark on https://github.com/bazelbuild/bazel/pull/12552? It has release 11.0. I don't have a workstation only cloudtop.

joeleba

comment created time in 9 hours

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

Let me know if you need help performing the benchmarks.

joeleba

comment created time in 9 hours

issue commentbazelbuild/bazel

java_tools 10.3 regressed bazel performance, before java_tools javac11 v10.4 improved it.

It would be great if the commits of the java_tools releases contain these changelogs, so it's easier to understand what went into them.

I think Ivo is correct, that the regression is completely fixed by the 10.4 release, because there's another regression in the 8fce67f..b1d1485 range that adds up: image

Ivo: it would be still good to know what caused the regression (and what fixed it) before we pull the trigger on the 4.0 release.

joeleba

comment created time in 9 hours

more