profile
viewpoint
If you are wondering where the data of this site comes from, please visit https://api.github.com/users/shawnl/events. GitMemory does not store any data, but only uses NGINX to cache data for a period of time. The idea behind GitMemory is simply to give users a better reading experience.

llvm/llvm-project 10799

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. Note: the repository does not accept github pull requests at this moment. Please submit your patches at http://reviews.llvm.org.

distcc/distcc 1504

distributed builds for C, C++ and Objective C

distcc/distcc.github.io 4

The distcc website

shawnl/command-not-found 3

command not found

shawnl/bedrock 1

Making mozilla.org awesome, one pebble at a time

shawnl/calagator 1

An event aggregator based in Portland, OR

shawnl/fortunes-wilde 1

Oscar Wilde Fortunes

shawnl/gitignore 1

A collection of useful .gitignore templates

mgerring/datafest-moneymap 0

Map of variables influencing money effectiveness in California

issue commentdistcc/distcc

Build Error - make deb

If you want a deb package, you should use the packaging the official debian or ubuntu packages use. only ./configure|make|make check|make install is supported.

squidgegmc

comment created time in 12 hours

startedTow-Boot/Tow-Boot

started time in a day

issue closeddistcc/distcc

Build Error - make deb

I have a clean VM running Ubuntu 20.04 srv.

I am hitting build errors when attempting make deb:

image

closed time in 2 days

squidgegmc

issue commentdistcc/distcc

Build Error - make deb

That is not supported.

squidgegmc

comment created time in 2 days

issue commentdistcc/distcc

Error when compiling with distcc a package that use autoconf and DISTCC_VERBOSE=1

check out the log on the server

sylmio

comment created time in 3 days

issue commentdistcc/distcc

Error when compiling with distcc a package that use autoconf and DISTCC_VERBOSE=1

What problem are you trying to solve?

sylmio

comment created time in 7 days

issue commentdistcc/distcc

Error when compiling with distcc a package that use autoconf and DISTCC_VERBOSE=1

DISTCC_VERBOSE=1 is really verbose, and you shouldn't be setting it globally.

El vie., 15 oct. 2021 15:23, sylmio ***@***.***> escribió:

Strictly speaking there is no bug in distcc. But the outputs in verbose mode perturbs autoconf. So I think there is a bug when combining autoconf+distcc with DISTCC_VERBOSE=1.

You can reproduce the problem by launching the configure script of a package and seeing that it finds -fPIC does not work, and then not using it when it should (for shared libraries for example). It is not due to Gentoo system, and I suppose it will happen when compiling a package whatever the linux distribution.

Since distcc detects it is called by autoconf, pb could be solved by reducing output to reproducible one during autoconf (removing instance number, computation time). Or more simply, remove all outputs during autoconf., and add an additional verbose level if ones need output during autoconf with DISTCC_VERBOSE=2 or DISTCC_VERBOSE_AUTOCONF=1.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/distcc/distcc/issues/441#issuecomment-944220933, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD4W4WF3FYPTWNUXYORCPLUHAFKLANCNFSM5F7IQ5QQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

sylmio

comment created time in 8 days

PR merged distcc/distcc

Improved cross-rewriting on non-x86 systems

Unfortunately autoconf and GCC don't agree the system name:

  • On arm (aarch64): GNU_HOST is aarch64-unknown-linux-gnu, and GCC triple is aarch64-linux-gnu instead.
  • On rpm-based x86_64 distros: GNU_HOST is x86_64-pc-linux-gnu, and GCC triple is x86_64-redhat-linux

Therefore ask the compiler (when running distcc configure script) how to correctly identify it.

Closes: #440 ALTBUG: 40425

+19 -7

1 comment

2 changed files

asheplyakov

pr closed time in 8 days

push eventdistcc/distcc

Alexey Sheplyakov

commit sha 850db9eec0d5dd7f47ade8ffca91b679081f6d85

Improved cross-rewriting on non-x86 systems Unfortunately autoconf and GCC don't agree on the system name: - On arm (aarch64): `GNU_HOST` is `aarch64-unknown-linux-gnu`, and GCC triple is `aarch64-linux-gnu` instead. - On rpm-based x86_64 distros: `GNU_HOST` is `x86_64-pc-linux-gnu`, and GCC triple is `x86_64-redhat-linux` Therefore ask the compiler (when running distcc configure script) how to correctly identify it. Closes: #440 ALTBUG: 40425

view details

Shawn Landden

commit sha fae8362d4cacc8ebecdbd9b7adf5de853a10ead5

Merge pull request #427 from asheplyakov/native-compiler-triple-fixup Improved cross-rewriting on non-x86 systems Tested on aarch64 (Debian).

view details

push time in 8 days

issue closeddistcc/distcc

cross-rewriting does not work on arm architecture

Client: arm64 (raspberry pi 4 with a native GCC compiler) Compile nodes: x86_64 with an arm64-targeted cross-compiler aarch64-linux-gnu-gcc

On raspberry pi:

cat > hello.c <<EOF
#include <stdio.h>
int main(int argc, char **argv) {
     printf("Hello, world!\n");
     return 0;
}
EOF
distcc gcc -c hello.c
file hello.o
hello.o: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped

Verbose distcc log:

env DISTCC_VERBOSE=1 ./distcc gcc -c  hello.c
distcc[17003] (dcc_trace_version) distcc 3.4 aarch64-unknown-linux-gnu; built Sep 30 2021 16:11:41
distcc[17003] (dcc_recursion_safeguard) safeguard level=0
distcc[17003] (main) compiler name is "distcc"
distcc[17003] (dcc_scan_args) scanning arguments: gcc -c hello.c
distcc[17003] (dcc_scan_args) found input file "hello.c"
distcc[17003] (dcc_scan_args) no visible output file, going to add "-o hello.o" at end
distcc[17003] compile from hello.c to hello.o
distcc[17003] (dcc_get_hostlist) read hosts from environment
distcc[17003] (dcc_parse_hosts) found tcp token "X86HOST/20"
distcc[17003] (dcc_lock_host) got cpu lock on X86HOST/20 slot 0 as fd3
distcc[17003] (dcc_lock_host) got cpu lock on localhost slot 0 as fd4
distcc[17003] (dcc_note_state) note state 3, file "(NULL)", host "localhost"
distcc[17003] (dcc_strip_dasho) result: gcc -c hello.c
distcc[17003] (dcc_spawn_child) forking to execute: gcc -E hello.c
distcc[17003] (dcc_spawn_child) child started as pid17004
distcc[17003] (dcc_strip_local_args) result: gcc -c hello.c -o hello.o
distcc[17003] exec on X86HOST/20: gcc -c hello.c -o hello.o
distcc[17003] (dcc_note_state) note state 2, file "hello.c", host "X86HOST"
distcc[17004] (dcc_new_pgrp) entered process group
distcc[17004] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[17003] (dcc_connect_by_addr) started connecting to NN.NN.NN.NN:3632
distcc[17003] (dcc_select_for_write) select for write on fd5
distcc[17003] (dcc_note_state) note state 4, file "(NULL)", host "(NULL)"
distcc[17003] (dcc_x_token_int) send DIST00000001
distcc[17003] (dcc_x_token_int) send ARGC00000005
distcc[17003] (dcc_x_token_int) send ARGV00000003
distcc[17003] (dcc_x_token_string) send string 'gcc'
distcc[17003] (dcc_x_token_int) send ARGV00000002
distcc[17003] (dcc_x_token_string) send string '-c'
distcc[17003] (dcc_x_token_int) send ARGV00000007
distcc[17003] (dcc_x_token_string) send string 'hello.c'
distcc[17003] (dcc_x_token_int) send ARGV00000002
distcc[17003] (dcc_x_token_string) send string '-o'
distcc[17003] (dcc_x_token_int) send ARGV00000007
distcc[17003] (dcc_x_token_string) send string 'hello.o'
distcc[17003] (dcc_note_state) note state 3, file "(NULL)", host "(NULL)"
distcc[17003] (dcc_collect_child) cpp child 17004 terminated with status 0
distcc[17003] (dcc_collect_child) cpp times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[17003] cpp hello.c on localhost completed ok
distcc[17003] (dcc_unlock) release lock fd4
distcc[17003] (dcc_x_file) send 16242 byte file /tmp/.private/asheplyakov/distcc_10cead9a.i with token DOTI and compression 69
distcc[17003] (dcc_x_token_int) send DOTI00003f72
distcc[17003] (dcc_compile_remote) client finished sending request to server
distcc[17003] (dcc_note_state) note state 5, file "(NULL)", host "X86HOST"
distcc[17003] (dcc_select_for_read) select for read on fd5 for 300s
distcc[17003] (dcc_r_token_int) got DONE00000001
distcc[17003] (dcc_r_result_header) got response header
distcc[17003] (dcc_note_state) note state 6, file "(NULL)", host "(NULL)"
distcc[17003] (dcc_r_token_int) got STAT00000000
distcc[17003] (dcc_r_token_int) got SERR00000000
distcc[17003] (dcc_r_file) received 0 bytes to file /tmp/.private/asheplyakov/distcc_server_stderr_13d9ad9a.txt
distcc[17003] (dcc_r_token_int) got SOUT00000000
distcc[17003] (dcc_r_token_int) got DOTO000005f0
distcc[17003] (dcc_r_file) received 1520 bytes to file hello.o
distcc[17003] (dcc_r_file_timed) 1520 bytes received in 0.000359s, rate 4135kB/s
distcc[17003] 16242 bytes from hello.c compiled on X86HOST in 0.2941s, rate 54kB/s
distcc[17003] (dcc_unlock) release lock fd3
distcc[17003] compile hello.c on X86HOST/20 completed ok
distcc[17003] elapsed compilation time 0.295698s
distcc[17003] (dcc_exit) exit: code 0; self: 0.001968 user 0.009844 sys; children: 0.021002 user 0.016232 sys
distcc[17003] (dcc_cleanup_tempfiles_inner) deleted 2 temporary files

So cross-rewriting fails and distcc runs the native (x86) compiler on the remote node.

According to strace -e faccessat ./distcc gcc -c hello.c distcc is trying to find the aarch64-unknown-linux-gnu binary. However the actual GCC triple is aarch64-linux-gnu.

closed time in 8 days

asheplyakov

issue commentdistcc/distcc

Error when compiling with distcc a package that use autoconf and DISTCC_VERBOSE=1

Those distcc logs have no errors.

distcc[12000] (dcc_source_needs_local) autoconf tests are run locally: conftest.cpp

I don't see the bug. Because you are a Gentoo user, I suspect the problem is due to ricing, and has nothing to do with distcc.

sylmio

comment created time in 8 days

issue closeddistcc/distcc

Error when compiling with distcc a package that use autoconf and DISTCC_VERBOSE=1

Hello,

I had problems updating Gentoo using distcc. Many packages had errors with relocation asking to add option -fPIC, for example:

/usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: mpn/.libs/modexact_1c_odd.o: warning: relocation against `__gmp_modlimb_invert_table' in read-only section `.text'
/usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: mpn/.libs/modexact_1c_odd.o: relocation R_X86_64_PC32 against symbol `__gmp_modlimb_invert_table' can not be used when making a shared object; recompile with -fPIC

One can effectively force this option, but it does not work for all packages.,I ended up finding the root cause is having verbose mode of distcc (that is DISTCC_VERBOSE=1) that perturbes configure script of autoconf. Configure found -fPIC not working (as well as -c and -o due to same problem):

checking for x86_64-pc-linux-gnu-gcc option to produce PIC... -fPIC -DPIC
checking if x86_64-pc-linux-gnu-gcc PIC flag -fPIC -DPIC works... no
checking if x86_64-pc-linux-gnu-gcc static flag -static works... no
checking if x86_64-pc-linux-gnu-gcc supports -c -o file.o... no
checking if x86_64-pc-linux-gnu-gcc supports -c -o file.o... (cached) no

This is due to configure comparing error output of some compilation test. But due to some outputs of distcc in verbose mode, there are differences interpreted as errors. First distcc output used in configure as reference :

distcc[12000] (dcc_trace_version) distcc 3.4 x86_64-pc-linux-gnu; built Oct 13 2021 16:15:47
distcc[12000] (dcc_recursion_safeguard) safeguard level=0
distcc[12000] (main) compiler name is "x86_64-pc-linux-gnu-g++"
distcc[12000] (dcc_set_path) setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib64/opencascade-7.5.2/bin:/usr/lib/llvm/12/bin:/usr/lib/llvm/11/bin:/usr/lib/llvm/10/bin
distcc[12000] (dcc_scan_args) scanning arguments: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 conftest.cpp
distcc[12000] (dcc_scan_args) found input file "conftest.cpp"
distcc[12000] (dcc_source_needs_local) autoconf tests are run locally: conftest.cpp
distcc[12000] (dcc_get_hostlist) not reading /home/sylvain/.distcc/hosts: No such file or directory
distcc[12000] (dcc_parse_hosts_file) load hosts from /etc/distcc/hosts
distcc[12000] (dcc_parse_hosts) found tcp token "10.42.0.2/24,cpp,lzo"
distcc[12000] (dcc_parse_options) got CPP option
distcc[12000] (dcc_parse_options) got LZO option
distcc[12000] (dcc_parse_hosts) found tcp token "10.42.0.1/12,cpp,lzo"
distcc[12000] (dcc_parse_options) got CPP option
distcc[12000] (dcc_parse_options) got LZO option
distcc[12000] (dcc_lock_host) got cpu lock on localhost slot 0 as fd3
distcc[12000] exec on localhost: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 conftest.cpp
distcc[12000] (dcc_note_state) note state 5, file "conftest.cpp", host "localhost"
distcc[12000] (dcc_spawn_child) forking to execute: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 conftest.cpp
distcc[12000] (dcc_spawn_child) child started as pid12001
distcc[12001] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[12000] (dcc_collect_child) cc child 12001 terminated with status 0
distcc[12000] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[12000] compile conftest.cpp on localhost completed ok
distcc[12000] (dcc_unlock) release lock fd3
distcc[12000] elapsed compilation time 0.009451s
distcc[12000] (dcc_exit) exit: code 0; self: 0.000721 user 0.000000 sys; children: 0.007215 user 0.002086 sys
distcc[12000] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files

Second output of distcc in configure to test -fPIC:

distcc[12114] (dcc_trace_version) distcc 3.4 x86_64-pc-linux-gnu; built Oct 13 2021 16:15:47
distcc[12114] (dcc_recursion_safeguard) safeguard level=0
distcc[12114] (main) compiler name is "x86_64-pc-linux-gnu-g++"
distcc[12114] (dcc_set_path) setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib64/opencascade-7.5.2/bin:/usr/lib/llvm/12/bin:/usr/lib/llvm/11/bin:/usr/lib/llvm/10/bin
distcc[12114] (dcc_scan_args) scanning arguments: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 -fPIC -DPIC -DPIC conftest.cpp
distcc[12114] (dcc_scan_args) found input file "conftest.cpp"
distcc[12114] (dcc_source_needs_local) autoconf tests are run locally: conftest.cpp
distcc[12114] (dcc_get_hostlist) not reading /home/sylvain/.distcc/hosts: No such file or directory
distcc[12114] (dcc_parse_hosts_file) load hosts from /etc/distcc/hosts
distcc[12114] (dcc_parse_hosts) found tcp token "10.42.0.2/24,cpp,lzo"
distcc[12114] (dcc_parse_options) got CPP option
distcc[12114] (dcc_parse_options) got LZO option
distcc[12114] (dcc_parse_hosts) found tcp token "10.42.0.1/12,cpp,lzo"
distcc[12114] (dcc_parse_options) got CPP option
distcc[12114] (dcc_parse_options) got LZO option
distcc[12114] (dcc_lock_host) got cpu lock on localhost slot 0 as fd3
distcc[12114] exec on localhost: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 -fPIC -DPIC -DPIC conftest.cpp
distcc[12114] (dcc_note_state) note state 5, file "conftest.cpp", host "localhost"
distcc[12114] (dcc_spawn_child) forking to execute: x86_64-pc-linux-gnu-g++ -c -m64 -O2 -march=k8 -mtune=k8 -fPIC -DPIC -DPIC conftest.cpp
distcc[12114] (dcc_spawn_child) child started as pid12115
distcc[12115] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc[12114] (dcc_collect_child) cc child 12115 terminated with status 0
distcc[12114] (dcc_collect_child) cc times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc[12114] compile conftest.cpp on localhost completed ok
distcc[12114] (dcc_unlock) release lock fd3
distcc[12114] elapsed compilation time 0.010154s
distcc[12114] (dcc_exit) exit: code 0; self: 0.000000 user 0.000742 sys; children: 0.005493 user 0.003699 sys
distcc[12114] (dcc_cleanup_tempfiles_inner) deleted 0 temporary files

configure script expect same error output. If possible it would be better to output all this debug information of distcc somewhere else, as they are not errors. I am not familiar with dealing with different outputs, so I am not sure it is a root problem of distcc or autoconf.

Anyway I think it is important, as the only workaround is to remove verbose mode of distcc. And then it is not possible to know what happens with distcc for packages using autoconf.

Best regards,

Sylvain

closed time in 8 days

sylmio

issue closeddistcc/distcc

segv in cross-rewriting on x86 (with amd64 remotes)

I am repeatedly hitting some segfaults while building software with new distcc versions (tested 3.3 and 3.3.2) on my x86 machine. It's configured to use the following hosts:

localhost/1
machine1/5,lzo
machine2/3,lzo

where machine1/machine2 (hostnames obfuscated for privacy reasons) are both amd64 hosts.

Sadly, I've been unable to reproduce it by running the distcc process within gdb (I guess because it didn't use the remote host). However, the core dump may give some suggestions:

           PID: 28149 (gcc)
           UID: 250 (portage)
           GID: 250 (portage)
        Signal: 11 (SEGV)
     Timestamp: Wed 2018-12-12 16:44:26 CET (59min ago)
  Command Line: gcc -g -O2 -Wall -pthread -I/usr/include/libsecret-1 -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -o git-credential-libsecret.o -c git-credential-libsecret.c
    Executable: /usr/bin/distcc
 Control Group: /user.slice/user-1001.slice/session-2.scope
          Unit: session-2.scope
         Slice: user-1001.slice
       Session: 2
     Owner UID: 1001 (mgorny)
       Boot ID: 3a88b38273914236bbdf3890cc54cf01
    Machine ID: 61c0c15d779c0276357628ac545387fc
      Hostname: p86
       Storage: /var/lib/systemd/coredump/core.gcc.250.3a88b38273914236bbdf3890cc54cf01.28149.1544629466000000.lz4
       Message: Process 28149 (gcc) of user 250 dumped core.

Core was generated by `gcc -g -O2 -Wall -pthread -I/usr/include/libsecret-1 -I/usr/include/glib-2.0 -I'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0xb7e009b9 in __strcat_ia32 () from /lib/libc.so.6
(gdb) bt
#0  0xb7e009b9 in __strcat_ia32 () from /lib/libc.so.6
#1  0x0047f4ee in dcc_gcc_rewrite_fqn (argv=0x13125d0) at src/compile.c:578
#2  0x0047f7d0 in dcc_build_somewhere (argv=0x13125d0, sg_level=0, status=0xbfa9e094) at src/compile.c:691
#3  0x0047ff0d in dcc_build_somewhere_timed (argv=0x1312480, sg_level=0, status=0xbfa9e094) at src/compile.c:989
#4  0x004806f7 in main (argc=12, argv=0xbfa9e174) at src/distcc.c:352
(gdb) up
#1  0x0047f4ee in dcc_gcc_rewrite_fqn (argv=0x13125d0) at src/compile.c:578
578	    strcat(newcmd, argv[0]);
(gdb) p *argv
$1 = 0x756e672d <error: Cannot access memory at address 0x756e672d>
(gdb) up
#2  0x0047f7d0 in dcc_build_somewhere (argv=0x13125d0, sg_level=0, status=0xbfa9e094) at src/compile.c:691
691	        dcc_gcc_rewrite_fqn(new_argv);
(gdb) p new_argv[0]
$2 = 0x756e672d <error: Cannot access memory at address 0x756e672d>
(gdb) p new_argv[1]
$3 = 0x131002d <error: Cannot access memory at address 0x131002d>
(gdb) p new_argv[2]
$4 = 0x1312640 "-O2"

So apparently the first two elements of new_argv were either filled wrong or freed. Please let me know what else may I provide to help.

closed time in 19 days

mgorny

issue commentdistcc/distcc

segv in cross-rewriting on x86 (with amd64 remotes)

I have not tested, but I can say with high certainly that the mentioned patch fixed it. It was adding 1 to a pointer, which will cause problems if the string it referrs to is shorter than 4 characters long.

mgorny

comment created time in 19 days

issue commentdistcc/distcc

distcc 3.3.2: SIGABRT when building w. ninja/meson (ppc, 32bit, BE)

powerpc-unknown-linux-gnu-gcc

Is this cross-compiler available on the target system?

Can you attach the log of distccd.?

El mié., 29 sept. 2021 14:19, ernsteiswuerfel ***@***.***> escribió:

Thanks for your comments @asheplyakov https://github.com/asheplyakov! Your implication that the bug is about failure when building other software with ninja and distcc (not about building distcc itself) is correct.

However atm I can't investigate any further as the G4s' PSU is being serviced. The machine should be usable again in a few weeks. It has got whopping 2 GiB RAM, not 512 MiB btw. ;)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/distcc/distcc/issues/354#issuecomment-930044167, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD4W4SHX73ZVG5TPWEF6DDUELRZTANCNFSM4H6WMT2Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

ernsteiswuerfel

comment created time in 24 days

issue commentdistcc/distcc

distcc 3.3.2: SIGABRT when building w. ninja/meson (ppc, 32bit, BE)

You need to specify exactly how you are calling distcc, as this is not the ninja/meson bugtracker.

El mié., 29 sept. 2021 13:34, Alexey Sheplyakov ***@***.***> escribió:

  1. In fact GNU_HOST as reported by autoconf is not always the same as the compiler triple. For instance, on rpm-based distros GNU_HOST might be something like x86_64-pc-linux-gnu, and compiler triple (as reported by gcc -dumpmachine) is x86_64-redhat-linux. So I think it's better to ask the compiler (like #427 https://github.com/distcc/distcc/pull/427 does): it's more accurate and works with whatever build system.
  2. This bug is about failure when building other software with ninja and distcc. As far as I understand it has nothing to do about (possible) misbehavior of distcc built with meson.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/distcc/distcc/issues/354#issuecomment-930011138, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAD4W4U5XUNYDQTAUCCHT33UELMQXANCNFSM4H6WMT2Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

ernsteiswuerfel

comment created time in 24 days

issue commentdistcc/distcc

DISTCC_NO_CROSS_REWRITE not working?

That feature predates my cross-compiler rewrite logic, wherein you provide explicit remote compiler paths to use. I never used it.

solskogen

comment created time in 25 days

push eventdistcc/distcc

Alexey Sheplyakov

commit sha fd6f688120e4f18decee6a49120837f27df53b35

backoff.c: added dcc_backof_is_enabled() helper function

view details

Alexey Sheplyakov

commit sha a59c8a58caa912e85867749cbd03861f4654acc1

Restrict the number of retries when DISTCC_BACKOFF is disabled Since the commit a7dd5cf90e8c ("Try a new host after failing to connect instead of immediately falling back to localhost") distcc tries to choose a different host on connection/protocol errors. However when backoff is disabled (DISTCC_BACKOFF_PERIOD=0) distcc keeps retrying forever. To avoid the problem limit the number of retries (currently to 3) when backoff is disabled. Closes: #434

view details

Shawn Landden

commit sha 3d543c06263744452d099e2b1860052ab061a842

Merge pull request #438 from asheplyakov/fix-434 Restrict number of retries when backoff is disabled

view details

push time in 25 days

PR merged distcc/distcc

Restrict number of retries when backoff is disabled

Since the commit a7dd5cf90e8c ("Try a new host after failing to connect instead of immediately falling back to localhost") distcc tries to choose a different host on connection/protocol errors. However when backoff is disabled (DISTCC_BACKOFF_PERIOD=0) distcc keeps retrying forever.

To avoid the problem limit the number of retries (currently to 3) when backoff is disabled.

Closes: #434

+41 -17

2 comments

3 changed files

asheplyakov

pr closed time in 25 days

issue closeddistcc/distcc

DISTCC_SSH regression in v3.4

Version 3.4 seems to be affected by a regression regarding DISTCC_SSH, according to which the environment variable is only honored during the first connection attempt of distcc and ignored during any subsequent runs (assuming the distcc server is temporarily unavailable).

I accidentally stumbled upon this issue when my CI started failing once Gentoo stabilized the relevant version.

Relevant CI log (truncated): <details>

distcc-ssh-client_1  | distcc[10] (dcc_trace_version) distcc 3.4 i686-pc-linux-gnu; built Jul  4 2021 03:18:57
distcc-ssh-client_1  | distcc[10] (dcc_recursion_safeguard) safeguard level=0
distcc-ssh-client_1  | distcc[10] (main) compiler name is "distcc"
distcc-ssh-client_1  | distcc[10] (dcc_scan_args) scanning arguments: gcc -c test.c -o test-distcc.o
distcc-ssh-client_1  | distcc[10] (dcc_scan_args) found input file "test.c"
distcc-ssh-client_1  | distcc[10] (dcc_scan_args) found object/output file "test-distcc.o"
distcc-ssh-client_1  | distcc[10] compile from test.c to test-distcc.o
distcc-ssh-client_1  | distcc[10] (dcc_get_hostlist) read hosts from environment
distcc-ssh-client_1  | distcc[10] (dcc_parse_hosts) found ssh token "distcc-ssh@distcc-ssh-server"
distcc-ssh-client_1  | distcc[10] (dcc_lock_host) got cpu lock on distcc-ssh@distcc-ssh-server slot 0 as fd3
distcc-ssh-client_1  | distcc[10] (dcc_lock_host) got cpu lock on localhost slot 0 as fd4
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 3, file "(NULL)", host "localhost"
distcc-ssh-client_1  | distcc[10] (dcc_strip_dasho) result: gcc -c test.c
distcc-ssh-client_1  | distcc[10] (dcc_spawn_child) forking to execute: gcc -E test.c
distcc-ssh-client_1  | distcc[10] (dcc_spawn_child) child started as pid11
distcc-ssh-client_1  | distcc[10] (dcc_strip_local_args) result: gcc -c test.c -o test-distcc.o
distcc-ssh-client_1  | distcc[10] exec on distcc-ssh@distcc-ssh-server: gcc -c test.c -o test-distcc.o
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 2, file "test.c", host "distcc-ssh-server"
distcc-ssh-client_1  | distcc[11] (dcc_new_pgrp) entered process group
distcc-ssh-client_1  | distcc[11] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc-ssh-client_1  | distcc[10] (dcc_ssh_connect) connecting to distcc-ssh-server using ssh
distcc-ssh-client_1  | distcc[10] (dcc_run_piped_cmd) execute: ssh -o StrictHostKeyChecking=no -v -l distcc-ssh distcc-ssh-server distccd --inetd --enable-tcp-insecure
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 4, file "(NULL)", host "(NULL)"
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send DIST00000001
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGC00000005
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000003
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string 'gcc'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000002
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string '-c'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000006
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string 'test.c'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000002
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string '-o'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV0000000d
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string 'test-distcc.o'
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 3, file "(NULL)", host "(NULL)"
distcc-ssh-client_1  | OpenSSH_8.6p1, OpenSSL 1.1.1k  25 Mar 2021
distcc-ssh-client_1  | debug1: Reading configuration data /etc/ssh/ssh_config
distcc-ssh-client_1  | debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) cpp child 11 terminated with status 0
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) cpp times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc-ssh-client_1  | distcc[10] cpp test.c on localhost completed ok
distcc-ssh-client_1  | distcc[10] (dcc_unlock) release lock fd4
distcc-ssh-client_1  | distcc[10] (dcc_x_file) send 16555 byte file /tmp/distcc_fec22928.i with token DOTI and compression 69
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send DOTI000040ab
distcc-ssh-client_1  | distcc[10] (dcc_compile_remote) client finished sending request to server
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 5, file "(NULL)", host "distcc-ssh-server"
distcc-ssh-client_1  | distcc[10] (dcc_select_for_read) select for read on fd7 for 300s
distcc-ssh-client_1  | debug1: Connecting to distcc-ssh-server [172.18.0.2] port 22.
distcc-ssh-client_1  | debug1: connect to address 172.18.0.2 port 22: Connection refused
distcc-ssh-client_1  | ssh: connect to host distcc-ssh-server port 22: Connection refused
distcc-ssh-client_1  | distcc[10] (dcc_readx) ERROR: unexpected eof on fd7
distcc-ssh-client_1  | distcc[10] (dcc_r_token_int) ERROR: read failed while waiting for token "DONE"
distcc-ssh-client_1  | distcc[10] (dcc_r_result_header) ERROR: server provided no answer. Is the server configured to allow access from your IP address? Is the server performing authentication and your client isn't? Does the server have the compiler installed? Is the server configured to access the compiler?
distcc-ssh-client_1  | distcc[10] 16555 bytes from test.c compiled on distcc-ssh-server in 0.0266s, rate 607kB/s
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) ssh child 13 terminated with status 0xff00
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) ssh times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc-ssh-client_1  | distcc[10] (dcc_unlock) release lock fd3
distcc-ssh-client_1  | distcc[10] (dcc_get_hostlist) read hosts from environment
distcc-ssh-client_1  | distcc[10] (dcc_parse_hosts) found ssh token "distcc-ssh@distcc-ssh-server"
distcc-ssh-client_1  | distcc[10] (dcc_lock_host) got cpu lock on distcc-ssh@distcc-ssh-server slot 0 as fd3
distcc-ssh-client_1  | distcc[10] (dcc_lock_host) got cpu lock on localhost slot 0 as fd4
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 3, file "(NULL)", host "localhost"
distcc-ssh-client_1  | distcc[10] (dcc_strip_dasho) result: gcc -c test.c
distcc-ssh-client_1  | distcc[10] (dcc_spawn_child) forking to execute: gcc -E test.c
distcc-ssh-client_1  | distcc[10] (dcc_spawn_child) child started as pid14
distcc-ssh-client_1  | distcc[14] (dcc_new_pgrp) entered process group
distcc-ssh-client_1  | distcc[14] (dcc_increment_safeguard) setting safeguard: _DISTCC_SAFEGUARD=1
distcc-ssh-client_1  | distcc[10] (dcc_strip_local_args) result: gcc -c test.c -o test-distcc.o
distcc-ssh-client_1  | distcc[10] exec on distcc-ssh@distcc-ssh-server: gcc -c test.c -o test-distcc.o
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 2, file "test.c", host "distcc-ssh-server"
distcc-ssh-client_1  | distcc[10] (dcc_ssh_connect) connecting to distcc-ssh-server using ssh
distcc-ssh-client_1  | distcc[10] (dcc_run_piped_cmd) execute: ssh -l distcc-ssh distcc-ssh-server distccd --inetd --enable-tcp-insecure
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 4, file "(NULL)", host "(NULL)"
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send DIST00000001
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGC00000005
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000003
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string 'gcc'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000002
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string '-c'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000006
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string 'test.c'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV00000002
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string '-o'
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send ARGV0000000d
distcc-ssh-client_1  | distcc[10] (dcc_x_token_string) send string 'test-distcc.o'
distcc-ssh-client_1  | distcc[10] (dcc_note_state) note state 3, file "(NULL)", host "(NULL)"
distcc-ssh-client_1  | ssh: connect to host distcc-ssh-server port 22: Connection refused
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) cpp child 14 terminated with status 0
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) cpp times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc-ssh-client_1  | distcc[10] cpp test.c on localhost completed ok
distcc-ssh-client_1  | distcc[10] (dcc_unlock) release lock fd4
distcc-ssh-client_1  | distcc[10] (dcc_x_file) send 16555 byte file /tmp/distcc_6b452928.i with token DOTI and compression 69
distcc-ssh-client_1  | distcc[10] (dcc_x_token_int) send DOTI000040ab
distcc-ssh-client_1  | distcc[10] (dcc_writex) ERROR: failed to write: Broken pipe
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) ssh child 15 terminated with status 0xff00
distcc-ssh-client_1  | distcc[10] (dcc_collect_child) ssh times: user 0.000000s, system 0.000000s, 0 minflt, 0 majflt
distcc-ssh-client_1  | distcc[10] (dcc_unlock) release lock fd3

</details>

closed time in 25 days

KSmanis

push eventdistcc/distcc

Alexey Sheplyakov

commit sha 7f0d80dfe16577f55822ba86a1539e53f9d54417

Improved cross-compilation with clang With this patch I can cross-compile Linux kernel with clang and distcc. Otherwise both remote compilation and local fallback fail like this: cat > foo.c <<-EOF int main(int argc, char **argv) { return 0; } EOF env DISTCC_HOSTS=127.0.0.1 distcc clang -c --target=aarch64-linux-gnu- -o foo.o foo.c error: unknown target triple 'unknown', please use -triple or -arch distcc[304773] ERROR: compile foo.c on 127.0.0.1 failed distcc[304773] (dcc_build_somewhere) Warning: remote compilation of 'foo.c' failed, retrying locally distcc[304773] Warning: failed to distribute foo.c to 127.0.0.1, running locally instead error: unknown target triple 'unknown', please use -triple or -arch distcc[304773] ERROR: compile foo.c on localhost failed Linux' kernel makefiles specify the target arch with `--target=foo`, and distcc expects `-target foo`, hence the problem. Closes: #416

view details

Shawn Landden

commit sha da7412679d16d161f1e6d23e8f93f15f11fcbf73

Merge pull request #417 from asheplyakov/clang-distcc-linuxkernel-compile Improved cross-compilation with clang

view details

push time in 25 days

PR merged distcc/distcc

Improved cross-compilation with clang

With this patch I can cross-compile Linux kernel with clang and distcc. Otherwise both remote compilation and local fallback fail like this:

cat > foo.c <<-EOF
int main(int argc, char **argv) { return 0; }
EOF
env DISTCC_HOSTS=127.0.0.1 distcc clang -c --target=aarch64-linux-gnu- foo.c

error: unknown target triple 'unknown', please use -triple or -arch
distcc[304773] ERROR: compile foo.c on 127.0.0.1 failed
distcc[304773] (dcc_build_somewhere) Warning: remote compilation of 'foo.c' failed, retrying locally
distcc[304773] Warning: failed to distribute foo.c to 127.0.0.1, running locally instead
error: unknown target triple 'unknown', please use -triple or -arch
distcc[304773] ERROR: compile foo.c on localhost failed

Linux' kernel makefiles specify the target arch with --target=foo, and distcc expects -target foo, hence the problem.

Closes: #416

+19 -0

0 comment

3 changed files

asheplyakov

pr closed time in 25 days

issue closeddistcc/distcc

cross-compilation with clang fails

$ cat > foo.c <<-EOF
int main(int argc, char **argv) { return 0; }
EOF
$ env DISTCC_HOSTS=127.0.0.1 distcc clang --target=aarch64-linux-gnu- -c foo.c
error: unknown target triple 'unknown', please use -triple or -arch
distcc[304936] ERROR: compile foo.c on 127.0.0.1 failed
distcc[304936] (dcc_build_somewhere) Warning: remote compilation of 'foo.c' failed, retrying locally
distcc[304936] Warning: failed to distribute foo.c to 127.0.0.1, running locally instead
error: unknown target triple 'unknown', please use -triple or -arch
distcc[304936] ERROR: compile foo.c on localhost failed

Linux' kernel makefiles specify target arch with --target=foo rather than -target foo, hence cross-compilation with distcc and clang fails.

closed time in 25 days

asheplyakov

Pull request review commentdistcc/distcc

Improved cross-compilation with clang

 int dcc_argv_search(char **a, }  +/**+ * Return true if argv contains argument starting with needle.+ */+int dcc_argv_startswith(char **a,+                        const char *needle)+{+    size_t needle_len = strlen(needle);+    for (; *a; a++)

oh my bad, I didn't see the double star.

asheplyakov

comment created time in 25 days

PullRequestReviewEvent

push eventdistcc/distcc

Matt Whitlock

commit sha 9699301fc3f16ef34570fa53e0863a2b91cdd3a6

distccd: add --oom-score-adj option (Linux only) This new option sets /proc/self/oom_score_adj in prefork worker child processes. This can afford users a bit of protection against a scenario in which many concurrent compilation jobs cause an out-of-memory condition that prompts the kernel's OOM killer to kill a process that is not so "disposable" as a distccd worker. This commit adds a new Autoconf test to check whether the host system type (that is, the type of the system on which the compiled binaries are to be run) is Linux. The new --oom-score-adj option is implemented only if the host system type is Linux.

view details

Shawn Landden

commit sha a91aef9cf64b259536d0dbfbe4cc58f45a6b594e

Merge pull request #439 from whitslack/oom-score-adj distccd: add --oom-score-adj option (Linux only)

view details

push time in a month

PR merged distcc/distcc

distccd: add --oom-score-adj option (Linux only)

This new option sets /proc/self/oom_score_adj in prefork worker child processes. This can afford users a bit of protection against a scenario in which many concurrent compilation jobs cause an out-of-memory condition that prompts the kernel's OOM killer to kill a process that is not so "disposable" as a distccd worker.

This commit adds a new Autoconf test to check whether the host system type (that is, the type of the system on which the compiled binaries are to be run) is Linux. The new --oom-score-adj option is implemented only if the host system type is Linux.

+54 -0

2 comments

6 changed files

whitslack

pr closed time in a month

pull request commentdistcc/distcc

Restrict number of retries when backoff is disabled

Why not just revert a7dd5cf ?

asheplyakov

comment created time in a month

PullRequestReviewEvent

Pull request review commentdistcc/distcc

Improved cross-compilation with clang

 int dcc_argv_search(char **a, }  +/**+ * Return true if argv contains argument starting with needle.+ */+int dcc_argv_startswith(char **a,+                        const char *needle)+{+    size_t needle_len = strlen(needle);+    for (; *a; a++)

This is unnecessary. strncmp does this for you.

asheplyakov

comment created time in a month