profile
viewpoint

Ask questionsProblems with install python from source

🐛 Bug

For Pytorch 1.3.0 a from source installation is necessary, if one only have cuda 9.0. I cant update the version, because I do not have root access.

To Reproduce

I have done everything what they say in the tutorial. The result is the following output of CMakerError:

Performing C++ SOURCE FILE Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_b8375 
[1/2] Building CXX object CMakeFiles/cmTC_b8375.dir/src.cxx.o
FAILED: CMakeFiles/cmTC_b8375.dir/src.cxx.o 
/usr/bin/c++    -DCAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING -std=c++11 -o CMakeFiles/cmTC_b8375.dir/src.cxx.o -c src.cxx
src.cxx:1:30: fatal error: glog/stl_logging.h: Datei oder Verzeichnis nicht gefunden
 #include <glog/stl_logging.h>
                              ^
compilation terminated.
ninja: build stopped: subcommand failed.

Source file was:
#include <glog/stl_logging.h>
    int main(int argc, char** argv) {
      return 0;
    }
Determining if the pthread_create exist failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_7fcd6 
[1/2] Building C object CMakeFiles/cmTC_7fcd6.dir/CheckSymbolExists.c.o
[2/2] Linking C executable cmTC_7fcd6
FAILED: cmTC_7fcd6 
: && /usr/bin/gcc   -rdynamic CMakeFiles/cmTC_7fcd6.dir/CheckSymbolExists.c.o  -o cmTC_7fcd6   && :
CMakeFiles/cmTC_7fcd6.dir/CheckSymbolExists.c.o: In function `main':
CheckSymbolExists.c:(.text+0x1b): undefined reference to `pthread_create'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.

File /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp/CheckSymbolExists.c:
/* */
#include <pthread.h>

int main(int argc, char** argv)
{
  (void)argv;
#ifndef pthread_create
  return ((int*)(&pthread_create))[argc];
#else
  (void)argc;
  return 0;
#endif
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_95667 
[1/2] Building C object CMakeFiles/cmTC_95667.dir/CheckFunctionExists.c.o
[2/2] Linking C executable cmTC_95667
FAILED: cmTC_95667 
: && /usr/bin/gcc -DCHECK_FUNCTION_EXISTS=pthread_create  -rdynamic CMakeFiles/cmTC_95667.dir/CheckFunctionExists.c.o  -o cmTC_95667  -lpthreads && :
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.


Performing C SOURCE FILE Test NNPACK_ARCH_IS_X86_32 failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_94e41 
[1/2] Building C object CMakeFiles/cmTC_94e41.dir/src.c.o
FAILED: CMakeFiles/cmTC_94e41.dir/src.c.o 
/usr/bin/gcc   -DNNPACK_ARCH_IS_X86_32 -o CMakeFiles/cmTC_94e41.dir/src.c.o   -c src.c
src.c:3:10: error: #error AVX only on x86_64
         #error AVX only on x86_64
          ^~~~~
ninja: build stopped: subcommand failed.

Source file was:

      #if ! (defined(__i386) || defined(_M_IX86))
        #error AVX only on x86_64
      #endif
      int main() {
        return 0;
      }
Performing C++ SOURCE FILE Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_c20e9 
[1/2] Building CXX object CMakeFiles/cmTC_c20e9.dir/src.cxx.o
FAILED: CMakeFiles/cmTC_c20e9.dir/src.cxx.o 
/usr/bin/c++    -Wno-deprecated -fvisibility-inlines-hidden  -std=c++11  -Wall  -Wextra  -Wshadow  -pedantic  -pedantic-errors -DHAVE_CXX_FLAG_WSHORTEN_64_TO_32  -Wshorten-64-to-32   -Wshorten-64-to-32 -o CMakeFiles/cmTC_c20e9.dir/src.cxx.o -c src.cxx
c++: error: unrecognized command line option '-Wshorten-64-to-32'
c++: error: unrecognized command line option '-Wshorten-64-to-32'
ninja: build stopped: subcommand failed.

Source file was:
int main() { return 0; }
Performing C++ SOURCE FILE Test HAVE_CXX_FLAG_WD654 failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_bfc93 
[1/2] Building CXX object CMakeFiles/cmTC_bfc93.dir/src.cxx.o
FAILED: CMakeFiles/cmTC_bfc93.dir/src.cxx.o 
/usr/bin/c++    -Wno-deprecated -fvisibility-inlines-hidden  -std=c++11  -Wall  -Wextra  -Wshadow  -pedantic  -pedantic-errors  -Wfloat-equal  -fstrict-aliasing  -Wno-deprecated-declarations  -Wstrict-aliasing -DHAVE_CXX_FLAG_WD654  -wd654   -wd654 -o CMakeFiles/cmTC_bfc93.dir/src.cxx.o -c src.cxx
c++: error: unrecognized command line option '-wd654'
c++: error: unrecognized command line option '-wd654'
ninja: build stopped: subcommand failed.

Source file was:
int main() { return 0; }
Performing C++ SOURCE FILE Test HAVE_CXX_FLAG_WTHREAD_SAFETY failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_9f5d6 
[1/2] Building CXX object CMakeFiles/cmTC_9f5d6.dir/src.cxx.o
FAILED: CMakeFiles/cmTC_9f5d6.dir/src.cxx.o 
/usr/bin/c++    -Wno-deprecated -fvisibility-inlines-hidden  -std=c++11  -Wall  -Wextra  -Wshadow  -pedantic  -pedantic-errors  -Wfloat-equal  -fstrict-aliasing  -Wno-deprecated-declarations  -Wstrict-aliasing -DHAVE_CXX_FLAG_WTHREAD_SAFETY  -Wthread-safety   -Wthread-safety -o CMakeFiles/cmTC_9f5d6.dir/src.cxx.o -c src.cxx
c++: error: unrecognized command line option '-Wthread-safety'; did you mean '-fthread-jumps'?
c++: error: unrecognized command line option '-Wthread-safety'; did you mean '-fthread-jumps'?
ninja: build stopped: subcommand failed.

Source file was:
int main() { return 0; }
Performing C SOURCE FILE Test C_HAS_AVX_1 failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_b6753 
[1/2] Building C object CMakeFiles/cmTC_b6753.dir/src.c.o
FAILED: CMakeFiles/cmTC_b6753.dir/src.c.o 
/usr/bin/gcc  -I"/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/include" -I/usr/local/cuda/include -fopenmp -DC_HAS_AVX_1 -fPIE -o CMakeFiles/cmTC_b6753.dir/src.c.o   -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
     a = _mm256_set1_ps(0);
     ~~^~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/6/include/immintrin.h:41:0,
                 from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/6/include/avxintrin.h:1285:1: error: inlining failed in call to always_inline ‘_mm256_set1_ps’: target specific option mismatch
 _mm256_set1_ps (float __A)
 ^~~~~~~~~~~~~~
src.c:7:7: note: called from here
     a = _mm256_set1_ps(0);
     ~~^~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

  #include <immintrin.h>

  int main()
  {
    __m256 a;
    a = _mm256_set1_ps(0);
    return 0;
  }

Performing C SOURCE FILE Test C_HAS_AVX2_1 failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_0bc98 
[1/2] Building C object CMakeFiles/cmTC_0bc98.dir/src.c.o
FAILED: CMakeFiles/cmTC_0bc98.dir/src.c.o 
/usr/bin/gcc  -I"/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/include" -I/usr/local/cuda/include -fopenmp -DC_HAS_AVX2_1 -fPIE -o CMakeFiles/cmTC_0bc98.dir/src.c.o   -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
     a = _mm256_abs_epi16(a);
     ~~^~~~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/6/include/immintrin.h:43:0,
                 from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/6/include/avx2intrin.h:63:1: error: inlining failed in call to always_inline ‘_mm256_abs_epi16’: target specific option mismatch
 _mm256_abs_epi16 (__m256i __A)
 ^~~~~~~~~~~~~~~~
src.c:7:7: note: called from here
     a = _mm256_abs_epi16(a);
     ~~^~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

  #include <immintrin.h>

  int main()
  {
    __m256i a = {0};
    a = _mm256_abs_epi16(a);
    __m256i x;
    _mm256_extract_epi64(x, 0); // we rely on this in our AVX2 code
    return 0;
  }

Performing C SOURCE FILE Test CXX_HAS_AVX_1 failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_dcabd 
[1/2] Building C object CMakeFiles/cmTC_dcabd.dir/src.c.o
FAILED: CMakeFiles/cmTC_dcabd.dir/src.c.o 
/usr/bin/gcc  -I"/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/include" -I/usr/local/cuda/include -fopenmp -DCXX_HAS_AVX_1 -fPIE -o CMakeFiles/cmTC_dcabd.dir/src.c.o   -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
     a = _mm256_set1_ps(0);
     ~~^~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/6/include/immintrin.h:41:0,
                 from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/6/include/avxintrin.h:1285:1: error: inlining failed in call to always_inline ‘_mm256_set1_ps’: target specific option mismatch
 _mm256_set1_ps (float __A)
 ^~~~~~~~~~~~~~
src.c:7:7: note: called from here
     a = _mm256_set1_ps(0);
     ~~^~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

  #include <immintrin.h>

  int main()
  {
    __m256 a;
    a = _mm256_set1_ps(0);
    return 0;
  }

Performing C SOURCE FILE Test CXX_HAS_AVX2_1 failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_b0cad 
[1/2] Building C object CMakeFiles/cmTC_b0cad.dir/src.c.o
FAILED: CMakeFiles/cmTC_b0cad.dir/src.c.o 
/usr/bin/gcc  -I"/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/include" -I/usr/local/cuda/include -fopenmp -DCXX_HAS_AVX2_1 -fPIE -o CMakeFiles/cmTC_b0cad.dir/src.c.o   -c src.c
src.c: In function ‘main’:
src.c:7:7: warning: AVX vector return without AVX enabled changes the ABI [-Wpsabi]
     a = _mm256_abs_epi16(a);
     ~~^~~~~~~~~~~~~~~~~~~~~
In file included from /usr/lib/gcc/x86_64-linux-gnu/6/include/immintrin.h:43:0,
                 from src.c:2:
/usr/lib/gcc/x86_64-linux-gnu/6/include/avx2intrin.h:63:1: error: inlining failed in call to always_inline ‘_mm256_abs_epi16’: target specific option mismatch
 _mm256_abs_epi16 (__m256i __A)
 ^~~~~~~~~~~~~~~~
src.c:7:7: note: called from here
     a = _mm256_abs_epi16(a);
     ~~^~~~~~~~~~~~~~~~~~~~~
ninja: build stopped: subcommand failed.

Source file was:

  #include <immintrin.h>

  int main()
  {
    __m256i a = {0};
    a = _mm256_abs_epi16(a);
    __m256i x;
    _mm256_extract_epi64(x, 0); // we rely on this in our AVX2 code
    return 0;
  }

Performing C SOURCE FILE Test BLAS_F2C_DOUBLE_WORKS failed with the following compile output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_6dd89 
[1/2] Building C object CMakeFiles/cmTC_6dd89.dir/src.c.o
[2/2] Linking C executable cmTC_6dd89

...and run output:

Return value: 1
Source file was:

#include <stdlib.h>
#include <stdio.h>
float x[4] = { 1, 2, 3, 4 };
float y[4] = { .1, .01, .001, .0001 };
int four = 4;
int one = 1;
extern double sdot_();
int main() {
  int i;
  double r = sdot_(&four, x, &one, y, &one);
  exit((float)r != (float).1234);
}
Performing C SOURCE FILE Test COMPILER_SUPPORTS_SVE failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_feaf1 
[1/2] Building C object CMakeFiles/cmTC_feaf1.dir/src.c.o
FAILED: CMakeFiles/cmTC_feaf1.dir/src.c.o 
/usr/bin/gcc  -I"/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/include" -I/usr/local/cuda/include -fopenmp -Wno-ignored-qualifiers -Wno-absolute-value -DCOMPILER_SUPPORTS_SVE -march=armv8-a+sve -fPIE -o CMakeFiles/cmTC_feaf1.dir/src.c.o   -c src.c
src.c:1:0: error: bad value (armv8-a+sve) for -march= switch
 
 
cc1: warning: unrecognized command line option ‘-Wno-absolute-value’
ninja: build stopped: subcommand failed.

Source file was:

  #include <arm_sve.h>
  int main() {
    svint32_t r = svdup_n_s32(1); }
Performing C SOURCE FILE Test COMPILER_SUPPORTS_AVX512F failed with the following output:
Change Dir: /group/ag_cmb/scratch/msalomon#/pytorch/build/CMakeFiles/CMakeTmp

Run Build Command(s):/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/bin/ninja cmTC_1782d 
[1/2] Building C object CMakeFiles/cmTC_1782d.dir/src.c.o
FAILED: CMakeFiles/cmTC_1782d.dir/src.c.o 
/usr/bin/gcc  -I"/group/ag_cmb/scratch/msalomon#/anaconda3/envs/pytorch-build/include" -I/usr/local/cuda/include -fopenmp -Wno-ignored-qualifiers -Wno-absolute-value -DCOMPILER_SUPPORTS_AVX512F -mavx512f -fPIE -o CMakeFiles/cmTC_1782d.dir/src.c.o   -c src.c
src.c: In function ‘main’:
src.c:14:5: error: unrecognizable insn:
     __m512i r = _mm512_andnot_si512(a, a); }
     ^~~~~~~
(insn 22 21 23 2 (set (mem/c:V4DI (plus:DI (reg/f:DI 82 virtual-stack-vars)
                (const_int -96 [0xffffffffffffffa0])) [3 ymm+0 S32 A256])
        (vec_merge:V4DI (vec_select:V4DI (reg:V8DI 103)
                (parallel [
                        (const_int 0 [0])
                        (const_int 1 [0x1])
                        (const_int 2 [0x2])
                        (const_int 3 [0x3])
                    ]))
            (reg:V4DI 104)
            (reg:QI 105))) src.c:12 -1
     (nil))
src.c:14:5: internal compiler error: in extract_insn, at recog.c:2287
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-6/README.Bugs> for instructions.
ninja: build stopped: subcommand failed.

Source file was:

  #if defined(_MSC_VER)
  #include <intrin.h>
  #else
  #include <x86intrin.h>
  #endif
  __m512 addConstant(__m512 arg) {
    return _mm512_add_ps(arg, _mm512_set1_ps(1.f));
  }
  int main() {
    __m512i a = _mm512_set1_epi32(1);
    __m256i ymm = _mm512_extracti64x4_epi64(a, 0);
    __mmask16 m = _mm512_cmp_epi32_mask(a, a, _MM_CMPINT_EQ);
    __m512i r = _mm512_andnot_si512(a, a); }

Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

Environment

PyTorch version: From Source Is debug build: N/A CUDA used to build PyTorch: 9.0

OS: Debian GNU/Linux 9.9 (stretch) GCC version: (Debian 6.3.0-18+deb9u1) 6.3.0 20170516 CMake version: version 3.14.0

Python version: 3.7 Is CUDA available: yes CUDA runtime version: Could not collect GPU models and configuration: GPU 0: GeForce GTX 1080 GPU 1: NVS 310

Nvidia driver version: 384.111 cuDNN version: Could not collect

Versions of relevant libraries: [pip3] numpy==1.17.0 [pip3] torch==1.2.0 [pip3] torchvision==0.4.0 [conda] blas 1.0 mkl
[conda] magma-cuda90 2.5.0 1 pytorch [conda] mkl 2019.4 243
[conda] mkl-include 2019.4 243
[conda] mkl-service 2.0.2 py37h7b6447c_0
[conda] mkl_fft 1.0.14 py37ha843d7b_0
[conda] mkl_random 1.0.2 py37hd81dba3_0

Additional context

<!-- Add any other context about the problem here. -->

pytorch/pytorch

Answer questions BioFreak95

Hey. The tutorial on pytorch.org for a "From source" installation. And yes, the master branch.

useful!

Related questions

TensorBoard logging requires TensorBoard with Python summary writer installed. This should be available in 1.14 or above hot 3
AttributeError: module 'torch.jit' has no attribute 'unused' hot 3
Script freezes with no output when using DistributedDataParallel hot 2
Adding Pixel Unshuffle hot 2
DataLoader leaking Semaphores. hot 2
[feature request] Add matrix exponential hot 2
cublas runtime error on torch.bmm() with CUDA10 and RTX2080Ti hot 2
libtorch does not initialize OpenMP/MKL by default hot 2
Use torch.device() with torch.load(..., map_location=torch.device()) hot 2
Cuda required when loading a TorchScript with map_location='cpu' hot 2
PyTorch 1.5 failed to import c:miniconda3-x64envs estlibsite-packages orchlibcaffe2_nvrtc.dll - pytorch hot 2
Error during import torch, NameError: name &#39;_C&#39; is not defined - pytorch hot 2
Quantisation of object detection models. hot 2
torch.utils.tensorboard.SummaryWriter.add_graph do not support non-tensor inputs - pytorch hot 2
a retrained and saved jit module could not be reload. hot 2
source:https://uonfu.com/
Github User Rank List