[petsc-dev] AVX kernels, old gcc, still broken

[Smith, Barry F.]

   This needs to be fixed properly with a configure test(s) and not with huge and inconsistent checks like this  

#if defined(PETSC_HAVE_IMMINTRIN_H) && defined(__AVX512F__) && defined(PETSC_USE_REAL_DOUBLE) && !defined(PETSC_USE_COMPLEX) && !defined(PETSC_USE_64BIT_INDICES)  or this

#elif defined(PETSC_USE_AVX512_KERNELS) && defined(PETSC_HAVE_IMMINTRIN_H) && defined(__AVX512F__) && defined(PETSC_USE_REAL_DOUBLE) && !defined(PETSC_USE_COMPLEX) && !defined(PETSC_USE_64BIT_INDICES) && !defined(PETSC_SKIP_IMMINTRIN_H_CUDAWORKAROUND)



   self.useAVX512Kernels = self.framework.argDB['with-avx512-kernels']
    if self.useAVX512Kernels:
      self.addDefine('USE_AVX512_KERNELS', 1)

     Here you should check that the needed include files are available, that it is 32 bit integers, that defined(__AVX512F__) exists and that appropriate functions exist. 

    Maybe you need two configure checks if there are two different types of functionality you are trying to catch? 

    Continuing to hack away with gross #if def means this crud is unmaintainable and will always haunt you. Yes the incremental cost of doing a proper configure test is there but once that is done the maintenance costs (which have been haunting use for months) will be gone.

  Barry









> On Oct 25, 2019, at 2:49 AM, Lisandro Dalcin via petsc-dev <petsc-dev at mcs.anl.gov> wrote:
> 
> 
> 
> On Fri, 25 Oct 2019 at 01:40, Balay, Satish <balay at mcs.anl.gov> wrote:
> I'm curious why this issue comes up for you. The code was unrelated to --with-avx512-kernels=0 option.
> 
> Its relying on __AVX512F__and PETSC_HAVE_IMMINTRIN_H flags. And
> assumes immintrin.h has a definition for _mm512_reduce_add_pd()
> 
> Is the flag __AVX512F__ always set on your machine by gcc? 
> 
> And does this change based on the hardware? I just tried this build
> [same os/compiler] on "Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz" - and
> can't reproduce the issue.
> 
> I do see _mm512_reduce_add_pd is missing from immintrin.h - but the
> flag __AVX512F__ is not set for me.
> 
> 
> Of course it is not set, you are just invoking the preprocessor. Try this way:
> 
> $ cat xyz.c
> #if defined __AVX512F__
> #error "avx512f flag set"
> #endif
> 
> $ gcc -march=native -c xyz.c
> xyz.c:2:2: error: #error "avx512f flag set"
>  #error "avx512f flag set"
> 
> I forgot to mention my XXXOPTFLAGS, full reconfigure script below.
> Do we have some Ubuntu 16 builder using system GCC?
>  Maybe we should use `-march=native -O3 -g3` in one of these builders?
> 
> $ cat arch-gnu-opt/lib/petsc/conf/reconfigure-arch-gnu-opt.py 
> #!/usr/bin/python
> if __name__ == '__main__':
>   import sys
>   import os
>   sys.path.insert(0, os.path.abspath('config'))
>   import configure
>   configure_options = [
>     '--COPTFLAGS=-march=native -mtune=native -O3',
>     '--CXXOPTFLAGS=-march=native -mtune=native -O3',
>     '--FOPTFLAGS=-march=native -mtune=native -O3',
>     '--download-metis=1',
>     '--download-p4est=1',
>     '--download-parmetis=1',
>     '--with-avx512-kernels=0',
>     '--with-debugging=0',
>     '--with-fortran-bindings=0',
>     '--with-zlib=1',
>     'CC=mpicc',
>     'CXX=mpicxx',
>     'FC=mpifort',
>     'PETSC_ARCH=arch-gnu-opt',
>   ]
>   configure.petsc_configure(configure_options)
> 
> 
> -- 
> Lisandro Dalcin
> ============
> Research Scientist
> Extreme Computing Research Center (ECRC)
> King Abdullah University of Science and Technology (KAUST)
> http://ecrc.kaust.edu.sa/