[petsc-users] Problem with BLASdot in 3.19.4

Satish Balay balay at mcs.anl.gov
Tue Sep 19 09:29:40 CDT 2023 

On Tue, 19 Sep 2023, Matthew Knepley wrote:

> On Tue, Sep 19, 2023 at 7:04 AM Thuc Bui <bui at calcreek.com> wrote:
> 
> > Hi Barry,
> >
> >
> >
> > Visual Studio 2022 is the problem! The code linked to Petsc 3.18.6 built
> > with VS 2022 also crashes at the same place. The same errors are shown
> > below. I don’t remember for sure, but I don’t think I was able to configure
> > Petsc 3.19.4 with VS 2019. However, I will still try that next.
> >
> 
> It is so easy to hate VS. This suggests that VS is secretly linking to
> another BLAS, perhaps with different calling semantics.

There could be multiple issues:

- code bugs - that show up in different settings
- Intel Compilers/OneAPI are have tight dependency on native compilers [on linux and windows]
So perhaps the current install is over VS19 - and not VS22. If this is the case - a reinstall [of Intel Compilers] over VS22 might help?
Or there could be different compiler dlls in VS vs Intel installs - and one of them is misbehaving.. [so reorder PATH location for Intel/MS?]
- MLK bugs that show up in some setups? [here using alternate blas - aka fblaslapack might help?]
- Latest VS with latest OneAPI might help? [however our test setup is with older versions of these compilers..

>>>
Compilers:
  C Compiler:         /cygdrive/e/svcpetsc/builds/Cw-cvdV3/0/petsc/petsc/lib/petsc/bin/win32fe/win_cl  -GF -MD -wd4996 -Zc:preprocessor  -O2 
    Version: Win32 Development Tool Front End, version 1.11.4 Fri, Sep 10, 2021  6:33:40 PM\nMicrosoft (R) C/C++ Optimizing Compiler Version 19.32.31329 for x64
  C++ Compiler:         /cygdrive/e/svcpetsc/builds/Cw-cvdV3/0/petsc/petsc/lib/petsc/bin/win32fe/win_cl  -GF -MD -GR -EHsc -Zc:preprocessor  -Zc:__cplusplus -O2 -Zm200  -std:c++20 -TP 
    Version: Win32 Development Tool Front End, version 1.11.4 Fri, Sep 10, 2021  6:33:40 PM\nMicrosoft (R) C/C++ Optimizing Compiler Version 19.32.31329 for x64
  Fortran Compiler:         /cygdrive/e/svcpetsc/builds/Cw-cvdV3/0/petsc/petsc/lib/petsc/bin/win32fe/win_ifort  -MD -O3 -fpp 
    Version: Win32 Development Tool Front End, version 1.11.4 Fri, Sep 10, 2021  6:33:40 PM\nIntel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.6.0 Build 20220226_000000
<<<

Satish

> 
>   Thanks,
> 
>      Matt
> 
> 
> > Thanks for your help,
> >
> > Thuc
> >
> >
> >
> > [0]PETSC ERROR:
> > ------------------------------------------------------------------------
> >
> > [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
> > probably memory access out of range
> >
> > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> >
> > [0]PETSC ERROR: or see https://petsc.org/release/faq/#valgrind and
> > https://petsc.org/release/faq/
> >
> > [0]PETSC ERROR: ---------------------  Stack Frames
> > ------------------------------------
> >
> > [0]PETSC ERROR: The line numbers in the error traceback are not always
> > exact.
> >
> > [0]PETSC ERROR: #1 BLASdot()
> >
> > [0]PETSC ERROR: #2 VecNorm_Seq() at
> > D:\Users\bbwannabe\Documents\Petsc\petsc-3.18.6\src\vec\vec\impls\seq\bvec2.c:216
> >
> > [0]PETSC ERROR: #3 VecNorm() at
> > D:\Users\bbwannabe\Documents\Petsc\petsc-3.18.6\src\vec\vec\interface\rvector.c:237
> >
> > [0]PETSC ERROR: #4 VecNormalize() at
> > D:\Users\bbwannabe\Documents\Petsc\petsc-3.18.6\src\vec\vec\interface\rvector.c:318
> >
> > [0]PETSC ERROR: #5 KSPGMRESCycle() at
> > D:\Users\bbwannabe\Documents\Petsc\petsc-3.18.6\src\ksp\ksp\impls\gmres\gmres.c:111
> >
> > [0]PETSC ERROR: #6 KSPSolve_GMRES() at
> > D:\Users\bbwannabe\Documents\Petsc\petsc-3.18.6\src\ksp\ksp\impls\gmres\gmres.c:228
> >
> > [0]PETSC ERROR: #7 KSPSolve_Private() at
> > D:\Users\bbwannabe\Documents\Petsc\petsc-3.18.6\src\ksp\ksp\interface\itfunc.c:899
> >
> > [0]PETSC ERROR: #8 KSPSolve() at
> > D:\Users\bbwannabe\Documents\Petsc\petsc-3.18.6\src\ksp\ksp\interface\itfunc.c:1071
> >
> >
> >
> > job aborted:
> >
> > [ranks] message
> >
> >
> >
> > [0] application aborted
> >
> > aborting MPI_COMM_WORLD (comm=0x44000000), error 59, comm rank 0
> >
> >
> >
> >
> >
> >
> >
> > *From:* petsc-users [mailto:petsc-users-bounces at mcs.anl.gov] *On Behalf
> > Of *Thuc Bui
> > *Sent:* Monday, September 18, 2023 4:24 PM
> > *To:* 'Barry Smith'
> > *Cc:* 'PETSc users list'
> > *Subject:* Re: [petsc-users] Problem with BLASdot in 3.19.4
> >
> >
> >
> > Thanks a lot Barry, for getting back to me. Will do what you have
> > suggested, and get back with the results.
> >
> >
> >
> > Best regards,
> >
> > Thuc
> >
> >
> >
> > *From:* Barry Smith [mailto:bsmith at petsc.dev]
> > *Sent:* Monday, September 18, 2023 3:43 PM
> > *To:* Thuc Bui
> > *Cc:* PETSc users list
> > *Subject:* Re: [petsc-users] Problem with BLASdot in 3.19.4
> >
> >
> >
> >
> >
> >   Ok, two things are being changed at the same time: the version  of PETSc
> > and the version of Visual Studio.
> >
> >
> >
> >   Could you please try with the new Visual Studio version but the same
> > older PETSc version? If that works could you try with the old Visual Studio
> > version but the new PETSc version?
> >
> >
> >
> >   Barry
> >
> >
> >
> >
> >
> >
> >
> > On Sep 18, 2023, at 6:26 PM, Thuc Bui <bui at calcreek.com> wrote:
> >
> >
> >
> > Dear Petsc users and experts,
> >
> >
> >
> > If someone can direct me how to track this bug, I would really appreciate
> > it.
> >
> >
> >
> > The Petsc DLL library version 3.19.4 was built on Windows 10 with Visual
> > Studio 2022, and with Microsoft MPI 10.1.2 and Intel MKL 2020.3.279. The
> > same code works fine with Petsc 3.18.6 using the same versions of MS MPI
> > and Intel MKL, and built with Visual Studio 2019.
> >
> >
> >
> > When my code calls PetscCall(KSPSolve(...)), it gets to
> > PetscCall(KSPGMRESCycle(...)), PetscCall(VecNormalize(VEC_VV(0), &res)),
> > PetscCall(VecNorm(x, NORM_2, &norm)) and then crashes in VecNorm_Seq() at:
> >
> >
> >
> >                 PetscCallBLAS("BLASdot", ztmp[type == NORM_1_AND_2] =
> > PetscSqrtReal(PetscRealPart(BLASdot_(&bn, xx, &one, xx, &one))));
> >
> >
> >
> > I tried to step into BLASdot_, but was unable to. I assume BLASdot_ belong
> > to Intel MKL, and its library does not contain debug information. After the
> > code exits, the errors are shown below the hash line.
> >
> >
> >
> > Should I install the latest Intel MKL to work with Petsc 3.19.4?
> >
> >
> >
> > Many thanks in advance for your help,
> >
> > Thuc Bui
> >
> > Senior R&D Engineer
> >
> > Calabazas Creek Research, Inc
> >
> > (650) 948-5361
> >
> >
> >
> >
> >
> > ###############################################
> >
> >
> >
> > [0]PETSC ERROR:
> > ------------------------------------------------------------------------
> >
> > [0]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation,
> > probably memory access out of range
> >
> > [0]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger
> >
> > [0]PETSC ERROR: or see https://petsc.org/release/faq/#valgrind and
> > https://petsc.org/release/faq/
> >
> > [0]PETSC ERROR: ---------------------  Stack Frames
> > ------------------------------------
> >
> > [0]PETSC ERROR: The line numbers in the error traceback are not always
> > exact.
> >
> > [0]PETSC ERROR: #1 BLASdot()
> >
> > [0]PETSC ERROR: #2 VecNorm_Seq() at
> > D:\Users\bbwannabe\Documents\Petsc\latest\src\vec\vec\impls\seq\bvec2.c:213
> >
> > [0]PETSC ERROR: #3 VecNorm() at
> > D:\Users\bbwannabe\Documents\Petsc\latest\src\vec\vec\interface\rvector.c:237
> >
> > [0]PETSC ERROR: #4 VecNormalize() at
> > D:\Users\bbwannabe\Documents\Petsc\latest\src\vec\vec\interface\rvector.c:318
> >
> > [0]PETSC ERROR: #5 KSPGMRESCycle() at
> > D:\Users\bbwannabe\Documents\Petsc\latest\src\ksp\ksp\impls\gmres\gmres.c:110
> >
> > [0]PETSC ERROR: #6 KSPSolve_GMRES() at
> > D:\Users\bbwannabe\Documents\Petsc\latest\src\ksp\ksp\impls\gmres\gmres.c:227
> >
> > [0]PETSC ERROR: #7 KSPSolve_Private() at
> > D:\Users\bbwannabe\Documents\Petsc\latest\src\ksp\ksp\interface\itfunc.c:898
> >
> > [0]PETSC ERROR: #8 KSPSolve() at
> > D:\Users\bbwannabe\Documents\Petsc\latest\src\ksp\ksp\interface\itfunc.c:1070
> >
> >
> >
> > job aborted:
> >
> > [ranks] message
> >
> >
> >
> > [0] application aborted
> >
> > aborting MPI_COMM_WORLD (comm=0x44000000), error 59, comm rank 0
> >
> >
> >
> 
> 
>

More information about the petsc-users mailing list