[petsc-users] slepc NHEP error

Barry Smith bsmith at mcs.anl.gov
Wed Jun 14 16:31:37 CDT 2017 

  Here is the line that generates an error:

    ierr = MPI_Allreduce(bv->work,y,len,MPIU_SCALAR,MPIU_SUM,PetscObjectComm((PetscObject)bv));CHKERRQ(ierr);

let's see what the MPI error is by running again with the additional command line option -on_error_abort

hopefully MPI will say something useful.

   Barry

> On Jun 14, 2017, at 4:24 PM, Kannan, Ramakrishnan <kannanr at ornl.gov> wrote:
> 
> 
> 
> -- 
> Regards,
> Ramki
> 
> 
> On 6/14/17, 5:21 PM, "Barry Smith" <bsmith at mcs.anl.gov> wrote:
> 
> 
>       Send the file autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvblas.c as an attachment.
> 
>       Barry
> 
>> On Jun 14, 2017, at 4:17 PM, Kannan, Ramakrishnan <kannanr at ornl.gov> wrote:
>> 
>> Barry,
>> 
>> Appreciate your kind help. It compiles fine. I am still getting the following error.
>> 
>> [0]PETSC ERROR: #1 BVDotVec_BLAS_Private() line 272 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvblas.c
>> [0]PETSC ERROR: #2 BVDotVec_Svec() line 150 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/impls/svec/svec.c
>> [0]PETSC ERROR: #3 BVDotVec() line 191 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvglobal.c
>> [0]PETSC ERROR: #4 BVOrthogonalizeCGS1() line 81 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [0]PETSC ERROR: #5 BVOrthogonalizeCGS() line 214 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [0]PETSC ERROR: #6 BVOrthogonalizeColumn() line 371 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [0]PETSC ERROR: [8]PETSC ERROR: #1 BVDotVec_BLAS_Private() line 272 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvblas.c
>> [8]PETSC ERROR: #2 BVDotVec_Svec() line 150 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/impls/svec/svec.c
>> [8]PETSC ERROR: #3 BVDotVec() line 191 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvglobal.c
>> [8]PETSC ERROR: #4 BVOrthogonalizeCGS1() line 81 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [8]PETSC ERROR: #5 BVOrthogonalizeCGS() line 214 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [8]PETSC ERROR: #6 BVOrthogonalizeColumn() line 371 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [8]PETSC ERROR: #7 EPSBasicArnoldi() line 59 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/epskrylov.c
>> [8]PETSC ERROR: #8 EPSSolve_KrylovSchur_Default() line 203 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/krylovschur/krylovschur.c
>> [8]PETSC ERROR: #9 EPSSolve() line 101 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/interface/epssolve.c
>> [8]PETSC ERROR: #10 count() line 266 in /lustre/atlas/proj-shared/csc040/gryffin/gryffndor/miniapps/cmake/../algorithms/tricount.hpp
>> [2]PETSC ERROR: #1 BVDotVec_BLAS_Private() line 272 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvblas.c
>> [2]PETSC ERROR: #2 BVDotVec_Svec() line 150 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/impls/svec/svec.c
>> [2]PETSC ERROR: #3 BVDotVec() line 191 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvglobal.c
>> [2]PETSC ERROR: #4 BVOrthogonalizeCGS1() line 81 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [2]PETSC ERROR: #5 BVOrthogonalizeCGS() line 214 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [2]PETSC ERROR: #6 BVOrthogonalizeColumn() line 371 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [2]PETSC ERROR: #7 EPSBasicArnoldi() line 59 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/epskrylov.c
>> [2]PETSC ERROR: #8 EPSSolve_KrylovSchur_Default() line 203 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/krylovschur/krylovschur.c
>> [2]PETSC ERROR: #9 EPSSolve() line 101 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/interface/epssolve.c
>> [2]PETSC ERROR: #10 count() line 266 in /lustre/atlas/proj-shared/csc040/gryffin/gryffndor/miniapps/cmake/../algorithms/tricount.hpp
>> #7 EPSBasicArnoldi() line 59 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/epskrylov.c
>> [0]PETSC ERROR: #8 EPSSolve_KrylovSchur_Default() line 203 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/krylovschur/krylovschur.c
>> [0]PETSC ERROR: #9 EPSSolve() line 101 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/interface/epssolve.c
>> [0]PETSC ERROR: #10 count() line 266 in /lustre/atlas/proj-shared/csc040/gryffin/gryffndor/miniapps/cmake/../algorithms/tricount.hpp
>> [7]PETSC ERROR: #1 BVDotVec_BLAS_Private() line 272 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvblas.c
>> [7]PETSC ERROR: #2 BVDotVec_Svec() line 150 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/impls/svec/svec.c
>> [7]PETSC ERROR: #3 BVDotVec() line 191 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvglobal.c
>> [7]PETSC ERROR: #4 BVOrthogonalizeCGS1() line 81 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [7]PETSC ERROR: #5 BVOrthogonalizeCGS() line 214 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [7]PETSC ERROR: [15]PETSC ERROR: #1 BVDotVec_BLAS_Private() line 272 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvblas.c
>> [15]PETSC ERROR: #2 BVDotVec_Svec() line 150 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/impls/svec/svec.c
>> [15]PETSC ERROR: #3 BVDotVec() line 191 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvglobal.c
>> [15]PETSC ERROR: #4 BVOrthogonalizeCGS1() line 81 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [15]PETSC ERROR: #5 BVOrthogonalizeCGS() line 214 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [15]PETSC ERROR: #6 BVOrthogonalizeColumn() line 371 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [15]PETSC ERROR: #7 EPSBasicArnoldi() line 59 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/epskrylov.c
>> [15]PETSC ERROR: #8 EPSSolve_KrylovSchur_Default() line 203 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/krylovschur/krylovschur.c
>> [15]PETSC ERROR: #9 EPSSolve() line 101 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/interface/epssolve.c
>> [15]PETSC ERROR: #10 count() line 266 in /lustre/atlas/proj-shared/csc040/gryffin/gryffndor/miniapps/cmake/../algorithms/tricount.hpp
>> #6 BVOrthogonalizeColumn() line 371 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>> [7]PETSC ERROR: #7 EPSBasicArnoldi() line 59 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/epskrylov.c
>> [7]PETSC ERROR: #8 EPSSolve_KrylovSchur_Default() line 203 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/krylovschur/krylovschur.c
>> [7]PETSC ERROR: #9 EPSSolve() line 101 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/interface/epssolve.c
>> [7]PETSC ERROR: #10 count() line 266 in /lustre/atlas/proj-shared/csc040/gryffin/gryffndor/miniapps/cmake/../algorithms/tricount.hpp
>> 
>> -- 
>> Regards,
>> Ramki
>> 
>> 
>> On 6/14/17, 4:48 PM, "Barry Smith" <bsmith at mcs.anl.gov> wrote:
>> 
>> 
>>> On Jun 14, 2017, at 3:45 PM, Kannan, Ramakrishnan <kannanr at ornl.gov> wrote:
>>> 
>>> Barry,
>>> 
>>> All the functions here are standard SLEPC functions and there are no user-defined or custom code here. As you can see, when I uncomment the CHKERRQ macros in my code, I am getting the compilation error. 
>> 
>>      Yes that is because YOUR function that calls the SLEPc functions is void and doesn't return an error code. It is that function I recommend changing to return error codes.
>> 
>>       Barry
>> 
>>> 
>>> -- 
>>> Regards,
>>> Ramki
>>> 
>>> 
>>> On 6/14/17, 4:40 PM, "Barry Smith" <bsmith at mcs.anl.gov> wrote:
>>> 
>>> 
>>>> On Jun 14, 2017, at 3:33 PM, Kannan, Ramakrishnan <kannanr at ornl.gov> wrote:
>>>> 
>>>> Can I use CHKERRV instead of CHKERRQ? Will that help?
>>> 
>>>     You can do that. But I question you having functions in your code that return void instead of an error code. Without error codes you are just hurting your own productivity.
>>> 
>>>     Barry
>>> 
>>>> 
>>>> -- 
>>>> Regards,
>>>> Ramki
>>>> 
>>>> 
>>>> On 6/14/17, 4:25 PM, "Kannan, Ramakrishnan" <kannanr at ornl.gov> wrote:
>>>> 
>>>> I get the following compilation error when I have CHKERRQ.
>>>> 
>>>> /opt/cray/petsc/3.7.4.0/real/GNU64/5.1/sandybridge/include/petscerror.h:433:154: error: return-statement with a value, in function returning 'void' [-fpermissive]
>>>>  #define CHKERRQ(n)             do {if (PetscUnlikely(n)) return PetscError(PETSC_COMM_SELF,__LINE__,PETSC_FUNCTION_NAME,__FILE__,n,PETSC_ERROR_REPEAT," ");} while (0)
>>>> 
>>>> 
>>>> -- 
>>>> Regards,
>>>> Ramki
>>>> 
>>>> 
>>>> On 6/14/17, 4:14 PM, "Barry Smith" <bsmith at mcs.anl.gov> wrote:
>>>> 
>>>> 
>>>>       Why do you have the CHKERRQ(ierr); commented out in your code? 
>>>> 
>>>>        Because of this you are getting mangled confusing error messages. 
>>>> 
>>>>        Put a ierr = in front of all calls and a CHKERRQ(ierr); after each call. 
>>>> 
>>>>        Then resend the new error message which will be much clearer.
>>>> 
>>>> 
>>>> 
>>>>> On Jun 14, 2017, at 2:58 PM, Kannan, Ramakrishnan <kannanr at ornl.gov> wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> I am running NHEP across 16 MPI processors over 16 nodes in a matrix of global size of 1,000,000x1,000,000 with approximately global 16,000,000 non-zeros. Each node has approximately 1million non-zeros.
>>>>> 
>>>>> The following is my slepc code for EPS.
>>>>> 
>>>>> PetscInt nev;
>>>>> ierr = EPSCreate(PETSC_COMM_WORLD, &eps);  // CHKERRQ(ierr);
>>>>> ierr = EPSSetOperators(eps, A, NULL);  // CHKERRQ(ierr);
>>>>> ierr = EPSSetProblemType(eps, EPS_NHEP);  // CHKERRQ(ierr);
>>>>> EPSSetWhichEigenpairs(eps, EPS_LARGEST_REAL);
>>>>> EPSSetDimensions(eps, 100, PETSC_DEFAULT, PETSC_DEFAULT);
>>>>> PRINTROOT("calling epssolve");
>>>>> ierr = EPSSolve(eps);  // CHKERRQ(ierr);
>>>>> ierr = EPSGetType(eps, &type);  // CHKERRQ(ierr);
>>>>> ierr = PetscPrintf(PETSC_COMM_WORLD, " Solution method: %s\n\n", type);
>>>>> // CHKERRQ(ierr);
>>>>> ierr = EPSGetDimensions(eps, &nev, NULL, NULL);  // CHKERRQ(ierr);
>>>>> ierr = PetscPrintf(PETSC_COMM_WORLD,
>>>>>                    " Number of requested eigenvalues: %D\n",
>>>>>                    nev);  // CHKERRQ(ierr);
>>>>> 
>>>>> I am getting the following error. Attached is the entire error file for your reference. Please let me know what should I fix in this code.
>>>>> 
>>>>> 2]PETSC ERROR: Argument out of range
>>>>> [2]PETSC ERROR: Argument 2 out of range
>>>>> [2]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting.
>>>>> [2]PETSC ERROR: Petsc Release Version 3.7.4, Oct, 02, 2016
>>>>> [2]PETSC ERROR: ./miniapps on a sandybridge named nid00300 by d3s Wed Jun 14 15:32:00 2017
>>>>> [13]PETSC ERROR: #1 BVDotVec_BLAS_Private() line 272 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvblas.c
>>>>> [13]PETSC ERROR: #2 BVDotVec_Svec() line 150 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/impls/svec/svec.c
>>>>> [13]PETSC ERROR: #3 BVDotVec() line 191 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvglobal.c
>>>>> [13]PETSC ERROR: #4 BVOrthogonalizeCGS1() line 81 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>>>>> [13]PETSC ERROR: #5 BVOrthogonalizeCGS() line 214 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>>>>> [13]PETSC ERROR: #6 BVOrthogonalizeColumn() line 371 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/sys/classes/bv/interface/bvorthog.c
>>>>> [13]PETSC ERROR: #7 EPSBasicArnoldi() line 59 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/epskrylov.c
>>>>> [13]PETSC ERROR: #8 EPSSolve_KrylovSchur_Default() line 203 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/impls/krylov/krylovschur/krylovschur.c
>>>>> [13]PETSC ERROR: #9 EPSSolve() line 101 in /autofs/nccs-svm1_home1/ramki/libraries/slepc-3.7.3/src/eps/interface/epssolve.c
>>>>> 
>>>>> -- 
>>>>> Regards,
>>>>> Ramki
>>>>> 
>>>>> <test.pbs.e613713>
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> 
>> 
> 
> 
> 
> 
> <bvblas.c>

More information about the petsc-users mailing list