[petsc-dev] SuperLU link error on BGP FEN

[John R. Cary]

On 12/2/2010 1:21 PM, Barry Smith wrote:
> On Dec 2, 2010, at 1:59 PM, John R. Cary wrote:
>
>> Anyway, given the pokeyness of the FENs on surveyor, it should be some
>> time before midnight!
>    This is completely unacceptable! PETSc takes less than 10 minutes to configure and compile on my $2000 laptop (and probably half as long on Satish's $1000 laptop). On a $100 million machine it takes one half a day?
>
>    That is frankly disgusting and indicates extreme problems with the configuration of that machine. Why has this problem not been fixed years ago?
>

Well, I may be exaggerating a bit :-), but I started configuring at:

-rwxrw-r--  1 cary users     924 Dec  2 13:55 
surveyor.alcf.anl.gov-petsc-ser.sh

and it is not done at

Thu Dec  2 14:33:00 CST 2010

This is just the petsc part, not all the packages, so it really does 
take a long time,
I find.

I do find that the XL compilers are both slow and buggy.  I have 
submitted bug
reports but they seem to disappear into a black hole at IBM.  For 
example, I submitted alcf-support
ticket #32756 on August 11, 2009, complaining that the mpi.mod in
/bgsys/drivers/V1R3M0_460_2008-081112P/ppc/comm/default/include/mpi.mod
was built with gfortran and incompatible with XLF, even though it is
called by the XLF compiler wrapper.  The guys at ALCF contacted IBM 
about this,
but here it is, over a year later, and the same problem is still 
present.  It seems
like IBM is just not responsive.

OTOH, both Ray and Vitali have been very helpful, without their help I might
not have found my way around several compiler bugs and other issues.

.....John







>> John
>>
>>> satish
>>>
>>> On Thu, 2 Dec 2010, Matthew Knepley wrote:
>>>
>>>> We always always always always need configure.log.
>>>>
>>>>     Matt
>>>>
>>>> On Thu, Dec 2, 2010 at 8:41 AM, John R. Cary<cary at txcorp.com>   wrote:
>>>>
>>>>> I am trying to link facets on a FEN of surveyor.alcf.anl.gov.  It ends
>>>>> with the error,
>>>>>
>>>>> /gpfs/home/projects/facets/surveyor/contrib-xlc-9.0/petsc-3.1-p4-ser/lib/libsuperlu_4.0.a(dldperm.o):
>>>>> In function `dldperm':
>>>>> /gpfs/home/cary/facetspkgs/builds/petsc-3.1-p4/ser/externalpackages/SuperLU_4.0/SRC/dldperm.c:127:
>>>>> undefined reference to `mc64id_'
>>>>> /gpfs/home/cary/facetspkgs/builds/petsc-3.1-p4/ser/externalpackages/SuperLU_4.0/SRC/dldperm.c:134:
>>>>> undefined reference to `mc64ad_'
>>>>>
>>>>> which indicates that the SuperLU compiled with PETSc did
>>>>> not get the fortran underscoring flag correct (which should
>>>>> be no underscore with xlf).  nm shows
>>>>>
>>>>> login1.surveyor$ nm
>>>>> /gpfs/home/projects/facets/surveyor/contrib-xlc-9.0/petsc-3.1-p4-ser/lib/libsuperlu_4.0.a
>>>>> | grep mc64
>>>>>                 U mc64ad_
>>>>>                 U mc64id_
>>>>> mc64ad.o:
>>>>> 0000000000000018 D mc64ad
>>>>> 0000000000000030 D mc64bd
>>>>> 0000000000000048 D mc64dd
>>>>> 0000000000000060 D mc64ed
>>>>> 0000000000000078 D mc64fd
>>>>> 0000000000000000 D mc64id
>>>>> 00000000000000c0 D mc64qd
>>>>> 0000000000000090 D mc64rd
>>>>> 00000000000000a8 D mc64sd
>>>>> 00000000000000d8 D mc64ud
>>>>> 00000000000000f0 D mc64wd
>>>>>                 U mc64ad_
>>>>>                 U mc64id_
>>>>>
>>>>> that the underscored symbol is being called, but the
>>>>> underscore-free symbol is what was defined.
>>>>>
>>>>> My PETSc configure line was
>>>>>
>>>>> #!/bin/sh
>>>>> /gpfs/home/cary/facetspkgs/builds-surveyor-xlc/facetspkgs/petsc-3.1-p4/ser/configure
>>>>> \
>>>>> --prefix=/gpfs/home/projects/facets/surveyor/contrib-xlc-9.0/petsc-3.1-p4-ser
>>>>> \
>>>>> --with-mpi=0 \
>>>>> --with-debugging=0 \
>>>>> --with-x=0 \
>>>>> --with-cc='xlc_r' \
>>>>> --with-cxx='xlC_r' \
>>>>> --with-fc='xlf_r' \
>>>>> --COPTFLAGS='-O2 -g' \
>>>>> --download-superlu \
>>>>> --with-lapack-lib=/home/projects/facets/intrepid/contrib/lapack-ser/lib/liblapack.a
>>>>> \
>>>>> --with-blas-lib=/home/projects/facets/intrepid/contrib/lapack-ser/lib/libblas.a
>>>>> \
>>>>> PETSC_DIR=/gpfs/home/cary/facetspkgs/builds-surveyor-xlc/facetspkgs/petsc-3.1-p4/ser
>>>>> \
>>>>> PETSC_ARCH=facets-ser \
>>>>> --CFLAGS='-q64 -qlanglvl=redefmac' \
>>>>> --CXXFLAGS='-q64 -qlanglvl=redefmac' \
>>>>> --FFLAGS='-q64 -qextname=flush' \
>>>>> --with-shared=1
>>>>>
>>>>> so I think that PETSc had enough info to figure out the underscoring.
>>>>> Perhaps this is a bug.
>>>>>
>>>>> But regardless, is there a workaround?
>>>>>
>>>>> Thx....John
>>>>>
>>>>
>>>>
>