On Thu, Aug 18, 2011 at 3:31 AM, Xuefei (Rebecca) Yuan <span dir="ltr"><<a href="mailto:xyuan@lbl.gov">xyuan@lbl.gov</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Dear all,<br>
<br>
I have a problem for compile PDSLin with PETSc, the configure.log says<br>
<br>
------------------------------------------------------<br>
====== Compiling PDSLin; this may take several minutes ===============================================================================<br>
sh: cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0 &&<br>
PDSLIN_INSTALL_DIR=/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib && export PDSLIN_INSTALL_DIR && make clean && make lib LAAUX="" && mv -f *.a /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib && cp -f include/*.h /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/include/.Executing: cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0 && PDSLIN_INSTALL_DIR=/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-<br>
xt6-pkgs-opt-doubleprecision-pdslin/lib && export PDSLIN_INSTALL_DIR && make clean && make lib LAAUX="" && mv -f *.a /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib && cp -f include/*.h /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/include/.<br>
sh: (cd src/; make clean; cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran; make clean; cd /scratch2/scratchdirs/xyuan/Software_Hopper/pet<br>
sc-dev/externalpackages/pdslin_0.0/fortran/examples; make clean; cd /scratch2/scratchdirs/x<br>
yuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/examples; make clean; rm -f ; )<br>
<br>
make[1]: Entering directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/src'<br>
( rm -f *.o; rm -f ./slu/*.o;)<br>
make[1]: Leaving directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/src'<br>
make[1]: Entering directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran'<br>
(rm -f *.o; rm -f *.mod;)<br>
make[1]: Leaving directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran'<br>
make[1]: Entering directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran/examples'<br>
(rm -f *.o; rm -f *.mod; rm -f dtest_pdslinf; rm -f ztest_pdslinf;)<br>
make[1]: Leaving directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran/examples'<br>
make[1]: Entering directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/examples'<br>
(rm -f *.o; rm -f *.dat; rm -f dtest_pdslin; rm -f ztest_pdslin; rm -f dtest; rm -f ztest;)<br>
make[1]: Leaving directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/examples'<br>
make: Nothing to be done for `lib'.<br>
<br>
*******************************************************************************<br>
UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):<br>
-------------------------------------------------------------------------------<br>
Error running make on PDSLIN: Could not execute "cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0 && PDSLIN_INSTALL_DIR=/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib && export PDSLIN_INSTALL_DIR && make clean && make lib LAAUX="" && mv -f *.a /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib && cp -f include/*.h /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/include/.":<br>
(cd src/; make clean; cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran; make clean; cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran/examples; make clean; cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/examples; make clean; rm -f ; )<br>
make[1]: Entering directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/src'<br>
( rm -f *.o; rm -f ./slu/*.o;)<br>
make[1]: Leaving directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/src'<br>
make[1]: Entering directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran'<br>
(rm -f *.o; rm -f *.mod; rm -f dtest_pdslinf; rm -f ztest_pdslinf;)<br>
make[1]: Leaving directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/fortran/examples'<br>
make[1]: Entering directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/external<br>
packages/pdslin_0.0/examples'<br>
(rm -f *.o; rm -f *.dat; rm -f dtest_pdslin; rm -f ztest_pdslin; rm -f dtest; rm -f ztest;)<br>
make[1]: Leaving directory `/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0/examples'<br>
make: Nothing to be done for `lib'.<br>
mv: cannot stat `*.a': No such file or directory*******************************************************************************<br></blockquote><div>^^^^^^^^^^^^^^^^^^^^^</div><div><br></div><div>Your install is failing.</div>
<div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/configure.py", line 27<br>
2, in petsc_configure framework.configure(out = sys.stdout)<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/BuildSystem/config/fra<br>
mework.py", line 925, in configure child.configure()<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/BuildSystem/config/package.py", line 500, in configure<br>
self.executeTest(self.configureLibrary)<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/BuildSystem/config/base.py", line 115, in executeTest<br>
ret = apply(test, args,kargs)<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/BuildSystem/config/package.py", line 427, in configureLibrary<br>
for location, directory, lib, incl in self.generateGuesses():<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/BuildSystem/config/package.py", line 223, in generateGuesses<br>
d = self.checkDownload(1)<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/BuildSystem/config/package.py", line 308, in checkDownload<br>
return self.getInstallDir()<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/BuildSystem/config/package.py", line 181, in getInstallDir<br>
return os.path.abspath(self.Install())<br>
File "/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/config/PETSc/packages/PDSLin.py", line 74, in Install<br>
raise RuntimeError('Error running make on PDSLIN: '+str(e))<br>
<br>
<br>
<br>
-----------------------------------------------------<br>
I started to copy and paste the error message including commands:<br>
<br>
<br>
Error running make on PDSLIN: Could not execute "cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0 && PDSLIN_INSTALL_DIR=/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib && export PDSLIN_INSTALL_DIR && make clean && make lib LAAUX="" && mv -f *.a /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib && cp -f include/*.h /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/include/.":<br>
<br>
<br>
And found that<br>
<br>
Software_Hopper/petsc-dev> cd /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0<br>
Directory: /scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/externalpackages/pdslin_0.0<br>
externalpackages/pdslin_0.0> PDSLIN_INSTALL_DIR=/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib<br>
PDSLIN_INSTALL_DIR=/scratch2/scratchdirs/xyuan/Software_Hopper/petsc-dev/arch-cray-xt6-pkgs-opt-doubleprecision-pdslin/lib: Command not found.<br>
externalpackages/pdslin_0.0><br>
<br>
Then I realized that it is a csh not a bash. However, there is no error message for other packages, such as SuperLU_DIST. My PDSLin.py looks like (modified from SuperLU_DIST.py):<br>
<br>
<br>
-------------------------------------<br>
import PETSc.package<br>
<br>
class Configure(PETSc.package.NewPackage):<br>
def __init__(self, framework): PETSc.package.NewPackage.__init__(self, framework)<br>
self.downloadname = self.name.lower()<br>
self.download = ['<a href="http://portal.nersc.gov/project/nerscpd/pdslin_0.0.tar.gz" target="_blank">http://portal.nersc.gov/project/nerscpd/pdslin_0.0.tar.gz</a>']<br>
self.functions = ['dpdslin_solver']<br>
self.includes = ['dpdslin_solver.h']<br>
self.liblist = [['libpdslin.a']]<br>
#<br>
# PDSLin supports 64 bit integers but uses ParMetis which does not, see the comment in<br>
ParMetis.py<br>
# in the method configureLibrary()<br>
self.requires32bitint = 0<br>
self.complex = 1<br>
return<br>
def setupDependencies(self, framework):<br>
PETSc.package.NewPackage.setupDependencies(self, framework)<br>
self.blasLapack = framework.require('config.packages.BlasLapack',self)<br>
self.parmetis = framework.require('PETSc.packages.ParMetis',self)<br>
self.scotch = framework.require('PETSc.packages.Scotch',self)<br>
self.superlu_dist = framework.require('PETSc.packages.SuperLU_DIST',self)<br>
self.deps = [self.mpi,self.blasLapack,self.parmetis,self.scotch,self.superlu_dist]<br>
return<br>
<br>
def Install(self):<br>
import os<br>
import sys<br>
<br>
g = open(os.path.join(self.packageDir,'make.inc'),'w')<br>
g.write('SHELL = '+self.programs.SHELL+'\n')<br>
g.write('DPDSLINroot = '+self.packageDir+'\n')<br>
g.write('HOME = $(DPDSLINroot)\n')<br>
g.write('DPDSLINLIB = $(DPDSLINroot)/lib/libpdslin.a\n')<br>
g.write('BLASDEF = -DUSE_VENDOR_BLAS\n')<br>
g.write('BLASLIB = '+self.libraries.toString(self.blasLapack.dlib)+'\n')<br>
g.write('IMPI = '+self.headers.toString(self.mpi.include)+'\n')<br>
g.write('MPILIB = '+self.libraries.toString(self.mpi.lib)+'\n')<br>
g.write('PMETISLIB = '+self.libraries.toString(self.parmetis.lib)+'\n')<br>
g.write('PSCOTCHLIB = '+self.libraries.toString(self.scotch.lib)+'\n')<br>
g.write('SUPERLUDISTLIB = '+self.libraries.toString(self.superlu_dist.lib)+'\n')<br>
g.write('LIBS = $(DPDSLINLIB) $(BLASLIB) $(PMETISLIB) $(MPILIB) $(PSCOTCHLIB) $(SUPERLUDISTLIB)\n')<br>
g.write('ARCH = '+<a href="http://self.setCompilers.AR" target="_blank">self.setCompilers.AR</a>+'\n')<br>
g.write('ARCHFLAGS = '+self.setCompilers.AR_FLAGS+'\n')<br>
g.write('RANLIB = '+self.setCompilers.RANLIB+'\n')<br>
self.setCompilers.pushLanguage('C')<br>
g.write('CC = '+self.setCompilers.getCompiler()+' $(IMPI)\n') #build fails without $(IMPI)<br>
g.write('CFLAGS = '+self.setCompilers.getCompilerFlags()+'\n')<br>
g.write('LOADER = '+self.setCompilers.getLinker()+'\n')<br>
g.write('LOADOPTS = \n')<br>
self.setCompilers.popLanguage()<br>
# set blas/lapack name mangling<br>
if self.blasLapack.mangling == 'underscore':<br>
g.write('CDEFS = -DAdd_')<br>
elif self.blasLapack.mangling == 'caps':<br>
g.write('CDEFS = -DUpCase')<br>
else:<br>
g.write('CDEFS = -DNoChange')<br>
if self.framework.argDB['with-64-bit-indices']:<br>
g.write(' -D_LONGINT')<br>
g.write('\n')<br>
# not sure what this is for<br>
g.write('NOOPTS = '+self.blasLapack.getSharedFlag(self.setCompilers.getCompilerFlags())+' '+self.blasLapack.getPrecisionFlag(self.setCompilers.getCompilerFlags())+' '+self.blasLapack.getWindowsNonOptFlags(self.setCompilers.getCompilerFlags())+'\n')<br>
g.close()<br>
<br>
if self.installNeeded('make.inc'):<br>
try:<br>
self.logPrintBox('Compiling PDSLin; this may take several minutes')<br>
if not os.path.exists(os.path.join(self.packageDir,'lib')):<br>
os.makedirs(os.path.join(self.packageDir,'lib'))<br>
output,err,ret = PETSc.package.NewPackage.executeShellCommand('cd '+self.packageDir+' && PDSLIN_INSTALL_DIR='+self.installDir+'/lib && export PDSLIN_INSTALL_DIR && make clean && make lib LAAUX="" && mv -f *.a '+os.path.join(self.installDir,'lib')+' && cp -f include/*.h '+os.path.join(self.installDir,'include')+'/.', timeout=2500, log = self.framework.log)<br>
except RuntimeError, e:<br>
raise RuntimeError('Error running make on PDSLIN: '+str(e))<br>
self.postInstall(output+err,'make.inc')<br>
return self.installDir<br>
<br>
def consistencyChecks(self):<br>
PETSc.package.NewPackage.consistencyChecks(self)<br>
if self.framework.argDB['with-'+self.package]:<br>
if not self.blasLapack.checkForRoutine('slamch'):<br>
raise RuntimeError('PDSLin requires the BLAS routine slamch()')<br>
self.framework.log.write('Found slamch() in BLAS library as needed by PDSLin\n')<br>
if not self.blasLapack.checkForRoutine('dlamch'):<br>
raise RuntimeError('PDSLin requires the BLAS routine dlamch()')<br>
self.framework.log.write('Found dlamch() in BLAS library as needed by PDSLin\n')<br>
if not self.blasLapack.checkForRoutine('xerbla'):<br>
raise RuntimeError('PDSLin requires the BLAS routine xerbla()')<br>
self.framework.log.write('Found xerbla() in BLAS library as needed by PDSLin\n')<br>
return<br>
<br>
<br>
------------------------------------<br>
<br>
What could be wrong here?<br>
<br>
Thanks very much!<br>
<br>
Cheers,<br>
<br>
Rebecca<br>
<br>
<br>
On Aug 17, 2011, at 1:46 PM, Barry Smith wrote:<br>
<br>
><br>
> Rebecca,<br>
><br>
> It would probably be best if you email to petsc-maint the PDSLin tarball so we can look at the PDSlin structure and come up with the best way to proceed. The way to handle the packaging will depend on how PDSLin interacts with PETSc.<br>
><br>
> Thanks<br>
><br>
> Barry<br>
><br>
> On Aug 17, 2011, at 3:19 PM, Xuefei (Rebecca) Yuan wrote:<br>
><br>
>> Dear all,<br>
>><br>
>> We are trying to add a PETSc dependent package called PDSLin to PETSc as an external package. PDSLin is a hybrid linear solver and we would like to have it been imported to PETSc as SuperLU_DIST used to be, and it provides an alternative way to solving a linear system in parallel.<br>
>><br>
>> Even though the package has not been released yet (should be released soon) and it is under testing for some problems, we are still interesting in putting it as an external package to PETSc for now.<br>
>><br>
>> I did some search of the old threads to the petsc-dev team and there is a request from Lious for the PaStiX a few years ago.<br>
>><br>
>> My understanding to add a package is the following steps:<br>
>><br>
>> 1) create a python code called "PDSLin.py" under /petsc-dev/config/PETSc/packages<br>
>><br>
>> 2) write an interface file between PETSc and PDSLin.<br>
>><br>
>> As PDSLin is very similar to SuperLU_DIST, and it also depending on PETSc (similar to Prometheus), the PDSLin.py is created based on SuperLU_DIST.py and Prometheus.py.<br>
>><br>
>> Before putting the interface file in some places under petsc-dev directory, would you please help me understand how those option calls go to this interface file?<br>
>><br>
>> a) am I missing another file to create/modify to have those options connected to PDSLin in a very similar way to SuperLU_DIST?<br>
>><br>
>> b) do I have some flexibility to add more options called from PETSc to this package?<br>
>><br>
>> Moreover, I am not sure how to put the interface file in the dev package.<br>
>><br>
>> c) where shall it go? The SuperLU_DIST is located at<br>
>><br>
>> ./src/mat/impls/aij/mpi/superlu_dist/superlu_dist.c<br>
>><br>
>> shall I put it as<br>
>><br>
>> ./src/mat/impls/aij/mpi/pdslin/pdslin.c ?<br>
>><br>
>> d) what is the rule of writing this interface file?<br>
>><br>
>> In superlu_dist.c, the extern functions are<br>
>><br>
>> extern PetscErrorCode MatFactorInfo_SuperLU_DIST(Mat,PetscViewer);<br>
>> extern PetscErrorCode MatLUFactorNumeric_SuperLU_DIST(Mat,Mat,const MatFactorInfo *);<br>
>> extern PetscErrorCode MatDestroy_SuperLU_DIST(Mat);<br>
>> extern PetscErrorCode MatView_SuperLU_DIST(Mat,PetscViewer);<br>
>> extern PetscErrorCode MatSolve_SuperLU_DIST(Mat,Vec,Vec);<br>
>> extern PetscErrorCode MatLUFactorSymbolic_SuperLU_DIST(Mat,Mat,IS,IS,const MatFactorInfo *);<br>
>> extern PetscErrorCode MatDestroy_MPIAIJ(Mat);<br>
>><br>
>> What kind of functions should I have in the interface file for PDSLin?<br>
>><br>
>> I am not sure if I have stated the problem clearly, in short is that we would like to add some package to PETSc but not sure the right way and steps to approach the goal.<br>
>><br>
>> Thanks very much!<br>
>><br>
>> Best regards,<br>
>><br>
>> Rebecca<br>
>><br>
>> Xuefei (Rebecca) Yuan<br>
>> Postdoctoral Fellow<br>
>> Lawrence Berkeley National Laboratory<br>
>> Tel: <a href="tel:1-510-486-7031" value="+15104867031">1-510-486-7031</a><br>
>><br>
>><br>
><br>
<br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>