On Fri, Jan 20, 2012 at 11:21 AM, Anton Popov <span dir="ltr"><<a href="mailto:popov@uni-mainz.de">popov@uni-mainz.de</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Well, I simply gave up on Cray compilers, as well as on the other compilers installed on this machine (Intel, Portland, Pathscale). After many attempts only GNU compilers were functional with PETSc.</blockquote><div><br></div>
<div>1) If you don't send in the error reports, we can't fix it.</div><div><br></div><div>2) Dominik, I have pushed a possible fix for this to petsc-dev. Can you try running with that on this machine?</div><div><br>
</div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5">
On 1/20/12 6:12 PM, Dominik Szczerba wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Anton,<br>
<br>
Many thanks for your hints. I am on the same machine. The key<br>
difference here is that I want to use the cray environment to compare<br>
performance with gnu. And I have no issues with gnu and my experience<br>
is very similar to yours.<br>
<br>
Many thanks and best regards,<br>
Dominik<br>
<br>
On Fri, Jan 20, 2012 at 6:08 PM, Anton Popov<<a href="mailto:popov@uni-mainz.de" target="_blank">popov@uni-mainz.de</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi Dominik,<br>
<br>
It really depends on which Cray machine you are installing. If you like I<br>
can describe my recent installation experience on Cray XE6 "Rosa" in Zurich<br>
(it uses Linux).<br>
First make sure which compiler you are using. For me, GNU compilers did the<br>
job well. I also prefer using ACML BLAS/LAPACK Library, because XE6 uses AMD<br>
processors.<br>
<br>
So just go to your .bashrc and add something like:<br>
<br>
module load PrgEnv-gnu<br>
module load acml<br>
<br>
# PETSC 3.X<br>
export PETSC_DIR="$HOME/LIB/petsc-3.<u></u>X-pY"<br>
export PETSC_ARCH="PETSC_CRAY_XE6_<u></u>GNU_OPT"<br>
export PATH=$PATH:$PETSC_DIR/bin/<br>
<br>
Don't forget to logout/login for changes to take effect. Also, you should<br>
only have one PrgEnv module in your .bashrc.<br>
<br>
Depending on the packages you need, you can modify the configuration command<br>
below:<br>
<br>
./config/configure.py \<br>
--with-batch=1 \<br>
--known-mpi-shared=0 \<br>
--known-memcmp-ok \<br>
--with-blas-lapack-lib="$ACML_<u></u>DIR/gfortran64/lib/libacml.a" \<br>
--COPTFLAGS="-O3" \<br>
--FOPTFLAGS="-O3" \<br>
--CXXOPTFLAGS="-O3" \<br>
--with-x=0 \<br>
--with-debugging=0 \<br>
--with-clib-autodetect=0 \<br>
--with-cxxlib-autodetect=0 \<br>
--with-fortranlib-autodetect=0 \<br>
--with-shared=0 \<br>
--with-dynamic=0 \<br>
--with-mpi-compilers=1 \<br>
--with-cc=cc \<br>
--with-cxx=CC \<br>
--with-fc=ftn \<br>
--download-blacs=1 \<br>
--download-scalapack=1 \<br>
--download-mumps=1 \<br>
--download-superlu_dist=1 \<br>
--download-parmetis=1 \<br>
--download-ml=1<br>
<br>
If you don't need ACML, just make PETSc download and install BLAS/LAPACK for<br>
you.<br>
<br>
What is important here is to always keep the following:<br>
--with-batch=1 \<br>
--known-mpi-shared=0 \<br>
--with-clib-autodetect=0 \<br>
--with-cxxlib-autodetect=0 \<br>
--with-fortranlib-autodetect=0 \<br>
<br>
NOTE: on CRAY machine you have to submit "conftest" executable on single<br>
processor via batch system (Rosa uses slurm) and run "reconfigure.py" to<br>
finalize configuration.<br>
<br>
After you proceed with this, you have to MANUALLY DELETE the following keys<br>
in $PETSC_DIR/$PETSC_ARCH/<u></u>include/petscconf.h<br>
<br>
PETSC_HAVE_SYS_PROCFS_H<br>
PETSC_HAVE_DLFCN_H<br>
PETSC_HAVE_SYS_SOCKET_H<br>
PETSC_HAVE_SYS_UTSNAME_H<br>
PETSC_HAVE_PWD_H<br>
PETSC_HAVE_GETWD<br>
PETSC_HAVE_UNAME<br>
PETSC_HAVE_GETHOSTNAME<br>
PETSC_HAVE_GETDOMAINNAME<br>
PETSC_HAVE_NETINET_IN_H<br>
PETSC_HAVE_NETDB_H<br>
PETSC_USE_SOCKET_VIEWER<br>
PETSC_HAVE_GETPWUID<br>
<br>
Otherwise nothing will run, and you'll get segfaults in the very beginning.<br>
The reason behind it is that Cray version of Linux only supports reduced set<br>
of system calls.<br>
<br>
NOW you can do "make all"<br>
<br>
Hope it'll be helpful,<br>
<br>
Anton<br>
<br>
------------------------<br>
<br>
<br>
On 1/20/12 4:45 PM, Matthew Knepley wrote:<br>
<br>
On Fri, Jan 20, 2012 at 9:43 AM, Dominik Szczerba<<a href="mailto:dominik@itis.ethz.ch" target="_blank">dominik@itis.ethz.ch</a>><br>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I am building on a Cray system with crayftn fortran compiler. I<br>
specify --with-fc=crayftn and --FFLAGS=-em. Despite the latter, I am<br>
still getting the below cited error. Interestingly, the mod file IS<br>
THERE:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
find /tmp/petsc-0Bnd1w -iname \*.mod<br>
</blockquote>
/tmp/petsc-0Bnd1w/config.<u></u>compilers/CONFIGTEST.mod<br>
<br>
How do I go on from here?<br>
</blockquote>
<br>
The problem is not that is doesn't create a module file, but that it will<br>
not put it<br>
where we ask it to. Please send entire configure.log (don't Cc users).<br>
<br>
Matt<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Thanks<br>
Dominik<br>
<br>
<br>
<br>
<br>
==============================<u></u>==============================<u></u>====================<br>
TEST checkFortranModuleInclude from<br>
<br>
config.compilers(/users/dsz/<u></u>pack/petsc-3.2-p5/config/<u></u>BuildSystem/config/compilers.<u></u>py:1155)<br>
TESTING: checkFortranModuleInclude from<br>
config.compilers(config/<u></u>BuildSystem/config/compilers.<u></u>py:1155)<br>
Figures out what flag is used to specify the include path for Fortran<br>
modules<br>
Pushing language FC<br>
sh: crayftn -c -o /tmp/petsc-0Bnd1w/config.<u></u>compilers/conftest.o<br>
-I/tmp/petsc-0Bnd1w/config.<u></u>compilers<br>
-I/tmp/petsc-0Bnd1w/config.<u></u>setCompilers -em -O3<br>
/tmp/petsc-0Bnd1w/config.<u></u>compilers/conftest.F<br>
Executing: crayftn -c -o /tmp/petsc-0Bnd1w/config.<u></u>compilers/conftest.o<br>
-I/tmp/petsc-0Bnd1w/config.<u></u>compilers<br>
-I/tmp/petsc-0Bnd1w/config.<u></u>setCompilers -em -O3<br>
/tmp/petsc-0Bnd1w/config.<u></u>compilers/conftest.F<br>
sh:<br>
Successful compile:<br>
Source:<br>
module configtest<br>
integer testint<br>
parameter (testint = 42)<br>
end module configtest<br>
<br>
******************************<u></u>******************************<u></u>*******************<br>
UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log<br>
for details):<br>
<br>
------------------------------<u></u>------------------------------<u></u>-------------------<br>
Fortran module was not created during the compile.<br>
configtest.mod/CONFIGTEST.mod not found<br>
<br>
******************************<u></u>******************************<u></u>*******************<br>
File "/users/dsz/pack/petsc-3.2-p5/<u></u>config/configure.py", line 283,<br>
in petsc_configure<br>
framework.configure(out = sys.stdout)<br>
File<br>
"/users/dsz/pack/petsc-3.2-p5/<u></u>config/BuildSystem/config/<u></u>framework.py",<br>
line 925, in configure<br>
child.configure()<br>
File<br>
"/users/dsz/pack/petsc-3.2-p5/<u></u>config/BuildSystem/config/<u></u>compilers.py",<br>
line 1338, in configure<br>
self.executeTest(self.<u></u>checkFortranModuleInclude)<br>
File "/users/dsz/pack/petsc-3.2-p5/<u></u>config/BuildSystem/config/<u></u>base.py",<br>
line 115, in executeTest<br>
ret = apply(test, args,kargs)<br>
File<br>
"/users/dsz/pack/petsc-3.2-p5/<u></u>config/BuildSystem/config/<u></u>compilers.py",<br>
line 1187, in checkFortranModuleInclude<br>
raise RuntimeError('Fortran module was not created during the<br>
compile. configtest.mod/CONFIGTEST.mod not found')<br>
<br>
</blockquote>
<br>
<br>
--<br>
What most experimenters take for granted before they begin their experiments<br>
is infinitely more interesting than any results to which their experiments<br>
lead.<br>
-- Norbert Wiener<br>
<br>
<br>
</blockquote></blockquote>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>