[petsc-dev] download problem p4est on Cori at NERSC

Barry Smith bsmith at mcs.anl.gov
Fri Feb 17 13:05:26 CST 2017 

> On Feb 17, 2017, at 12:46 PM, Satish Balay <balay at mcs.anl.gov> wrote:
> 
> Added the blas change now. There is also the following which petsc
> configure should probably control.. [and specify to p4est configure..]

   Satish,

    Specifically which ones do you mean?  

checking size of void *... 8 

checking for metis.  

Any others?

  Barry


> 
> Satish
> 
> o---------------------------------------
> | Checking libraries
> o---------------------------------------
> checking for library containing fabs... none required
> checking for library containing adler32_combine... -lz
> checking for library containing lua_createtable... no
> configure: Checking BLAS
> checking for dgemm_... yes
> checking for BLAS by linking a C program without FLIBS... successful
> configure: Checking LAPACK
> checking for dgecon_... yes
> checking for LAPACK by linking w/ BLAS_FLIBS but w/o FLIBS... successful
> checking for getopt... successful
> checking for obstack... successful
> checking for library containing pthread_create... -lpthread
> checking for POSIX threads... not used
> checking for library containing omp_get_thread_num... -lgomp
> checking for OpenMP... not used
> checking size of void *... 8
> checking for aligned_alloc... yes
> checking for posix_memalign... yes
> checking for memory alignment option... P4EST_SIZEOF_VOID_P
> checking for metis... not used
> checking for petscarch... no
> checking for PETSc... not used
> o---------------------------------------
> 
> On Fri, 17 Feb 2017, Barry Smith wrote:
> 
>> 
>>   Satish,
>> 
>>    Whoever wrote p4est.py messed up. p4est has a dependency on blas that is missing in p4est.py so p4est is off hunting for its own blas. Can you please add the blas dependency to p4est.py?
>> 
>>   Barry
>> 
>>> On Feb 17, 2017, at 12:28 PM, Mark Adams <mfadams at lbl.gov> wrote:
>>> 
>>> p4est seems to work now, but I am getting a BLAS failure.  But I download blas....
>>> 
>>> On Fri, Feb 17, 2017 at 12:48 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>>>>>> 
>>>  File "/global/u2/m/madams/petsc/config/BuildSystem/config/packages/p4est.py", line 51, in updateGitDir
>>>    raise RuntimeError('Could not initialize sc submodule needed by p4est')
>>> <<<
>>> 
>>> So 'git submoudle' stuff failed. Presumably this install of git doesn't support submodule?
>>> 
>>>>>>>>> 
>>> Checking for program /usr/common/software/git/2.9.1/bin/git...found
>>>              Defined make macro "GIT" to "git"
>>> Executing: git --version
>>> stdout: git version 2.9.1
>>> <<<<<<<
>>> 
>>> You should be able to use the taball - if you use the patch I mentioned eariler [which is already in master]
>>> 
>>> If you want everything else to be from git - and use tarball with only p4est - then do:
>>> 
>>> --download-p4est=https://github.com/p4est/p4est.github.io/raw/master/release/p4est-2.0.tar.gz
>>> 
>>> Satish
>>> 
>>> On Fri, 17 Feb 2017, Tobin Isaac wrote:
>>> 
>>>> The configure.log doesn't really tell me why it is failing.  Did you
>>>> start with a clean git.p4est after trying to use the tarball?
>>>> 
>>>> On Fri, Feb 17, 2017 at 12:21:50PM -0500, Mark Adams wrote:
>>>>> It turns out that Cori-KNL does have git, it is just not loaded by default.
>>>>> 
>>>>> It still does not work, but the error looks different.
>>>>> 
>>>>> *******************************************************************************
>>>>>         UNABLE to CONFIGURE with GIVEN OPTIONS    (see configure.log for
>>>>> details):
>>>>> -------------------------------------------------------------------------------
>>>>> Could not initialize sc submodule needed by p4est
>>>>> *******************************************************************************
>>>>> 
>>>>> 
>>>>> On Thu, Feb 16, 2017 at 10:20 PM, Tobin Isaac <tisaac at uchicago.edu> wrote:
>>>>> 
>>>>>> 
>>>>>> On Thu, Feb 16, 2017 at 05:48:57PM -0600, Barry Smith wrote:
>>>>>>> 
>>>>>>>> On Feb 16, 2017, at 5:01 PM, Tobin Isaac <tisaac at uchicago.edu> wrote:
>>>>>>>> 
>>>>>>>> The bootstrap script predates/supercedes  autoreconf to get the
>>>>>> necessary m4 scripts and write configure.  The tarballs have the m4 scripts
>>>>>> and configure, so they should be treated as standard gnu configure && make
>>>>>> && make install scripts.  I think that's what I accomplished in p4est.py.
>>>>>> I'm on my phone, when I'm at a computer I'll look at this thread's history
>>>>>> to see if I can figure out what's going on.
>>>>>>> 
>>>>>>>   You are confusing two errors. One is the issue you mention above, we
>>>>>> don't care about that. The only error we care about is that p4est
>>>>>> ./configure needs to run a compiled program to gather some information
>>>>>> about the machine. On systems with cross-compiling this doesn't work since
>>>>>> after ./configure compiles the program it cannot run it because it is not
>>>>>> compatible with the hardware. So the question is: is there an easy way to
>>>>>> hack the p4est configure so that we can provide the needed information
>>>>>> directly and not have ./configure try to run an executable?
>>>>>> 
>>>>>> Okay, I get it now.  There should be `config.log` in
>>>>>> externalpackags/git.p4est that would let me know what particular check
>>>>>> it was working on when cross-compilation errors cropped up, and I
>>>>>> could try to help come up with a solution.
>>>>>> 
>>>>>>> 
>>>>>>>   Barry
>>>>>>> 
>>>>>>> Note that this problem is common for many autoconf based build systems,
>>>>>> its solution depends on the details of how the particular system is using
>>>>>> using executables to gather information.
>>>>>>> 
>>>>>>>> 
>>>>>>>> On February 16, 2017 4:34:35 PM CST, Barry Smith <bsmith at mcs.anl.gov>
>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> The best long term fix is to find out what things p4est needs to
>>>>>>>>> determine by a running a program and providing them some other way.
>>>>>>>>> 
>>>>>>>>> Toby
>>>>>>>>> 
>>>>>>>>> So what things is p4est trying to determine by running a program?
>>>>>>>>> 
>>>>>>>>> Barry
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On Feb 16, 2017, at 3:25 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>>>>>>>>>> 
>>>>>>>>>> I don't understand autoconf well enough for cross compiling. Some
>>>>>>>>>> packages appear to require this - but others don't.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> https://www.gnu.org/software/autoconf/manual/autoconf-2.69/
>>>>>> html_node/Hosts-and-Cross_002dCompilation.html
>>>>>>>>>> 
>>>>>>>>>> So one would run 'config.guess' script on the compute node - and use
>>>>>>>>> this value on the compile node with configure?
>>>>>>>>>> 
>>>>>>>>>> I don't know how we could automate that..
>>>>>>>>>> 
>>>>>>>>>> Satish
>>>>>>>>>> 
>>>>>>>>>> On Wed, 15 Feb 2017, Mark Adams wrote:
>>>>>>>>>> 
>>>>>>>>>>> OK, that did not work. It says something about --host now.
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, Feb 15, 2017 at 5:06 PM, Mark Adams <mfadams at lbl.gov>
>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> I think this will work because I have git when I configure on a
>>>>>>>>> login node
>>>>>>>>>>>> (Haswell). I build KNL by submitting a batch script (it takes to
>>>>>>>>> long to
>>>>>>>>>>>> fit in 30 minutes of an interactive session). That fails because
>>>>>>>>> git is not
>>>>>>>>>>>> available (can I just load the module ...) on a KNL compute node.
>>>>>>>>> It is
>>>>>>>>>>>> reconfiguring now, I'll let you know if it fails.
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Feb 15, 2017 at 4:59 PM, Satish Balay <balay at mcs.anl.gov>
>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Sorry - this won't work - as p4est appears to require git
>>>>>>>>> [submodules?]..
>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>>>>> p4est.py <<<<<<<<<<<<<
>>>>>>>>>>>>> def updateGitDir(self):
>>>>>>>>>>>>>  config.package.GNUPackage.updateGitDir(self)
>>>>>>>>>>>>>  Dir = self.getDir()
>>>>>>>>>>>>>  try:
>>>>>>>>>>>>>    libsc = self.libsc
>>>>>>>>>>>>>  except AttributeError:
>>>>>>>>>>>>>    try:
>>>>>>>>>>>>>      self.executeShellCommand([self.sourceControl.git,
>>>>>>>>> 'submodule',
>>>>>>>>>>>>> 'update', '--init'], cwd=Dir, log=self.log)
>>>>>>>>>>>>>      import os
>>>>>>>>>>>>>      if os.path.isfile(os.path.join(Dir,'sc','README')):
>>>>>>>>>>>>>        self.libsc = os.path.join(Dir,'sc')
>>>>>>>>>>>>>      else:
>>>>>>>>>>>>>        raise RuntimeError
>>>>>>>>>>>>>    except RuntimeError:
>>>>>>>>>>>>>      raise RuntimeError('Could not initialize sc submodule
>>>>>>>>> needed by
>>>>>>>>>>>>> p4est')
>>>>>>>>>>>>>  return
>>>>>>>>>>>>> <<<<<<
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Satish
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, 15 Feb 2017, Satish Balay wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> batch won't help here. its git vs tarball - and p4est curently
>>>>>>>>> doesn't
>>>>>>>>>>>>> have a taball.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The following should give you a tarball - if you don't have git.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> diff --git a/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>> b/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>>> index 4735ff4..1f9507c 100644
>>>>>>>>>>>>>> --- a/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>>> +++ b/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>>> @@ -4,7 +4,7 @@ class Configure(config.package.GNUPackage):
>>>>>>>>>>>>>> def __init__(self, framework):
>>>>>>>>>>>>>>   config.package.GNUPackage.__init__(self, framework)
>>>>>>>>>>>>>>   self.gitcommit         = 'v2.0'
>>>>>>>>>>>>>> -    self.download          = ['git://https://github.com/cbu
>>>>>>>>>>>>> rstedde/p4est']
>>>>>>>>>>>>>> +    self.download          = ['git://https://github.com/cbu
>>>>>>>>>>>>> rstedde/p4est','https://github.com/cburstedde/p4est/archive/
>>>>>>>>>>>>> '+self.gitcommit+'.tar.gz']
>>>>>>>>>>>>>>   self.functions         = ['p4est_init']
>>>>>>>>>>>>>>   self.includes          = ['p4est_bits.h']
>>>>>>>>>>>>>>   self.liblist           = [['libp4est.a', 'libsc.a']]
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I  can push this to master.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Satish
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Wed, 15 Feb 2017, Mark Adams wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Oh, this is because git does not work on KNL. Maybe I should do
>>>>>>>>>>>>>>> with-batch=1,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Wed, Feb 15, 2017 at 4:36 PM, Mark Adams <mfadams at lbl.gov>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I get this error on the KNL partition at NERSC.  P4est works on
>>>>>>>>> the
>>>>>>>>>>>>>>>> Haswell partition and other downloads seem to work on the KNL
>>>>>>>>>>>>> partition.
>>>>>>>>>>>>>>>> Any ideas?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> <configure.log>
>> 
>> 
>

More information about the petsc-dev mailing list