[petsc-dev] download problem p4est on Cori at NERSC
Barry Smith
bsmith at mcs.anl.gov
Fri Feb 17 12:42:09 CST 2017
Satish,
Did you forget the patch?
> On Feb 17, 2017, at 12:40 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>
> Can you try the attached patch? If it works - I'll push it to master.
>
> BTW: why use --download-fblaslapack=1 on cray? Shouldn't you be using
> one of the blas impls provied by cray? [libsci?]
>
> Satish
>
> On Fri, 17 Feb 2017, Mark Adams wrote:
>
>> p4est seems to work now, but I am getting a BLAS failure. But I download
>> blas....
>>
>> On Fri, Feb 17, 2017 at 12:48 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>>
>>>>>>
>>> File "/global/u2/m/madams/petsc/config/BuildSystem/config/packages/p4est.py",
>>> line 51, in updateGitDir
>>> raise RuntimeError('Could not initialize sc submodule needed by p4est')
>>> <<<
>>>
>>> So 'git submoudle' stuff failed. Presumably this install of git doesn't
>>> support submodule?
>>>
>>>>>>>>>
>>> Checking for program /usr/common/software/git/2.9.1/bin/git...found
>>> Defined make macro "GIT" to "git"
>>> Executing: git --version
>>> stdout: git version 2.9.1
>>> <<<<<<<
>>>
>>> You should be able to use the taball - if you use the patch I mentioned
>>> eariler [which is already in master]
>>>
>>> If you want everything else to be from git - and use tarball with only
>>> p4est - then do:
>>>
>>> --download-p4est=https://github.com/p4est/p4est.github.
>>> io/raw/master/release/p4est-2.0.tar.gz
>>>
>>> Satish
>>>
>>> On Fri, 17 Feb 2017, Tobin Isaac wrote:
>>>
>>>> The configure.log doesn't really tell me why it is failing. Did you
>>>> start with a clean git.p4est after trying to use the tarball?
>>>>
>>>> On Fri, Feb 17, 2017 at 12:21:50PM -0500, Mark Adams wrote:
>>>>> It turns out that Cori-KNL does have git, it is just not loaded by
>>> default.
>>>>>
>>>>> It still does not work, but the error looks different.
>>>>>
>>>>> ************************************************************
>>> *******************
>>>>> UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log
>>> for
>>>>> details):
>>>>> ------------------------------------------------------------
>>> -------------------
>>>>> Could not initialize sc submodule needed by p4est
>>>>> ************************************************************
>>> *******************
>>>>>
>>>>>
>>>>> On Thu, Feb 16, 2017 at 10:20 PM, Tobin Isaac <tisaac at uchicago.edu>
>>> wrote:
>>>>>
>>>>>>
>>>>>> On Thu, Feb 16, 2017 at 05:48:57PM -0600, Barry Smith wrote:
>>>>>>>
>>>>>>>> On Feb 16, 2017, at 5:01 PM, Tobin Isaac <tisaac at uchicago.edu>
>>> wrote:
>>>>>>>>
>>>>>>>> The bootstrap script predates/supercedes autoreconf to get the
>>>>>> necessary m4 scripts and write configure. The tarballs have the m4
>>> scripts
>>>>>> and configure, so they should be treated as standard gnu configure
>>> && make
>>>>>> && make install scripts. I think that's what I accomplished in
>>> p4est.py.
>>>>>> I'm on my phone, when I'm at a computer I'll look at this thread's
>>> history
>>>>>> to see if I can figure out what's going on.
>>>>>>>
>>>>>>> You are confusing two errors. One is the issue you mention
>>> above, we
>>>>>> don't care about that. The only error we care about is that p4est
>>>>>> ./configure needs to run a compiled program to gather some
>>> information
>>>>>> about the machine. On systems with cross-compiling this doesn't work
>>> since
>>>>>> after ./configure compiles the program it cannot run it because it
>>> is not
>>>>>> compatible with the hardware. So the question is: is there an easy
>>> way to
>>>>>> hack the p4est configure so that we can provide the needed
>>> information
>>>>>> directly and not have ./configure try to run an executable?
>>>>>>
>>>>>> Okay, I get it now. There should be `config.log` in
>>>>>> externalpackags/git.p4est that would let me know what particular
>>> check
>>>>>> it was working on when cross-compilation errors cropped up, and I
>>>>>> could try to help come up with a solution.
>>>>>>
>>>>>>>
>>>>>>> Barry
>>>>>>>
>>>>>>> Note that this problem is common for many autoconf based build
>>> systems,
>>>>>> its solution depends on the details of how the particular system is
>>> using
>>>>>> using executables to gather information.
>>>>>>>
>>>>>>>>
>>>>>>>> On February 16, 2017 4:34:35 PM CST, Barry Smith <
>>> bsmith at mcs.anl.gov>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> The best long term fix is to find out what things p4est needs to
>>>>>>>>> determine by a running a program and providing them some other
>>> way.
>>>>>>>>>
>>>>>>>>> Toby
>>>>>>>>>
>>>>>>>>> So what things is p4est trying to determine by running a
>>> program?
>>>>>>>>>
>>>>>>>>> Barry
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Feb 16, 2017, at 3:25 PM, Satish Balay <balay at mcs.anl.gov>
>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I don't understand autoconf well enough for cross compiling.
>>> Some
>>>>>>>>>> packages appear to require this - but others don't.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> https://www.gnu.org/software/autoconf/manual/autoconf-2.69/
>>>>>> html_node/Hosts-and-Cross_002dCompilation.html
>>>>>>>>>>
>>>>>>>>>> So one would run 'config.guess' script on the compute node -
>>> and use
>>>>>>>>> this value on the compile node with configure?
>>>>>>>>>>
>>>>>>>>>> I don't know how we could automate that..
>>>>>>>>>>
>>>>>>>>>> Satish
>>>>>>>>>>
>>>>>>>>>> On Wed, 15 Feb 2017, Mark Adams wrote:
>>>>>>>>>>
>>>>>>>>>>> OK, that did not work. It says something about --host now.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Feb 15, 2017 at 5:06 PM, Mark Adams <mfadams at lbl.gov>
>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think this will work because I have git when I configure
>>> on a
>>>>>>>>> login node
>>>>>>>>>>>> (Haswell). I build KNL by submitting a batch script (it
>>> takes to
>>>>>>>>> long to
>>>>>>>>>>>> fit in 30 minutes of an interactive session). That fails
>>> because
>>>>>>>>> git is not
>>>>>>>>>>>> available (can I just load the module ...) on a KNL compute
>>> node.
>>>>>>>>> It is
>>>>>>>>>>>> reconfiguring now, I'll let you know if it fails.
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Feb 15, 2017 at 4:59 PM, Satish Balay <
>>> balay at mcs.anl.gov>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Sorry - this won't work - as p4est appears to require git
>>>>>>>>> [submodules?]..
>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> p4est.py <<<<<<<<<<<<<
>>>>>>>>>>>>> def updateGitDir(self):
>>>>>>>>>>>>> config.package.GNUPackage.updateGitDir(self)
>>>>>>>>>>>>> Dir = self.getDir()
>>>>>>>>>>>>> try:
>>>>>>>>>>>>> libsc = self.libsc
>>>>>>>>>>>>> except AttributeError:
>>>>>>>>>>>>> try:
>>>>>>>>>>>>> self.executeShellCommand([self.sourceControl.git,
>>>>>>>>> 'submodule',
>>>>>>>>>>>>> 'update', '--init'], cwd=Dir, log=self.log)
>>>>>>>>>>>>> import os
>>>>>>>>>>>>> if os.path.isfile(os.path.join(Dir,'sc','README')):
>>>>>>>>>>>>> self.libsc = os.path.join(Dir,'sc')
>>>>>>>>>>>>> else:
>>>>>>>>>>>>> raise RuntimeError
>>>>>>>>>>>>> except RuntimeError:
>>>>>>>>>>>>> raise RuntimeError('Could not initialize sc submodule
>>>>>>>>> needed by
>>>>>>>>>>>>> p4est')
>>>>>>>>>>>>> return
>>>>>>>>>>>>> <<<<<<
>>>>>>>>>>>>>
>>>>>>>>>>>>> Satish
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, 15 Feb 2017, Satish Balay wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> batch won't help here. its git vs tarball - and p4est
>>> curently
>>>>>>>>> doesn't
>>>>>>>>>>>>> have a taball.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The following should give you a tarball - if you don't
>>> have git.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> diff --git a/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>> b/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>>> index 4735ff4..1f9507c 100644
>>>>>>>>>>>>>> --- a/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>>> +++ b/config/BuildSystem/config/packages/p4est.py
>>>>>>>>>>>>>> @@ -4,7 +4,7 @@ class Configure(config.package.
>>> GNUPackage):
>>>>>>>>>>>>>> def __init__(self, framework):
>>>>>>>>>>>>>> config.package.GNUPackage.__init__(self, framework)
>>>>>>>>>>>>>> self.gitcommit = 'v2.0'
>>>>>>>>>>>>>> - self.download = ['git://
>>> https://github.com/cbu
>>>>>>>>>>>>> rstedde/p4est']
>>>>>>>>>>>>>> + self.download = ['git://
>>> https://github.com/cbu
>>>>>>>>>>>>> rstedde/p4est','https://github.com/cburstedde/p4est/
>>> archive/
>>>>>>>>>>>>> '+self.gitcommit+'.tar.gz']
>>>>>>>>>>>>>> self.functions = ['p4est_init']
>>>>>>>>>>>>>> self.includes = ['p4est_bits.h']
>>>>>>>>>>>>>> self.liblist = [['libp4est.a', 'libsc.a']]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I can push this to master.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Satish
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, 15 Feb 2017, Mark Adams wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Oh, this is because git does not work on KNL. Maybe I
>>> should do
>>>>>>>>>>>>>>> with-batch=1,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Feb 15, 2017 at 4:36 PM, Mark Adams <
>>> mfadams at lbl.gov>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I get this error on the KNL partition at NERSC. P4est
>>> works on
>>>>>>>>> the
>>>>>>>>>>>>>>>> Haswell partition and other downloads seem to work on
>>> the KNL
>>>>>>>>>>>>> partition.
>>>>>>>>>>>>>>>> Any ideas?
More information about the petsc-dev
mailing list