[petsc-dev] slow "ar Scq" on cygwin

[Chetan Jhurani]

Matt,

 

I was surprised too on both counts. Couldn't find any libpetsc.lib after the python build

and I did not run "clean" in between that could have deleted the .o files.

 

Here's one more try below to show it.

 

$ ls test_8/lib/

libpetsc-obj

 

$ ls test_8/lib/libpetsc-obj/ | wc -l

560

 

$ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py

(The same happens (existing .o are deleted) if ./config/builder2.py build is used)

 

Rebuilding Dependencies

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm

Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll

 

I stopped the build using control-c.  It would have deleted all files first.

 

Traceback (most recent call last):

  File "./config/builder.py", line 1534, in <module>

    PETScMaker().run()

  File "./config/builder.py", line 1526, in run

    if self.buildLibraries('libpetsc', self.rootDir):

  File "./config/builder.py", line 1331, in buildLibraries

    objects += self.buildDir(root, files, objDir)

  File "./config/builder.py", line 1251, in buildDir

    objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir))

  File "./config/builder.py", line 1025, in compileC

    return self.compile(self.configInfo.languages.clanguage, source, objDir)

  File "./config/builder.py", line 1012, in compile

    (output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, log=self.log)

  File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in executeShellCommand

    (output, error, status) = runInShell(command, log, cwd)

  File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell

    thread.join(timeout)

  File "/usr/lib/python2.6/threading.py", line 655, in join

    self.__block.wait(delay)

  File "/usr/lib/python2.6/threading.py", line 258, in wait

    _sleep(delay)

KeyboardInterrupt

 

$ ls test_8/lib/libpetsc-obj/

bag.o

 

All the previous .o were deleted.  Perhaps I'm not doing something right?

 

Chetan

 

 

From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at mcs.anl.gov] On Behalf Of Matthew Knepley
Sent: Thursday, June 28, 2012 2:46 PM
To: For users of the development version of PETSc
Subject: Re: [petsc-dev] slow "ar Scq" on cygwin

 

On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <chetan.jhurani at gmail.com> wrote:

Thanks Matt, Satish.  A few issues though.

 

Like you said, the latest cygwin has python 2.6.8 which

does not have argparse.  When I place the following

in path http://argparse.googlecode.com/svn/trunk/argparse.py

I can run the python build with 2.6.8.

 

However, the "ar" based build had worked fine and the python

based build fails for these two files.

 

  src/snes/impls/MULTIB~1/multiblock.c

  src/snes/impls/vi/rsaug/virsaug.c

 

Yep, it fails, but the build succeeds, so no problem.

 

It looks like these two files were NOT compiled in the ar based

build. The two make.log files are attached.

 

One more question. I tried to run the python based build

the second time (after it was unsuccessful the first time)

and then it deleted all the previous .o files it had created in 

ARCH/lib/libpetsc-obj.  How can that be avoided?

 

It does not do that unless you run 'clean'.

 

   Matt

 

Chetan

 

 

From: petsc-dev-bounces at mcs.anl.gov [mailto:petsc-dev-bounces at mcs.anl.gov] On Behalf Of Matthew Knepley
Sent: Thursday, June 28, 2012 1:35 PM
To: For users of the development version of PETSc
Subject: Re: [petsc-dev] slow "ar Scq" on cygwin

 

On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <chetan.jhurani at gmail.com> wrote:

Hi devs,

Is there a way to avoid running the archiver (ar)
every time a petsc directory is compiled?

The reason I'm asking this is that when libpetsc.lib
becomes large enough (say 100MB+, esp. for debug build),
"ar Sqc" on cygwin takes very long in each directory.
Depending on OS caching, very long can be 10 seconds per
PETSc directory just to run ar.

It creates a new temp file from scratch, puts stuff from
libpetsc.lib and new stuff in it and then renames the
temp file.  As you can imagine, the compilation process is
fast in the beginning but crawls by the end.  I've seen
the same IO sequence on Linux (with Sqc flags to ar), but
since cygwin IO is slower the effect is more clearly visible.
Building within a ramdisk leads to 25% faster compilation
but it was more of a curiosity.

 

Build with Python. It never uses antique things like ar :)

 

  python2.7 ./config/builder2.py build

 

     Matt

 


Chetan





 

-- 
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to
which their experiments lead.
-- Norbert Wiener





 

-- 
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to
which their experiments lead.
-- Norbert Wiener

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20120628/f1b5f5d1/attachment.html>