On Thu, Jun 28, 2012 at 4:10 PM, Chetan Jhurani <span dir="ltr"><<a href="mailto:chetan.jhurani@gmail.com" target="_blank">chetan.jhurani@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Matt,<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">I was surprised too on both counts. Couldn’t find any libpetsc.lib after the python build<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">and I did not run “clean” in between that could have deleted the .o files.<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Here’s one more try below to show it.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">$ ls test_8/lib/<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">libpetsc-obj<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">$ ls test_8/lib/libpetsc-obj/ | wc -l<u></u><u></u></span></b></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">560<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">$ PETSC_DIR=/cygdrive/d/petsc-dev-2012-06-26 PETSC_ARCH=test_8 ./config/builder.py<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">(The same happens (existing .o are deleted) if ./config/builder2.py build is used)</span></b></p></div></blockquote>
<div><br></div><div>This is not what I wrote. You used builder.py, which is python 2.4 compatible, but has to do everything without options,</div><div>so yes it deletes all the objects and rebuilds everything. That is why I did not suggest you use it.</div>
<div><br></div><div>I said to use builder2.py, which does proper dependency tracking.</div><div><br></div><div>   Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""> </span></b></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Rebuilding Dependencies<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/bag<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/comm<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Building in /cygdrive/d/petsc-dev-2012-06-26/src/sys/dll<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">I stopped the build using control-c.  It would have deleted all files first.<u></u><u></u></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Traceback (most recent call last):<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "./config/builder.py", line 1534, in <module><u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    PETScMaker().run()<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "./config/builder.py", line 1526, in run<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    if self.buildLibraries('libpetsc', self.rootDir):<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "./config/builder.py", line 1331, in buildLibraries<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    objects += self.buildDir(root, files, objDir)<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "./config/builder.py", line 1251, in buildDir<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    objects.extend(getattr(self, 'compile'+language)(sourceMap[language], objDir))<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "./config/builder.py", line 1025, in compileC<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    return self.compile(self.configInfo.languages.clanguage, source, objDir)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "./config/builder.py", line 1012, in compile<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    (output, error, status) = self.executeShellCommand(cmd, checkCommand = noCheckCommand, log=self.log)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 254, in executeShellCommand<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    (output, error, status) = runInShell(command, log, cwd)<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "/cygdrive/d/petsc-dev-2012-06-26/config/BuildSystem/script.py", line 243, in runInShell<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    thread.join(timeout)<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "/usr/lib/python2.6/threading.py", line 655, in join<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    self.__block.wait(delay)<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  File "/usr/lib/python2.6/threading.py", line 258, in wait<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">    _sleep(delay)<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">KeyboardInterrupt<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">$ ls test_8/lib/libpetsc-obj/<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">bag.o<u></u><u></u></span></b></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">All the previous .o were deleted.  Perhaps I’m not doing something right?<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Chetan<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Matthew Knepley<br>
<b>Sent:</b> Thursday, June 28, 2012 2:46 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] slow "ar Scq" on cygwin<u></u><u></u></span></p></div></div><p class="MsoNormal">
<u></u> <u></u></p><p class="MsoNormal">On Thu, Jun 28, 2012 at 3:31 PM, Chetan Jhurani <<a href="mailto:chetan.jhurani@gmail.com" target="_blank">chetan.jhurani@gmail.com</a>> wrote:<u></u><u></u></p><div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Thanks Matt, Satish.  A few issues though.</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Like you said, the latest cygwin has python 2.6.8 which</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">does not have argparse.  When I place the following</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">in path <a href="http://argparse.googlecode.com/svn/trunk/argparse.py" target="_blank">http://argparse.googlecode.com/svn/trunk/argparse.py</a></span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">I can run the python build with 2.6.8.</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">However, the “ar” based build had worked fine and the python</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">based build fails for these two files.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  src/snes/impls/MULTIB~1/multiblock.c</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">  src/snes/impls/vi/rsaug/virsaug.c</span><u></u><u></u></p></div></blockquote><div><p class="MsoNormal"><u></u> <u></u></p>
</div><div><p class="MsoNormal">Yep, it fails, but the build succeeds, so no problem.<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">It looks like these two files were NOT compiled in the ar based</span><u></u><u></u></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">build. The two make.log files are attached.</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">One more question. I tried to run the python based build</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">the second time (after it was unsuccessful the first time)</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">and then it deleted all the previous .o files it had created in </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">ARCH/lib/libpetsc-obj.  How can that be avoided?</span><u></u><u></u></p>
</div></blockquote><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">It does not do that unless you run 'clean'.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div>
<div><p class="MsoNormal">   Matt<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif"">Chetan</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Verdana","sans-serif""> </span><u></u><u></u></p><div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a> [mailto:<a href="mailto:petsc-dev-bounces@mcs.anl.gov" target="_blank">petsc-dev-bounces@mcs.anl.gov</a>] <b>On Behalf Of </b>Matthew Knepley<br>
<b>Sent:</b> Thursday, June 28, 2012 1:35 PM<br><b>To:</b> For users of the development version of PETSc<br><b>Subject:</b> Re: [petsc-dev] slow "ar Scq" on cygwin</span><u></u><u></u></p></div></div><div><p class="MsoNormal">
 <u></u><u></u></p><p class="MsoNormal">On Thu, Jun 28, 2012 at 2:23 PM, Chetan Jhurani <<a href="mailto:chetan.jhurani@gmail.com" target="_blank">chetan.jhurani@gmail.com</a>> wrote:<u></u><u></u></p></div><div><div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt"><p class="MsoNormal">Hi devs,<br><br>Is there a way to avoid running the archiver (ar)<br>
every time a petsc directory is compiled?<br><br>The reason I'm asking this is that when libpetsc.lib<br>becomes large enough (say 100MB+, esp. for debug build),<br>"ar Sqc" on cygwin takes very long in each directory.<br>
Depending on OS caching, very long can be 10 seconds per<br>PETSc directory just to run ar.<br><br>It creates a new temp file from scratch, puts stuff from<br>libpetsc.lib and new stuff in it and then renames the<br>temp file.  As you can imagine, the compilation process is<br>
fast in the beginning but crawls by the end.  I've seen<br>the same IO sequence on Linux (with Sqc flags to ar), but<br>since cygwin IO is slower the effect is more clearly visible.<br>Building within a ramdisk leads to 25% faster compilation<br>
but it was more of a curiosity.<u></u><u></u></p></blockquote><div><p class="MsoNormal"> <u></u><u></u></p></div></div><div><p class="MsoNormal">Build with Python. It never uses antique things like ar :)<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">  python2.7 ./config/builder2.py build<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">
     Matt<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="color:#888888"><br>Chetan</span><u></u><u></u></p></blockquote></div><p class="MsoNormal"><span style="color:#888888"><br><br clear="all"><u></u><u></u></span></p>
<div><p class="MsoNormal"><span style="color:#888888"> <span class="HOEnZb"><font color="#888888"><u></u><u></u></font></span></span></p></div><span class="HOEnZb"><font color="#888888"><p class="MsoNormal"><span style="color:#888888">-- <br>
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>-- Norbert Wiener<u></u><u></u></span></p></font></span></div>
</div></blockquote></div><span class="HOEnZb"><font color="#888888"><p class="MsoNormal"><br><br clear="all"><u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal">-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<u></u><u></u></p></font></span></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>