[petsc-dev] BuildSystem performance

[Jed Brown]

Here are the top 15 tests in BuildSystem, accounting for 96% of total run-time (82 seconds).

TIME:   12.862      <class 'config.packages.BlasLapack.Configure'> /home/jed/petsc/config/BuildSystem/config/packages/BlasLapack.py:561
TIME:   11.006             <class 'config.setCompilers.Configure'> /home/jed/petsc/config/BuildSystem/config/setCompilers.py:1521
TIME:   10.638                <class 'config.compilers.Configure'> /home/jed/petsc/config/BuildSystem/config/compilers.py:1323
TIME:   10.021                <class 'config.functions.Configure'> /home/jed/petsc/config/BuildSystem/config/functions.py:189
TIME:    8.270             <class 'config.packages.MPI.Configure'> /home/jed/petsc/config/BuildSystem/config/package.py:544
TIME:    5.830                 <class 'PETSc.Configure.Configure'> /home/jed/petsc/config/PETSc/Configure.py:907
TIME:    4.717           <class 'PETSc.packages.sowing.Configure'> /home/jed/petsc/config/PETSc/packages/sowing.py:94
TIME:    3.483            <class 'config.compilerFlags.Configure'> /home/jed/petsc/config/BuildSystem/config/compilerFlags.py:112
TIME:    3.336                    <class 'config.types.Configure'> /home/jed/petsc/config/BuildSystem/config/types.py:368
TIME:    2.995                  <class 'config.headers.Configure'> /home/jed/petsc/config/BuildSystem/config/headers.py:227
TIME:    2.297         <class 'PETSc.utilities.missing.Configure'> /home/jed/petsc/config/PETSc/utilities/missing.py:149
TIME:    2.089                <class 'config.libraries.Configure'> /home/jed/petsc/config/BuildSystem/config/libraries.py:438
TIME:    1.561  <class 'PETSc.utilities.fortranCommandLine.Configure'> /home/jed/petsc/config/PETSc/utilities/fortranCommandLine.py:63
TIME:    0.644    <class 'PETSc.utilities.cacheDetails.Configure'> /home/jed/petsc/config/PETSc/utilities/cacheDetails.py:107
TIME:    0.609     <class 'PETSc.utilities.scalarTypes.Configure'> /home/jed/petsc/config/PETSc/utilities/scalarTypes.py:97

Surely we can find it within our heart to confirm that there are no
missing functions in BlasLapack in less than 12 seconds.  We are testing
the same few functions many times:

     12 char ddot();
     12 char dgeev();
     12 char dgetrs();
     13 char dgeev_();
     14 char df2cblaslapack311_id();
     14 char dgetrs_();
     25 char ddot_();

This is a system with normal underscore name mangling and -llapack
-lblas, with no missing functions.  At worst, I would expect a few
failures with one function, then only one test to confirm that the same
mangling works for all the others.


diff --git i/config/BuildSystem/config/framework.py w/config/BuildSystem/config/framework.py
index 6ce70d2..2c2e64b 100644
--- i/config/BuildSystem/config/framework.py
+++ w/config/BuildSystem/config/framework.py
@@ -928,9 +928,13 @@ class Framework(config.base.Configure, script.LanguageProcessor):
     self.outputBanner()
     self.updateDependencies()
     self.executeTest(self.configureExternalPackagesDir)
+    times = []
     for child in graph.DirectedGraph.topologicalSort(self.childGraph):
       if not hasattr(child, '_configured'):
+        import time,inspect,sys
+        t0 = time.time()
         child.configure()
+        times.append((time.time()-t0, type(child), inspect.getsourcefile(child.configure), inspect.getsourcelines(child.configure)[1]))
       else:
         child.no_configure()
       child._configured = 1
@@ -938,4 +942,6 @@ class Framework(config.base.Configure, script.LanguageProcessor):
       self.configureBatch()
     self.dumpConfFiles()
     self.cleanup()
+    for item in sorted(times)[-15:]:
+      sys.stderr.write('TIME: %8.3f  %50s %s:%d\n' % item)
     return 1


Why do we check for the same program multiple times?

      2 Checking for program /bin/nvcc...not found
      2 Checking for program /home/jed/bin/ar...not found
      2 Checking for program /home/jed/bin/mpicxx...not found
      2 Checking for program /home/jed/bin/mpiexec...not found
      2 Checking for program /home/jed/bin/mpirun...not found
      2 Checking for program /home/jed/bin/mprun...not found
      2 Checking for program /home/jed/bin/nvcc...not found
      2 Checking for program /home/jed/bin/ranlib...not found
      2 Checking for program /opt/brlcad/bin/nvcc...not found
      2 Checking for program /opt/cuda/bin/nvcc...found
      2 Checking for program /sbin/nvcc...not found
      2 Checking for program /usr/bin/ar...found
      2 Checking for program /usr/bin/mpicxx...found
      2 Checking for program /usr/bin/nvcc...not found
      2 Checking for program /usr/bin/ranlib...found
      2 Checking for program /usr/sbin/nvcc...not found
      4 Checking for program /home/jed/bin/mpicc...not found
      4 Checking for program /usr/bin/mpicc...found

The number of times we re-include some headers in configure.log is
incredible.

  sort configure.log | uniq -c | sort -n

     43 # 10 "/usr/include/gnu/stubs.h" 3 4
     43 # 11 "/usr/include/gnu/stubs.h" 2 3 4
     43 # 1 "/usr/include/features.h" 1 3 4
     43 # 1 "/usr/include/gnu/stubs-64.h" 1 3 4
     43 # 1 "/usr/include/gnu/stubs.h" 1 3 4
     43 # 1 "/usr/include/sys/cdefs.h" 1 3 4
     43 # 375 "/usr/include/features.h" 3 4
     43 # 376 "/usr/include/features.h" 2 3 4
     43 # 385 "/usr/include/sys/cdefs.h" 3 4
     43 # 386 "/usr/include/sys/cdefs.h" 2 3 4
     43 # 399 "/usr/include/features.h" 3 4
     43 # 400 "/usr/include/features.h" 2 3 4
     60 # 1 "/usr/include/stdc-predef.h" 1 3 4
    132 # 1 "/usr/include/bits/wordsize.h" 1 3 4

Surely we could cut down on configure.log size.  Do we need to include
mpicc -E output for all of these?

    51 sh: mpicc -E  -I/tmp/petsc-PTKA0T/config.setCompilers -I/tmp/petsc-PTKA0T/config.headers  /tmp/petsc-PTKA0T/config.headers/conftest.c