[petsc-dev] new configuration/compile system for PETSc

[Barry Smith]

   Ok, here's the current list of requirements. What is missing?


 Requirements: 
     X    1) Portability to Windows, Cygwin, Unix (all versions)
         2) Compatible with GUI development systems (Xcode, Eclipse, Emacs, ...)
         3) Able to run parallel configures and builds (on shared memory system enough?, distcc?)
         4) Able to work with batch systems
         5) Able to handle dependencies between packages (given dependencies between packages builds everything in the correct order)
         6) Able to utilize clang and similar systems
         7) Works seamlessly with GNU autoconf, Cmake, pkconfig packages (that is will build these packages automatically and maybe generate them)
         8) Can download and install packages by given URL
         9) Easy to add configurations for packages such as SuperLU, etc that have no decent configuration
     x   10) Can test for available functionalities (include files, etc)
     X   11) Easy to add new types of compilers, new tests
        12) Does dependency analysis and rebuilds only what needs to be rebuilt
        13) Able to manage test suites
        14) Compatible with and able to use revision control systems
        15) Completely command-line controlled but also with a GUI frontend that gives one full control as well
        16) Able to build for bizarre-assed systems like the iPAD and GPUs.
        17) Able to manage the concept of abstract packages such as MPI and BLAS/LAPACK and particular implementations
        18) Able to handle already installed packages and detect if that package uses something incompatable with other packages: for example HDF5 installed with OpenMPI and someone building other packages with MPICH. 
        19) Able to manage several different compilers of the same type (for example 2 C compilers) being used for different packages.
        20) Incrementable builds, that is more packages can be installed later based on packages already installed. Say add SLEPc later.
        

    X  - does,   x - does some


On Jul 9, 2011, at 7:45 PM, Dmitry Karpeev wrote:

> Observe that PETSc's configure deals with the problem of consistently
> configuring MULTIPLE
> packages, while most tools I have heard of typically deal with a
> SINGLE package.  That's a
> qualitative difference, in my opinion.
> 
> Separately, I would suggest having the new build system allow the user
> to install PETSc in one
> go.  Something like 'cll install --with-clanguage=c++ ...', because
> it's rare for users to stop after
> configuring, and then end up executing an easily automatable sequence of steps:
> configure --with-clanguage=c++ ...
> make PETSC_DIR= ... all
> make PETSC_DIR= ... PETSC_ARCH=... test
> 
> Of course, we should provide intermediate stages as well -- 'cll
> configure' -- etc.
> In particular, if cll were able to configure "arbitrary" collections
> of packages, it may be desirable
> to leave the "top" package configured, but not built, as it has to be
> built repeatedly (as part of
> a development cycle) using make or cmake, etc.
> 
> Dmitry.
> 
> On Sat, Jul 9, 2011 at 7:37 PM, Dmitry Karpeev <karpeev at mcs.anl.gov> wrote:
>> On Sat, Jul 9, 2011 at 7:18 PM, Harald Pfeiffer
>> <pfeiffer at cita.utoronto.ca> wrote:
>>> We happen to use Petsc with our own makefiles.  PETSc is compiled entirely
>>> separately (or we use system-provided PETSc-modules).   We then manually set
>>> the appropriate include- and library-paths in our makefiles, for example:
>>> 
>>> PETSC_INCLUDE = -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include
>>> PETSC_LIB = -L$(PETSC_DIR)/$(PETSC_ARCH)/lib -lpetsc -lX11
>>> ...
>>> other stuff here
>>> ...
>>> ARCH_INCLUDE = $(PETSC_INCLUDE)  ...
>>> ARCH_LIB_PARALLEL   = $(HDF_LIB) $(FFTW_LIB) $(PETSC_LIB)  ....
>>> 
>>> 
>>> Not elegant, but has been working like a charm for the last 12 years.
>> 
>> I'm not saying it can't be made to work.
>> I'm saying it might not be trivial for someone just starting to use
>> PETSc and not even necessarily
>> that familiar with make: there are a lot of potential users like that.
>>  And even for more sophisticated
>> users it might take a while to work out the kinks.
>> 
>> Dmitry.
>> 
>>> 
>>> 
>>> Harald
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> On 7/10/11 1:02 AM, Dmitry Karpeev wrote:
>>>> 
>>>> On Sat, Jul 9, 2011 at 6:42 PM, Jed Brown<jedbrown at mcs.anl.gov>  wrote:
>>>>> 
>>>>> On Sat, Jul 9, 2011 at 18:30, Dmitry Karpeev<karpeev at mcs.anl.gov>  wrote:
>>>>>> 
>>>>>>  If a user could write a
>>>>>> simple foo.py that configured/built their
>>>>>> favorite package with PETSc as a dependency, they could then proceed
>>>>>> to modify the FOO code and jack
>>>>>> PETSc into it.
>>>>> 
>>>>> But this just a different form of asking them to adopt our build system,
>>>>> except that makefiles are a de-facto standard and our system won't be
>>>>> familiar to anyone. Any system has to be easily usable from other systems
>>>>> (at least makefiles, automake, and cmake).
>>>> 
>>>> Yes, but there is also a practical problem: an engineer or a scientist,
>>>> not
>>>> intimately familiar with automake, cmake, etc., has taken the effort
>>>> to understand
>>>> PETSc so that our linear solvers can be substituted for FOO's.  How can we
>>>> make it easier for these people to insert PETSc into their code?  Here
>>>> I only mean
>>>> the build process.  Maybe making them write foo.py is not the right
>>>> approach,
>>>> but we should have a usable approach (and, maybe, an example of doing it).
>>>> 
>>>> In particular, is it sufficient to export linklibs, or do we also need
>>>> to export the
>>>> compiler (e.g., to ensure consistent Fortrant name mangling)?  There are
>>>> a good number of Fortran users that want to use PETSc too.
>>>> 
>>>> Dmitry.
>>> 
>>> --
>>> Harald Pfeiffer
>>> Assistant Professor
>>> Cdn. Institute for Theoretical Astrophysics
>>> University of Toronto
>>> pfeiffer at cita.utoronto.ca
>>> Tel. ++1 416-978-8497
>>> 
>>> 
>>