BuildSystem question

[S V N Vishwanathan]

Hi!

>> I am unclear about where external packages required by PETSc should
>> go. Many packages (like PETSc, TAO, OOQP) might want to use some common
>> external package (like MPICH). But it seems like a wasted effort if we
>> replicate the PETSc build scripts which already install these packages
>> into $PETSC_DIR/externalpackages.

Matt> PETSc does have --with-external-packages-dir, which defaults to
Matt> $PETSC_DIR/externalpackages, but you can initialize it to whatever
Matt> you want.

This I realized after I had a careful look at the configure options
yesterday night. 

>> When I try to grab the framework object from RDict.db and use a
>> require method on it, it seems to bomb because many of the scripts in
>> $PETSC_DIR/python/PETSc seem to make an implicit assumption that the
>> configure scripts are run from $PETSC_DIR (a bad assumption I think,
>> esp given that you have the PETSC_DIR variable available to you).

Matt> We would be willing to work on this with you. That requirement has
Matt> been discussed. The point of view from the other side is that
Matt> users (very) often get confused and we would like to do as much as
Matt> possible to make sure they are doing the correct thing (like
Matt> installing from PETSC_DIR). Also, we never assume that PETSC_DIR
Matt> is defined during configure. That said, I'm sure these outright
Matt> dependencies can be fixed.

Yes that would be very good. I am sure there are other pieces of
software (TAO and SLEPc come to mind) which might benefit from just
making their build depend on the PETSc configure. As for the PETSC_DIR
problem, I think your configure.py script already does enough sanity
checking that you can safely set a variable and use it. I am sure it is
more portable than the current approach. 


Matt> I tried to document the configure (Python) part as much as I
Matt> could. I know the make part is not documented, but it is an
Matt> artifact of an earlier age when make systems were considered the
Matt> equivalent of hammers, which are rarely documented. Feel free to
Matt> mail questions.

Good. I will probably mail you and bug you a bit more :)

>> So my question is simple, if you were in my position i.e. designing the
>> build scripts for LINEAL, how would approach the problem?

Matt>   I would decide up front whether I wanted to use:

Matt>   a) make

For heavens sake NO!

Matt>   b) autotools

I know they are quite powerful but never got my head around to actually
using them. They are still a bit of a black box to me and I would keep
away from them.

Matt>   c) something totally new

I am seriously considering using your BuildSystem. I like it since it is
Python and modular. My only concern is whether it might be a bit of an
overkill to begin with. But going forward it might be more extensible
and more comprehensive. I am also considering Scons. The main advantage
of Scons is that it is well documented and hence the learning curve is
not too steep.

Ideally, I would like to use your BuildSystem to hack up a quick build
for Numerix. Then as we add and integrate more tools in, I would work on
making it fancier. 


Matt> From this basic design decision flows almost every other
Matt> decision. Also, it gives you an idea of what kind of investment is
Matt> necessary. I will try and indicate why I choose c). The kind of
Matt> work involved in making a new system is different and appeals to

The idea of working with a system which has good support appeals to
me. For me having good support is more important than having good
documentation :) While I would not mind trying out new things I am
generally loathe to develop new things unless there is a clear and
pressing need for me to do so. 

Matt> me. If you use a), generally you spend lots of time writing shell
Matt> and working out bizzare tricks with stamp files to get flow

I hate Make too. I tried out only simple things with it but already
things got very very messy. 


Matt> work things out, but you can fix many things and perhaps introduce
Matt> a more powerful paradigm. However, all projects are different and
Matt> all options shuold be considered every time.

I find that many of the aims I outlined in my last mail are very similar
to what the PETSc build already does. I am willing to live with a build
completely done in Python. 


Here is the plan I came up with (after sleeping over it for a night):

1) I assume that the PETSc Framework object can be made modular i.e. can
   be invoked as long as the PETSC_DIR is set and points to a sane
   directory. 

2) I will create a new Configure object for Numerix and add it to the
   PETSc framework using require(). My configure script will also set
   the PETSC_DIR env variable to a sane directory. 

3) All options that are passed to my build are first passed to the PETSc
   build. I also pass it
   --with-external-packages-dir=$LINEAL_DIR/externalpackages 

4) Any extra flags which are not handled by PETSc (or are shared between
   packages) will be handled and passed to appropriate builds by the
   Numerix configure. 

Does this make sense?

How do we handle the case when say PETSc libs are installed by using a
package manager like rpm or apt?

I guess my problem at a high level is as follows:

What is the best approach to take for a package which depends on PETSc
to use 
a) The BuildSystem module
b) The PETSc framework object itself

I am sorry if some of my questions are incoherent. I am still trying to
figure out how to structure out build and I realize it is not an easy
task. 

vishy

----- 
We're mortal -- which is to say, we're ignorant, stupid, and sinful --
but those are only handicaps.  Our pride is that nevertheless, now and
then, we do our best.  A few times we succeed.  What more dare we ask for?
		-- Ensign Flandry