[petsc-dev] Sean is going to love this

[Sean Farley]

Barry Smith writes:

>> On Dec 29, 2014, at 9:56 PM, Geoff Oxberry <goxberry at gmail.com> wrote:
>> 
>> > Date: Sun, 28 Dec 2014 21:27:21 -0600
>> > From: Barry Smith <bsmith at mcs.anl.gov>
>> > To: Sean Farley <sean.michael.farley at gmail.com>
>> > Cc: petsc-dev <petsc-dev at mcs.anl.gov>
>> > Subject: Re: [petsc-dev] Sean is going to love this
>> > Message-ID: <BA03ABA6-C693-4404-9FA5-2211BC371836 at mcs.anl.gov>
>> > Content-Type: text/plain; charset="utf-8"
>> >
>> >
>> >   Sean,
>> >
>> >     brew install /homebrew/science/petsc
>> >     brew install /homebrew/science/petsc --HEAD --with-x11
>> >
>> >   Is there any reason not to use home-brew for everything now? Should we be working with the homebrew/science guys to work out the rough edges? (Like their hypre build is sequential and they don't pass the MPI compiler wrappers properly to build packages and ... and ...)
>> 
>> I contributed most of the hypre recipe; brew install hypre --with-mpi should get you a parallel build, and it should also run some parallel exampled as smoke tests. If there's anything missing that should be added, let me know and I can get to it when I return from vacation. 
>
>    Geoff,
>
>     Thanks, I noticed your name on a bunch of things there. I may bother you later with some issues when I understand brew better but, what the heck, here are a few things I noticed. (By the way, overall I was impressed with brew working with all these "nasty" scientific packages)

No love for the work I did with all the nasty scientific packages? Oh,
ok :-(

> 1)   I ran brew install /homebrew/science/petsc --HEAD --with-hypre and it failed because it just used the bottled hypre (which is sequential) as a dependent package and then tried to build the parallel PETSc with the sequential hypre so failed in the middle of compiling PETSc because of inconsistent hypre include files (ones for sequential).

Yep, this is a problem that doesn't exist for PETSc in MacPorts.

>    Presumably had I done a brew install hypre --with-mpi first and then the brew install /homebrew/science/petsc --HEAD it may have worked properly? (but how is a naive new user to know?)

Indeed.

>    Second meta issue, is there a way to force a compiled of the package instead of it using the prepared bottle? Or even better to just tell brew I never want it to use bottles but always want it to compile for all packages? One reason I ask this is because PETSc has so many optional dependencies it seems maybe? better to just have it build based on the choices the user made rather than using some pre-built bottle that only has certain things turned on? But maybe I don't understand bottles and when they are used instead of building.

This is done in MacPorts.

> 2) I think PETSc is just "lucky" that it builds at all. brew seems to set the environmental variables CC, CXX, and FC to clang, clang++ and gfortran but then PETSc actually ignores these variables (printing a warning if you run with -vd) and hunts for mpicc, mpicxx, mpifc which it happens to find in /usr/local/bin since open-mpi is a dependency of PETSc and thus must have been installed first.

This is why sandboxing is important for package managers.

>    So this leads the third meta issue, it seems brew doesn't seem to have any concept of some packages requiring mpi wrapper compilers to be used to compile the package and so it is kind of catch as catch can if MPI based packages will build properly instead of having a systematic way of handling it. For example if PETSc did not ignore CC, CXX, and FC  it would try to use clang, clang++ and gfortran and then crap out that it didn't have MPI compilers. Any chance that brew would ever be smart enough to "reset" the compilers to the MPI ones before building packages that depend on MPI?

A hard graph problem that I hacked in MacPorts. I have no idea about how
to do it in brew (hence why I haven't used it).

> 3) PETSc has a bunch of other optional dependencies such as hdf5, netcdf ... that it would be nice to support. Presumably I could try to make a pull request with support for them but I'm concerned that my lack of understanding of meta issue two might mess me up.

All of those are supported in MacPorts.

> 4) The issue of MPICH2 and open-mpi conflicting is a pain. Some package managers handle this by having a concept of a meta-package or abstract-package (such as MPI) that can have multiple implementation. Then, for example, one could say PETSc is dependent on MPI (and not open-mpi or MPICH) and the package system would work properly if either open-mpi or MPICH had already been installed (instead of crapping out like it does if I first install MPICH and then install PETSc; since it tries to install open-mpi for PETSc which conflicts with MPICH and thus stops).

Also, this is done in MacPorts.

With that being said, I must also declare that I have no love for
MacPorts. It gets the job done for these specific cases and works just
well enough but still has other problems. MacPorts will never be able to
solve harder DAG problems for dependencies (just hacks). I wish I had
the time to build a new package manager, but alas. The quote from ESR
seems to ring true:

The most dangerous enemy of a better solution is an existing codebase
that is just good enough.