Petsc on Blue Gene

[Brian Biskeborn]

Hello,

Does anyone here have experience with using Petsc on a Blue Gene system?
I'm at IBM's Almaden Research Center and am currently involved in porting
some code that uses Petsc to Blue Gene.  I've managed to compile and run
the code, and it seems to produce the correct results, but it generates
many floating point alignment exceptions at runtime.  I traced some of
these warnings to a call to MatAssemblyBegin, so I'm thinking maybe the
problem is in Petsc.  The Blue Gene architecture requires 8- or 16-byte
memory alignment for floating point operations, depending on whether they
are issued in parallel, so I tried bumping up the alignment value in
src/sys/src/memory/mal.c.  This had no effect on the problem.

I didn't write any of the code I'm porting, and I'm not too familiar with
it yet, so I suppose the problem could really be anywhere.  My question
here is simply this: has anyone seen this problem?  Does it sound like an
issue with Petsc, or is it more likely to be in the top-level code?

Thanks, and best regards,
Brian