Petsc on Blue Gene

[Satish Balay]

On Wed, 11 Jul 2007, Brian Biskeborn wrote:

> Well, what do you know? I'm getting floating point exceptions with
> mat/examples/tests/ex2. There are two relevant lines in the RAS event log.
> They are:
> Kernel detected 6 floating point alignment exceptions (1) iar 0x0026f4a8,
> dear 0x0069c22c (2) iar 0x0026f4a8, dear 0x0069c23c (3) iar 0x0026f4a8,
> dear 0x0069c24c (4) iar 0x0026f4a8, dear 0x0069c25c (5) iar 0x0026f4a8,
> dear 0x0069c26c (6) iar 0x0026f4a8, dear 0x0069c27c
> Kernel detected 36 floating point alignment exceptions (29) iar 0x0026f4a8,
> dear 0x006b48ac (30) iar 0x0026f4a8, dear 0x006b48bc (31) iar 0x0026f4a8,
> dear 0x006b48cc (32) iar 0x0026f4a8, dear 0x006b48dc (33) iar 0x0026f4a8,
> dear 0x006b48ec (34) iar 0x0026f4a8, dear 0x006b48fc (35) iar 0x0026f4a8,
> dear 0x006b490c (36) iar 0x0026f4a8, dear 0x006b491c
> 
> The 42 exceptions above break down as follows: 10 in the test of MatNorm,
> 12 in MatTranspose, 10 in the 2nd MatNorm, and 10 during the test of
> MatAXPY.

How do you know the location of these exceptions? Can you narrow down further
to the correct function name/source line?

Also do you use --with-debugging=0 for this build? Do you get the smae
errors wih '--with-debugging=1' build?

> I compiled the Petsc I used above with a modified version of mal.c that
> #defines PETSC_MEMALIGN to 32 (to be on the safe side) and uses the
> posix_memalign call to allocate aligned memory. As far as I know, the only
> way to produce alignment exceptions would be to manually produce a
> misalignment somewhere (for example, storing an int immediately followed by
> a double at the beginning of an aligned memory block).
> 
> Sorry, I totally forgot to mention this earlier: the code I'm working with
> requires Petsc 2.3.0, so I'm not using the latest version.
> 
> Any suggestions on where this data misalignment might be occurring?

Currently I have no clue as to where these alignment errors might come from.

Satish