Petsc on Blue Gene

[Lisandro Dalcin]

On 7/11/07, Brian Biskeborn <bbiskebo at us.ibm.com> wrote:
> Lisandro:
> The compiler guarantees proper alignment of stack-allocated and
> statically-allocated data. Also, I think the Blue Gene implementation of
> malloc always returns 16-byte aligned addresses.

In my 32bits linux FC6 box, I compile the following

#include <stdio.h>
#include <malloc.h>
typedef unsigned us;
#define PRINTP(p) \
  printf("p=%p p mod 4=%u p mod 8=%u \n",\
	 p, (us)(((us)p)%4), (us)(((us)p)%8))
int main(int argc, char* argv[]) {
  int iv1,iv2,iv3;
  int *ip1 = &iv1;
  int *ip2 = &iv2;
  int *ip3 = &iv3;
  printf("stacked\n");
  PRINTP(ip1); PRINTP(ip2); PRINTP(ip3);
  ip1 = (int*) malloc(sizeof(int));
  ip2 = (int*) malloc(sizeof(int));
  ip3 = (int*) malloc(sizeof(int));
  printf("malloced\n");
  PRINTP(ip1); PRINTP(ip2); PRINTP(ip3);
  return 0;
}

and next run
$ a.out
stacked
p=0xbff66184 p mod 4=0 p mod 8=4
p=0xbff66180 p mod 4=0 p mod 8=0
p=0xbff6617c p mod 4=0 p mod 8=4
malloced
p=0x9caf008 p mod 4=0 p mod 8=0
p=0x9caf018 p mod 4=0 p mod 8=0
p=0x9caf028 p mod 4=0 p mod 8=0

So malloced mem is 8 bytes aligned (is this the definition for
aligned?), but addresses to local, stacked integer variables are 4
bytes.

So this makes me think that perhaps PETSc is treating all adresses as
malloced and in some place this can cause problems (you said you were
using a custom PETSC_MEMALIGN)...

As I said, my knowledge of this is 0 (zero), this is just an idea, and
forget me if all this does not make any sense


-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594