[petsc-users] Read in sequential, solve in parallel

[Moinier, Pierre (UK)]

Jed,

Thanks for your help and thanks also to all of the others who have replied!. I made some progress and wrote a new code that runs in parallel. However the results seems to show that the time requires to solve the linear systems is the same whether I use 1, 2 or 4 processors... Surely I am missing something. I copied the code below. For info, I run the executable as: ./test -ksp_type cg -ksp_rtol 1.e-6 -pc_type none 

The code is:


  PetscInitialize(&argc,&argv,(char *)0,help);
  ierr = MPI_Comm_size(PETSC_COMM_WORLD,&size);CHKERRQ(ierr);

  /* Read in matrix */  
  ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD,"A.dat",FILE_MODE_READ,&view);CHKERRQ(ierr);
  ierr = MatCreate(PETSC_COMM_WORLD,&A);CHKERRQ(ierr);
  ierr = MatLoad(view,MATMPIAIJ,&A);CHKERRQ(ierr);
  ierr = PetscViewerDestroy(view);CHKERRQ(ierr);

  ierr = PetscPrintf(PETSC_COMM_SELF,"Reading matrix completed.\n");CHKERRQ(ierr);

  /* Read in RHS */
  ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD,"RHS.dat",FILE_MODE_READ,&view);CHKERRQ(ierr);
  ierr = VecCreate(PETSC_COMM_WORLD,&b);CHKERRQ(ierr);
  ierr = VecLoad(view,VECMPI,&b);CHKERRQ(ierr);
  ierr = PetscViewerDestroy(view);CHKERRQ(ierr);

  ierr = PetscPrintf(PETSC_COMM_SELF,"Reading RHS completed.\n");CHKERRQ(ierr);

  VecCreate(PETSC_COMM_WORLD,&x);
  VecDuplicate(b,&x);


/* 
   Create linear solver context
*/
  ierr = PetscPrintf(PETSC_COMM_SELF,"Solving ... \n");CHKERRQ(ierr);


  KSPCreate(PETSC_COMM_WORLD,&ksp);
  KSPSetOperators(ksp, A, A, SAME_NONZERO_PATTERN);
  KSPSetFromOptions(ksp);

             timeval tim;
             gettimeofday(&tim, NULL);
             double t1=tim.tv_sec+(tim.tv_usec/1000000.0);

  KSPSolve(ksp, b, x);

             gettimeofday(&tim, NULL);
             double t2=tim.tv_sec+(tim.tv_usec/1000000.0);
             printf("%.6lf seconds elapsed\n", t2-t1);

  ierr = MatDestroy(A);CHKERRQ(ierr);
  ierr = VecDestroy(b);CHKERRQ(ierr);

  ierr = PetscFinalize();CHKERRQ(ierr);

-----
Dr Pierre Moinier
Principal Research Scientist
Office ': +44 (0)117 302 8223  >  pierre.moinier at baesystems.com | ü www.baesystems.com
BAE Systems ¦ Advanced Technology Centre ¦ Sowerby Building (20R) ¦ FPC 267 ¦ PO Box 5 ¦ Filton ¦ Bristol ¦ BS34 7QW
BAE Systems (Operations) Limited
Registered Office: Warwick House, PO Box 87, Farnborough Aerospace Centre, Farnborough, Hants, GU14 6YU, UK
Registered in England & Wales No: 1996687

-----Original Message-----
From: petsc-users-bounces at mcs.anl.gov [mailto:petsc-users-bounces at mcs.anl.gov] On Behalf Of Jed Brown
Sent: 29 September 2010 11:18
To: PETSc users list
Subject: Re: [petsc-users] Read in sequential, solve in parallel

                    *** WARNING ***

  This message has originated outside your organisation,
  either from an external partner or the Global Internet. 
      Keep this in mind if you answer this message.
 

On Wed, Sep 29, 2010 at 11:46, Moinier, Pierre (UK) <Pierre.Moinier at baesystems.com> wrote:
> I have a Matrix and a Right Hand Side (RHS) in an ASCII format stored 
> in 2 different files. I have written the code that reads the data and 
> solve the system in sequential. I would like to solve the same problem 
> in parallel. I read the FAQ section that says:
>
> Never read or write in parallel an ASCII matrix file, instead for reading:
> read in sequentially then save the matrix with the binary viewer
> PetscBinaryViewerOpen() and load the matrix in parallel with MatLoad().
>
> So far, I did not manage to implement this. Could any one help me?

With your matrix assembled in serial:

    ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD,"A.dat",FILE_MODE_WRITE,&viewer);CHKERRQ(ierr);
    ierr = MatView(A,viewer);CHKERRQ(ierr);
    ierr = PetscViewerDestroy(viewer);CHKERRQ(ierr);

In parallel:

    ierr = PetscViewerBinaryOpen(PETSC_COMM_WORLD,"A.dat",FILE_MODE_READ,&viewer);CHKERRQ(ierr);
    ierr = MatCreate(PETSC_COMM_WORLD,&A);CHKERRQ(ierr);
    ierr = MatLoad(A,viewer);CHKERRQ(ierr);
    ierr = PetscViewerDestroy(viewer);CHKERRQ(ierr);
    ierr = KSPSetOperators(ksp,A,A,DIFFERENT_NONZERO_PATTERN);CHKERRQ(ierr);


Jed

********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************