[petsc-users] pctype hmpi

[George Pau]

Hi Barry,

I tried with the addition of 

-hmpi_spawn_size 3

but I am still getting the same error though.  I am using mpich2.  Any other options to try?

George


On Aug 30, 2012, at 7:28 PM, Barry Smith wrote:

> 
> On Aug 30, 2012, at 7:24 PM, George Pau <gpau at lbl.gov> wrote:
> 
>> Hi,
>> 
>> I have some issues using the -pctype hmpi.  I used the same setting found at 
>> 
>> http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCHMPI.html
>> 
>> i.e. 
>> -pc_type hmpi 
>> -ksp_type preonly
>> -hmpi_ksp_type cg 
>> -hmpi_pc_type hypre 
>> -hmpi_pc_hypre_type boomeramg
>> 
>> My command  is
>> 
>> mpiexec -n 1 myprogram
> 
>   Sorry the documentation doesn't make this clearer. You need to start PETSc with special options to get the "worker" processes initialized. From the manual page for PCHMPI it has
> 
>   See PetscHMPIMerge() and PetscHMPISpawn() for two ways to start up MPI for use with this preconditioner
> 
>   This will tell you want option to start PETSc up with.
> 
>    I will fix the PC so that it prints a far more useful error message.
> 
> 
> 
>   Barry
> 
> 
>> 
>> But, I get 
>> 
>> [gilbert:4041] *** An error occurred in MPI_Bcast
>> [gilbert:4041] *** on communicator MPI_COMM_WORLD
>> [gilbert:4041] *** MPI_ERR_COMM: invalid communicator
>> [gilbert:4041] *** MPI_ERRORS_ARE_FATAL (your MPI job will now abort)
>> 
>> with openmpi.  I get similar error with mpich2
>> 
>> Fatal error in PMPI_Bcast: Invalid communicator, error stack:
>> PMPI_Bcast(1478): MPI_Bcast(buf=0x7fffb683479c, count=1, MPI_INT, root=0, comm=0x0) failed
>> PMPI_Bcast(1418): Invalid communicator
>> 
>> I couldn't figure out what is wrong.    My petsc is  version 3.3.3 and the configuration is -with-debugging=0 --with-mpi-dir=/usr/lib/openmpi --download-hypre=1 and I am on a Ubuntu machine.
>> 
>> Note that with the default pc_type and ksp_type, everything is fine.  It was also tested with multiple processors.  I wondering whether there are some options that I am not specifying correctly?
>> 
>> -- 
>> George Pau
>> Earth Sciences Division
>> Lawrence Berkeley National Laboratory
>> One Cyclotron, MS 74-120
>> Berkeley, CA 94720
>> 
>> (510) 486-7196
>> gpau at lbl.gov
>> http://esd.lbl.gov/about/staff/georgepau/
>> 
>