[petsc-users] coupling with Matlab and parallel solution

[Barry Smith]

On Sep 3, 2010, at 4:32 PM, Benjamin Sanderse wrote:

> Hi Barry,
> 
> Thanks for your help! However, there are still some issues left. In other to test things, I simplified the program even more and now I am just sending a vector back and forth: matlab->petsc->matlab:
> 
>  fd   = PETSC_VIEWER_SOCKET_WORLD;
> 
>  // load rhs vector
>  ierr = VecLoad(fd,VECMPI,&b);CHKERRQ(ierr);
> 
>  // send to matlab
>  ierr = VecView(b,fd);CHKERRQ(ierr);
>  ierr = VecDestroy(b);CHKERRQ(ierr);
> 
> 
> - Your approach with two windows works *sometimes*. I removed the 'launch' statement and I executed my program 10 times, the first 2 times worked, and in all other cases I got this:
> 
> petscmpiexec -n 2 ./petsc_poisson_par_barry2 -info
> [1] PetscInitialize(): PETSc successfully started: number of processors = 2
> [1] PetscInitialize(): Running on machine: borr.mas.cwi.nl
> [0] PetscInitialize(): PETSc successfully started: number of processors = 2
> [0] PetscInitialize(): Running on machine: borr.mas.cwi.nl
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850688 -2080374784 max tags = 2147483647
> [1] PetscCommDuplicate(): Duplicating a communicator 1140850688 -2080374784 max tags = 2147483647
> [1] PetscCommDuplicate():   returning tag 2147483647
> [0] PetscCommDuplicate():   returning tag 2147483647
> [0] PetscViewerSocketSetConnection(): Connecting to socket process on port 5005 machine borr.mas.cwi.nl
> [0] PetscOpenSocket(): Connection refused in attaching socket, trying again[0] PetscOpenSocket(): Connection refused in attaching socket, trying again[0] PetscOpenSocket(): Connection refused in attaching socket, trying again
> [0] PetscOpenSocket(): Connection refused in attaching socket, trying again^C
> -bash-4.0$ [0]0:Return code = 0, signaled with Interrupt
> [0]1:Return code = 0, signaled with Interrupt
> 
> Every time I start the program I use close(socket) and clear all in Matlab, so the socket from the previous run should not be present anymore. It seems that the port gets corrupted after a couple of times? Matlab does not respond and I have to kill it and restart it manually.

   Sometimes when you close a socket connection it doesn't close for a very long time so that if you try to open it again it doesn't work. When it appears the socket can not be used try using netstat | grep 5005 to see if the socket is still active. 

> 
> - If I include the launch statement, or just type
> system('mpiexec -n 2 ./petsc_poisson_par_barry2 &')
> the program never works. 

   Are you sure mpiexec is in the path of system and it is the right one? The problem is that we are kind of cheating with system because we start a new job in the background and have no idea what the output is. Are you using unix and running Matlab on the command line or in a GUI?

  Barry


> 
> Hope you can figure out what is going wrong.
> 
> Ben
> 
> 
> Op 3 sep 2010, om 13:25 heeft Barry Smith het volgende geschreven:
> 
>> 
>>  Ben
>> 
>>  Ok, I figured out the problem. It is not fundamental and mostly comes from not having a create way to debug this.
>> 
>>  The test vector you create is sequential then you try to view it back to Matlab with the parallel fd viewer. If you change to 
>> ierr = VecCreateMPI(PETSC_COMM_WORLD,PETSC_DECIDE,1,&test);CHKERRQ(ierr);
>> then the code runs.
>> 
>>  I've found (just now) that when I use launch all the output from the .c program gets lost which makes it impossible to figure out what has gone wrong. You can debug by running the two parts of the computation in two different windows. So comment out the launch from the matlab script and then in Matlab run the script (it will hang waiting for the socket to work) and in a separate terminal window run the .c program; for example petscmpiexec -n 2 ./ex1 -info Now you see exactly what is happening in the PETSc program. You can even use -start_in_debugger on the PETSc side to run the debugger on crashes.
>> 
>>  I'll add this to the docs for launch
>> 
>>  Barry
>> 
>> 
>> On Sep 2, 2010, at 3:28 PM, Benjamin Sanderse wrote:
>> 
>>> Hi Barry,
>>> 
>>> I attached my matlab file, c file and makefile. First I generate the executable with 'make petsc_poisson_par_barry' and then I run test_petsc_par_barry.m. 
>>> If you change MATMPIAIJ to MATAIJ and VECMPI to VECSEQ the code works fine.
>>> 
>>> Thanks a lot,
>>> 
>>> Benjamin
>>> 
>>> <makefile><test_petsc_par_barry.m><petsc_poisson_par_barry.c>
>>> 
>>> Op 2 sep 2010, om 13:45 heeft Barry Smith het volgende geschreven:
>>> 
>>>> 
>>>> 
>>>> Matlab is never aware the vector is parallel. Please send me the code and I'll figure out what is going on.
>>>> 
>>>> Barry
>>>> 
>>>> On Sep 2, 2010, at 2:07 PM, Benjamin Sanderse wrote:
>>>> 
>>>>> That sounds great, but there is one issue I am encountering. I switched vector types to VECMPI and matrix type to MATMPIAIJ, but when running Matlab I get the following error:
>>>>> 
>>>>> Found unrecogonized header 0 in file. If your file contains complex numbers
>>>>> then call PetscBinaryRead() with "complex" as the second argument
>>>>> Error in ==> PetscBinaryRead at 27
>>>>> if nargin < 2
>>>>> 
>>>>> ??? Output argument "varargout" (and maybe others) not assigned during call to 
>>>>> "/ufs/sanderse/Software/petsc-3.1-p4/bin/matlab/PetscBinaryRead.m>PetscBinaryRead".
>>>>> 
>>>>> Error in ==> test_petsc_par at 57
>>>>> 	x4 = PetscBinaryReady(PS);
>>>>> 
>>>>> Could it be that Matlab does not understand the "parallel" vector which is returned by Petsc? Currently I have this done with VecView as follows:
>>>>> 
>>>>> fd = PETSC_VIEWER_SOCKET_WORLD;
>>>>> ...
>>>>> KSPSolve(ksp,b,x);
>>>>> ...
>>>>> VecView(fd,x);
>>>>> 
>>>>> Thanks for the help!
>>>>> 
>>>>> Ben
>>>>> 
>>>>> Op 2 sep 2010, om 10:09 heeft Barry Smith het volgende geschreven:
>>>>> 
>>>>>> 
>>>>>> On Sep 2, 2010, at 10:51 AM, Benjamin Sanderse wrote:
>>>>>> 
>>>>>>> Hello all,
>>>>>>> 
>>>>>>> I figured out the coupling with Matlab and I can send back and forth matrices and vectors between Petsc and Matlab. Actually, I send only once a matrix from Matlab to Petsc and then repeatedly send new right hand sides from Matlab->Petsc and the solution vector from Petsc->Matlab. That works great.
>>>>>>> I know want to see if the matrix that is send from (serial) Matlab to Petsc can be stored as a parallel matrix in Petsc so that subsequent computations with different right hand sides can be performed in parallel by Petsc. Does this simply work by using MatLoad and setting Mattype MPIAIJ? Or is something more fancy required?
>>>>>> 
>>>>>> In theory this can be done using the same code as sequential only with parallel vectors VECMPI  and matrices. MATMPIAIJ
>>>>>> 
>>>>>> Barry
>>>>>> 
>>>>>>> 
>>>>>>> Thanks,
>>>>>>> 
>>>>>>> Ben
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
>