[petsc-users] About MatGetOwnershipRange()

[Joon Hee Choi]

Thank you for your reply. Actually, I succeeded in running with the PetscSpliOwnership() code that you gave to me. I think I didn't understand how to write a code in parallel. Thank you again.


----- Original Message -----
From: "Jed Brown" <jedbrown at mcs.anl.gov>
To: "Joon Hee Choi" <choi240 at purdue.edu>
Cc: petsc-users at mcs.anl.gov
Sent: Friday, September 13, 2013 5:34:33 PM
Subject: Re: [petsc-users] About MatGetOwnershipRange()

Joon Hee Choi <choi240 at purdue.edu> writes:

>>>Did you run in parallel?
>>
>> Yes. I tested my code with 2 nodes and 4 ppn(processors per node).

>Then print what you are passing in and what you get back on each
>process.  The PetscSplitOwnership code is short.

>>>> Also, tuples are the total pairs of (row dimension, column dimension,
>>>> element), values of sparse matrix, which are read from a file. 
>>
>>>Using files this way is a massive bottleneck that you'll have to
>>>eliminate if you want your code to be scalable.
>>
>> What do you mean a massive bottleneck? 

>Writing to a file is at least 1000 times slower than accessing memory
>and it does not parallelize as well.

>> Actually, I succeeded in setting up the large matrix using seqaij type
>> and calculating products between huge matrices. However, I need to use
>> multiple machines because I have to run loops of the product many
>> times. And I am a novice of mpi.

>I'm not sure what you're asking, but you should create the matrices in
>memory, preferably in parallel and mostly where the entries will need to
>be stored.

>>>> The tuples are sorted and distributed. 
>>
>>>When you distribute, are you sure that each process really gets the
>>>entire row, or would it be possible to cut in the middle of a row?
>>
>> I think each process gets the entire row. The following is the code for getting >>tuples from text file:
>>
>>    ierr = PetscFOpen(PETSC_COMM_WORLD, inputFile, "r", &file); CHKERRQ(ierr);
>>    while (fscanf(file, "%d %d %d", &ii, &jj, &vv) == 3)
>>    {
>>       tups.push_back(std::tr1::make_tuple (ii-1, jj-1, vv));
>>       if (ii > I) I = ii;
>>       if (jj > J) J = jj;
>>    }
>>    ierr = PetscFClose(PETSC_COMM_WORLD, file); CHKERRQ(ierr);

>Nothing in this code snippet indicates that the row partition is non-overlapping.