[petsc-users] Strange error(?) message

[Mohammad Mirzadeh]

Thanks Matt. When I use -start_in_debugger, it just executes the program
without any extra message whatsoever! so its like I did not use the flag at
all.

Also, thought to share my conclusions on using ParMetis with others. The
two important things are:

1- Make sure you have a symmetric adjacency matrix for partitioning.
Chaning my non-symmetric matrix into a symmetric one solved the "Key 10716
not found!" problem.
2- Unlike PETSc, ParMetis does not equal number of nodes/elements to each
processors. The numbers are close, but not the same as you start
partitioning the grid with. It, however, does produce contigious
distribution of nodes/elements. You can use  ISPartitioningCount to figure
out how many nodes/elements are assigned to each processor.

Mohammad


On Wed, Apr 11, 2012 at 5:43 PM, Matthew Knepley <knepley at gmail.com> wrote:

> On Wed, Apr 11, 2012 at 10:49 PM, Mohammad Mirzadeh <mirzadeh at gmail.com>wrote:
>
>> I see. Well the reason for non-symmetry is that I'm using a finite
>> difference discretization on quadtree AMR grids. When discretizing at a
>> hanging node, i.e. a node that is missing a direct neighbor, I use
>>  interpolation from nodes of the parent cell. When it comes to discretizing
>> at those nodes (i.e. nodes of the parent), however, they are on a coarser
>> grid and will have direct neighbors so they would not see the hanging node
>> in their discretization. This leads to a non-symmetric discretization
>> matrix which I was using (structurally) to define the adjacency matrix of
>> the graph. I think I can fix this by requiring the parent nodes to also
>> have a link to the hanging node (although they are not used in the
>> discretization anyway)
>>
>> I'm just surprised that ParMetis does not complain about non-symmetric
>> graphs at lower resolutions and partitions them just fine!
>>
>
> ParMetis does not burden the user with a whole lot of diagnostic output :)
>
>
>> As for the debugger, -start_in_debugger does not start the debugger for
>> me :(. I'm using mpirun -np 2 ./a.out -start_in_debugger. Is this the
>> correct way?
>>
>
> Send the entire output. It is likely that at the beginning there are lines
> showing a failure to open an X window.
>
>    Matt
>
>
>>
>> On Wed, Apr 11, 2012 at 3:07 PM, Matthew Knepley <knepley at gmail.com>wrote:
>>
>>> On Wed, Apr 11, 2012 at 8:24 PM, Mohammad Mirzadeh <mirzadeh at gmail.com>wrote:
>>>
>>>> I was reading the FAQ list when I came across the following:
>>>>
>>>> http://www.mcs.anl.gov/petsc/documentation/faq.html#key
>>>>
>>>> When calling MatPartitioningApply() you get a message Error! Key 16615
>>>> not found The graph of the matrix you are using is not symmetric. You
>>>> must use symmetric matrices for partitioning.
>>>>
>>>> Is this a limitation on ParMetis side? I set up the adjacency matrix
>>>> based on the discretization that I will be performing on the grid which is
>>>> non-symmetric; both numerically and structurally. What's the solution here?
>>>> Make an "approximate" adjacency matrix that sort of looks like
>>>> (structurally) my discretization but is symmetric? What I don't understand
>>>> is my matrix IS non-symmetric when the code runs on coarser grids!
>>>>
>>>
>>> I don't quite understand how you can have a non-symmetric adjacency
>>> description. But Metis/ParMetis partitions undirected graphs, which
>>> by definition have symmetric adjacency matrices. Non-symmetric adjacency
>>> would seem to imply a directed graph of some sort, or more plainly,
>>> something is a adjacent to another thing which is not adjacent to it.
>>> That is a very strange concept.
>>>
>>>
>>>> Also, I was reading the FAQ hoping I can find something regarding using
>>>> gdb in parallel. I found this:
>>>> http://scicomp.stackexchange.com/a/410/485 but I'm not sure how I
>>>> should be using gdb in parallel. Could you (maybe Matt?) please explain a
>>>> little bit?
>>>>
>>>
>>> -start_in_debugger spawns a gdb windows for EVERY process and attaches it
>>>
>>> -debbuger_nodes a,b,c spawns gdb windows ONLY for ranks a, b, and c
>>>
>>>   Thanks,
>>>
>>>      Matt
>>>
>>>
>>>> Thanks
>>>>
>>>> On Tue, Apr 10, 2012 at 12:08 AM, Mohammad Mirzadeh <mirzadeh at gmail.com
>>>> > wrote:
>>>>
>>>>> Just built petsc-dev and it did not help. I'm going to look into the
>>>>> code to see if my graph is "ill-formed" in some sense. Just hope the
>>>>> problem is from my side not a real bug in ParMetis!
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Apr 9, 2012 at 8:46 PM, Mohammad Mirzadeh <mirzadeh at gmail.com>wrote:
>>>>>
>>>>>> ok. Thanks Jed. I'll try petsc-dev to see if it fixes the problem.
>>>>>>
>>>>>> Thanks everyone.
>>>>>>
>>>>>>
>>>>>> On Mon, Apr 9, 2012 at 8:42 PM, Jed Brown <jedbrown at mcs.anl.gov>wrote:
>>>>>>
>>>>>>> On Mon, Apr 9, 2012 at 22:37, Mohammad Mirzadeh <mirzadeh at gmail.com>wrote:
>>>>>>>
>>>>>>>> Thanks Sean. I'm using Petsc 3.2-p6 along with ParMetis 4.0.2.
>>>>>>>> Since this was not supported with 3.2-p6, and previous versions had bugs, I
>>>>>>>> built parmetis myself and used --with-parmetis-include and
>>>>>>>> --with-parmetis-lib flags to build petsc.
>>>>>>>>
>>>>>>>> Should I switch to petsc-dev?
>>>>>>>>
>>>>>>>
>>>>>>> Yes, and use --download-metis --download-parmetis because the
>>>>>>> version upstream has some bugs for which the patches have not been applied.
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> What most experimenters take for granted before they begin their
>>> experiments is infinitely more interesting than any results to which their
>>> experiments lead.
>>> -- Norbert Wiener
>>>
>>
>>
>
>
> --
> What most experimenters take for granted before they begin their
> experiments is infinitely more interesting than any results to which their
> experiments lead.
> -- Norbert Wiener
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120412/c388e7a0/attachment.htm>