On Sat, Oct 29, 2011 at 6:58 PM, Shao-Ching Huang <span dir="ltr"><<a href="mailto:huangsc@gmail.com">huangsc@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Hi, I have additional questions:<br>
<br>
1. Suppose I create DMDA0 on two processes {0,1} (communicator comm0)<br>
and DMDA1 on another two processes {2,3} (communicator comm1). Can I<br>
DMCompositeAddDM() them into DMComposite (created using communicator<br>
MPI_COMM_WORLD, containing all 4 processes, 0-3)?<br></blockquote><div><br></div><div>No. This is due to the complexity of MPI for collectives, etc between communicators.</div><div>Instead, you should use the full communicator for both, but give no vertices to</div>
<div>ranks you want to leave out. This means you will have to partition the DMDA yourself,</div><div>but this is straightforward. There are no performance hits when communicating ghost</div><div>values, and the reductions inside the solve would need all the procs anyway.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
2. Suppose DMDA0 and DMDA1 are 2D Cartesian domains, and that the<br>
right-hand-side of DMDA0 is "connected" to DMDA1 (just like the<br>
subdomains within a regular DMDA). Which API should I use to tell<br>
DMComposite that the "right side" (say i=Nx, all j) of DMDA0 is<br>
connected to the "left side" (say i=0, all j) of DMDA1. I suppose I<br>
need to use IS index set somewhere.<br></blockquote><div><br></div><div>This is more complicated. All our examples (like SNES ex28) are not grids or scalar which</div><div>are not coupled. You would need to construct the LocalToGlobal mapping for this collection</div>
<div>of grids (which is a set of two ISes). Here is the current code:</div><div><br></div><div> <a href="http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/src/dm/impls/composite/pack.c.html#DMCompositeGetISLocalToGlobalMappings">http://www.mcs.anl.gov/petsc/petsc-as/snapshots/petsc-current/src/dm/impls/composite/pack.c.html#DMCompositeGetISLocalToGlobalMappings</a></div>
<div><br></div><div>Notice that the mapping for each DMDA is just concatenated. You code would look similar,</div><div>except that you would knit together one edge.</div><div><br></div><div>Since we have never had anyone ask for this, the interface is still primitive. If</div>
<div>you have a suggestion for a nice way to construct this IS, please let us know.</div><div><br></div><div> Thanks,</div><div><br></div><div> Matt</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
Thanks,<br>
<br>
Shao-Ching<br>
<br>
On Fri, Oct 28, 2011 at 5:17 PM, Matthew Knepley <<a href="mailto:knepley@gmail.com">knepley@gmail.com</a>> wrote:<br>
> On Fri, Oct 28, 2011 at 11:48 PM, Shao-Ching Huang <<a href="mailto:huangsc@gmail.com">huangsc@gmail.com</a>><br>
> wrote:<br>
>><br>
>> Hi<br>
>><br>
>> We are planning a new (finite volume) multiblock code, in which each<br>
>> block has logically structured mesh. We plan to create one DMDA for<br>
>> one block (which could span across 1 or more processes; we already<br>
>> have code). What would be the recommended PETSc-way to couple these<br>
>> blocks together for implicit solves? We also need ghosted region<br>
>> between two connected blocks (just like the ghost regions among the<br>
>> subdomains within a DMDA) for interpolation.<br>
><br>
> I think the idea here is to use a DMComposite to couple together these<br>
> DMDAs. You<br>
> would have to specify the coupling explicitly since we have no way of<br>
> knowing how they<br>
> are connected, but after that, the GlobalToLocal() should work just the<br>
> same.<br>
> Thanks,<br>
> Matt<br>
><br>
>><br>
>> Thanks.<br>
>><br>
>> Shao-Ching<br>
><br>
><br>
><br>
> --<br>
> What most experimenters take for granted before they begin their experiments<br>
> is infinitely more interesting than any results to which their experiments<br>
> lead.<br>
> -- Norbert Wiener<br>
><br>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br>
-- Norbert Wiener<br>