[MOAB-dev] synchronisation of ranges

[Vijay S. Mahadevan]

This functionality looks quite similar to what exchange_tags routine
provides on ParallelComm. Have you tried this to synchronize the tag
data already ?

http://ftp.mcs.anl.gov/pub/fathom/moab-docs/classmoab_1_1ParallelComm.html#ab27be002508fa7b3bf2ad1f68461f1e9

I'll look over the function you implemented again to better understand
your use-case if exchange_tags doesn't solve the issue.

Vijay



On Wed, Apr 1, 2015 at 3:15 PM, Lukasz Kaczmarczyk
<Lukasz.Kaczmarczyk at glasgow.ac.uk> wrote:
> Hello Vijay,
>
> Yes, this is first step, next one is to synchronise entities in ranges between processors.
>
> Imagine that you partitioned tetrahedral, then you take subset of those tetrahedrals (on each part), and next you take adjacencies, f.e. faces of those tetrahedral. On each processor you need to do collectively set data on tags of those faces. On some processors you can any tetrahedral which is part of the subset, but you can have some faces which are on skin of that subset. In such a case you need synchronise such entities in the range.
>
> I already made some eclectic implementation of this, see line,
> 491 PetscErrorCode Core::synchronise_entities(Range &ents,int verb) {
>
> in
> http://userweb.eng.gla.ac.uk/lukasz.kaczmarczyk/MoFem/html/_core_8cpp_source.html
>
> It looks that it working, but not tested properly.
>
> Kind regards,
> Lukasz
>
>
>> On 1 Apr 2015, at 20:30, Vijay S. Mahadevan <vijay.m at gmail.com> wrote:
>>
>> Lukasz,
>>
>> The Range data-structure stores locally visible data (EntityHandles)
>> only and you can store these from various queries on the Interface
>> class. If you specifically want to store the shared entities in a
>> range, you can apply a Pstatus filter like below:
>>
>> Range vlocal, vowned, vghost, adjs;
>> // Get all local vertices: owned + ghosted (including shared)
>> merr = mbiface->get_entities_by_dimension(0, 0, vlocal, false);MB_CHK_ERR(merr);
>>
>> // filter based on parallel status
>> merr = pcomm->filter_pstatus(vlocal,PSTATUS_NOT_OWNED,PSTATUS_NOT,-1,vowned);MB_CHK_ERR(merr);
>>
>> // filter all the non-owned and shared entities out of the list
>> adjs = moab::subtract(vlocal, vowned);
>> merr = pcomm->filter_pstatus(adjs,PSTATUS_GHOST|PSTATUS_INTERFACE,PSTATUS_OR,-1,&vghost);MB_CHK_ERR(merr);
>> adjs = moab::subtract(adjs, vghost);
>> vlocal = moab::subtract(vlocal, adjs);
>>
>> Instead of PSTATUS_GHOST, you could use PSTATUS_SHARED if you wanted
>> to filter different set of (shared) entities locally. Hope this helps.
>> If not, let us know what is unclear.
>>
>> Vijay
>>
>> On Wed, Apr 1, 2015 at 6:43 AM, Lukasz Kaczmarczyk
>> <Lukasz.Kaczmarczyk at glasgow.ac.uk> wrote:
>>> Hello,
>>>
>>> It is quick method to synchronise ranges.
>>>
>>> 1) Range on each processor keeps entities, some of those entities are shared between one or more processors.
>>>
>>> 2) I need collectively synchronise that ranges, such that if entity is in range on one of processors is as well in range on other processors (only if it is shared).
>>>
>>> 3) I do not need to send entity, only synchronise entities in range which are shared.
>>>
>>> This is needed for collective operations on tags, instead of exchanging tags values, I need to synchronise range and then collectively set tags.
>>>
>>>
>>> Kind regards,
>>> Lukasz Kaczmarczyk
>>>
>