[petsc-users] Doubt about mesh size distribution in DMDACreate3d using periodic boundary conditions

[Barry Smith]

  What do you mean by discretization size, and how do I see it in the code?

  Barry


> On Nov 20, 2024, at 12:48 PM, MIGUEL MOLINOS PEREZ <mmolinos at us.es> wrote:
> 
> Sorry, I meant that the discretisation size is not constant across the edges of the cube. 
> 
> Miguel
> 
>> On 20 Nov 2024, at 18:36, Barry Smith <bsmith at petsc.dev> wrote:
>> 
>> 
>>    I am sorry, I don't understand the problem. When I run by default with -da_view I get 
>> 
>> Processor [0] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 0 2, Z range of indices: 0 2
>> Processor [1] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 3, Y range of indices: 0 2, Z range of indices: 0 2
>> Processor [2] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 2 3, Z range of indices: 0 2
>> Processor [3] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 3, Y range of indices: 2 3, Z range of indices: 0 2
>> Processor [4] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 0 2, Z range of indices: 2 3
>> Processor [5] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 3, Y range of indices: 0 2, Z range of indices: 2 3
>> Processor [6] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 2 3, Z range of indices: 2 3
>> Processor [7] M 3 N 3 P 3 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 3, Y range of indices: 2 3, Z range of indices: 2 3
>> 
>> which seems right because you are trying to have three cells in each direction. The distribution has to be uneven, hence 0 2 and 2 3
>> 
>> When I change the code to use ndiv_mesh_* = 4 and run with periodic or not I get 
>> 
>> $ PETSC_OPTIONS="" mpiexec -n 8 ./atoms-3D -dm_view
>> DM Object: 8 MPI processes
>>   type: da
>> Processor [0] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 0 2, Z range of indices: 0 2
>> Processor [1] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 4, Y range of indices: 0 2, Z range of indices: 0 2
>> Processor [2] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 2 4, Z range of indices: 0 2
>> Processor [3] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 4, Y range of indices: 2 4, Z range of indices: 0 2
>> Processor [4] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 0 2, Z range of indices: 2 4
>> Processor [5] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 4, Y range of indices: 0 2, Z range of indices: 2 4
>> Processor [6] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 0 2, Y range of indices: 2 4, Z range of indices: 2 4
>> Processor [7] M 4 N 4 P 4 m 2 n 2 p 2 w 1 s 1
>> X range of indices: 2 4, Y range of indices: 2 4, Z range of indices: 2 4
>> 
>> so it is splitting as expected each rank gets a 2 by 2 by 2 set of indices.
>> 
>> Could you please let me know what the problem is that I should be seeing.
>> 
>>   Barry
>> 
>> 
>>> On Nov 20, 2024, at 7:06 AM, MIGUEL MOLINOS PEREZ <mmolinos at us.es> wrote:
>>> 
>>> Dear Barry,
>>> 
>>> Please, find attached to this email a minimal example of the problem. Run it using 8 MPI processes. 
>>> 
>>> Thanks,
>>> Miguel
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On 20 Nov 2024, at 11:48, Miguel Molinos <mmolinos at us.es> wrote:
>>>> 
>>>> Hi Bary:
>>>> 
>>>> I will check the example you suggest. Anyhow, I’ll send a reproducible example ASAP.
>>>> 
>>>> Thanks,
>>>> Miguel
>>>> 
>>>>> On 19 Nov 2024, at 18:55, Barry Smith <bsmith at petsc.dev> wrote:
>>>>> 
>>>>> 
>>>>>    I modify src/dm/tests/ex25.c and always see a nice even split when possible with both DM_BOUNDARY_NONE and DM_BOUNDARY_PERIODIC
>>>>> 
>>>>>     Can you please send a reproducible example?
>>>>> 
>>>>>     Thanks
>>>>> 
>>>>>      Barry
>>>>> 
>>>>> 
>>>>>> On Nov 19, 2024, at 6:14 AM, MIGUEL MOLINOS PEREZ <mmolinos at us.es> wrote:
>>>>>> 
>>>>>> Dear all:
>>>>>> 
>>>>>> It seems that if I mesh a cubic domain with “DMDACreate3d” using 8 bricks for discretization and with periodic boundaries, each of the bricks has a different size. In contrast, if I use DM_BOUNDARY_NONE, all 8 bricks have the same size. 
>>>>>> 
>>>>>> I have used this together with the DMSWarm discretization. And as you can see the number of particles per rank is not evenly distributed: 
>>>>>> 210 420 366 732 420 840 732 1464
>>>>>> 
>>>>>> Am I missing something?
>>>>>> 
>>>>>> Thanks,
>>>>>> Miguel
>>>>>> 
>>>>>> 
>>>>>> <Screenshot 2024-11-19 at 10.56.36.png>
>>>>> 
>>>> 
>>> 
>>> <atoms-3D.cpp><Mg-hcp-cube-x17-x10-x10.dump>
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20241120/63e4d4b8/attachment-0001.html>