[petsc-users] Using DM for a refined level-set grid

[Åsmund Ervik]

Hi Jed,

Thanks for the answer, sorry for the long delay here, I got knocked out by the flu. Oh well.

>Fra: Jed Brown [jed at jedbrown.org]
>Sendt: 22. februar 2014 01:50
>Til: Åsmund Ervik; petsc-users at mcs.anl.gov
>Emne: Re: [petsc-users] Using DM for a refined level-set grid
>
>
>Is this 2D or 3D, what fraction of the domain is likely to be within
>5*dxCoarse of the interface, and how large do you intend your subdomains
>to be?

This is 3D. Of the global domain only a small fraction would be within 5*dxCoarse of the interface; we're not studying atomization or anything extreme like that. I guess it's a little tricky to say how large the subdomains will be, but I'd say around 300 subdomains would be a reasonable guess at the number of them. Then the size would be roughly 1/7th of the global domain in each dimension, so for a typical case around 100*dxCoarse in each dimension. So still the subdomains should not be nearly-full refined.

>My worry is that even if you implement the dynamic algorithm you're
>thinking of (which is nontrivial to do well), some subdomains will have
>nearly-full refined sections which will limit your max problem size and
>performance.  If you can't load balance dynamically (in terms of peak
>memory and time) at the needed granularity, there is no point trying to
>save some storage in benign regions of your domain.

This is a good point. It's probably worth it to implement just a plain refined grid (without any "sparse allocation") and see if that will be sufficiently fast. Should be quick to do.

>What sort of methods will you be using for your CFD?  Does it involve
>assembled matrices?

We're solving a pressure Poisson equation, so yes, it involves assembled matrices.

>From your questions I now realize that perhaps the worst-scaling part of our code will be velocity extrapolation for level-set advection. This takes some time (not as much as the Poisson equation, maybe 5% of that) and only happens in a band around the interface. I guess we should look into alternatives to velocity extrapolation. 

Regards,
Åsmund