[petsc-users] a question about DAE Function

[Barry Smith]

On Jun 4, 2013, at 2:13 PM, "Jin, Shuangshuang" <Shuangshuang.Jin at pnnl.gov> wrote:

> Hello, I have a question regarding how to set up the IFunction for TSSetIFunction. Please see the detailed illustration below:
>  
> Inside my IFunction, I have one of the f equations defined as:
>  
> if (i >= xstart && i < xstart+xlen) {
>       f[i+2] = sin(x[i])*x[i+2] + cos(x[i])*x[i+3];
>       for (k = 0; k < n; k++) {
>           f[i+2] += -(cos(x[k])*a[k])*b) - (sin(x[k])*c[k])*d);               (1)
>       }
> }
>  
> Where x is an distributed array, a and c are constant arrays, b and d are constants. n is the length of the global length of x.
>  
> The problem is, since x is distributed across all the processes, each process only owns a piece of items of x, when I do (1) as written above, no matter which processor f[i+2] is on, it has to grab all the x array data from other processors, which means I have to do a scattering of x array in advance to make the x[k] value available. This is quite an inefficient way given the scattering of vector would happen in each iteration when the IFuntion is called.

   If f[i] is dependent on all the x[i] then you have no choice but to have all the x[] on each process. Yes that requires a good chunk of communication but that is life. Note that the Jacobian will be dense and the linear solves with the dense Jacobian will dominate the time anyways. PETSc has an interface with Elemental to do the dense matrix solves.



   Barry

>  
> I am wondering if this is the only way I have to deal with this problem, or am I trapped in a wrong direction?
>  
> Your comments are highly appreciated!
>  
> Thanks,
> Shuangshuang
>  
>