[Ad-dev] Thoughts on partial separability implementation

[Sri Hari Krishna Narayanan]

Hi,

During our intermittent discussions about the partial separability  
work, both before
the submission and during the AMR proposal writing we floated the idea  
of eliminating
a loop that was considered wasteful by you, but convenient from an  
implementation
standpoint by me.

Specifically:
The statements:
   #pragma $adic_partiallyseparable, fquad, flin
	*f = area*(p5*fquad+flin);

were converted originally to:
#pragma $adic_partiallyseparable, fquad, flin
  ADIC_SPARSE_Create1DimArray(&__adic_temp_f, ad_var_max);
  for (__adic_temp0 = 0; __adic_temp0 < ad_var_max; __adic_temp0++) {
     __adic_temp_f[__adic_temp0] = (area * ((p5 * fquad[__adic_temp0])
                                       + flin[__adic_temp0]));
   }
   ADIC_SPARSE_Summation(f,__adic_temp_f,ad_var_max);


where the grad_array of f is dense and large in size. f is an object  
of the class SparseDerivSeparable.
The grad_array of __adic_temp_f, fquad, and  flin
are of size p, where p is the number of colors used by ColPack to  
compress the sparse elemental Jacobian.
These variables are objects of the class SparseDeriv. DERIV_TYPE is  
typedeffed to SparseDeriv
SparseDerivSeparable is a child of SparseDeriv.SparseDeriv
contains a static array:
double grad[ADIC_GRADVEC_LENGTH];
and SparseDerivSeparable contains a dynamically allocated array:
   double *grad;.
The only location where a SparseDeriv object is used to compute a   
SparseDerivSeparable
object is in the ADIC_SPARSE_Summation function.  In the
sparsity detection mode ADIC_SPARSE_Summation registers the sparsity  
patterns of __adic_temp_f.
In the computation mode,  uses the registered patterns along with the  
ColPack routines to
compute f from __adic_temp_f.

I went ahead an implemented a version that did not require a loop that
calculates each element of  __adic_temp_f.
The output is given below:

#pragma $adic_partiallyseparable, fquad, flin
   DERIV_TYPE *__adic_temp_flin;
   DERIV_TYPE *__adic_temp_fquad;
   ad_var_26 = ADIC_SPARSE_max(ad_var_0,ad_var_1);
   ADIC_SPARSE_Summation(__adic_temp_fquad,ad_fquad,ad_var_26);
   ad_var_27 = ADIC_SPARSE_max(ad_var_0,ad_var_1);
   ADIC_SPARSE_Summation(__adic_temp_flin,ad_flin,ad_var_27);
   ad_TempVartmp_7 = DERIV_val(ad_p5) * DERIV_val( *__adic_temp_fquad)  
+ DERIV_val( *__adic_temp_flin);
   ad_TempVarSymbol_12 = DERIV_val(ad_area) * ad_TempVartmp_7;
   ad_TempVarlin_30 = ad_TempVartmp_7;
   ad_TempVarlin_32 = DERIV_val( *__adic_temp_fquad);
   ad_TempVarlin_33 = DERIV_val(ad_p5);
   ad_TempVarlin_31 = DERIV_val(ad_area);
   ADIC_SetValue( *f,ad_TempVarSymbol_12);
    
ADIC_SetDeriv 
(DERIV_TYPE_ref(ad_area),DERIV_TYPE_ref(ad_TempVarprop_55));
    
ADIC_SetDeriv(DERIV_TYPE_ref(ad_p5),DERIV_TYPE_ref(ad_TempVarprop_56));
    
ADIC_SetDeriv 
(DERIV_TYPE_ref( *__adic_temp_fquad),DERIV_TYPE_ref(ad_TempVarprop_57));
    
ADIC_SetDeriv 
(DERIV_TYPE_ref( *__adic_temp_flin),DERIV_TYPE_ref(ad_TempVarprop_58));
    
ADIC_Sax_3 
(1 
,DERIV_TYPE_ref 
(ad_TempVarprop_58 
),ad_TempVarlin_32 
,DERIV_TYPE_ref 
(ad_TempVarprop_56 
),ad_TempVarlin_33 
,DERIV_TYPE_ref(ad_TempVarprop_57),DERIV_TYPE_ref(ad_TempVarprop_59));
    
ADIC_Sax_2 
(ad_TempVarlin_30 
,DERIV_TYPE_ref 
(ad_TempVarprop_55 
),ad_TempVarlin_31 
,DERIV_TYPE_ref(ad_TempVarprop_59),DERIV_TYPE_ref( *f));
}

Right now, all temporary variables are objects of SparseDeriv.
There are two calls to ADIC_SPARSE_Summation
that assign values to the temporary variables __adic_temp_flin and  
__adic_temp_fquad
These variables will be defined as  SparseDerivSeparable
within ADIC_SPARSE_Summation.

Here begins the problem, the XAIFBooster output for the statement
area*(p5*fquad+flin);
requires ADIC_Sax_# and SetDeriv calls which causes
assignments of SparseDeriv to SparseDerivSeparable
and vice-versa.

Statements such as:
    
ADIC_SetDeriv 
(DERIV_TYPE_ref( *__adic_temp_fquad),DERIV_TYPE_ref(ad_TempVarprop_57));
can probably be handled by intelligent declaration of temporaries.

However, the statement
    
ADIC_Sax_3 
(1 
,DERIV_TYPE_ref 
(ad_TempVarprop_58 
),ad_TempVarlin_32 
,DERIV_TYPE_ref 
(ad_TempVarprop_56 
),ad_TempVarlin_33 
,DERIV_TYPE_ref(ad_TempVarprop_57),DERIV_TYPE_ref(ad_TempVarprop_59));
is trickier to handle as it requires us to define the computation  
between  a SparseDeriv type and a SparseDerivSeparable type.
ad_TempVarprop_56 is just a temporary holding the grad_arrary of ad_area
which is  a SparseDeriv object.

What can be noted is that ad_area is a non_elemental value that is  
also in effect inactive.
So if we integrate ActivityAnalysis, we should be able to keep ad_area  
inactive (double type) and so
the above situation should not occur.

Are there any thoughts on this?

I have not yet talked about possibly using f later on in the code in  
this email. I am assuming that
the succeeding code is all SparseDerivSeparable.
We will have to work out how to have different types and variables  
declaration co-existing harmoniously.

Thanks,
Krishna

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/ad-dev/attachments/20120227/0b61a14f/attachment.htm>