precision woes

[Joachim Worringen]

Joachim Worringen:
> Currently, with most issues in nf_test fixed, I still get this error
> message on our local SX-6i (mpirun ... nf_test -v):
> *** Testing nfmpi_put_att_real ...
>  ERROR 2
  [^^^^^^^ This is one of my diagnostic messages]
>  Numeric conversion not representable
>
>
>  8   good comparisons.
>
>
>    ###   1   FAILURES TESTING nfmpi_put_att_real! ###

I digged deeper into this one and discovered that the problem is in fact a 
rounding error. The test wants to use INT_MAX (2^31 - 1), which seemingly can 
not be properly represented as a float/real value (which seems reasonable). 
On SX, 2^31 is used (rounded), while on other platforms smaller values (2^31 
- 1) are used (probably cut off). 

The valus are already incorrect when passed to nfmpi_put_att_real(). Thus, 
this error is not a real error. Should the test be changed in any way? I'd 
say yes, because the outcome is not really defined due to the inherent lack 
of precision. I did fix it with some preprocessor statements (test_put.F):

        subroutine test_nfmpi_put_att_real()
[...]
                    do 3, k = 1, ATT_LEN(j,i)
                        ndx(1) = k
                        value(k) = hash_real(ATT_TYPE(j,i), -1, ndx, 
     +                                     NFT_REAL)
                        val = value(k)
                        allInExtRange = allInExtRange .and.
     +                      inRange3(val, ATT_TYPE(j,i), 
     +                               NFT_REAL)
3                   continue
#if defined SX
                    allInExtRange = allInExtRange .AND. 
     +                  (ATT_TYPE(j,i) .ne. NF_INT)
#endif
                    err = nfmpi_put_att_real(ncid, i, ATT_NAME(j,i), 
     +                                  ATT_TYPE(j,i), ATT_LEN(j,i), 
     +                                  value)

Of course, a real fix would have to happen in test_put.m4.

  Joachim

-- 
Joachim Worringen - NEC C&C research lab St.Augustin
fon +49-2241-9252.20 - fax .99 - http://www.ccrl-nece.de