Problems porting code to an IBM power5+ machine

[Matthew Knepley]

  This smells like a memory overwrite or use of an uninitialized
variable. The initial norm is 1e28.

1) Try Jacobi instead of ICC to see if it is localized to the PC

2) Run with valgrind or something similar to check for a memory overwrite

3) Maybe insert CHKMEMQ statements into the code

  Thanks,

    Matt

On 3/19/07, Knut Erik Teigen <knutert at stud.ntnu.no> wrote:
> Hello
>
> I have got a working code on my local machine( Pentium 4), but when I
> try to run the code on a power5+ machine, the equation solver won't
> converge. It seems like it calculates the wrong right hand side norm.
> Below is the result with run-time options
> "-ksp_type cg -pc_type icc -ksp_monitor -ksp_view -info
> First with the code running on the power5+ machine, then on my local
> machine. I've also printed the right hand side, jacobian matrix and
> solution for a small 3x3 grid.
>
> Can anyone help me figure out what's wrong?
>
> Regards,
> Knut Erik Teigen
>
> Code running on Power5+:
> rhs:
> 980
> 980
> 980
> -0
> -0
> -0
> -980
> -980
> -980
> jacobian:
> row 0: (0, 20)  (1, -10)  (3, -10)
> row 1: (0, -10)  (1, 30)  (2, -10)  (4, -10)
> row 2: (1, -10)  (2, 20)  (5, -10)
> row 3: (0, -10)  (3, 30)  (4, -10)  (6, -10)
> row 4: (1, -10)  (3, -10)  (4, 40)  (5, -10)  (7, -10)
> row 5: (2, -10)  (4, -10)  (5, 30)  (8, -10)
> row 6: (3, -10)  (6, 20)  (7, -10)
> row 7: (4, -10)  (6, -10)  (7, 30)  (8, -10)
> row 8: (5, -10)  (7, -10)  (8, 20)
> [0] PCSetUpSetting up new PC
> [0] PetscCommDuplicateDuplicating a communicator 1 4 max tags =
> 1073741823
> [0] PetscCommDuplicateUsing internal PETSc communicator 1 4
> [0] PetscCommDuplicateUsing internal PETSc communicator 1 4
>   0 KSP Residual norm 7.410163701832e+28
>   1 KSP Residual norm 6.464393707520e+11
> [0] KSPDefaultConvergedLinear solver has converged. Residual norm
> 6.46439e+11 is less than relative tolerance 1e-07 times initial right
> hand side norm 7.41016e+28 at iteration 1
> KSP Object:
>   type: cg
>   maximum iterations=10000, initial guess is zero
>   tolerances:  relative=1e-07, absolute=1e-50, divergence=10000
>   left preconditioning
> PC Object:
>   type: icc
>     ICC: 0 levels of fill
>     ICC: factor fill ratio allocated 1
>     ICC: factor fill ratio needed 0.636364
>          Factored matrix follows
>         Matrix Object:
>           type=seqsbaij, rows=9, cols=9
>           total: nonzeros=21, allocated nonzeros=21
>               block size is 1
>   linear system matrix = precond matrix:
>   Matrix Object:
>     type=seqaij, rows=9, cols=9
>     total: nonzeros=33, allocated nonzeros=45
>       not using I-node routines
> solution:
> 6.37205e-09
> 7.13167e-09
> 7.49911e-09
> -2.48277e-09
> -4.56885e-10
> 0
> 0
> 0
> 0
>
> Code running on local machine:
> rhs:
> 980
> 980
> 980
> -0
> -0
> -0
> -980
> -980
> -980
> jacobian:
> row 0: (0, 20)  (1, -10)  (3, -10)
> row 1: (0, -10)  (1, 30)  (2, -10)  (4, -10)
> row 2: (1, -10)  (2, 20)  (5, -10)
> row 3: (0, -10)  (3, 30)  (4, -10)  (6, -10)
> row 4: (1, -10)  (3, -10)  (4, 40)  (5, -10)  (7, -10)
> row 5: (2, -10)  (4, -10)  (5, 30)  (8, -10)
> row 6: (3, -10)  (6, 20)  (7, -10)
> row 7: (4, -10)  (6, -10)  (7, 30)  (8, -10)
> row 8: (5, -10)  (7, -10)  (8, 20)
> [0] PCSetUp(): Setting up new PC
> [0] PetscCommDuplicate(): Duplicating a communicator 1140850689
> -2080374783 max tags = 2147483647
> [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689
> -2080374783
> [0] PetscCommDuplicate(): Using internal PETSc communicator 1140850689
> -2080374783
>   0 KSP Residual norm 2.505507810276e+02
>   1 KSP Residual norm 3.596555656581e+01
>   2 KSP Residual norm 2.632672485513e+00
>   3 KSP Residual norm 1.888285055287e-01
>   4 KSP Residual norm 7.029433008806e-03
>   5 KSP Residual norm 3.635267067420e-14
> [0] KSPDefaultConverged(): Linear solver has converged. Residual norm
> 3.63527e-14 is less than relative tolerance 1e-07 times initial right
> hand side norm 250.551 at iteration 5
> KSP Object:
>   type: cg
>   maximum iterations=10000, initial guess is zero
>   tolerances:  relative=1e-07, absolute=1e-50, divergence=10000
>   left preconditioning
> PC Object:
>   type: icc
>     ICC: 0 levels of fill
>     ICC: factor fill ratio allocated 1
>     ICC: factor fill ratio needed 0.636364
>          Factored matrix follows
>         Matrix Object:
>           type=seqsbaij, rows=9, cols=9
>           total: nonzeros=21, allocated nonzeros=21
>               block size is 1
>   linear system matrix = precond matrix:
>   Matrix Object:
>     type=seqaij, rows=9, cols=9
>     total: nonzeros=33, allocated nonzeros=45
>       not using I-node routines
> solution:
> 92.3023
> 92.3023
> 92.3023
> -5.69767
> -5.69767
> -5.69767
> -103.698
> -103.698
> -103.698
>
>
>
>


-- 
One trouble is that despite this system, anyone who reads journals widely
and critically is forced to realize that there are scarcely any bars to eventual
publication. There seems to be no study too fragmented, no hypothesis too
trivial, no literature citation too biased or too egotistical, no design too
warped, no methodology too bungled, no presentation of results too
inaccurate, too obscure, and too contradictory, no analysis too self-serving,
no argument too circular, no conclusions too trifling or too unjustified, and
no grammar and syntax too offensive for a paper to end up in print. --
Drummond Rennie