Question with pc_type redundant
Nguyen, Hung V ERDC-ITL-MS
Hung.V.Nguyen at usace.army.mil
Thu Nov 20 11:47:05 CST 2008
Hong,
Thank you for the answer.
-Hung
-----Original Message-----
From: owner-petsc-users at mcs.anl.gov [mailto:owner-petsc-users at mcs.anl.gov] On
Behalf Of Hong Zhang
Sent: Thursday, November 20, 2008 9:31 AM
To: petsc-users at mcs.anl.gov
Subject: Re: Question with pc_type redundant
Hung,
PC redundant is used for solving coarse-grid linear problem, for which the
problem is too small for parallel processing.
Thus, we assign each(default) or a subset of procesors to solve the problem
redundantly with LU direct solver.
In your case, it is likely all processors implement LU sequentially. Adding
the time of broadcasting matrix entries to all processors, more processors
take longer time is anticipated.
Hong
On Thu, 20 Nov 2008, Nguyen, Hung V ERDC-ITL-MS wrote:
> All,
>
> I try to test the pc_type of redundant with ksp_type of stcg and cg.
> For both cases, why does the elapsed times are large when running with
> large number of processors?
>
> Thanks for your help.
>
> -Hung
>
> hvnguyen:jade17% head A.m
> % Size = 59409 59409
> % Nonzeros = 1113875
> zzz = zeros(1113875,3);
>
> 1. Test with stcg and redundant:
>
> hvnguyen:jade17% aprun -n 1 ./test_matrix_read -ksp_type stcg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04 0 KSP Residual norm
> 1.117929341660e-06 0 KSP Residual norm 2.394961013957e-16 Time in
> PETSc solver: 26.436834 seconds
> The number of iteration = 2
> The solution residual error = 2.394961e-16
> 2 norm 1.848728e-08
> infinity norm 4.014424e-10
> 1 norm 2.288013e-06
>
> hvnguyen:jade17% aprun -n 2 ./test_matrix_read -ksp_type stcg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04 0 KSP Residual norm
> 1.108986752645e-06 0 KSP Residual norm 2.388254585478e-16 Time in
> PETSc solver: 28.498422 seconds
> The number of iteration = 2
> The solution residual error = 2.388255e-16
> 2 norm 1.763764e-08
> infinity norm 3.500773e-10
> 1 norm 2.543252e-06
>
> hvnguyen:jade17% aprun -n 4 ./test_matrix_read -ksp_type stcg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04 0 KSP Residual norm
> 1.076617230240e-06 0 KSP Residual norm 2.364985060528e-16 Time in
> PETSc solver: 34.782896 seconds
> The number of iteration = 2
> The solution residual error = 2.364985e-16
> 2 norm 3.544232e-08
> infinity norm 4.848555e-10
> 1 norm 5.478173e-06
>
> hvnguyen:jade17% setenv MPICH_UNEX_BUFFER_SIZE 100M
> hvnguyen:jade17% setenv MPICH_PTL_MATCH_OFF 1
> hvnguyen:jade17% aprun -n 8 ./test_matrix_read -ksp_type stcg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04 0 KSP Residual norm
> 1.076187797630e-06 0 KSP Residual norm 2.327317778542e-16 Time in
> PETSc solver: 34.810215 seconds
> The number of iteration = 2
> The solution residual error = 2.327318e-16
> 2 norm 3.448040e-08
> infinity norm 4.582703e-10
> 1 norm 5.306073e-06
>
> hvnguyen:jade17% aprun -n 16 ./test_matrix_read -ksp_type stcg
> -pc_type redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0
> KSP Residual norm 1.573381744277e+04 0 KSP Residual norm
> 1.170520566185e-06 0 KSP Residual norm 2.480424058413e-16 Time in
> PETSc solver: 34.851701 seconds
> The number of iteration = 2
> The solution residual error = 2.480424e-16
> 2 norm 6.318255e-08
> infinity norm 8.093555e-10
> 1 norm 9.681583e-06
>
>
> hvnguyen:jade17% aprun -n 32 ./test_matrix_read -ksp_type stcg
> -pc_type redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0
> KSP Residual norm 1.573381744277e+04 0 KSP Residual norm
> 1.072600099318e-06 0 KSP Residual norm 2.340608377807e-16 Time in
> PETSc solver: 34.951315 seconds
> The number of iteration = 2
> The solution residual error = 2.340608e-16
> 2 norm 3.767959e-08
> infinity norm 5.213003e-10
> 1 norm 5.790317e-06
>
>
> 2. Test with cg and redundant:
>
> hvnguyen:jade17% aprun -n 1 ./test_matrix_read -ksp_type cg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04
> 1 KSP Residual norm 1.117929341660e-06
> 2 KSP Residual norm 2.394961013957e-16 Time in PETSc solver:
> 26.460811 seconds
> The number of iteration = 2
> The solution residual error = 2.394961e-16
> 2 norm 1.848728e-08
> infinity norm 4.014424e-10
> 1 norm 2.288013e-06
>
> hvnguyen:jade17% aprun -n 2 ./test_matrix_read -ksp_type cg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04
> 1 KSP Residual norm 1.108986752645e-06
> 2 KSP Residual norm 2.388254585478e-16 Time in PETSc solver:
> 28.494262 seconds
> The number of iteration = 2
> The solution residual error = 2.388255e-16
> 2 norm 1.763764e-08
> infinity norm 3.500773e-10
> 1 norm 2.543252e-06
>
> hvnguyen:jade17% aprun -n 4 ./test_matrix_read -ksp_type cg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04
> 1 KSP Residual norm 1.076617230240e-06
> 2 KSP Residual norm 2.364985060528e-16 Time in PETSc solver:
> 34.746018 seconds
> The number of iteration = 2
> The solution residual error = 2.364985e-16
> 2 norm 3.544232e-08
> infinity norm 4.848555e-10
> 1 norm 5.478173e-06
>
> hvnguyen:jade17% aprun -n 8 ./test_matrix_read -ksp_type cg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04
> 1 KSP Residual norm 1.076187797630e-06
> 2 KSP Residual norm 2.327317778542e-16 Time in PETSc solver:
> 34.830512 seconds
> The number of iteration = 2
> The solution residual error = 2.327318e-16
> 2 norm 3.448040e-08
> infinity norm 4.582703e-10
> 1 norm 5.306073e-06
>
> hvnguyen:jade17% aprun -n 16 ./test_matrix_read -ksp_type cg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04
> 1 KSP Residual norm 1.170520566185e-06
> 2 KSP Residual norm 2.480424058413e-16 Time in PETSc solver:
> 34.866815 seconds
> The number of iteration = 2
> The solution residual error = 2.480424e-16
> 2 norm 6.318255e-08
> infinity norm 8.093555e-10
> 1 norm 9.681583e-06
>
> hvnguyen:jade17% aprun -n 32 ./test_matrix_read -ksp_type cg -pc_type
> redundant -ksp_rtol 1.0e-12 -ksp_max_it 50000 -ksp_monitor 0 KSP
> Residual norm 1.573381744277e+04
> 1 KSP Residual norm 1.072600099318e-06
> 2 KSP Residual norm 2.340608377807e-16 Time in PETSc solver:
> 34.968400 seconds
> The number of iteration = 2
> The solution residual error = 2.340608e-16
> 2 norm 3.767959e-08
> infinity norm 5.213003e-10
> 1 norm 5.790317e-06
>
>
>
More information about the petsc-users
mailing list