[petsc-users] problem in using MatCreateMPIAIJWithArrays routine
Priyank Patel
ppate024 at odu.edu
Tue Aug 12 20:02:36 CDT 2014
Hello,
I am trying to do sparse - dense matrix multiplication and for that I am using a CSR Matrix(that are three array) in PETSC using the function MatCreateMPIAIJWithArrays() to create a petsc matrix.
My call to routine is as below
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
//Initialize PETSC
MPI_Init(&argc, &argv);
PetscInitialize(&argc, &argv, (char*) 0, help);
MPI_Comm_rank(PETSC_COMM_WORLD, &taskid);
MPI_Comm_size(PETSC_COMM_WORLD, &numtasks);
//Create A Sparse Matrix fro the files
ierr = MatCreateMPIAIJWithArrays(PETSC_COMM_WORLD, m, m, PETSC_DETERMINE, PETSC_DETERMINE, h_iA, h_jA, h_valA, &A);CHKERRQ(ierr);
//ierr = MatCreateSeqAIJWithArrays(PETSC_COMM_WORLD, m, m, h_iA, h_jA, h_valA, &A);CHKERRQ(ierr);
MatView(A,PETSC_VIEWER_STDOUT_WORLD);
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
My make file is as below
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
PETSC_DIR=/export/software/petsc-3.3
INTEL_ROOT=/export/software/intel/composerxe-2011.4.191/
MKL_LIB_PATH_64=/export/software/intel/mkl/lib/intel64/
INTEL_LIB_PATH_64=/export/software/intel/composerxe-2011.4.191/compiler/lib/intel64/
MKL_INC_PATH=/export/software/intel/mkl/include/
SCALAPACK_LIB_PATH=/export/software/scalapack-2.0.2/lib/
SCALAPACK_INC_PATH=/export/software/scalapack-2.0.2/
GENERAL_LIB_PATH=/export/software/lib/
CLINKER=mpicc
PKG_INC = -I${MKL_INC_PATH}
PKG_LIB = -Wl,-rpath,${MKL_LIB_PATH_64} -L${MKL_LIB_PATH_64} -lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core -Wl,-rpath,${INTEL_LIB_PATH_64} -L${INTEL_LIB_PATH_64} -liomp5 -Wl,-rpath,${SCALAPACK_LIB_PATH} -L${SCALAPACK_LIB_PATH} -lscalapack -llapack -lblas
CFLAGS =
FFLAGS =
CPPFLAGS = ${PKG_INC}
FPPFLAGS =
CLEANFILES =
include ${PETSC_DIR}/conf/variables
include ${PETSC_DIR}/conf/rules
blockCG_1: blockCG_1.o chkopts
-${CLINKER} -o blockCG_1 blockCG_1.o ${PKG_LIB} ${PETSC_LIB}
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
And I tried to run the program with following command
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
/opt/openmpi/bin/mpirun -np 3 ./blockCG_1 /home/ppatel/MPI/LF10_2
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
but the routine is not distributing the matrix and give me the result as below.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
loading matrix from : /home/ppatel/MPI/LF10_2
nRows: 18, nCoefs: 82.
loading matrix from : /home/ppatel/MPI/LF10_2
nRows: 18, nCoefs: 82.
loading matrix from : /home/ppatel/MPI/LF10_2
nRows: 18, nCoefs: 82.
loading matrix from : /home/ppatel/MPI/LF10_2
nRows: 18, nCoefs: 82.
loading matrix from : /home/ppatel/MPI/LF10_2
nRows: 18, nCoefs: 82.
loading matrix from : /home/ppatel/MPI/LF10_2
nRows: 18, nCoefs: 82.
Matrix Object: 1 MPI processes
type: mpiaij
row 0: (0, 3.53448) (1, -477.155) (2, 1.76724)
row 1: (0, -477.155) (1, 171776) (3, -85887.9) (4, 477.155)
row 2: (0, 1.76724) (2, 7.06896) (3, -477.155) (4, 1.76724)
row 3: (1, -85887.9) (2, -477.155) (3, 171776) (5, -85887.9) (6, 477.155)
row 4: (1, 477.155) (2, 1.76724) (4, 7.06896) (5, -477.155) (6, 1.76724)
row 5: (3, -85887.9) (4, -477.155) (5, 171776) (7, -85887.9) (8, 477.155)
row 6: (3, 477.155) (4, 1.76724) (6, 7.06896) (7, -477.155) (8, 1.76724)
row 7: (5, -85887.9) (6, -477.155) (7, 171776) (9, -85887.9) (10, 477.155)
row 8: (5, 477.155) (6, 1.76724) (8, 7.06896) (9, -477.155) (10, 1.76724)
row 9: (7, -85887.9) (8, -477.155) (9, 171776) (11, -85887.9) (12, 477.155)
row 10: (7, 477.155) (8, 1.76724) (10, 7.06896) (11, -477.155) (12, 1.76724)
row 11: (9, -85887.9) (10, -477.155) (11, 171776) (13, -85887.9) (14, 477.155)
row 12: (9, 477.155) (10, 1.76724) (12, 7.06896) (13, -477.155) (14, 1.76724)
row 13: (11, -85887.9) (12, -477.155) (13, 171776) (15, -85887.9) (16, 477.155)
row 14: (11, 477.155) (12, 1.76724) (14, 7.06896) (15, -477.155) (16, 1.76724)
row 15: (13, -85887.9) (14, -477.155) (15, 171776) (17, 477.155)
row 16: (13, 477.155) (14, 1.76724) (16, 7.06896) (17, 1.76724)
row 17: (15, 477.155) (16, 1.76724) (17, 3.53448)
row 18: (0, 3.53448) (1, -477.155) (2, 1.76724)
row 19: (0, -477.155) (1, 171776) (3, -85887.9) (4, 477.155)
row 20: (0, 1.76724) (2, 7.06896) (3, -477.155) (4, 1.76724)
row 21: (1, -85887.9) (2, -477.155) (3, 171776) (5, -85887.9) (6, 477.155)
row 22: (1, 477.155) (2, 1.76724) (4, 7.06896) (5, -477.155) (6, 1.76724)
row 23: (3, -85887.9) (4, -477.155) (5, 171776) (7, -85887.9) (8, 477.155)
row 24: (3, 477.155) (4, 1.76724) (6, 7.06896) (7, -477.155) (8, 1.76724)
row 25: (5, -85887.9) (6, -477.155) (7, 171776) (9, -85887.9) (10, 477.155)
row 26: (5, 477.155) (6, 1.76724) (8, 7.06896) (9, -477.155) (10, 1.76724)
row 27: (7, -85887.9) (8, -477.155) (9, 171776) (11, -85887.9) (12, 477.155)
row 28: (7, 477.155) (8, 1.76724) (10, 7.06896) (11, -477.155) (12, 1.76724)
row 29: (9, -85887.9) (10, -477.155) (11, 171776) (13, -85887.9) (14, 477.155)
row 30: (9, 477.155) (10, 1.76724) (12, 7.06896) (13, -477.155) (14, 1.76724)
row 31: (11, -85887.9) (12, -477.155) (13, 171776) (15, -85887.9) (16, 477.155)
row 32: (11, 477.155) (12, 1.76724) (14, 7.06896) (15, -477.155) (16, 1.76724)
row 33: (13, -85887.9) (14, -477.155) (15, 171776) (17, 477.155)
row 34: (13, 477.155) (14, 1.76724) (16, 7.06896) (17, 1.76724)
row 35: (15, 477.155) (16, 1.76724) (17, 3.53448)
row 36: (0, 3.53448) (1, -477.155) (2, 1.76724)
row 37: (0, -477.155) (1, 171776) (3, -85887.9) (4, 477.155)
row 38: (0, 1.76724) (2, 7.06896) (3, -477.155) (4, 1.76724)
row 39: (1, -85887.9) (2, -477.155) (3, 171776) (5, -85887.9) (6, 477.155)
row 40: (1, 477.155) (2, 1.76724) (4, 7.06896) (5, -477.155) (6, 1.76724)
row 41: (3, -85887.9) (4, -477.155) (5, 171776) (7, -85887.9) (8, 477.155)
row 42: (3, 477.155) (4, 1.76724) (6, 7.06896) (7, -477.155) (8, 1.76724)
row 43: (5, -85887.9) (6, -477.155) (7, 171776) (9, -85887.9) (10, 477.155)
row 44: (5, 477.155) (6, 1.76724) (8, 7.06896) (9, -477.155) (10, 1.76724)
row 45: (7, -85887.9) (8, -477.155) (9, 171776) (11, -85887.9) (12, 477.155)
row 46: (7, 477.155) (8, 1.76724) (10, 7.06896) (11, -477.155) (12, 1.76724)
row 47: (9, -85887.9) (10, -477.155) (11, 171776) (13, -85887.9) (14, 477.155)
row 48: (9, 477.155) (10, 1.76724) (12, 7.06896) (13, -477.155) (14, 1.76724)
row 49: (11, -85887.9) (12, -477.155) (13, 171776) (15, -85887.9) (16, 477.155)
row 50: (11, 477.155) (12, 1.76724) (14, 7.06896) (15, -477.155) (16, 1.76724)
row 51: (13, -85887.9) (14, -477.155) (15, 171776) (17, 477.155)
row 52: (13, 477.155) (14, 1.76724) (16, 7.06896) (17, 1.76724)
row 53: (15, 477.155) (16, 1.76724) (17, 3.53448)
row 54: (0, 3.53448) (1, -477.155) (2, 1.76724)
row 55: (0, -477.155) (1, 171776) (3, -85887.9) (4, 477.155)
row 56: (0, 1.76724) (2, 7.06896) (3, -477.155) (4, 1.76724)
row 57: (1, -85887.9) (2, -477.155) (3, 171776) (5, -85887.9) (6, 477.155)
row 58: (1, 477.155) (2, 1.76724) (4, 7.06896) (5, -477.155) (6, 1.76724)
row 59: (3, -85887.9) (4, -477.155) (5, 171776) (7, -85887.9) (8, 477.155)
row 60: (3, 477.155) (4, 1.76724) (6, 7.06896) (7, -477.155) (8, 1.76724)
row 61: (5, -85887.9) (6, -477.155) (7, 171776) (9, -85887.9) (10, 477.155)
row 62: (5, 477.155) (6, 1.76724) (8, 7.06896) (9, -477.155) (10, 1.76724)
row 63: (7, -85887.9) (8, -477.155) (9, 171776) (11, -85887.9) (12, 477.155)
row 64: (7, 477.155) (8, 1.76724) (10, 7.06896) (11, -477.155) (12, 1.76724)
row 65: (9, -85887.9) (10, -477.155) (11, 171776) (13, -85887.9) (14, 477.155)
row 66: (9, 477.155) (10, 1.76724) (12, 7.06896) (13, -477.155) (14, 1.76724)
row 67: (11, -85887.9) (12, -477.155) (13, 171776) (15, -85887.9) (16, 477.155)
row 68: (11, 477.155) (12, 1.76724) (14, 7.06896) (15, -477.155) (16, 1.76724)
row 69: (13, -85887.9) (14, -477.155) (15, 171776) (17, 477.155)
row 70: (13, 477.155) (14, 1.76724) (16, 7.06896) (17, 1.76724)
row 71: (15, 477.155) (16, 1.76724) (17, 3.53448)
row 72: (0, 3.53448) (1, -477.155) (2, 1.76724)
row 73: (0, -477.155) (1, 171776) (3, -85887.9) (4, 477.155)
row 74: (0, 1.76724) (2, 7.06896) (3, -477.155) (4, 1.76724)
row 75: (1, -85887.9) (2, -477.155) (3, 171776) (5, -85887.9) (6, 477.155)
row 76: (1, 477.155) (2, 1.76724) (4, 7.06896) (5, -477.155) (6, 1.76724)
row 77: (3, -85887.9) (4, -477.155) (5, 171776) (7, -85887.9) (8, 477.155)
row 78: (3, 477.155) (4, 1.76724) (6, 7.06896) (7, -477.155) (8, 1.76724)
row 79: (5, -85887.9) (6, -477.155) (7, 171776) (9, -85887.9) (10, 477.155)
row 80: (5, 477.155) (6, 1.76724) (8, 7.06896) (9, -477.155) (10, 1.76724)
row 81: (7, -85887.9) (8, -477.155) (9, 171776) (11, -85887.9) (12, 477.155)
row 82: (7, 477.155) (8, 1.76724) (10, 7.06896) (11, -477.155) (12, 1.76724)
row 83: (9, -85887.9) (10, -477.155) (11, 171776) (13, -85887.9) (14, 477.155)
row 84: (9, 477.155) (10, 1.76724) (12, 7.06896) (13, -477.155) (14, 1.76724)
row 85: (11, -85887.9) (12, -477.155) (13, 171776) (15, -85887.9) (16, 477.155)
row 86: (11, 477.155) (12, 1.76724) (14, 7.06896) (15, -477.155) (16, 1.76724)
row 87: (13, -85887.9) (14, -477.155) (15, 171776) (17, 477.155)
row 88: (13, 477.155) (14, 1.76724) (16, 7.06896) (17, 1.76724)
row 89: (15, 477.155) (16, 1.76724) (17, 3.53448)
row 90: (0, 3.53448) (1, -477.155) (2, 1.76724)
row 91: (0, -477.155) (1, 171776) (3, -85887.9) (4, 477.155)
row 92: (0, 1.76724) (2, 7.06896) (3, -477.155) (4, 1.76724)
row 93: (1, -85887.9) (2, -477.155) (3, 171776) (5, -85887.9) (6, 477.155)
row 94: (1, 477.155) (2, 1.76724) (4, 7.06896) (5, -477.155) (6, 1.76724)
row 95: (3, -85887.9) (4, -477.155) (5, 171776) (7, -85887.9) (8, 477.155)
row 96: (3, 477.155) (4, 1.76724) (6, 7.06896) (7, -477.155) (8, 1.76724)
row 97: (5, -85887.9) (6, -477.155) (7, 171776) (9, -85887.9) (10, 477.155)
row 98: (5, 477.155) (6, 1.76724) (8, 7.06896) (9, -477.155) (10, 1.76724)
row 99: (7, -85887.9) (8, -477.155) (9, 171776) (11, -85887.9) (12, 477.155)
row 100: (7, 477.155) (8, 1.76724) (10, 7.06896) (11, -477.155) (12, 1.76724)
row 101: (9, -85887.9) (10, -477.155) (11, 171776) (13, -85887.9) (14, 477.155)
row 102: (9, 477.155) (10, 1.76724) (12, 7.06896) (13, -477.155) (14, 1.76724)
row 103: (11, -85887.9) (12, -477.155) (13, 171776) (15, -85887.9) (16, 477.155)
row 104: (11, 477.155) (12, 1.76724) (14, 7.06896) (15, -477.155) (16, 1.76724)
row 105: (13, -85887.9) (14, -477.155) (15, 171776) (17, 477.155)
row 106: (13, 477.155) (14, 1.76724) (16, 7.06896) (17, 1.76724)
row 107: (15, 477.155) (16, 1.76724) (17, 3.53448)
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
that is creating a matrix with rows repeated 6 times instead of distributed over 6 process.
Can you please tell me where I am doing wrong??
Thanks for your time
Priyank Patel
ppate024 at odu.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20140812/1bc98571/attachment-0001.html>
More information about the petsc-users
mailing list