[petsc-users] 3-dimension to matrix

[Joon hee Choi]

Thank you for your reply.
I will calculate the multiplication of block matrices. I made the vector with tuples from dataset and then set up the matrix by reading each tuple. 
And the matrix has 144*10^6 non-zero entries.

Sincerely,
Joon

----- Original Message -----
From: "Jed Brown" <jedbrown at mcs.anl.gov>
To: "Joon hee Choi" <choi240 at purdue.edu>
Cc: petsc-users at mcs.anl.gov
Sent: Friday, May 17, 2013 4:48:28 PM
Subject: Re: [petsc-users] 3-dimension to matrix

Joon hee Choi <choi240 at purdue.edu> writes:

> My petsc version is 3.3.5.
> And I made nnz using the number of each row. When I wrote the matrix
> using MatSetValues, I wrote from low row and column to high row and
> column. However, when I changed the order writing the matrix, it took
> much more time (up to 3hrs). So I am concerned that the slowness is
> because the big size of matrix (2.6*10^7, 1.248*10^15) is related to
> reading from or writing to the cache, ram, or hard.

What are you going to do with a matrix of that dimension?  You can't
apply it to a vector because you can't store the vector.

How many entries does it have?

> Anyway, the following code is the part that I read the data from a
> file and set up tuples and nnz:
>
>         FILE *fp = fopen("data.txt", "r"); 
>         while (fscanf(fp, "%d %d %d %d", &x, &y, &z, &v) == 4) 
>         {
>                 tups.push_back(std::tr1::make_tuple (x-1, z-1, y-1, v)); 
>                 nzrow[i-1] += 1; 
>                 if (x > X) X = x; 
>                 if (y > Y) Y = y; 
>                 if (z > Z) Z = z; 
>         } 
>         fclose(fp);
>         PetscMalloc(X*sizeof(PetscInt), &nnz); 
>         memset(nnz, 0, X); 
>         for (itnz=nzrow.begin(); itnz!=nzrow.end(); ++itnz) { 
>                 nnz[itnz->first] = itnz->second; 

What is this trying to do?  nnz will just be the column of the last
point to be processed...

>         }
>         sort(tups.begin(), tups.end());
>
> If my code is wrong, then please let me know.
>
> Thank you,
> Joon
>
> ----- Original Message -----
> From: "Jed Brown" <jedbrown at mcs.anl.gov>
> To: "Joon hee Choi" <choi240 at purdue.edu>
> Cc: petsc-users at mcs.anl.gov
> Sent: Friday, May 17, 2013 3:24:09 PM
> Subject: Re: [petsc-users] 3-dimension to matrix
>
> Joon hee Choi <choi240 at purdue.edu> writes:
>
>> Thank you for your fast reply. Your last comments looks like my first
>> code. The full size of the matrix is (X, Y*Z). Also, I used SEQAIJ as
>> matrix type and (X, Y) as block size. Also, I implemented
>> SeqAIJpreallocating with nnz. Nevertheless, it was very slow. The
>> Matrix-Set-Up part of the first code is as follows:
>>
>>         sort(tups.begin(), tups.end());
>>         MatCreate(PETSC_COMM_SELF, &A);
>>         MatSetType(A, MATSEQAIJ);
>>         MatSetSizes(A, PETSC_DECIDE, PETSC_DECIDE, X, Y*Z);
>>         MatSetBlockSizes(A, X, Y);
>>         MatSeqAIJSetPreallocation(A, PETSC_DEFAULT, nnz);
>>
>>         sz = tups.size();
>>         for (i=0; i<sz; i++) {
>>                 x = std::tr1::get<0>(tups[i]);
>>                 y = std::tr1::get<2>(tups[i]) + std::tr1::get<1>(tups[i])*Y;
>>                 val = std::tr1::get<3>(tups[i]);
>>                 MatSetValues(A, 1, &x, 1, &y, &val, INSERT_VALUES);
>>         }
>>         MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY);
>>         MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY);
>>
>>
>> I used tuple (x, z, y, value) and vector of c++. I didn't get any
>> errors from this code. However, it took about 9 minutes in this
>> part. 
>
> What version of PETSc?  Your preallocation was almost certainly not
> sufficient.