[petsc-users] segfault in MatAssemblyEnd() when using large matrices on multi-core MAC OS-X

[Ronald M. Caplan]

Yes that is correct.  That is the updated code with each node storing its
own values.  See my previous email to Matt for the old version which
segfaults with processors more than 1 and npts =25.

 - Ron

On Mon, Jul 30, 2012 at 4:43 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> I ran
>
> $ mpirun -n 2 ./a.out
>  ---------------------------------------------------
>  Running MAS PETSc tests with:
>  nr:                   23
>  np:                   27
>  nt:                   25
>  N:                 46575
>  num_steps:          2000
>  MPI cores:             2
>  ---------------------------------------------------
>  MPI TEST:  My rank is:           0
>  MPI TEST:  My rank is:           1
>  Rank            0  has rows            0  to        23288
>  Rank            1  has rows        23288  to        46575
>  Number of non-zero entries in matrix:       690339
>  Computing y=Ax with RON_CSR_AX...
>  ...done!
>  ||y||:     821.67460825997637
>  y(5)=      2.7454534359667053
>  Storing MAS matrix into PETSc matrix...
>  ...done!
>  rank            1  about to call MatAssemblyEnd()...
>  rank            0  about to call MatAssemblyEnd()...
>  Computing y=Ax with PETSc MatMult...
>  ...done!
>  ||y||:     821.67460825997568
>  y(5)=      2.7454534359667053
>  RON_CSR_AX y=Ax time:      201.704498     nsec/mp.
>  PETSc y=Ax time:           198.269424     nsec/mp.
>  PETSc y=Ax flops:         0.417739183     GFLOPS.
>  Did not converge! Number of iteratioins:            0
>
>
> On Mon, Jul 30, 2012 at 3:12 PM, Ronald M. Caplan <caplanr at predsci.com>wrote:
>
>> Attached is the code.  The original code which segfaults with more than
>> one core is the code I sent last week.
>>
>>  - Ron C
>>
>>
>> On Mon, Jul 30, 2012 at 3:09 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
>>
>>> On Mon, Jul 30, 2012 at 3:04 PM, Ronald M. Caplan <caplanr at predsci.com>wrote:
>>>
>>>> I seem to have solved the problem.
>>>>
>>>> I was storing my entire matrix on node 0 and then calling MatAssembly
>>>> (begin and end) on all nodes (which should have worked...).
>>>>
>>>> Apparently I was using too much space for the buffering or the like,
>>>> because when I change the code so each node sets its own matrix values,
>>>> than the MatAssemblyEnd does not seg fault.
>>>>
>>>
>>> Can you send the test case. It shouldn't seg-fault unless the machine
>>> runs out of memory (and most desktop systems have overcommit, so the system
>>> will kill arbitrary processes, not necessarily the job that did the latest
>>> malloc.
>>>
>>> In practice, you should call MatAssemblyBegin(...,MAT_FLUSH_ASSEMBLY)
>>> periodically.
>>>
>>>
>>>>
>>>> Why should this be the case?   How many elements of a vector or matrix
>>>> can a single node "set" before Assembly to distribute over all nodes?
>>>>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20120730/c2042f58/attachment.html>