[petsc-dev] I hate PetscThreadComm

[Jed Brown]

Barry Smith <bsmith at mcs.anl.gov> writes:

> It is next to impossible to debug situations like this

It is too intrusive and too mixed in with PETSc memory logging.  A
fundamental question we have to deal with is "where do the threads come
from?"  The original thinking was that they would be created
automatically by PETSc and would live after the destruction of the last
object, so that thread pools don't have to be initialized on each solve.
I think we soon agreed that was a bad idea, and perhaps they should come
explicitly from the user or at least only exist when PETSc objects
exist.  But if we use PetscMalloc and someone forgets to destroy an
object, they also get memory leaks from the thread pool.

Back to the central problem: we are trying to make a shared resource
implicit within a library that is not widely-used.  I think it has to be
explicit, but what is a good interface and is it capable of decent
performance?

If MPI came through with endpoints, it would be natural to have each
thread hold a separate handle for Mat, Vec, etc.  I would prefer that
model (the user always creates the threads however they like and calls
PETSc collectively with each thread holding a separate handle).  But in
lieu of endpoints, we would have to wrap the MPI_Comm or have a separate
argument for the thread context.

> Continuing.
> Number of processors = 1, rank = 0
> [0] Jumbled Hello World
> [0]Total space allocated 4896 bytes
> [ 0]16 bytes PetscCommDuplicate() line 151 in /sandbox/bsmith/petsc/src/sys/objects/tagm.c
> [ 0]16 bytes PetscCommDuplicate() line 151 in /sandbox/bsmith/petsc/src/sys/objects/tagm.c
> [ 0]16 bytes PetscThreadCommReductionCreate() line 448 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcommred.c
> [ 0]512 bytes PetscThreadCommReductionCreate() line 440 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcommred.c
> [ 0]128 bytes PetscThreadCommReductionCreate() line 436 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcommred.c
> [ 0]1280 bytes PetscThreadCommReductionCreate() line 435 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcommred.c
> [ 0]32 bytes PetscThreadCommReductionCreate() line 432 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcommred.c
> [ 0]64 bytes PetscThreadCommWorldInitialize() line 1241 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcomm.c
> [ 0]2432 bytes PetscThreadCommWorldInitialize() line 1240 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcomm.c
> [ 0]16 bytes PetscThreadCommWorldInitialize() line 1232 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcomm.c
> [ 0]16 bytes PetscThreadCommSetAffinities() line 423 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcomm.c
> [ 0]48 bytes PetscThreadCommCreate() line 150 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcomm.c
> [ 0]320 bytes PetscThreadCommCreate() line 146 in /sandbox/bsmith/petsc/src/sys/threadcomm/interface/threadcomm.c
> [Inferior 1 (process 31050) exited normally]
>
>
> We should just take it out.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20150113/f71cc544/attachment.sig>