[petsc-dev] Modify 3rd party lib

[Mark Adams]

Sherry, I found the problem.

I added this print statement to dDestroy_LU

    nb = CEILING(nsupers, grid->npcol);
    for (i = 0; i < nb; ++i)
if ( Llu->Lrowind_bc_ptr[i] ) {

*  fprintf(stderr,"dDestroy_LU: GPU free Llu->Lnzval_bc_ptr[%d/%d] = %p,
CPU free Llu->Lrowind_bc_ptr =
%p\n",i,nb,Llu->Lnzval_bc_ptr[i],Llu->Lrowind_bc_ptr[i]);*    SUPERLU_FREE
(Llu->Lrowind_bc_ptr[i]);
#ifdef GPU_ACC
   checkCuda(cudaFreeHost(Llu->Lnzval_bc_ptr[i]));
#else
   SUPERLU_FREE (Llu->Lnzval_bc_ptr[i]);
#endif
}

And I see:

   1 SNES Function norm 1.245977692562e-04

*dDestroy_LU: GPU free Llu->Lnzval_bc_ptr[0/134] = 0x4ff9b000, CPU free
Llu->Lrowind_bc_ptr = 0x4ff9a000*ex112d: cudahook.cc:762: CUresult
host_free_callback(void*): Assertion `cacheNode != __null' failed.

THis looks like Lnzval_bc_ptr is on the CPU so I removed the GPU_ACC stuff
and it works now.

I see this in distribution. Perhaps this a serial run bug?

On Sun, Apr 19, 2020 at 5:58 PM Xiaoye S. Li <xsli at lbl.gov> wrote:

> Mark,
> you should fork a branch of your own to do this.
>
> Sherry
>
> On Sun, Apr 19, 2020 at 2:54 PM Stefano Zampini <stefano.zampini at gmail.com>
> wrote:
>
>> First, commit your changes to the superlu_dist branch, then rerun
>> configure with
>>
>> —download-superlu_dist-commit=HEAD
>>
>>
>> > On Apr 20, 2020, at 12:50 AM, Mark Adams <mfadams at lbl.gov> wrote:
>> >
>> > I would like to modify SuperLU_dist but if I change the source and
>> configure it says no need to reconfigure, use --force. I use --force and it
>> seems to clobber my changes. Can I tell configure to use build but not
>> download SuperLU?
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20200419/697230e3/attachment.html>