possible new problem: dynlibs retrieved twice, need advice for fix

[Lisandro Dalcin]

Barry, your arguments were really convincing. So I would propose the
following API changes:

1) Private Interface:

    1.a) PetscDLLibraryRetrieve(comm, path, ... ):
           Implements automatic file retrieval.
    1.b) PetscDLLibraryOpen(comm, path, flags, &entry):
            - call 1.a for file retrieval
            - next dlopen()/LoadLibrary() (flags are for dlopen)
            - and finally malloc and fill a new PetscDLLibrary entry.
    1.c) PetscDLLibraryClose(comm, &entry):
           - first dlclose()/FreeLibrary()
           - and then free then PetscDLLibrary entry allocated in 1.b
           *change: this is the oposite to 1.b

1) Public Interface

2.a) PetscDLLibrary:
        a list of loaded dynlibs
2.b) PetscDLLibrary{Append|Prepend}(comm, list, path, flags)
       automatic file retrieval + dlopen()/LoadLibrary() +
append/prepend in list
       * change: add arg for dlopen() flags RTLD_{LAZY|NOW}/{LOCAL_GLOBAL}
2.c) PetscDLLibrarySym(comm, list, symbol, &value):
       look for symbol in list (and also program code if not found in list)
       *change: path removed, use 1.b for adding new libs to the list.
2.d) PetscDLLibraryClear(comm, &list):
       free list entries calling 1.c, and finally set list to NULL.
       *change 1: former name was PetscDLLibraryClose
       *change 2: trully call dlclose()/FreeLibrary() on handles.

Some additional notes:
- In some cases, the comm argument is unused, but let it there just in
case we need it in the near future.
- As I know you hate abreviations, perhaps we could rename
PetscDLLibrarySym to PetscDLLibraryLookSymbol ?

That's all :-) . What do you think? Feel free to suggest better names
for the routines, you are far better than me for that.


On Tue, Oct 21, 2008 at 3:59 PM, Barry Smith <bsmith at mcs.anl.gov> wrote:
> Lisandro,
>
>   I've briefly reviewed the code to refresh my memory on how it work. Here
> is
> my interpretation of the present structure.
>
>   Public interface
>       PetscDLLibraryAppend() and Prepend()
>       PetscDLLibrarySym()
>       PetscDLLibraryClose()
>
>   "Private" interface
>       PetscDLLibraryOpen() (note that this is not really the opposite of
> PetscDLLibraryClose()
>       PetscDLLibraryRetrieve()
>
> These are intended as THE replacement to dlopen(), dlsym(), dlclose(). Hence
> having
> a PetscDLOpen(), PetscDLSym(), PetscDLClose() would be redundant. You know I
> hate
> to have two different ways to do the same thing.
>
> _n_PetscDLLibrary is then a REPLACEMENT for the void** handle you get from
> dlopen(),
> but it has the additional functionality that several handles can be kept and
> used together.
> PetscDLLibraryAppend() and Prepend() are then THE replacement for dlopen()
> and
> PetscDLLibrarySym() is THE replacement for dlsym().
>
> PetscDLLibrarySym() is a replacement for dlsym() with the flexibility to
> find a symbol is
> several handles/files and to look in a particular file for the symbol. The
> idea is Unix paths,
> when I run cc it looks for cc in that path and when I run /bin/cc I tell it
> which particular one
> to use.
>
> The ALL have the IMPORTANT dependence on a MPI_Comm, that dlopen(),... do
> not.
>
> This is all independent of whether one is using PETSc dynamic libraries or
> not , except I
> have the single PETSC_HAVE_DYNAMIC flag that turns everything on. The entire
> dl.c
> file should be TURNED on if the PETSC_HAVE_DYNAMIC flag is on.
>
> Traditional dlopen(), dlsym() dlclose() can be emulated with the above using
> PetscDLLibrary *library = PETSC_NULL
> PetscDLLibraryAppend(.... &library,filename)
> PetscDLLibrarySym(....library, symbolname)
> PetscDLLibrarySym on other symbols
> PetscDLLibraryClose(library)
>
> ----------------------------------------------------------------------------------------------------------------------
>
> Now for PETSc's dynamic libraries, PETSc maintains A _n_PetscDLLibrary
> object for managing ITS dynamic libraries.
> this is handled through the PetscLoadDynamicLibrary() interface.  This
> interface should
> be turned on ONLY if PETSC_USE_DYNAMIC libraries is turned on.
>
> --------------------------------------------------------------------------------------------------------------------
>
> My question to you is then: what changes/additions/fixes do we need to add
> to the
> PetscDLLibraryAppend() and Prepend()      PetscDLLibrarySym()
> PetscDLLibraryClose()
> public interface so that it may be used for all of your needs???? Ok, the
> current interface
> may be a little clumsy, let's make it less clumsy rather than adding a whole
> new PetscDLOpen()...
> interface next to it that has the same functionality. I'm even fine with
> changing names and adding
> new methods to the current interface.
>
>
>  Barry
>
>
>
> On Oct 21, 2008, at 10:04 AM, Lisandro Dalcin wrote:
>
>> Barry, I catched a new problem. At PetscInitialize(), dynamic
>> libraries are 'retrieved' twice.  The calls that leads to this are the
>> following:
>>
>> PetscInitialize()
>> PetscInitialize_DynamicLibraries()
>> PetscLoadDynamicLibrary() // helper routine for opening each  libpescXXX
>> lib
>> PetscDLLibraryRetrieve() // FIRST call
>> PetscDLLibraryAppend()
>> PetscDLLibraryOpen()
>> PetscDLLibraryRetrieve() // SECOND call
>
>  That first use of Retrieve() is probably not needed and should be removed.
>
>>
>>
>> However, I'm not sure if this is a real issue. It seems that
>> ./bin/urlget.py is smart enough as to skip downloading based on
>> timestamps (though I'm not sure if the same applies to the wget
>> fallback in 'bin/urlget' shell script)
>>
>>
>> --
>> Lisandro Dalcín
>> ---------------
>> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
>> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
>> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
>> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>>
>
>



-- 
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594