[petsc-dev] dynamic loading

[Lisandro Dalcin]

On 28 July 2010 21:33, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
> On Jul 28, 2010, at 6:03 PM, Lisandro Dalcin wrote:
>
>> Could any of you tell me exactly the intention of
>> "--with-dynamic-loading" in the context of PETSc?
>>
>> I tend to think that dynamic loading of user-defined stuff should
>> always be supported if dlopen() or equivalent calls are available. For
>> example, I'm able to build Python from sources with static libraries,
>> but I can still use Python .so extension modules loaded via dlopen().
>> So usage of dynamic loading is somewhat orthogonal to static/shared
>> libraries.
>>
>> If PETSc-builtin Vec/Mat/KSP/etc subtypes implementations could be
>> built on separate .so files intended to be loaded at runtime,
>
>    This is "in theory" possible but requires changes to the "buildsystem" that compiles code and sticks it into libraries.
>

I understand. This is somewhat hard to make it robust across platforms.

>> then
>> --with-dynamic-loading would ask for such kind of build. And then a
>> normal build would be just like now (let say, single petsc.so library
>> with everything bundled). However, regardless of the way PETSc would
>> be built, users should still be able to dlopen() modules and register
>> custom subtypes at runtime, even if PETSc was built with static
>> libraries!!
>
>   We should be able to support this with a small number of changes. Currently the flag PETSC_USE_DYNAMIC_LIBRARIES is used for two purposes:
>
> 1) that KSPRegisterDynamic() eliminates the function pointer so PETSc registers the library and string name and not directly the function pointer. Then the functions are accessed via sym()
>
> 2) the function registration stuff that keeps the registered function pointers and libraries (if PETSC_USE_DYNAMIC_LIBRARIES is off then managing the library access is turned off.
>

I never fully understood the rationale of the current implementation.
IMHO, PETSc should NEVER dlopen()&dlsym() itself, and the function
pointer should NEVER be eliminated. Every time you lookup for a
XXXCreate_YYY routine, then you first try to find it in the DLL
library list (by default empty, unless users append to it). If the
search success, it means the user wanted to override the PETSc
implementation. If the search fails (remember I'm proposing
libpetsc.so not being in the library list), then just use the stored
function pointer.

>  If we introduce PETSC_HAVE_DLOPEN_OR_WINDOWS_VERSION that we use in the underlying code then I think we will get the effect you want.
>

Yes, but I still think that we should review the usages of
XXXRegister()/XXXRegisterDynamic() and the somewhat nonsensical
auto-dlopen of libpetsc.so


Ah! Let me add something. Regarding dynamic loading, we could do more
or less the same that Python does for loading extension modules.
Instead of dlsym() onto libs for every XXXCreate_YYY routine, we
should instead lookup right after dlopen() lib<name>.so for a
PetscInitialize_<name>() and call it (anyway, PETSc currently does
something like this). That entry-point routine would be in charge of
calling any XXXRegister() for user-provided subtypes of Vec/Mat/KSP,
etc. The only thing that I do not like about this is coupling the file
name lib<name>.so to the PetscInitialize_<name>() symbol. Anyway, we
could implement this more or less independently o all the current
stuff, think about a PetscImportModule("modname"), then we are free to
decide the filename convention <prefix?><name><suffix?>.so and the
naming convention for the PetscInitialize_<name>() entry point...



-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169