[petsc-dev] dynamic loading

[Lisandro Dalcin]

On 28 July 2010 23:04, Barry Smith <bsmith at mcs.anl.gov> wrote:
>
> On Jul 28, 2010, at 8:40 PM, Lisandro Dalcin wrote:
>
>> On 28 July 2010 21:33, Barry Smith <bsmith at mcs.anl.gov> wrote:
>>>
>>> On Jul 28, 2010, at 6:03 PM, Lisandro Dalcin wrote:
>>>
>>>> Could any of you tell me exactly the intention of
>>>> "--with-dynamic-loading" in the context of PETSc?
>>>>
>>>> I tend to think that dynamic loading of user-defined stuff should
>>>> always be supported if dlopen() or equivalent calls are available. For
>>>> example, I'm able to build Python from sources with static libraries,
>>>> but I can still use Python .so extension modules loaded via dlopen().
>>>> So usage of dynamic loading is somewhat orthogonal to static/shared
>>>> libraries.
>>>>
>>>> If PETSc-builtin Vec/Mat/KSP/etc subtypes implementations could be
>>>> built on separate .so files intended to be loaded at runtime,
>>>
>>>    This is "in theory" possible but requires changes to the "buildsystem" that compiles code and sticks it into libraries.
>>>
>>
>> I understand. This is somewhat hard to make it robust across platforms.
>>
>>>> then
>>>> --with-dynamic-loading would ask for such kind of build. And then a
>>>> normal build would be just like now (let say, single petsc.so library
>>>> with everything bundled). However, regardless of the way PETSc would
>>>> be built, users should still be able to dlopen() modules and register
>>>> custom subtypes at runtime, even if PETSc was built with static
>>>> libraries!!
>>>
>>>   We should be able to support this with a small number of changes. Currently the flag PETSC_USE_DYNAMIC_LIBRARIES is used for two purposes:
>>>
>>> 1) that KSPRegisterDynamic() eliminates the function pointer so PETSc registers the library and string name and not directly the function pointer. Then the functions are accessed via sym()
>>>
>>> 2) the function registration stuff that keeps the registered function pointers and libraries (if PETSC_USE_DYNAMIC_LIBRARIES is off then managing the library access is turned off.
>
>   We actually already have a PETSC_HAVE_DYNAMIC_LIBRARIES so I have changed PetscFListFind() etc to always build the dynamic library access parts when dynamic libraries work even if PETSc is not built with dynamic libraries.  Now you can dynamically load types with static PETSc (let me know if you have problems).
>
>
>>>
>>
>> I never fully understood the rationale of the current implementation.
>> IMHO, PETSc should NEVER dlopen()&dlsym() itself, and the function
>> pointer should NEVER be eliminated. Every time you lookup for a
>> XXXCreate_YYY routine, then you first try to find it in the DLL
>> library list (by default empty, unless users append to it). If the
>> search success, it means the user wanted to override the PETSc
>> implementation. If the search fails (remember I'm proposing
>> libpetsc.so not being in the library list), then just use the stored
>> function pointer.
>
>  The rationale is that we have intended for well over a decade to separate the "interface" part of PETSc from the implementation. This means all the particular instances would go into a different (or several different) dynamic libraries from libpetsc.[so/a] We did all the logic for dynamic loading etc but never separate the two parts of PETSc into independent parts.
>

Ah! OK. I understand now.

>
>>
>>>  If we introduce PETSC_HAVE_DLOPEN_OR_WINDOWS_VERSION that we use in the underlying code then I think we will get the effect you want.
>>>
>>
>> Yes, but I still think that we should review the usages of
>> XXXRegister()/XXXRegisterDynamic() and the somewhat nonsensical
>> auto-dlopen of libpetsc.so
>>
>>
>> Ah! Let me add something. Regarding dynamic loading, we could do more
>> or less the same that Python does for loading extension modules.
>> Instead of dlsym() onto libs for every XXXCreate_YYY routine, we
>> should instead lookup right after dlopen() lib<name>.so for a
>> PetscInitialize_<name>() and call it (anyway, PETSc currently does
>> something like this). That entry-point routine would be in charge of
>> calling any XXXRegister() for user-provided subtypes of Vec/Mat/KSP,
>> etc.
>
>   We want the flexibility of not requiring the user to actually register their function, just stick the function into the library and then use it. -ksp_type libraryname.so:mycreatefunction this is why we also look for the raw symbol.
>

Yep, you are right, that's very convenient.

>
>> The only thing that I do not like about this is coupling the file
>> name lib<name>.so to the PetscInitialize_<name>() symbol. Anyway, we
>> could implement this more or less independently o all the current
>> stuff, think about a PetscImportModule("modname"), then we are free to
>> decide the filename convention <prefix?><name><suffix?>.so and the
>> naming convention for the PetscInitialize_<name>() entry point...
>>
>  I cannot remember why I had the function name depend on the library name but I know I had a good reason for it.
>

PETSc dlopen()'s libraries with RTLD_GLOBAL. Then the entry points
should have different names to avoid collisions in the global symbol
namespace.


-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169