[petsc-dev] Why --with-dynamic-loading?

[Barry Smith]

On Aug 29, 2013, at 9:41 AM, Barry Smith <bsmith at mcs.anl.gov> wrote:

> 
> On Aug 29, 2013, at 9:07 AM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> 
>> Lisandro asks why dynamic loading needs a different initialization procedure
>> 
>> #if !defined(PETSC_USE_DYNAMIC_LIBRARIES)
>> ierr = KSPInitializePackage();CHKERRQ(ierr);
>> #endif
>> 
>> Meanwhile, we use normal shared libraries (--with-dynamic-loading=0)
>> with plugins (petsc4py dynamically loads PETSc and the petsc-eig plugin
>> is loaded by PETSc and calls back into SLEPc and PETSc).
>> 
>> What is the use case that --with-dynamic-loading is intended to support?
> 
>   One early plan was that all the PETSc impls would be in plug-in libraries; so for example the CG code could be in libpetsckspimpl.dylib etc.  Thus requires some refactoring of the code, we've always been too lazy to do, for example the solver specific interface functions like KSPGMRESSetRestart() have to be completely in header files or in the libpetscksp.dylib and this would require moving lots of stuff around. Putting KSPGMRESSetRestart() in the base library and the GMRES in the plugin library is also kind of silly since KSPGMRESSetRestart() is specific to GMRES and really doesn't belong in the base library. And note that even some logging stuff is specific to a certain solver and thus "properly" should be refactored to be an add-on plugin and not in the base code. In a dreamland we could support perhaps multiple precisions in the same application by loading some single impls and some double but I've never figured out a way to do that reasonably.
> 

   I forgot a couple of things:

>> 
>> Can we support that use case without having the configuration option?
> 
> 
>   The reason the PetscInitialize_DynamicLibraries()  is called in PetscInitialize() instead of delayed until needed is if dynamic loading required some collective MPI action rather than something each MPI process can do completely asynchronously. Of course we have no experience with that case.
> 
>   I would say we currently don't really have a use for the --with-dynamic-loading it is just a stand in for future possibilities :-)
> 
>   We could drop the configure option and add a runtime option -dynamic_loading that triggered the loads in PetscInitialize()

  Originally  BuildSystem should have checked that dynamic libraries worked. But now we have PETSC_HAVE_DYNAMIC_LIBRARIES that checks if dynamic libraries work and and PETSC_USE_DYNAMIC_LIBRARIES that indicate PETSc should use them. The USE requires that the HAVE checked passed. So we would only support the -dynamic_library flag if HAVE is true.



> 
>    We could also just leave things as they are and say "why are you building things using --with-dynamic-libraries? there is nothing to be gained"
> 
> Also, given that the first line of KSPInitializePackage is:
> if (KSPPackageInitialized) PetscFunctionReturn(0);
> and PetscDLLibraryRegister_petscksp has already called
> KSPInitializePackage, isn't the preprocessor guard redundant?
> 
>   Yes, I think the guard is now redundant and could be removed. This stuff evolved over time without ever having a really strong case for dynamic libraries except future possibilities.

   Without the guard the symbols such as KSPRegisterAll() etc are directly visible at link time and this means all the symbols in all the implementations are visible. With the guard KSPRegisterAll() is only seen AFTER the dynamic load (and all symbols it knows about). The idea is that impls symbols are delayed in the dynamic case.  Now does anyone care about this, I don't know? 


> 
> 
>   Barry
> 
>