[petsc-dev] Why --with-dynamic-loading?

[Barry Smith]

On Aug 29, 2013, at 9:07 AM, Jed Brown <jedbrown at mcs.anl.gov> wrote:

> Lisandro asks why dynamic loading needs a different initialization procedure
> 
> #if !defined(PETSC_USE_DYNAMIC_LIBRARIES)
>  ierr = KSPInitializePackage();CHKERRQ(ierr);
> #endif
> 
> Meanwhile, we use normal shared libraries (--with-dynamic-loading=0)
> with plugins (petsc4py dynamically loads PETSc and the petsc-eig plugin
> is loaded by PETSc and calls back into SLEPc and PETSc).
> 
> What is the use case that --with-dynamic-loading is intended to support?

   One early plan was that all the PETSc impls would be in plug-in libraries; so for example the CG code could be in libpetsckspimpl.dylib etc.  Thus requires some refactoring of the code, we've always been too lazy to do, for example the solver specific interface functions like KSPGMRESSetRestart() have to be completely in header files or in the libpetscksp.dylib and this would require moving lots of stuff around. Putting KSPGMRESSetRestart() in the base library and the GMRES in the plugin library is also kind of silly since KSPGMRESSetRestart() is specific to GMRES and really doesn't belong in the base library. And note that even some logging stuff is specific to a certain solver and thus "properly" should be refactored to be an add-on plugin and not in the base code. In a dreamland we could support perhaps multiple precisions in the same application by loading some single impls and some double but I've never figured out a way to do that reasonably.

> 
> Can we support that use case without having the configuration option?

   
   The reason the PetscInitialize_DynamicLibraries()  is called in PetscInitialize() instead of delayed until needed is if dynamic loading required some collective MPI action rather than something each MPI process can do completely asynchronously. Of course we have no experience with that case.

   I would say we currently don't really have a use for the --with-dynamic-loading it is just a stand in for future possibilities :-)

   We could drop the configure option and add a runtime option -dynamic_loading that triggered the loads in PetscInitialize()

    We could also just leave things as they are and say "why are you building things using --with-dynamic-libraries? there is nothing to be gained"

Also, given that the first line of KSPInitializePackage is:
 if (KSPPackageInitialized) PetscFunctionReturn(0);
and PetscDLLibraryRegister_petscksp has already called
KSPInitializePackage, isn't the preprocessor guard redundant?

   Yes, I think the guard is now redundant and could be removed. This stuff evolved over time without ever having a really strong case for dynamic libraries except future possibilities. 

   Barry