[petsc-users] Confusion/failures about the tests involved in including Hypre
Daniel Stone
daniel.stone at opengosim.com
Tue Aug 1 04:12:06 CDT 2023
This still isn't working, and it looks like the md/mt distinciton might be
the culprit.
I use the Ninja generator with cmake, on the intel oneapi command line
(i.e. windows command line but with the pathing and other things sorted out
for the new intel compilers).
Looking at the build.ninja file produced, I see rules like:
-------------------
build CMakeFiles\HYPRE.dir\blas\dasum.c.obj: C_COMPILER__HYPRE_Debug
..\blas\dasum.c || cmake_object_order_depends_target_HYPRE
DEFINES = -D_CRT_SECURE_NO_WARNINGS
DEP_FILE = CMakeFiles\HYPRE.dir\blas\dasum.c.obj.d
FLAGS = /DWIN32 /D_WINDOWS /Zi /Ob0 /Od /RTC1 -MDd
INCLUDES = [etc....]
------------------
Note the -MDd flag (which is an -MD flag if I ask for a non debug build).
It's odd that it isn't an /MD flag, but I've done some quick checks on the
oneapi command line and it looks like -MD
is still interpreted as valid input by icx (e.g., >> icx /MD and >> icx -MD
both only complain about lack of input files, while >> icx
/MDblahblahnotarealflag and >> icx -MDblahblahnotarealflag
both give "unknown argument" errors). Thus it really seems like my Hypre
library has been building with the MD option, although MT is the default (
https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2023-0/md.html
and
https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2023-0/mt.html),
as suggested by Satish.
There seems to be no hint of the MD flag in any of the cmake files provided
with hypre, so it looks like the -MD flag is being added by the cmake Ninja
generator as a kind of default (and maybe in a confused way - why the -
instead of /?)
So the task now seems to be to convince cmake to stop doing this.
On Mon, Jul 24, 2023 at 11:21 AM Daniel Stone <daniel.stone at opengosim.com>
wrote:
> On the hypre versioning - aha. For this project I locked the petsc version
> a little while ago (3.19.1), but I've been using a fresh clone of hypre, so
> clearly
> it's too modern a version. Using the appropriate version of hypre (2.28.0,
> according to hypre.py) might fix some things.
> I may have other problems in the form of the confilicting compiler options
> as Satish mentioned, which I'll have to figure out too.
>
> Thanks,
>
> Daniel
>
> On Fri, Jul 21, 2023 at 9:32 PM Barry Smith <bsmith at petsc.dev> wrote:
>
>>
>>
>> hypre_Error was changed from an integer to a struct in hypre. PETSc code
>> was changed in main to work with that struct and not work if hypre_Error is
>> an int. This means main cannot work with previous hypre versions (where
>> hypre_Error is an int). Sure, instead of changing the minimum version one
>> could potentially change PETSc's main version source code to work with both
>> old and new hypre, but that would need to be done and has not been done.
>> main is simply broken, it allows configure to succeed with an older version
>> of hypre but PETSc main cannot compile with this, thus confusing developers
>> and users. This does need to be fixed, one way or another.
>>
>>
>>
>> > On Jul 21, 2023, at 1:32 PM, Satish Balay <balay at mcs.anl.gov> wrote:
>> >
>> > We don't have a good way to verify if the changes continue to work
>> > with the minver of the pkg [and if minor fixes can get this working -
>> > without increasing this requirement - sure without access to the new
>> > features provided by the new version]
>> >
>> > Having a single version dependency between pkgs makes the whole
>> > ecosystem [where many pkgs with all their dependencies mixed in]
>> > brittle..
>> >
>> > Satish
>> >
>> > On Thu, 20 Jul 2023, Barry Smith wrote:
>> >
>> >>
>> >> Satish,
>> >>
>> >> I think hypre.py in main needs minversion = 2.29 instead of 2.14
>> >>
>> >>
>> >>
>> >>> On Jul 20, 2023, at 11:05 AM, Satish Balay <balay at mcs.anl.gov> wrote:
>> >>>
>> >>> Can check config/BuildSystem/config/packages/hypre.py
>> >>>
>> >>> petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc
>> 'main' branch with 2.29.0
>> >>>
>> >>> Satish
>> >>>
>> >>> On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote:
>> >>>
>> >>>>
>> >>>> You cannot use this version of PETSc, 3.19, with the version of
>> hypre you installed. In hypre they recently changed hypre_Error from an
>> integer to a struct which completely breaks compatibility with previous
>> versions of hypre (and hence previous versions of PETSc). You must use the
>> main git branch of PETSc with the version of hypre you installed.
>> >>>>
>> >>>> Barry
>> >>>>
>> >>>>
>> >>>>> On Jul 20, 2023, at 5:10 AM, Daniel Stone <
>> daniel.stone at opengosim.com> wrote:
>> >>>>>
>> >>>>> Hi All,
>> >>>>>
>> >>>>> Many thanks for the detailed explainations and ideas!
>> >>>>>
>> >>>>> I tried skipping the test. When it came time to do the build itself
>> (make $PETSC_DIR... all) I get some failures, unsurprisingly:
>> >>>>>
>> >>>>> --------------------------------
>> >>>>>
>> >>>>> FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o
>> >>>>> CC
>> arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o
>> >>>>> CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o
>> >>>>> CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o
>> >>>>>
>> C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29):
>> error: assigning to 'hypre_Error' from incompatible type 'int'
>> >>>>> hypre__global_error = 0;
>> >>>>> ^ ~
>> >>>>> C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7):
>> note: expanded from macro 'PetscStackCallExternalVoid'
>> >>>>> __VA_ARGS__; \
>> >>>>> ^~~~~~~~~~~
>> >>>>>
>> C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29):
>> error: assigning to 'hypre_Error' from incompatible type 'int'
>> >>>>> hypre__global_error = 0;
>> >>>>> ^ ~
>> >>>>> C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7):
>> note: expanded from macro 'PetscStackCallExternalVoid'
>> >>>>> __VA_ARGS__; \
>> >>>>> ^~~~~~~~~~~
>> >>>>> 2 errors generated.
>> >>>>> make[3]: *** [gmakefile:195:
>> arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1
>> >>>>> make[3]: *** Waiting for unfinished jobs....
>> >>>>> FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o
>> >>>>> CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o
>> >>>>> CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o
>> >>>>> make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1'
>> >>>>> make[2]: ***
>> [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2
>> >>>>> make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1'
>> >>>>> **************************ERROR*************************************
>> >>>>> Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log
>> >>>>> Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to
>> petsc-maint at mcs.anl.gov <mailto:petsc-maint at mcs.anl.gov>
>> >>>>> ********************************************************************
>> >>>>> Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
>> >>>>>
>> >>>>> ----------------------------------------
>> >>>>>
>> >>>>> But wait - isn't this the compile stage, not the linking stage?
>> This seems to imply that I've made a hash of providing include file such
>> that a definition of "hypre_Error"
>> >>>>> cannot be seen - unless I'm misinterpreting. Interesting note about
>> Hypre and include files - if built using configure and make, all the
>> include files are conviniently copied
>> >>>>> into hypre/src/hypre/include. This is not done for a cmake build -
>> I had to do the copying myself. Maybe I missed one.
>> >>>>>
>> >>>>>
>> >>>>> On shared vs. static - if there a clear way of telling which I've
>> ended up with? I've checked the cmakelists for hypre and this seems to
>> imply that not-shared is the default,
>> >>>>> which I didn't change:
>> >>>>>
>> >>>>> # Configuration options
>> >>>>> option(HYPRE_ENABLE_SHARED "Build a shared library" OFF)
>> >>>>> option(HYPRE_ENABLE_BIGINT "Use long long int for
>> HYPRE_Int" OFF)
>> >>>>> option(HYPRE_ENABLE_MIXEDINT "Use long long int for
>> HYPRE_BigInt, int for HYPRE_INT" OFF)
>> >>>>> [....]
>> >>>>>
>> >>>>>
>> >>>>> checking again, I've noticed that the way that the stub-test fails
>> is different depending on whether it's called from the config script or
>> used in isolation - more details on that soon.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Thanks again,
>> >>>>>
>> >>>>> Daniel
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users <
>> petsc-users at mcs.anl.gov <mailto:petsc-users at mcs.anl.gov>> wrote:
>> >>>>>> I think it should work with static libraries and 64bit compilers.
>> >>>>>>
>> >>>>>> That's how I think --download-f2cblaslapack [etc] work.
>> >>>>>>
>> >>>>>> Also it works with MS-MPI - even-though its a dll install, the
>> library stub provides this symbol somehow..
>> >>>>>>
>> >>>>>> balay at ps5 /cygdrive/c/Program Files (x86)/Microsoft
>> SDKs/MPI/Lib/x64
>> >>>>>> $ nm -Ao msmpi.lib |grep " MPI_Init"
>> >>>>>> msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init
>> >>>>>> msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread
>> >>>>>> msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
>> >>>>>>
>> >>>>>> However - if the library symbol is somehow mangled - this
>> configure mode of checking library functions will fail.
>> >>>>>>
>> >>>>>> Checking PETSc dll build:
>> >>>>>>
>> >>>>>> balay at ps5 ~/petsc/arch-ci-mswin-uni/lib
>> >>>>>> $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays
>> >>>>>> libpetsc.lib:libpetsc.dll:0000000000000000 I
>> __imp_MatCreateSeqAIJWithArrays
>> >>>>>> libpetsc.lib:libpetsc.dll:0000000000000000 T
>> MatCreateSeqAIJWithArrays
>> >>>>>>
>> >>>>>> It also has the unmangled symbol - so I guess this mode can work
>> generally with dlls.
>> >>>>>>
>> >>>>>> Satish
>> >>>>>>
>> >>>>>>
>> >>>>>> On Wed, 19 Jul 2023, Barry Smith wrote:
>> >>>>>>
>> >>>>>>>
>> >>>>>>> Satish,
>> >>>>>>>
>> >>>>>>> So it will always fail on Windows with Windows compilers (both
>> with static and shared libraries)? Is this true for all PETSc external
>> packages? If so, why does the installation documentation say that some
>> external packages can work with Windows compilers? (Presumably PETSc cannot
>> since the configure tests will fail).
>> >>>>>>>
>> >>>>>>> Barry
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> On Jul 19, 2023, at 11:40 AM, Satish Balay <balay at mcs.anl.gov
>> <mailto:balay at mcs.anl.gov>> wrote:
>> >>>>>>>>
>> >>>>>>>> BTW: Some explanation of configure:
>> >>>>>>>>
>> >>>>>>>> It attempts the following on linux:
>> >>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>> Source:
>> >>>>>>>> #include "confdefs.h"
>> >>>>>>>> #include "conffix.h"
>> >>>>>>>> /* Override any gcc2 internal prototype to avoid an error. */
>> >>>>>>>> char HYPRE_IJMatrixCreate();
>> >>>>>>>> static void _check_HYPRE_IJMatrixCreate() {
>> HYPRE_IJMatrixCreate(); }
>> >>>>>>>>
>> >>>>>>>> int main(void) {
>> >>>>>>>> _check_HYPRE_IJMatrixCreate();
>> >>>>>>>> return 0;
>> >>>>>>>> }
>> >>>>>>>> <<<<<<<
>> >>>>>>>>
>> >>>>>>>> Note - it does not include 'HYPRE.h' here - but redefines the
>> prototype as 'char HYPRE_IJMatrixCreate();
>> >>>>>>>>
>> >>>>>>>> Compiling it manually:
>> >>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>> [balay at pj01 petsc]$ cat conftest.c
>> >>>>>>>> char HYPRE_IJMatrixCreate();
>> >>>>>>>> static void _check_HYPRE_IJMatrixCreate() {
>> HYPRE_IJMatrixCreate(); }
>> >>>>>>>>
>> >>>>>>>> int main(void) {
>> >>>>>>>> _check_HYPRE_IJMatrixCreate();
>> >>>>>>>> return 0;
>> >>>>>>>> }
>> >>>>>>>> [balay at pj01 petsc]$ gcc -c conftest.c
>> >>>>>>>> [balay at pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate
>> >>>>>>>> conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate
>> >>>>>>>> conftest.o: U HYPRE_IJMatrixCreate
>> >>>>>>>> [balay at pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so
>> |grep HYPRE_IJMatrixCreate
>> >>>>>>>> arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T
>> HYPRE_IJMatrixCreate
>> >>>>>>>> [balay at pj01 petsc]$
>> >>>>>>>> <<<<
>> >>>>>>>>
>> >>>>>>>> Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T
>> HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure
>> succeeds!
>> >>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>> [balay at pj01 petsc]$ gcc -o conftest conftest.o
>> arch-linux-c-debug/lib/libHYPRE.so
>> >>>>>>>> [balay at pj01 petsc]$ echo $?
>> >>>>>>>> 0
>> >>>>>>>> <<<<<
>> >>>>>>>>
>> >>>>>>>> On windows - [due to name mangling by cdecl/stdcall, (/MT vs
>> /MD) etc..] - this might not match - resulting in link failures.
>> >>>>>>>>
>> >>>>>>>> Satish
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
>> >>>>>>>>
>> >>>>>>>>> You could try skipping this test [and assume
>> --with-hypre-include and --with-hypre-lib options are correct] - and see if
>> this works.
>> >>>>>>>>>
>> >>>>>>>>> diff --git a/config/BuildSystem/config/packages/hypre.py
>> b/config/BuildSystem/config/packages/hypre.py
>> >>>>>>>>> index 5bc88322aa2..2d6c7932e17 100644
>> >>>>>>>>> --- a/config/BuildSystem/config/packages/hypre.py
>> >>>>>>>>> +++ b/config/BuildSystem/config/packages/hypre.py
>> >>>>>>>>> @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage):
>> >>>>>>>>> self.requiresversion = 1
>> >>>>>>>>> self.gitcommit = 'v'+self.version
>> >>>>>>>>> self.download = ['git://
>> https://github.com/hypre-space/hypre','
>> https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz']
>> >>>>>>>>> - self.functions = ['HYPRE_IJMatrixCreate']
>> >>>>>>>>> + self.functions = []
>> >>>>>>>>> self.includes = ['HYPRE.h']
>> >>>>>>>>> self.liblist = [['libHYPRE.a']]
>> >>>>>>>>> self.buildLanguages = ['C','Cxx']
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Satish
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> On Wed, 19 Jul 2023, Barry Smith wrote:
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> You don't indicate what type of libraries you built hypre
>> with; static or shared. My guess is you ended up with shared
>> >>>>>>>>>>
>> >>>>>>>>>> I think the answer to your difficulty is hidden in __cdecl
>> (Satish will know much better than me). When you are looking for symbols in
>> Windows shared libraries you have to prepend something to the function
>> prototype to have it successfully found. For example the PETSc include
>> files have these things __declspec(dllimport) The configure test fails
>> because it does not provide the needed prototype. Likely you built PTScotch
>> with static libraries so no problem.
>> >>>>>>>>>>
>> >>>>>>>>>> The simplest fix would be to build static hypre libraries. I
>> think it is a major project to get PETSc configure and macro system to work
>> properly with external packages that are in Windows shared libraries since
>> more use of __declspec would be needed.
>> >>>>>>>>>>
>> >>>>>>>>>> Barry
>> >>>>>>>>>>
>> >>>>>>>>>> The PETSc installation instructions should probably say
>> something about external packages with Windows shared libraries.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>> On Jul 19, 2023, at 10:52 AM, Daniel Stone <
>> daniel.stone at opengosim.com <mailto:daniel.stone at opengosim.com>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Hello,
>> >>>>>>>>>>>
>> >>>>>>>>>>> I'm working on getting a petsc build running on windows. One
>> necessary package to include is Hypre. I've been able to build Hypre
>> seperately using cmake, and confirmed that the library works
>> >>>>>>>>>>> by setting up a VS project to run some of the example
>> programs.
>> >>>>>>>>>>>
>> >>>>>>>>>>> My attempted petsc build is being done through cygwin. I've
>> been able to (with varying degrees of difficulty), build a fairly plain
>> petsc, and one that downloads and builds ptscotch (after some modifications
>> >>>>>>>>>>> to both ptscotch and the config script). I am now attempting
>> to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc).
>> Note that the same compilers are being used for both Hypre and for petsc
>> >>>>>>>>>>> through cygwin - the new intel oneapi compilers (icx and ifx,
>> after again varying amounts of pain to work around their awkwardness with
>> the config script).
>> >>>>>>>>>>>
>> >>>>>>>>>>> I'm seeing a problem when the config script does some tests
>> on the included hypre lib. The source code looks like:
>> >>>>>>>>>>>
>> >>>>>>>>>>> #include "confdefs.h"
>> >>>>>>>>>>> #include "conffix.h"
>> >>>>>>>>>>> /* Override any gcc2 internal prototype to avoid an error. */
>> >>>>>>>>>>>
>> >>>>>>>>>>> #include "HYPRE.h"
>> >>>>>>>>>>>
>> >>>>>>>>>>> char HYPRE_IJMatrixCreate();
>> >>>>>>>>>>> static void _check_HYPRE_IJMatrixCreate() {
>> HYPRE_IJMatrixCreate(); }
>> >>>>>>>>>>>
>> >>>>>>>>>>> int main() {
>> >>>>>>>>>>> _check_HYPRE_IJMatrixCreate();;
>> >>>>>>>>>>> return 0;
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> As I understand this is a fairly standard type of stub
>> program used by the config script to check that it is able to link to
>> certain symbols in given libraries. Tests like this have succeeded in my
>> builds that
>> >>>>>>>>>>> include PTScotch.
>> >>>>>>>>>>>
>> >>>>>>>>>>> I keep getting a linker error with the above test, including
>> if I seperate it out and try to build it seperately:
>> >>>>>>>>>>>
>> >>>>>>>>>>> unresolved external symbol "char __cdel
>> HYPRE_IJMatrixCreate(void)" ....
>> >>>>>>>>>>>
>> >>>>>>>>>>> Ok, it looks like a problem with either the library or linker
>> commands. But here's the interesting thing - If I transplant this code into
>> VS, with the same project setting that allows it to build the much more
>> >>>>>>>>>>> nontrivial Hypre example programs, I get the same error:
>> >>>>>>>>>>>
>> >>>>>>>>>>> Error LNK2001 unresolved external symbol "char __cdecl
>> HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1
>> C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
>> >>>>>>>>>>>
>> >>>>>>>>>>> So it seems like there might be something about this type of
>> stub program that is not working with my Hypre library. I don't fully
>> understand this program - it's able to call the function with no arguments,
>> but
>> >>>>>>>>>>> it also needs to be linked against a library containing the
>> function, apparently by wrapping it in a static void function? Not
>> something I've seen before.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Does anyone have any insight into what might be going wrong -
>> or really just any explaination of how the stub program works so I can
>> figure out why it isn't in this case?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Many thanks,
>> >>>>>>>>>>>
>> >>>>>>>>>>> Daniel
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>>>
>> >>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/petsc-users/attachments/20230801/bd026e82/attachment-0001.html>
More information about the petsc-users
mailing list