[petsc-dev] Experimental GNU make build system
Satish Balay
balay at mcs.anl.gov
Thu May 23 10:29:26 CDT 2013
posted to cygwin list - but there is a prior similar post. Appears to
be a windows process launcher issue. :(
http://cygwin.com/ml/cygwin/2013-05/msg00340.html
http://sourceware.org/ml/cygwin/2011-02/msg00416.html
>>>>>>>>>
The problem is that the stack is not created by Cygwin in the first
place, and for some reason the Windows loader appears to need some more
space in the low memory area below the process stack, so the process
stack is moved by 64K. However, for some other reason the Windows
loader doesn't move the stack from it's default location in the forked
child, despite the fact that $PATH hasn't changed. The problem now
is, that the memory area behind the stack is already taken, probably
by same data from one of the loaded system DLLs. So Cygwin's attempt
to match the child stack with the parent stack doesn't work, because
it can't reserve the upper 64K of the stack space of the parent in
the child. Therefore the fork failed, because it's not possible to
reproduce the parent stack.
Beats me why the Windows loader neglects to move the stack in the forked
child even though $PATH hasn't changed. Very puzzeling. I'm not sure
there's a good solution for this problem. After all, there's no way
to start a process and tell the Windows loader where you want the stack.
<<<<<<<<
Satish
On Thu, 23 May 2013, Satish Balay wrote:
> I can reproduce the issue with the attached makefile.
>
> It appears to be related to the following:
> - make [as the same command from shell works]
> - pipe or multiple commands [i.e 'cmd1|cmd2' or 'cmd1; cmd2']
> - the huge string in one of the commands
> - length of PATH string [works if PATH is not too long?]
> - happens on win 2003/x32 server, and win2008/x64 server - but not win7/x32?
>
> Perhaps its a cygwin bug - or the servers have some limits that cygwin
> is hitting with this example? wierd.
>
> Satish
>
> -----------------
> sbalay at ps3 ~
> $ echo $PATH
> /home/sbalay/bin:/usr/local/bin:/usr/bin:/usr/local/bin:/usr/bin:/cygdrive/c/Program Files/Microsoft Visual Studio/Common/Tools:/cygdrive/c/Program Files/Microsoft Visual Studio/Common/Msdev98/BIN:/cygdrive/c/Program Files/Microsoft Visual Studio/DF98/BIN:/cygdrive/c/Program Files/Microsoft Visual Studio/VC98/BIN:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/Program Files/Intel/MKL/ia32/bin:/cygdrive/c/Python25:/cygdrive/c/Program Files/TortoiseHg:/usr/lib/lapack:/cygdrive/c/Program Files/MPICH2/bin:/cygdrive/c/Borland/BCC55/Bin
>
> sbalay at ps3 ~
> $ make -f test.mk
> 30621
> e
> sbalay at ps3 ~
> $ export PATH=$PATH:$PATH
>
> sbalay at ps3 ~
> $ make -f test.mk
> 30621
>
> sbalay at ps3 ~
> $ export PATH=$PATH:$PATH
>
> sbalay at ps3 ~
> $ make -f test.mk
> 10 [main] sh 3644 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
> /bin/sh: fork: retry: Resource temporarily unavailable
> 10 [main] sh 2440 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
> /bin/sh: fork: retry: Resource temporarily unavailable
> 10 [main] sh 2120 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
> /bin/sh: fork: retry: Resource temporarily unavailable
> 10 [main] sh 620 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
> /bin/sh: fork: retry: Resource temporarily unavailable
> 10 [main] sh 2548 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
> /bin/sh: fork: Resource temporarily unavailable
> test.mk:2: recipe for target `all' failed
> make: *** [all] Error 254
>
> sbalay at ps3 ~
> $
>
>
>
>
> On Wed, 22 May 2013, Satish Balay wrote:
>
> > I don't think its a rebase issue. [cygwin setup runs rbaseall
> > successfully if no cygwin process is running during upgrade - and I
> > make sure of that. To be sure - I reran rebaseall again]
> >
> > [and I haven't seen this message with all-legacy builds which create
> > scores of processes recursively]
> >
> > And I mistakingly assumed its a '-j 8' vs '-j 1' issue. I see the
> > issue with 'make' but not with 'make V=1'. And it happens consistantly
> > at the exact same stage - either at:
> > CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so
> > or at:
> > AR /home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib
> >
> > This is strange..
> >
> > Satish
> > --------
> >
> > balay at msnehalem2 ~/petsc.clone
> > $ make -f gmakefile PETSC_ARCH=arch-gmake -j 8
> > Use "/usr/bin/make V=1" to see the verbose compile lines.
> > AR /home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib
> > 0 [main] sh 9560 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 7752 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 6772 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 6584 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 8724 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: Resource temporarily unavailable
> > gmakefile:62: recipe for target `/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib' failed
> > make: *** [/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib] Error 254
> >
> > balay at msnehalem2 ~/petsc.clone
> > $ make -f gmakefile PETSC_ARCH=arch-gmake
> > Use "/usr/bin/make V=1" to see the verbose compile lines.
> > AR /home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib
> > 0 [main] sh 7920 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 7532 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 2932 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 9096 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > 0 [main] sh 5856 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> > /usr/bin/sh: fork: Resource temporarily unavailable
> > gmakefile:62: recipe for target `/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib' failed
> > make: *** [/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib] Error 254
> >
> >
> >
> > > Satish Balay <balay at mcs.anl.gov> writes:
> > >
> > > > Ok. The 'child_info_fork::abort:' error below is due to gnumake somehow
> > > > trying to parallelize 'CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so'?
> > >
> > > What do you mean "trying to parallelize"? It's only running the linker
> > > once.
> > >
> > > Web search yields this, but it may not be relevant.
> > >
> > > http://stackoverflow.com/questions/9300722/cygwin-error-bash-fork-retry-resource-temporarily-unavailable
> > >
> > > > I see the same if I replace it with 'AR'. [here I'm having to use '-j
> > > > 1' for AR part]
> > > >
> > > > I have a successful build with 'make -j 8' - and then 'make -j '1 to
> > > > recover from AR errors. And 'make test' is successful.
> > > >
> > > > my hacky changes to get this working is below.
> > > >
> > > > Satish
> > > >
> > > > ---------
> > > > diff --git a/conf/variables b/conf/variables
> > > > index 79ce8ba..bb7746f 100644
> > > > --- a/conf/variables
> > > > +++ b/conf/variables
> > > > @@ -10,7 +10,7 @@
> > > > #
> > > > PETSC_LIB_DIR = ${PETSC_DIR}/${PETSC_ARCH}/lib
> > > >
> > > > -PETSC_CCPPFLAGS = ${PETSC_CC_INCLUDES} ${PETSCFLAGS} ${CPP_FLAGS} ${CPPFLAGS} -D__INSDIR__=${LOCDIR}
> > > > +PETSC_CCPPFLAGS = ${PETSC_CC_INCLUDES} ${PETSCFLAGS} ${CPP_FLAGS} ${CPPFLAGS}
> > > > PETSC_FCPPFLAGS = ${PETSC_FC_INCLUDES} ${PETSCFLAGS} ${FPP_FLAGS} ${FPPFLAGS}
> > > > PETSC_C_SH_LIB_PATH = ${CC_LINKER_SLFLAG}${PETSC_LIB_DIR}
> > > > PETSC_F_SH_LIB_PATH = ${FC_LINKER_SLFLAG}${PETSC_LIB_DIR}
> > > > diff --git a/gmakefile b/gmakefile
> > > > index 1577acb..e5c4a37 100644
> > > > --- a/gmakefile
> > > > +++ b/gmakefile
> > > > @@ -9,6 +9,7 @@ pkgs := sys vec mat dm ksp snes ts
> > > >
> > > > libpetsc_shared := $(LIBDIR)/libpetsc.so
> > > > libpetscpkgs_shared := $(foreach pkg, $(pkgs), $(LIBDIR)/libpetsc$(pkg).so)
> > > > +libpetsc_static := $(LIBDIR)/libpetsc.${AR_LIB_SUFFIX}
> > > >
> > > > ifeq ($(PETSC_WITH_EXTERNAL_LIB),)
> > > > libpetscall_shared := $(libpetscpkgs_shared)
> > > > @@ -16,7 +17,7 @@ else
> > > > libpetscall_shared := $(libpetsc_shared)
> > > > endif
> > > >
> > > > -all : $(libpetscall_shared)
> > > > +all : $(libpetsc_static)
> > > >
> > > > .SECONDEXPANSION: # to expand $$(@D)/.DIR
> > > >
> > > > @@ -40,9 +41,9 @@ else
> > > > cc_name := CC
> > > > endif
> > > >
> > > > -PETSC_DEPFLAGS.c := -MMD -MP
> > > > -PETSC_DEPFLAGS.cxx := -MMD -MP
> > > > -PETSC_DEPFLAGS.F := -MMD -MP
> > > > +PETSC_DEPFLAGS.c := #-MMD -MP
> > > > +PETSC_DEPFLAGS.cxx := #-MMD -MP
> > > > +PETSC_DEPFLAGS.F := #-MMD -MP
> > > >
> > > > PETSC_COMPILE.c = $(call quiet,$(cc_name)) -c $(PCC_FLAGS) $(CFLAGS) $(CCPPFLAGS) $(PETSC_DEPFLAGS.c)
> > > > PETSC_COMPILE.cxx = $(call quiet,CXX) -c $(PCC_FLAGS) $(CFLAGS) $(CCPPFLAGS) $(PETSC_DEPFLAGS.cxx)
> > > > @@ -57,6 +58,8 @@ allobj := $(foreach pkg, $(pkgs), $(call concatlang,$(pkg)))
> > > > # with-single-library=1 (default)
> > > > $(libpetsc_shared) : $(allobj) | $$(@D)/.DIR
> > > > $(call quiet,CLINKER) -shared -o $@ $^ $(PETSC_EXTERNAL_LIB_BASIC)
> > > > +$(libpetsc_static): $(allobj) | $$(@D)/.DIR
> > > > + $(call quiet,AR) $(FAST_AR_FLAGS) $@ $^
> > > >
> > > > # with-single-library=0
> > > > libpkg = $(foreach pkg, $1, $(LIBDIR)/libpetsc$(pkg).so)
> > > >
> > > >
> > > > On Wed, 22 May 2013, Satish Balay wrote:
> > > >
> > > >> Looks like it can will work on windows.
> > > >>
> > > >> For one it doesn't like gcc options like -MDD. [gives warnings] - so I removed that.
> > > >>
> > > >> And - cl [or win32fe?] is not liking empty LOCDIR for "-D__INSDIR__=${LOCDIR}"
> > > >> so I removed that.
> > > >>
> > > >> The build with 'make -j 8' goes through quickly until:
> > > >>
> > > >> CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so
> > > >>
> > > >> [so a static library target will perhaps fix this issue]
> > > >>
> > > >> Satish
> > > >>
> > > >> ---------
> > > >> CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so
> > > >> 0 [main] sh 3108 child_info_fork::abort: can't commit memory for stack 0x2
> > > >> 8A000(90112), Win32 error 487
> > > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > > >> 0 [main] sh 1824 child_info_fork::abort: can't commit memory for stack 0x2
> > > >> 8A000(90112), Win32 error 487
> > > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > > >> 0 [main] sh 3548 child_info_fork::abort: can't commit memory for stack 0x2
> > > >> 8A000(90112), Win32 error 487
> > > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > > >> 0 [main] sh 2908 child_info_fork::abort: can't commit memory for stack 0x2
> > > >> 8A000(90112), Win32 error 487
> > > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > > >> 0 [main] sh 2872 child_info_fork::abort: can't commit memory for stack 0x2
> > > >> 8A000(90112), Win32 error 487
> > > >> /usr/bin/sh: fork: Resource temporarily unavailable
> > > >> gmakefile:59: recipe for target `/home/balay/petsc.clone/arch-gmake/lib/libpetsc
> > > >> .so' failed
> > > >> make: *** [/home/balay/petsc.clone/arch-gmake/lib/libpetsc.so] Error 254
> > > >>
> > > >> balay at msnehalem2 ~/petsc.clone
> > > >> $
> > > >>
> > > >> On Mon, 20 May 2013, Barry Smith wrote:
> > > >>
> > > >> >
> > > >> > And does this solve the Windows problem?
> > > >> >
> > > >> > On May 20, 2013, at 4:46 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> > > >> >
> > > >> > > I just merged 'jed/gnumake' to 'next'. I haven't added support for
> > > >> > > static libraries yet, but it should work with pretty much all other
> > > >> > > configurations using compilers that support '-MMD -MP' (tested with gcc,
> > > >> > > clang, and intel; C, C++, and Fortran; single and multiple library).
> > > >> > >
> > > >> > > $ make -f gmakefile -j20 PETSC_ARCH=your-choice
> > > >> > >
> > > >> > > The makefile is 100 lines and the Python for figuring out what to
> > > >> > > include in the build is about the same (conf/gmakegen.py; neglecting
> > > >> > > python-2.4 compatibility stuff). Simplifying the rules would reduce
> > > >> > > that, naturally.
> > > >> > >
> > > >> > > Let me know how it works for you.
> > > >> >
> > > >> >
> > > >>
> > > >>
> > >
> >
> >
>
More information about the petsc-dev
mailing list