[petsc-dev] Experimental GNU make build system
Satish Balay
balay at mcs.anl.gov
Thu May 23 00:09:53 CDT 2013
I can reproduce the issue with the attached makefile.
It appears to be related to the following:
- make [as the same command from shell works]
- pipe or multiple commands [i.e 'cmd1|cmd2' or 'cmd1; cmd2']
- the huge string in one of the commands
- length of PATH string [works if PATH is not too long?]
- happens on win 2003/x32 server, and win2008/x64 server - but not win7/x32?
Perhaps its a cygwin bug - or the servers have some limits that cygwin
is hitting with this example? wierd.
Satish
-----------------
sbalay at ps3 ~
$ echo $PATH
/home/sbalay/bin:/usr/local/bin:/usr/bin:/usr/local/bin:/usr/bin:/cygdrive/c/Program Files/Microsoft Visual Studio/Common/Tools:/cygdrive/c/Program Files/Microsoft Visual Studio/Common/Msdev98/BIN:/cygdrive/c/Program Files/Microsoft Visual Studio/DF98/BIN:/cygdrive/c/Program Files/Microsoft Visual Studio/VC98/BIN:/cygdrive/c/WINDOWS/system32:/cygdrive/c/WINDOWS:/cygdrive/c/WINDOWS/System32/Wbem:/cygdrive/c/Program Files/Intel/MKL/ia32/bin:/cygdrive/c/Python25:/cygdrive/c/Program Files/TortoiseHg:/usr/lib/lapack:/cygdrive/c/Program Files/MPICH2/bin:/cygdrive/c/Borland/BCC55/Bin
sbalay at ps3 ~
$ make -f test.mk
30621
e
sbalay at ps3 ~
$ export PATH=$PATH:$PATH
sbalay at ps3 ~
$ make -f test.mk
30621
sbalay at ps3 ~
$ export PATH=$PATH:$PATH
sbalay at ps3 ~
$ make -f test.mk
10 [main] sh 3644 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable
10 [main] sh 2440 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable
10 [main] sh 2120 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable
10 [main] sh 620 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
/bin/sh: fork: retry: Resource temporarily unavailable
10 [main] sh 2548 child_info_fork::abort: can't commit memory for stack 0x22A000(90112), Win32 error 487
/bin/sh: fork: Resource temporarily unavailable
test.mk:2: recipe for target `all' failed
make: *** [all] Error 254
sbalay at ps3 ~
$
On Wed, 22 May 2013, Satish Balay wrote:
> I don't think its a rebase issue. [cygwin setup runs rbaseall
> successfully if no cygwin process is running during upgrade - and I
> make sure of that. To be sure - I reran rebaseall again]
>
> [and I haven't seen this message with all-legacy builds which create
> scores of processes recursively]
>
> And I mistakingly assumed its a '-j 8' vs '-j 1' issue. I see the
> issue with 'make' but not with 'make V=1'. And it happens consistantly
> at the exact same stage - either at:
> CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so
> or at:
> AR /home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib
>
> This is strange..
>
> Satish
> --------
>
> balay at msnehalem2 ~/petsc.clone
> $ make -f gmakefile PETSC_ARCH=arch-gmake -j 8
> Use "/usr/bin/make V=1" to see the verbose compile lines.
> AR /home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib
> 0 [main] sh 9560 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 7752 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 6772 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 6584 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 8724 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: Resource temporarily unavailable
> gmakefile:62: recipe for target `/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib' failed
> make: *** [/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib] Error 254
>
> balay at msnehalem2 ~/petsc.clone
> $ make -f gmakefile PETSC_ARCH=arch-gmake
> Use "/usr/bin/make V=1" to see the verbose compile lines.
> AR /home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib
> 0 [main] sh 7920 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 7532 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 2932 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 9096 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> 0 [main] sh 5856 child_info_fork::abort: can't commit memory for stack 0x28A000(90112), Win32 error 487
> /usr/bin/sh: fork: Resource temporarily unavailable
> gmakefile:62: recipe for target `/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib' failed
> make: *** [/home/balay/petsc.clone/arch-gmake/lib/libpetsc.lib] Error 254
>
>
>
> > Satish Balay <balay at mcs.anl.gov> writes:
> >
> > > Ok. The 'child_info_fork::abort:' error below is due to gnumake somehow
> > > trying to parallelize 'CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so'?
> >
> > What do you mean "trying to parallelize"? It's only running the linker
> > once.
> >
> > Web search yields this, but it may not be relevant.
> >
> > http://stackoverflow.com/questions/9300722/cygwin-error-bash-fork-retry-resource-temporarily-unavailable
> >
> > > I see the same if I replace it with 'AR'. [here I'm having to use '-j
> > > 1' for AR part]
> > >
> > > I have a successful build with 'make -j 8' - and then 'make -j '1 to
> > > recover from AR errors. And 'make test' is successful.
> > >
> > > my hacky changes to get this working is below.
> > >
> > > Satish
> > >
> > > ---------
> > > diff --git a/conf/variables b/conf/variables
> > > index 79ce8ba..bb7746f 100644
> > > --- a/conf/variables
> > > +++ b/conf/variables
> > > @@ -10,7 +10,7 @@
> > > #
> > > PETSC_LIB_DIR = ${PETSC_DIR}/${PETSC_ARCH}/lib
> > >
> > > -PETSC_CCPPFLAGS = ${PETSC_CC_INCLUDES} ${PETSCFLAGS} ${CPP_FLAGS} ${CPPFLAGS} -D__INSDIR__=${LOCDIR}
> > > +PETSC_CCPPFLAGS = ${PETSC_CC_INCLUDES} ${PETSCFLAGS} ${CPP_FLAGS} ${CPPFLAGS}
> > > PETSC_FCPPFLAGS = ${PETSC_FC_INCLUDES} ${PETSCFLAGS} ${FPP_FLAGS} ${FPPFLAGS}
> > > PETSC_C_SH_LIB_PATH = ${CC_LINKER_SLFLAG}${PETSC_LIB_DIR}
> > > PETSC_F_SH_LIB_PATH = ${FC_LINKER_SLFLAG}${PETSC_LIB_DIR}
> > > diff --git a/gmakefile b/gmakefile
> > > index 1577acb..e5c4a37 100644
> > > --- a/gmakefile
> > > +++ b/gmakefile
> > > @@ -9,6 +9,7 @@ pkgs := sys vec mat dm ksp snes ts
> > >
> > > libpetsc_shared := $(LIBDIR)/libpetsc.so
> > > libpetscpkgs_shared := $(foreach pkg, $(pkgs), $(LIBDIR)/libpetsc$(pkg).so)
> > > +libpetsc_static := $(LIBDIR)/libpetsc.${AR_LIB_SUFFIX}
> > >
> > > ifeq ($(PETSC_WITH_EXTERNAL_LIB),)
> > > libpetscall_shared := $(libpetscpkgs_shared)
> > > @@ -16,7 +17,7 @@ else
> > > libpetscall_shared := $(libpetsc_shared)
> > > endif
> > >
> > > -all : $(libpetscall_shared)
> > > +all : $(libpetsc_static)
> > >
> > > .SECONDEXPANSION: # to expand $$(@D)/.DIR
> > >
> > > @@ -40,9 +41,9 @@ else
> > > cc_name := CC
> > > endif
> > >
> > > -PETSC_DEPFLAGS.c := -MMD -MP
> > > -PETSC_DEPFLAGS.cxx := -MMD -MP
> > > -PETSC_DEPFLAGS.F := -MMD -MP
> > > +PETSC_DEPFLAGS.c := #-MMD -MP
> > > +PETSC_DEPFLAGS.cxx := #-MMD -MP
> > > +PETSC_DEPFLAGS.F := #-MMD -MP
> > >
> > > PETSC_COMPILE.c = $(call quiet,$(cc_name)) -c $(PCC_FLAGS) $(CFLAGS) $(CCPPFLAGS) $(PETSC_DEPFLAGS.c)
> > > PETSC_COMPILE.cxx = $(call quiet,CXX) -c $(PCC_FLAGS) $(CFLAGS) $(CCPPFLAGS) $(PETSC_DEPFLAGS.cxx)
> > > @@ -57,6 +58,8 @@ allobj := $(foreach pkg, $(pkgs), $(call concatlang,$(pkg)))
> > > # with-single-library=1 (default)
> > > $(libpetsc_shared) : $(allobj) | $$(@D)/.DIR
> > > $(call quiet,CLINKER) -shared -o $@ $^ $(PETSC_EXTERNAL_LIB_BASIC)
> > > +$(libpetsc_static): $(allobj) | $$(@D)/.DIR
> > > + $(call quiet,AR) $(FAST_AR_FLAGS) $@ $^
> > >
> > > # with-single-library=0
> > > libpkg = $(foreach pkg, $1, $(LIBDIR)/libpetsc$(pkg).so)
> > >
> > >
> > > On Wed, 22 May 2013, Satish Balay wrote:
> > >
> > >> Looks like it can will work on windows.
> > >>
> > >> For one it doesn't like gcc options like -MDD. [gives warnings] - so I removed that.
> > >>
> > >> And - cl [or win32fe?] is not liking empty LOCDIR for "-D__INSDIR__=${LOCDIR}"
> > >> so I removed that.
> > >>
> > >> The build with 'make -j 8' goes through quickly until:
> > >>
> > >> CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so
> > >>
> > >> [so a static library target will perhaps fix this issue]
> > >>
> > >> Satish
> > >>
> > >> ---------
> > >> CLINKER /home/balay/petsc.clone/arch-gmake/lib/libpetsc.so
> > >> 0 [main] sh 3108 child_info_fork::abort: can't commit memory for stack 0x2
> > >> 8A000(90112), Win32 error 487
> > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > >> 0 [main] sh 1824 child_info_fork::abort: can't commit memory for stack 0x2
> > >> 8A000(90112), Win32 error 487
> > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > >> 0 [main] sh 3548 child_info_fork::abort: can't commit memory for stack 0x2
> > >> 8A000(90112), Win32 error 487
> > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > >> 0 [main] sh 2908 child_info_fork::abort: can't commit memory for stack 0x2
> > >> 8A000(90112), Win32 error 487
> > >> /usr/bin/sh: fork: retry: Resource temporarily unavailable
> > >> 0 [main] sh 2872 child_info_fork::abort: can't commit memory for stack 0x2
> > >> 8A000(90112), Win32 error 487
> > >> /usr/bin/sh: fork: Resource temporarily unavailable
> > >> gmakefile:59: recipe for target `/home/balay/petsc.clone/arch-gmake/lib/libpetsc
> > >> .so' failed
> > >> make: *** [/home/balay/petsc.clone/arch-gmake/lib/libpetsc.so] Error 254
> > >>
> > >> balay at msnehalem2 ~/petsc.clone
> > >> $
> > >>
> > >> On Mon, 20 May 2013, Barry Smith wrote:
> > >>
> > >> >
> > >> > And does this solve the Windows problem?
> > >> >
> > >> > On May 20, 2013, at 4:46 PM, Jed Brown <jedbrown at mcs.anl.gov> wrote:
> > >> >
> > >> > > I just merged 'jed/gnumake' to 'next'. I haven't added support for
> > >> > > static libraries yet, but it should work with pretty much all other
> > >> > > configurations using compilers that support '-MMD -MP' (tested with gcc,
> > >> > > clang, and intel; C, C++, and Fortran; single and multiple library).
> > >> > >
> > >> > > $ make -f gmakefile -j20 PETSC_ARCH=your-choice
> > >> > >
> > >> > > The makefile is 100 lines and the Python for figuring out what to
> > >> > > include in the build is about the same (conf/gmakegen.py; neglecting
> > >> > > python-2.4 compatibility stuff). Simplifying the rules would reduce
> > >> > > that, naturally.
> > >> > >
> > >> > > Let me know how it works for you.
> > >> >
> > >> >
> > >>
> > >>
> >
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.mk
Type: application/octet-stream
Size: 30642 bytes
Desc:
URL: <http://lists.mcs.anl.gov/pipermail/petsc-dev/attachments/20130523/f7762b54/attachment.obj>
More information about the petsc-dev
mailing list