<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">futex is a Linux system call used for locking shared resources.<div class=""><br class=""></div><div class="">It could be indicative of an MPI problem. I wouldn't be surprised. If anyone has any idea how to get around it that would be great. We have dozens of applications on our compute cluster that use MPI, this version being our default. I'm wondering if there is something specific to the mix of MPI flavor / compiler, etc. that could be going on here.</div><div class=""><br class=""></div><div class="">This is the gdb stack trace:</div><div class=""><br class=""></div><div class=""><font face="Andale Mono" class="">#0 0x00000039c6a0e264 in __lll_lock_wait () from /lib64/libpthread.so.0<br class="">#1 0x00000039c6a09508 in _L_lock_854 () from /lib64/libpthread.so.0<br class="">#2 0x00000039c6a093d7 in pthread_mutex_lock () from /lib64/libpthread.so.0<br class="">#3 0x00002aaaaf13ddd4 in opal_mutex_lock (attr_hash=0x2aaaaf651c70, key=128, attribute=0x7fffffffc200, flag=0xffffffffffffffff)<br class=""> at ../opal/threads/mutex_unix.h:104<br class="">#4 ompi_attr_get_c (attr_hash=0x2aaaaf651c70, key=128, attribute=0x7fffffffc200, flag=0xffffffffffffffff)<br class=""> at attribute/attribute.c:758<br class="">#5 0x00002aaaaf17080e in PMPI_Attr_get (comm=0x2aaaaf651c70, keyval=128, attribute_val=0x7fffffffc200, flag=0xffffffffffffffff)<br class=""> at pattr_get.c:61<br class="">#6 0x00002aaaaacad0b3 in Petsc_DelComm_Outer (comm=0x2aaaaf6d4140, keyval=13, attr_val=0x7af160, extra_state=0x0)<br class=""> at /n/home08/lchristakis/petsc/petsc-3.5.4/src/sys/objects/pinit.c:409<br class="">#7 0x00002aaaaf13f1a4 in ompi_attr_delete_impl (type=2942639216, object=0x80, attr_hash=0x7fffffffc200, key=-1, predefined=112 'p')<br class=""> at attribute/attribute.c:970<br class="">#8 0x00002aaaaf13ee02 in ompi_attr_delete (type=2942639216, object=0x80, attr_hash=0x7fffffffc200, key=-1, predefined=112 'p')<br class=""> at attribute/attribute.c:1019<br class="">#9 0x00002aaaaf170710 in PMPI_Attr_delete (comm=0x2aaaaf651c70, keyval=128) at pattr_delete.c:59<br class="">#10 0x00002aaaaac61848 in PetscCommDestroy (comm=0x888cf0) at /n/home08/lchristakis/petsc/petsc-3.5.4/src/sys/objects/tagm.c:256<br class="">#11 0x00002aaaaac6a273 in PetscHeaderDestroy_Private (h=0x888ce0)<br class=""> at /n/home08/lchristakis/petsc/petsc-3.5.4/src/sys/objects/inherit.c:121<br class="">#12 0x00002aaaaaf51512 in VecDestroy (v=0x7fffffffcbd0)<br class=""> at /n/home08/lchristakis/petsc/petsc-3.5.4/src/vec/vec/interface/vector.c:434<br class="">#13 0x00002aaaab9c5c7f in DMSetUp_DA_2D (da=0x87b1b0) at /n/home08/lchristakis/petsc/petsc-3.5.4/src/dm/impls/da/da2.c:776<br class="">#14 0x00002aaaaba73bfd in DMSetUp_DA (da=0x87b1b0) at /n/home08/lchristakis/petsc/petsc-3.5.4/src/dm/impls/da/dareg.c:25<br class="">#15 0x00002aaaab93399a in DMSetUp (dm=0x87b1b0) at /n/home08/lchristakis/petsc/petsc-3.5.4/src/dm/interface/dm.c:560<br class="">#16 0x00002aaaab9c6941 in DMDACreate2d (comm=0x2aaaaf6d45c0, bx=DM_BOUNDARY_NONE, by=DM_BOUNDARY_NONE, <br class=""> stencil_type=DMDA_STENCIL_STAR, M=-4, N=-4, m=-1, n=-1, dof=1, s=1, lx=0x0, ly=0x0, da=0x7fffffffd668)<br class=""> at /n/home08/lchristakis/petsc/petsc-3.5.4/src/dm/impls/da/da2.c:862<br class="">#17 0x00000000004023d0 in main (argc=1, argv=0x7fffffffd8c8)<br class=""> at /n/home08/lchristakis/petsc/petsc-3.5.4/src/snes/examples/tutorials/ex5.c:116<br class=""></font><br class=""></div><div class=""><br class=""><div apple-content-edited="true" class="">
<div class="">Aaron Kitzmiller</div><div class="">Informatics and Scientific Applications</div><div class=""><a href="mailto:aaron_kitzmiller@harvard.edu" class="">aaron_kitzmiller@harvard.edu</a></div><div class=""><br class=""></div><br class="Apple-interchange-newline">
</div>
<br class=""><div><blockquote type="cite" class=""><div class="">On Jul 24, 2015, at 12:18 PM, Matthew Knepley <<a href="mailto:knepley@gmail.com" class="">knepley@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div class="gmail_extra"><div class="gmail_quote">On Fri, Jul 24, 2015 at 11:17 AM, Matthew Knepley<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><span class="">On Fri, Jul 24, 2015 at 11:09 AM, Aaron Kitzmiller<span class="Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:akitzmiller@g.harvard.edu" target="_blank" class="">akitzmiller@g.harvard.edu</a>></span><span class="Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class="">Doesn't run. Hangs just like the tests do.<div class=""><br class=""></div><div class="">I doubt it's helpful, but when I run it under strace, it hangs on a "futex". The last thing vaguely informative was an attempt to read the non-existent .petscrc.</div></div></blockquote><div class=""><br class=""></div></span><div class="">Run in the debugger and get a stack trace.</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">Also futex does not appear in the PETSc source:</div><div class=""><br class=""></div><div class=""> knepley/feature-snes-deflation *+$|MERGING:/PETSc3/petsc/petsc-dev$ find src -name "*.c" | xargs grep futex</div><div class=""> find src -name "*.c" | xargs grep futex</div><div class=""><br class=""></div><div class="">You have an MPI problem.</div><div class=""><br class=""></div><div class=""> Matt</div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""> Matt</div><span class=""><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;"><div style="word-wrap: break-word;" class=""><div class="">ajk</div><div class=""><br class=""><div class=""><div class="">Aaron Kitzmiller</div><div class="">Informatics and Scientific Applications</div><div class=""><a href="mailto:aaron_kitzmiller@harvard.edu" target="_blank" class="">aaron_kitzmiller@harvard.edu</a></div><div class=""><br class=""></div><br class=""></div><br class=""><div class=""><blockquote type="cite" class=""><div class="">On Jul 24, 2015, at 11:21 AM, Matthew Knepley <<a href="mailto:knepley@gmail.com" target="_blank" class="">knepley@gmail.com</a>> wrote:</div><br class=""><div class=""><span style="font-family: Helvetica; font-size: 14px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline !important;" class=""> <span class="Apple-converted-space"> </span>./ex5 -snes_monitor</span></div></blockquote></div><br class=""></div></div></blockquote></span></div><br class=""><br clear="all" class=""><span class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div class="">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div></span></div></div></blockquote></div><br class=""><br clear="all" class=""><div class=""><br class=""></div>--<span class="Apple-converted-space"> </span><br class=""><div class="gmail_signature">What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead.<br class="">-- Norbert Wiener</div></div></div></div></blockquote></div><br class=""></div></body></html>