<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix"><br>
Can you check in vanilla-g++ ? <br>
<br>
On 07/31/2014 02:29 PM, Ketan Maheshwari wrote:<br>
</div>
<blockquote
cite="mid:CAMUuviq2V8iKT+P9cvru4_iokqNaO4W9dxmD7cwot625v4W6_g@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<div dir="ltr">Yes indeed, I am loading from a shared library
which is causing segfault. I tested this with a single line tcl
as you suggested:
<div><br>
</div>
<div>load ./libdock_wrap.so<br>
</div>
<div><br>
</div>
<div>
<div>$ tclsh8.5 test.tcl</div>
<div>Segmentation fault (core dumped)</div>
</div>
<div><br>
</div>
<div>I do not know why should this happen and possible root
cause. This is how the .so is generated:</div>
<div><br>
</div>
<div>g++ -O2 -shared -o libdock_wrap.so extension.o dock_wrap.o
objfiles/*.o -L /home/ketan/tcl-install/lib -ltcl8.5 -lm
-lpthread -Wl,-rpath -Wl,/home/ketan/tcl-install/lib<br>
</div>
<div><br>
</div>
<div>Where the objfiles/*.o are the object files required by the
application. These object files are generated with the
application's config, make except that I added the -fPIC
compilation flag as required for generating shared lib.</div>
<div><br>
</div>
<div>Do you see anything suspicious in the above line by any
chance?</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Ketan</div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">
On Tue, Jul 29, 2014 at 4:20 PM, Tim Armstrong <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:tim.g.armstrong@gmail.com" target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div dir="ltr">
<div>
<div>The next logical step would be to look at what is
actually happening when you are loading the
package. I don't know exactly how the package is
set up. However, you can look in pkgIndex.tcl to
see what commands are run (separated by newlines) to
load the package. The stack trace also told use
that it happened in Tcl_LoadObjCmd, so it's probably
happened in a load command. E.g. in the turbine
pkgIndex.tcl you have this: [list load [file join
$dir libtclturbine.so]]<br>
<br>
</div>
<div>Are you loading the library from a shared
library? There appear to be multiple ways to load a
library.<br>
</div>
<div><br>
</div>
If you extract that out into a runnable Tcl file and
edit paths according you'll probably have an even more
minimal example, e.g.<br>
<br>
<span style="font-family:courier new,monospace">load
"./libwhatever.so</span>"<span class="HOEnZb"><font
color="#888888"><br>
<br>
</font></span></div>
<span class="HOEnZb"><font color="#888888">
- Tim</font></span>
<div>
<div class="h5"><br>
<div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Tue, Jul 29, 2014 at
4:01 PM, Ketan Maheshwari <span dir="ltr">
<<a moz-do-not-send="true"
href="mailto:ketan@mcs.anl.gov"
target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div dir="ltr">I tried a minimal tcl and
find the segfault occurs at:
<div><br>
</div>
<div><span
style="font-size:13px;font-family:'courier
new',monospace">package require
leaf_main 0.0</span><br
style="font-family:arial,sans-serif;font-size:13px">
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">
<div>On Tue, Jul 29, 2014 at 3:39 PM,
Tim Armstrong <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:tim.g.armstrong@gmail.com"
target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>Well, anyway, let's
backtrack. The
stacktrace already
told us that the
segfault is happening
in a package require
statement.<br>
<br>
</div>
I compiled
apps/dock/user-code.swift
and looked at the code.
There are two package
requires:<br>
<br>
<span
style="font-family:courier
new,monospace">package
require turbine 0.5.0<br>
package require
leaf_main 0.0</span><br>
<br>
</div>
They are up the top before
anything else really
runs. So if the problem
is in loading one of those
packages, whatever happens
later is irrelevant.<br>
<br>
</div>
So how about just running
user-code.tcl, or even
creating a minimal tcl file
with those two package
require lines.<br>
<br>
</div>
You may need to set TCLLIBPATH
(a space-separated list: <a
moz-do-not-send="true"
href="http://www.tcl.tk/man/tcl8.6/TclCmd/library.htm#M27"
target="_blank">
http://www.tcl.tk/man/tcl8.6/TclCmd/library.htm#M27</a>) to the
directories with the turbine
and leaf_main pkgIndex.tcl
files.<span><font
color="#888888"><br>
<br>
</font></span></div>
</div>
<span><font color="#888888">- Tim<br>
</font></span></div>
<div>
<div>
<div>
<div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On
Tue, Jul 29, 2014 at
3:16 PM, Ketan
Maheshwari <span
dir="ltr">
<<a
moz-do-not-send="true"
href="mailto:ketan@mcs.anl.gov" target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">We are
trying to narrow
down the cause of
segfault by running
the tcl out of
turbine thus getting
rid of the
swift/T/tcl and
turbine script. I
suppose this is the
tcl script that gets
invoked which in
turn invokes the
application.</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>On Tue, Jul
29, 2014 at 3:14
PM, Tim
Armstrong <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:tim.g.armstrong@gmail.com"
target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div dir="ltr">
<div>
<div>proc just
defines the
functions. You
need to call
them for it to
run.<br>
<br>
</div>
<div>What are
we trying to
achieve by
running this
file anyway?
This look like
a set of
library
functions
rather than
the entry
point for a
script.<span><font
color="#888888"><br>
</font></span></div>
</div>
<span><font
color="#888888">
<div><br>
</div>
- Tim<br>
</font></span></div>
<div>
<div>
<div>
<div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Tue, Jul 29,
2014 at 3:08
PM, Ketan
Maheshwari <span
dir="ltr">
<<a
moz-do-not-send="true"
href="mailto:ketan@mcs.anl.gov" target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">Here
is the tcl
script with
puts messages:
<div><br>
</div>
<div>
<div>package
provide
leaf_main 0.0</div>
<div><br>
</div>
<div># dnl
Receive
USER_LEAF from
environment
for m4
processing</div>
<div>set
USER_LEAF
dock_wrap</div>
<div>puts
hello1</div>
<div><br>
</div>
<div>namespace
eval leaf_main
{</div>
<div>puts
hello2</div>
<div><br>
</div>
<div> proc
leaf_main_wrap
{ rc A } {</div>
<div>
deeprule $A 1
0
"leaf_main::leaf_main_wrap_impl
$rc $A" type
$::turbine::WORK</div>
<div> }</div>
<div><br>
</div>
<div> proc
leaf_main_wrap_impl
{ rc A } {</div>
<div><br>
</div>
<div>
global
USER_LEAF</div>
<div><br>
</div>
<div>
set length [
adlb::container_size
$A ]</div>
<div>
set tds [
adlb::enumerate
$A dict all 0
]</div>
<div>
set argv [
list ]</div>
<div><br>
</div>
<div>
puts hello3</div>
<div><br>
</div>
<div> #
Fill argv with
blanks</div>
<div>
dict for { i v
} $tds {</div>
<div>
lappend argv
0</div>
<div> }</div>
<div> #
Set values at
ordered list
positions</div>
<div>
dict for { i v
} $tds {</div>
<div>
lset argv $i
$v</div>
<div> }</div>
<div>
set rc_value [
${USER_LEAF}_extension
{*}$argv ]</div>
<div>
turbine::store_integer
$rc $rc_value</div>
<div>
puts hello4</div>
<div> }</div>
<div> puts
hello5</div>
<div>}</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>It
prints:</div>
<div><br>
</div>
<div>
<div>hello1</div>
<div>hello2</div>
<div>hello5</div>
</div>
<div><br>
</div>
<div>I see
that it is not
going in the
proc_leaf_main_wrap_impl
but I am not
familiar
enough with
TCL to
understand
why.</div>
<div><br>
</div>
</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>On Tue,
Jul 29, 2014
at 2:41 PM,
Tim Armstrong
<span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:tim.g.armstrong@gmail.com"
target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div>
<div dir="ltr">I
don't see any
reason why
that
invocation of
tclsh would
silently fail
to run the tcl
script. Have
you attempted
to confirm
your
hypothesis
that it's not
running the
script, for
example by
modifying the
script to
print
something at
the beginning
or end?<br>
</div>
</div>
<div>
<div>
<div>
<div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Tue, Jul 29,
2014 at 1:42
PM, Ketan
Maheshwari <span
dir="ltr">
<<a
moz-do-not-send="true"
href="mailto:ketan@mcs.anl.gov" target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">I
expect it to
run the
application or
crash on
segfault.
Nothing
happens.
<div><br>
</div>
</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>On Tue,
Jul 29, 2014
at 1:39 PM,
Tim Armstrong
<span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:tim.g.armstrong@gmail.com"
target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div dir="ltr">
<div>
<div>That
looks right,
it should run
dock_wrap.tcl
fine. And it
runs
successfully
to completion
with no
output? Is
that what you
expected it to
do?<br>
<br>
</div>
<div>Backtracking
to your
original
problem, if
you could work
out which
"package
require"
statement was
failing and
provide some
info about
that package
it might help
understand the
issue.<span><font
color="#888888"><br>
</font></span></div>
</div>
<span><font
color="#888888">
<div><br>
</div>
- Tim<br>
</font></span></div>
<div>
<div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>On Tue,
Jul 29, 2014
at 1:32 PM,
Ketan
Maheshwari <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:ketan@mcs.anl.gov" target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
</div>
<div>
<div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">
<div>I run
tclsh as
follows:</div>
<div><br>
</div>
/home/ketan/tcl-install/bin/tclsh8.5
dock_wrap.tcl
-i <a
moz-do-not-send="true"
href="http://rigid.in" target="_blank">
rigid.in</a><br>
<div><br>
</div>
<div>and</div>
<div><br>
</div>
<div>mpiexec
-n 3
/home/ketan/tcl-install/bin/tclsh8.5
dock_wrap.tcl
-i <a
moz-do-not-send="true"
href="http://rigid.in" target="_blank">
rigid.in</a><br>
</div>
</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>On Tue,
Jul 29, 2014
at 1:28 PM,
Tim Armstrong
<span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:tim.g.armstrong@gmail.com"
target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div>
<div dir="ltr">
<div>I forgot
to reply all
earlier,
re-including
the list.<br>
<br>
</div>
How are you
running tclsh?<br>
</div>
</div>
<div>
<div>
<div>
<div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Tue, Jul 29,
2014 at 11:53
AM, Ketan
Maheshwari <span
dir="ltr">
<<a
moz-do-not-send="true"
href="mailto:ketan@mcs.anl.gov" target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">when
I try tclsh,
it does not do
anything. Just
returns with
an exit status
0.</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>On Tue,
Jul 29, 2014
at 11:02 AM,
Tim Armstrong
<span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:tim.g.armstrong@gmail.com"
target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div>
<div dir="ltr">
<div>You can
run it
directly with
tclsh or
mpiexec tclsh,
which is what
turbine
eventually
does after
setting up
environment
variables,
etc.<span><font
color="#888888"><br>
<br>
</font></span></div>
<span><font
color="#888888">-
Tim<br>
</font></span></div>
</div>
<div>
<div>
<div>
<div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Tue, Jul 29,
2014 at 10:57
AM, Ketan
Maheshwari <span
dir="ltr">
<<a
moz-do-not-send="true"
href="mailto:ketan@mcs.anl.gov" target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div dir="ltr">Is
it possible to
run the
dock_wrap.tcl
outside of
turbine just
as in the case
of static
build?
<div><br>
</div>
<div><br>
</div>
</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>On Tue,
Jul 29, 2014
at 10:45 AM,
Wozniak,
Justin M. <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:wozniak@mcs.anl.gov" target="_blank">wozniak@mcs.anl.gov</a>></span>
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
bgcolor="#FFFFFF"
text="#000000">
<div>
<div><br>
Ok, it's in.
The Swift/K
SVN is
apparently
down so it's
not on the web
yet but see
the asciidoc.
<br>
<div><br>
On 07/29/2014
10:21 AM,
Justin M
Wozniak wrote:<br>
</div>
</div>
</div>
<div>
<div>
<blockquote
type="cite">
<div>
<div><br>
I thought
VALGRIND was
in the manual
already but it
isn't. I will
add it now. I
will also talk
about our GDB
feature.
<br>
<br>
On 07/29/2014
10:17 AM,
Ketan
Maheshwari
wrote:<br>
</div>
</div>
<div>
<div>
<blockquote
type="cite">
<div dir="ltr">Thanks!
Seems turbine
script already
had a
placeholder
for Valgrind
so I tried
that and from
the output, it
seems tcl
libraries are
causing
segfault but I
may be wrong.
Attached is
the Valgrind
output.
<div><br>
</div>
</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">On
Tue, Jul 29,
2014 at 10:05
AM, Tim
Armstrong <span
dir="ltr">
<<a
moz-do-not-send="true"
href="mailto:tim.g.armstrong@gmail.com" target="_blank">tim.g.armstrong@gmail.com</a>></span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div dir="ltr">
<div>I don't
have any
particular
insight into
the cause of
the segfault,
I can help
with the
debugger
though.<br>
<br>
You need to
point gdb at
the tclsh that
is being used
by turbine
(which is just
a shell
script). You
can locate the
correct tclsh
by looking at
TCLSH in
scripts/turbine-config.sh
in the turbine
install
directory.<br>
<br>
</div>
- TIm<br>
</div>
<div
class="gmail_extra"><br>
<br>
<div
class="gmail_quote">
<div>
<div>On Tue,
Jul 29, 2014
at 10:00 AM,
Ketan
Maheshwari <span
dir="ltr"><<a
moz-do-not-send="true" href="mailto:ketan@mcs.anl.gov" target="_blank">ketan@mcs.anl.gov</a>></span>
wrote:<br>
</div>
</div>
<blockquote
class="gmail_quote"
style="margin:0
0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div>
<div dir="ltr">Hi,
<div><br>
</div>
<div>Trying to
main-wrap DOCK
6.6
application
for ATPESC, I
get the build
right (seems)
but things
fail at
runtime giving
segfault:</div>
<div><br>
</div>
<div>
<div>$ turbine
-n 4
user-code.tcl</div>
<div><br>
</div>
<div>===================================================================================</div>
<div>= BAD
TERMINATION OF
ONE OF YOUR
APPLICATION
PROCESSES</div>
<div>= EXIT
CODE: 139</div>
<div>=
CLEANING UP
REMAINING
PROCESSES</div>
<div>= YOU
CAN IGNORE THE
BELOW CLEANUP
MESSAGES</div>
<div>===================================================================================</div>
<div>YOUR
APPLICATION
TERMINATED
WITH THE EXIT
STRING:
Segmentation
fault (signal
11)</div>
<div>This
typically
refers to a
problem with
your
application.</div>
<div>Please
see the FAQ
page for
debugging
suggestions</div>
</div>
<div><br>
</div>
<div>This is
on MCS
machine. Any
suggestion to
debug this? I
tried gdb but
it gives:</div>
<div><br>
</div>
<div> "/nfs2/ketan/exm-install/turbine/bin/turbine":
not in
executable
format: File
format not
recognized</div>
<div><br>
</div>
<div>With
strace, I see
some signs of
missing files
but not sure
if that is the
cause of
segfault.
Attached is
the strace
output of:</div>
<div><br>
</div>
<div>strace -o
strace.out
turbine -n 4
user-code.tcl<br>
</div>
<div><br>
</div>
<div>The code
has some MPI
and pthread
elements but
does not use
them as far as
I understand.</div>
<div><br>
</div>
<div>Thanks
for any
suggestions.</div>
<div><br>
</div>
<div>--</div>
<div>Ketan</div>
</div>
<br>
</div>
</div>
_______________________________________________<br>
ExM-user
mailing list<br>
<a
moz-do-not-send="true"
href="mailto:ExM-user@lists.mcs.anl.gov" target="_blank">ExM-user@lists.mcs.anl.gov</a><br>
<a
moz-do-not-send="true"
href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user"
target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
ExM-user mailing list
<a moz-do-not-send="true" href="mailto:ExM-user@lists.mcs.anl.gov" target="_blank">ExM-user@lists.mcs.anl.gov</a>
<a moz-do-not-send="true" href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user" target="_blank">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a>
</pre>
</blockquote>
<br>
<br>
</div>
</div>
<span><font
color="#888888">
<pre cols="72">--
Justin M Wozniak</pre>
</font></span></blockquote>
</div>
</div>
<span><font
color="#888888"><br>
<span><font
color="#888888"><br>
<pre cols="72">--
Justin M Wozniak</pre>
</font></span></font></span></div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
ExM-user mailing list
<a class="moz-txt-link-abbreviated" href="mailto:ExM-user@lists.mcs.anl.gov">ExM-user@lists.mcs.anl.gov</a>
<a class="moz-txt-link-freetext" href="https://lists.mcs.anl.gov/mailman/listinfo/exm-user">https://lists.mcs.anl.gov/mailman/listinfo/exm-user</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Justin M Wozniak</pre>
</body>
</html>