<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: times new roman,new york,times,serif; font-size: 12pt; color: #000000'>So with -C vagrind would work even without exp option?<br>I just realized that valgrind found another one:<br><br>ErrorCode ScdInterface::get_neighbor_alljorkori(int np, int pfrom,<br> const int * const gdims, const int * const gperiodic, const int * const dijk, <br> int &pto, int *rdims, int *facedims, int *across_bdy)<br>{<br> ErrorCode rval = MB_SUCCESS;<br> pto = -1;<br> if (np == 1) return MB_SUCCESS;<br> <br> int pijk[3], lperiodic<span style="background-color: rgb(204, 0, 0);">[2]</span>, ldims[6];<br> rval = compute_partition_alljorkori(np, pfrom, gdims, gperiodic, ldims, lperiodic, pijk);<br> if (MB_SUCCESS != rval) return rval;<br><br>Can you fix this one too, Danqing?<br><br>Thanks,<br>Iulian<br><br><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;">I think having -C option for debug builds might be a good idea. It<br>will slow the computation down further but should catch a lot of such<br>errors during runtime.<br><br>I enable this option for most of the fortran codes and it has saved me<br>ton of headache in the past.<br><br>Vijay<br><br>On Tue, Sep 10, 2013 at 2:20 PM, Iulian Grindeanu <iulian@mcs.anl.gov> wrote:<br>> running the test with experimental thing --tool=exp-sgcheck found it:<br>> (I did not have to compile with -C)<br>><br>> I didn't know about this experimental option<br>><br>> Thanks,<br>> Iulian<br>><br>> iulian@T520-iuli:~/source/MOABp13/test$ valgrind --tool=exp-sgcheck<br>> scdseq_test<br>> ==6670== exp-sgcheck, a stack and global array overrun detector<br>> ==6670== NOTE: This is an Experimental-Class Valgrind Tool<br>> ==6670== Copyright (C) 2003-2011, and GNU GPL'd, by OpenWorks Ltd et al.<br>> ==6670== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info<br>> ==6670== Command: scdseq_test<br>> ==6670==<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> --6670-- warning: evaluate_Dwarf3_Expr: unhandled DW_OP_ 0x93<br>> Running test_parallel_partitions ...<br>> ==6670== Invalid write of size 4<br>> ==6670== at 0x418C09:<br>> moab::ScdInterface::compute_partition_alljorkori(int, int, int const*, int<br>> const*, int*, int*, int*) (ScdInterface.hpp:787)<br>> ==6670== by 0x470E31: moab::ScdInterface::get_neighbor_alljorkori(int,<br>> int, int const*, int const*, int const*, int&, int*, int*, int*)<br>> (ScdInterface.cpp:1154)<br>> ==6670== by 0x41AFD5: moab::ScdInterface::get_neighbor(int, int,<br>> moab::ScdParData const&, int const*, int&, int*, int*, int*)<br>> (ScdInterface.hpp:1216)<br>> ==6670== by 0x417726: test_parallel_partition(int*, int, int)<br>> (scdseq_test.cpp:1379)<br>> ==6670== by 0x41748A: test_parallel_partitions() (scdseq_test.cpp:1331)<br>> ==6670== by 0x40E664: run_test(void (*)(), char const*)<br>> (TestUtil.hpp:320)<br>> ==6670== by 0x410229: main (scdseq_test.cpp:267)<br>> ==6670== Address 0x7fefff878 expected vs actual:<br>> ==6670== Expected: stack array "lperiodic" of size 8 in frame 1 back from<br>> here<br>> ==6670== Actual: unknown<br>> ==6670== Actual: is 0 after Expected<br>> ==6670==<br>> ==6670== Invalid write of size 4<br>> ==6670== at 0x418C09:<br>> moab::ScdInterface::compute_partition_alljorkori(int, int, int const*, int<br>> const*, int*, int*, int*) (ScdInterface.hpp:787)<br>> ==6670== by 0x470E31: moab::ScdInterface::get_neighbor_alljorkori(int,<br>> int, int const*, int const*, int const*, int&, int*, int*, int*)<br>> (ScdInterface.cpp:1154)<br>> ==6670== by 0x41AFD5: moab::ScdInterface::get_neighbor(int, int,<br>> moab::ScdParData const&, int const*, int&, int*, int*, int*)<br>> (ScdInterface.hpp:1216)<br>> ==6670== by 0x4177E8: test_parallel_partition(int*, int, int)<br>> (scdseq_test.cpp:1392)<br>> ==6670== by 0x41748A: test_parallel_partitions() (scdseq_test.cpp:1331)<br>> ==6670== by 0x40E664: run_test(void (*)(), char const*)<br>> (TestUtil.hpp:320)<br>> ==6670== by 0x410229: main (scdseq_test.cpp:267)<br>> ==6670== Address 0x7fefff878 expected vs actual:<br>> ==6670== Expected: stack array "lperiodic" of size 8 in frame 1 back from<br>> here<br>> ==6670== Actual: unknown<br>> ==6670== Actual: is 0 after Expected<br>> ==6670==<br>> ==6670== Invalid write of size 4<br>> ==6670== at 0x41917E: moab::ScdInterface::compute_partition_alljkbal(int,<br>> int, int const*, int const*, int*, int*, int*) (ScdInterface.hpp:868)<br>> ==6670== by 0x46E655: moab::ScdInterface::get_neighbor_alljkbal(int, int,<br>> int const*, int const*, int const*, int&, int*, int*, int*)<br>> (ScdInterface.cpp:758)<br>> ==6670== by 0x41B01C: moab::ScdInterface::get_neighbor(int, int,<br>> moab::ScdParData const&, int const*, int&, int*, int*, int*)<br>> (ScdInterface.hpp:1219)<br>> ==6670== by 0x417726: test_parallel_partition(int*, int, int)<br>> (scdseq_test.cpp:1379)<br>> ==6670== by 0x4174AD: test_parallel_partitions() (scdseq_test.cpp:1336)<br>> ==6670== by 0x40E664: run_test(void (*)(), char const*)<br>> (TestUtil.hpp:320)<br>> ==6670== by 0x410229: main (scdseq_test.cpp:267)<br>> ==6670== Address 0x7fefff888 expected vs actual:<br>> ==6670== Expected: stack array "lperiodic" of size 8 in frame 1 back from<br>> here<br>> ==6670== Actual: unknown<br>> ==6670== Actual: is 0 after Expected<br>> ==6670==<br>> ==6670== Invalid write of size 4<br>> ==6670== at 0x41917E: moab::ScdInterface::compute_partition_alljkbal(int,<br>> int, int const*, int const*, int*, int*, int*) (ScdInterface.hpp:868)<br>> ==6670== by 0x46E655: moab::ScdInterface::get_neighbor_alljkbal(int, int,<br>> int const*, int const*, int const*, int&, int*, int*, int*)<br>> (ScdInterface.cpp:758)<br>> ==6670== by 0x41B01C: moab::ScdInterface::get_neighbor(int, int,<br>> moab::ScdParData const&, int const*, int&, int*, int*, int*)<br>> (ScdInterface.hpp:1219)<br>> ==6670== by 0x4177E8: test_parallel_partition(int*, int, int)<br>> (scdseq_test.cpp:1392)<br>> ==6670== by 0x4174AD: test_parallel_partitions() (scdseq_test.cpp:1336)<br>> ==6670== by 0x40E664: run_test(void (*)(), char const*)<br>> (TestUtil.hpp:320)<br>> ==6670== by 0x410229: main (scdseq_test.cpp:267)<br>> ==6670== Address 0x7fefff888 expected vs actual:<br>> ==6670== Expected: stack array "lperiodic" of size 8 in frame 1 back from<br>> here<br>> ==6670== Actual: unknown<br>> ==6670== Actual: is 0 after Expected<br>> ==6670==<br>> Running test_vertex_seq ...<br>> Running test_element_seq ...<br>> Running test_periodic_seq ...<br>><br>><br>> ________________________________<br>><br>> Hmm, I guess I'd still vote for using std::vector for statically-allocated<br>> arrays, rather than the alternative of<br>> building -C into all our debug builds. Thoughts?<br>><br>> - tim<br>><br>> On 09/10/2013 02:03 PM, Vijay S. Mahadevan wrote:<br>>> You need to compile with -C to catch static allocation errors. That<br>>> will specifically turn on range checks.<br>>><br>>> Good to know that memcheck doesn't give you invalid read errors on<br>>> statically allocated arrays. Look at faq:<br>>> http://valgrind.org/docs/manual/faq.html<br>>><br>>>> Why doesn't Memcheck find the array overruns in this program?<br>>>> Unfortunately, Memcheck doesn't do bounds checking on global or stack<br>>>> arrays. We'd like to, but it's just not possible to do in a reasonable way<br>>>> that fits with how Memcheck works. Sorry.<br>>><br>>>> However, the experimental tool SGcheck can detect errors like this. Run<br>>>> Valgrind with the --tool=exp-sgcheck option to try it, but be aware that it<br>>>> is not as robust as Memcheck.<br>>><br>>> Vijay<br>>><br>>> On Tue, Sep 10, 2013 at 1:51 PM, Tim Tautges <tautges@mcs.anl.gov> wrote:<br>>>> Good catch Danqing, I didn't know that (that valgrind wouldn't catch out<br>>>> of<br>>>> bounds errors on statically-allocated arrays).<br>>>><br>>>> The preferred way to do this, then, will be to use std::vector, with a<br>>>> static size set at instantiation. That makes it dynamically allocated<br>>>> but<br>>>> still static size. I'll remember that one.<br>>>><br>>>> - tim<br>>>><br>>>> On 09/10/2013 01:22 PM, Danqing Wu wrote:<br>>>>><br>>>>> Here is what I found online:<br>>>>><br>>>>> What Won't Valgrind Find?<br>>>>> Valgrind doesn't perform bounds checking on static arrays (allocated on<br>>>>> the stack). So if you declare an array inside your function:<br>>>>><br>>>>> int main()<br>>>>> {<br>>>>> char x[10];<br>>>>> x[11] = 'a';<br>>>>> }<br>>>>><br>>>>> then Valgrind won't alert you! One possible solution for testing<br>>>>> purposes<br>>>>> is simply to change your static arrays into dynamically allocated memory<br>>>>> taken from the heap, where you will get bounds-checking, though this<br>>>>> could<br>>>>> be a mess of unfreed memory.<br>>>>><br>>>>> ----- Original Message -----<br>>>>> From: "Iulian Grindeanu" <iulian@mcs.anl.gov><br>>>>> To: "Danqing Wu" <wuda@mcs.anl.gov><br>>>>> Cc: "Tim Tautges" <tautges@mcs.anl.gov><br>>>>> Sent: Tuesday, September 10, 2013 1:02:45 PM<br>>>>> Subject: Re: Simple code to reproduce ICC segmentation fault<br>>>>><br>>>>><br>>>>><br>>>>><br>>>>> ----- Original Message -----<br>>>>><br>>>>><br>>>>><br>>>>> After correcting that, moab-intel test works fine!<br>>>>> Good job again, Danqing!<br>>>>><br>>>>> Thanks,<br>>>>> Iulian<br>>>>><br>>>>> now the question is why valgrind did not find this ...<br>>>>><br>>>>><br>>>>><br>>>>><br>>>>> ----- Original Message -----<br>>>>><br>>>>><br>>>>> I think I found one possible reason.<br>>>>><br>>>>> ErrorCode ScdInterface::get_neighbor_alljkbal(int np, int pfrom,<br>>>>> const int * const gdims, const int * const gperiodic, const int * const<br>>>>> dijk,<br>>>>> int &pto, int *rdims, int *facedims, int *across_bdy)<br>>>>> {<br>>>>> ...<br>>>>> int ldims[6], pijk[3], lperiodic[2];<br>>>>> ErrorCode rval = compute_partition_alljkbal(np, pfrom, gdims, gperiodic,<br>>>>> ldims, lperiodic, pijk);<br>>>>> ...<br>>>>> }<br>>>>><br>>>>> Here lperiodic[2] should be lperiodic[3], as the third element will be<br>>>>> accessed inside compute_partition_alljkbal().<br>>>>><br>>>>> The behaviour could be dependent on compilers. Maybe only for ICC 12 and<br>>>>> O2, and when assert is disabled, this out of memory issue causes a<br>>>>> segmentation fault.<br>>>>><br>>>>> I will retest after this fix.<br>>>>><br>>>>> ----- Original Message -----<br>>>>> From: "Iulian Grindeanu" <iulian@mcs.anl.gov><br>>>>> To: "Danqing Wu" <wuda@mcs.anl.gov><br>>>>> Cc: "Tim Tautges" <tautges@mcs.anl.gov><br>>>>> Sent: Tuesday, September 10, 2013 10:17:28 AM<br>>>>> Subject: Re: Simple code to reproduce ICC segmentation fault<br>>>>><br>>>>><br>>>>> If it works on icc 13 / ubuntu 12, I suggest moving moab-intel build to<br>>>>> jenkins; we may have to rebuild netcdf with icc if there are issues with<br>>>>> libcurl.<br>>>>><br>>>>> Any suggestions?<br>>>>><br>>>>> Iulian<br>>>>> ----- Original Message -----<br>>>>><br>>>>><br>>>>> On gnep, icc 12.<br>>>>><br>>>>> Configure option<br>>>>> ./configure --prefix=/homes/fathom/libs/current/moabintel<br>>>>> --with-netcdf=/homes/fathom/3rdparty/netcdf-4.1.3-intel<br>>>>> --with-hdf5=/homes/fathom/3rdparty/hdf5-1.8.8-ser-intel<br>>>>> --with-zlib=/homes/fathom/3rdparty/zlib/zlib-1.2.4/gcc --enable-igeom<br>>>>> --enable-imesh CC=icc CXX=icpc F77=ifort FC=ifort F90=ifort<br>>>>><br>>>>> So the flags will include both -O2 and -DNDEBUG<br>>>>><br>>>>> Here since NDEBUG is enabled, all of the assert(...) will do nothing,<br>>>>> and<br>>>>> this could make some differences.<br>>>>><br>>>>> On gnep, icc 12, if only -O2, but no NDEBUG, the original test can pass.<br>>>>> I<br>>>>> guess ICC 12 would be affected by the assert stuff.<br>>>>><br>>>>> ----- Original Message -----<br>>>>> From: "Iulian Grindeanu" <iulian@mcs.anl.gov><br>>>>> To: "Danqing Wu" <wuda@mcs.anl.gov><br>>>>> Cc: "Tim Tautges" <tautges@mcs.anl.gov><br>>>>> Sent: Tuesday, September 10, 2013 10:04:39 AM<br>>>>> Subject: Re: Simple code to reproduce ICC segmentation fault<br>>>>><br>>>>><br>>>>> so this is with icc -O2 or what are the compile options?<br>>>>> Is this on gnep? icc 12? icc 13?<br>>>>><br>>>>> Should we try to use ubuntu 12 for intel builds?<br>>>>><br>>>>> (we can do that on jenkins auto build platform)<br>>>>><br>>>>> Iulian<br>>>>><br>>>>><br>>>>> ----- Original Message -----<br>>>>><br>>>>><br>>>>> I am still debugging, but it seems that the two calls of<br>>>>> ScdInterface::get_neighbor() caused the crash. If I comment out the<br>>>>> second<br>>>>> call, no segmentaion fault.<br>>>>><br>>>>><br>>>>> #include "moab/ScdInterface.hpp"<br>>>>> #include "moab/Core.hpp"<br>>>>><br>>>>> #include <iostream><br>>>>><br>>>>> using namespace moab;<br>>>>><br>>>>> int main()<br>>>>> {<br>>>>> Core moab;<br>>>>> ScdInterface* scdi;<br>>>>> ErrorCode rval = moab.Interface::query_interface(scdi);<br>>>>><br>>>>> int gdims[] = {0, 0, 0, 48, 40, 18};<br>>>>> int nprocs = 4;<br>>>>> int pto = 0;<br>>>>> int across_bdy_a[3] = {0};<br>>>>> int rdims_a[6] = {0};<br>>>>> int facedims_a[6] = {0};<br>>>>><br>>>>> ScdParData spd;<br>>>>> int n;<br>>>>> for (n = 0; n < 6; n++)<br>>>>> spd.gDims[n] = gdims[n];<br>>>>> for (n = 0; n < 3; n++)<br>>>>> spd.gPeriodic[n] = 0;<br>>>>><br>>>>> spd.partMethod = ScdParData::ALLJKBAL;<br>>>>><br>>>>> int dijka[3] = {0};<br>>>>><br>>>>> dijka[0] = -1;<br>>>>> dijka[1] = -1;<br>>>>> dijka[2] = -1;<br>>>>> rval = ScdInterface::get_neighbor(nprocs, 0, spd, dijka, pto, rdims_a,<br>>>>> facedims_a, across_bdy_a);<br>>>>><br>>>>> dijka[0] = 0;<br>>>>> dijka[1] = -1;<br>>>>> dijka[2] = -1;<br>>>>> rval = ScdInterface::get_neighbor(nprocs, 0, spd, dijka, pto, rdims_a,<br>>>>> facedims_a, across_bdy_a);<br>>>>><br>>>>> std::cout << "Return from main()" << std::endl;<br>>>>><br>>>>> return 0;<br>>>>> }<br>>>>><br>>>>><br>>>>><br>>>>><br>>>>><br>>>><br>>>> --<br>>>> ================================================================<br>>>> "You will keep in perfect peace him whose mind is<br>>>> steadfast, because he trusts in you." Isaiah 26:3<br>>>><br>>>> Tim Tautges Argonne National Laboratory<br>>>> (tautges@mcs.anl.gov) (telecommuting from UW-Madison)<br>>>> phone (gvoice): (608) 354-1459 1500 Engineering Dr.<br>>>> fax: (608) 263-4499 Madison, WI 53706<br>>>><br>>><br>><br>> --<br>> ================================================================<br>> "You will keep in perfect peace him whose mind is<br>> steadfast, because he trusts in you." Isaiah 26:3<br>><br>> Tim Tautges Argonne National Laboratory<br>> (tautges@mcs.anl.gov) (telecommuting from UW-Madison)<br>> phone (gvoice): (608) 354-1459 1500 Engineering Dr.<br>> fax: (608) 263-4499 Madison, WI 53706<br>><br>><br></blockquote><br></div></body></html>