[ExM Users] Passing two dimensional numerical arrays to leaf functions

Ozik, Jonathan jozik at anl.gov
Wed Sep 24 15:28:59 CDT 2014 

It looks like line 542 in files.tcl calls blobutils_cast_to_int so I replaced that with blobutils_cast_to_long and it seems to work.

Jonathan

On Sep 24, 2014, at 3:07 PM, Ozik, Jonathan <jozik at anl.gov<mailto:jozik at anl.gov>> wrote:

I believe I’ve applied the patches but it appears that the blobutils_cast_to_int is still being called with the same error resulting:
$ turbine test-b-simple.tcl
   0.000 MODE: SERVER
   0.000 MODE: WORKER
   0.000 MODE: WORKER
   0.000 WORKERS: 2 RANKS: 0 - 1
   0.000 SERVERS: 1 RANKS: 2 - 2
   0.001 function:swift:constants
   0.001 function:swift:constants
   0.001 enter function: main
   0.001 allocated u:b=<1> u:v=<2>
   0.001 allocated file: u:data=<3>
   0.001 store: <3>=path input.data
   0.010 blob_read: input.data
valgrind_assert(): failed: src/tcl/blob/blob.c:142
valgrind_assert(): pointer is too long for int!

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 69379 RUNNING AT nsit-dhcp-250-217.uchicago.edu<http://nsit-dhcp-250-217.uchicago.edu/>
=   EXIT CODE: 6
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================

FYI, this is by running the blob example (test-b-swift.tcl).

Jonathan

On Sep 24, 2014, at 2:41 PM, Tim Armstrong <tim.g.armstrong at gmail.com<mailto:tim.g.armstrong at gmail.com>> wrote:

Sorry, I should've checked that it applied cleanly. The change in the patch is ok, it just didn't have the right context to apply.  These are the lines that needed to be added to blob.h


+/** DOCD(blobutils_cast_long_to_int_ptr i, +long+ to +int*+.) */
+      int*    blobutils_cast_long_to_int_ptr      (long l);
+/** DOCD(blobutils_cast_long_to_const_int_ptr i, +long+ to +const int*+.) */
+const int*    blobutils_cast_long_to_const_int_ptr(long l);
+/** DOCD(blobutils_cast_long_to_dbl_ptr i, +long+ to +double*+.) */
+      double* blobutils_cast_long_to_dbl_ptr      (long l);
+/** DOCD(blobutils_cast_long_to_const_dbl_ptr i, +long+ to +const double*+.) */
+const double* blobutils_cast_long_to_const_dbl_ptr(long l);

You shouldn't need any additional patches.  An updated patch that includes all the fixes is attached.

- Tim

On Wed, Sep 24, 2014 at 2:00 PM, Ozik, Jonathan <jozik at anl.gov<mailto:jozik at anl.gov>> wrote:
Thanks Tim.
It looks like my blob.h file is a little shorter than yours. So I got a patching failure:
$ patch -p 5 < ~/Downloads/turbine-0.5.1-pointer-fix.patch
patching file lib/blob.tcl
patching file src/tcl/blob/blob.c
patching file src/tcl/blob/blob.h
Hunk #1 FAILED at 184.
1 out of 1 hunk FAILED -- saving rejects to file src/tcl/blob/blob.h.rej

I haven’t modified blob.h from the 0.6.1 version, so maybe there are more patches that I should apply?

Jonathan

On Sep 24, 2014, at 1:49 PM, Tim Armstrong <tim.g.armstrong at gmail.com<mailto:tim.g.armstrong at gmail.com>> wrote:

Hi Jonathan,
  This looks like another bug that hadn't manifested in our tests.  I've got a fix for it and am in the process of putting together a new release.  In the meantime, you can patch the source if needed with the attached patch.

cd turbine
patch -p 5 < turbine-0.5.1-pointer-fix.patch

- Tim

On Tue, Sep 23, 2014 at 3:54 PM, Ozik, Jonathan <jozik at anl.gov<mailto:jozik at anl.gov>> wrote:
I’m now seeing this error regarding pointer length:

$ turbine test-b-simple.tcl
   0.000 MODE: WORKER
   0.000 MODE: WORKER
   0.000 MODE: SERVER
   0.000 WORKERS: 2 RANKS: 0 - 1
   0.000 SERVERS: 1 RANKS: 2 - 2
   0.001 function:swift:constants
   0.001 function:swift:constants
   0.001 enter function: main
   0.002 allocated u:b=<1> u:v=<2>
   0.002 allocated file: u:data=<3>
   0.003 store: <3>=path input.data
   0.003 blob_read: input.data
valgrind_assert(): failed: src/tcl/blob/blob.c:142
valgrind_assert(): pointer is too long for int!

Jonathan

On Sep 23, 2014, at 11:18 AM, Tim Armstrong <tim.g.armstrong at gmail.com<mailto:tim.g.armstrong at gmail.com>> wrote:

Oops, one additional fix was needed:

-     proc blob_read_body { result input } {
+     proc blob_read_body { result src } {
        set val [ retrieve_decr_file $src ]
-       set input_name [ dict get $val path ]
+       set input_name [ local_file_path $val ]


On Tue, Sep 23, 2014 at 11:08 AM, Ozik, Jonathan <jozik at anl.gov<mailto:jozik at anl.gov>> wrote:
Thanks Tim!


On Sep 23, 2014, at 10:51 AM, Tim Armstrong <tim.g.armstrong at gmail.com<mailto:tim.g.armstrong at gmail.com>> wrote:

There's a bug in one of our library functions.  Thanks for reporting.  I've fixed it in the repository so it'll get into the next point release. If you want to quickly patch your version as a workaround, you can edit line 535 of turbine/lib/files.tcl in the Swift/T source as below, then run "make install" to install the fixed version of the file.

-    proc blob_read_body { result input } {
+   proc blob_read_body { result src } {

- Tim

On Tue, Sep 23, 2014 at 9:47 AM, Ozik, Jonathan <jozik at anl.gov<mailto:jozik at anl.gov>> wrote:
I’m attempting the blob example (http://swift-lang.org/Swift-T/leaf.html#_complete_example_3_c_function_with_binary_data) and am seeing this error:

>> turbine test-b-simple.tcl
   0.000 MODE: SERVER
   0.000 MODE: WORKER
   0.000 WORKERS: 2 RANKS: 0 - 1
   0.000 SERVERS: 1 RANKS: 2 - 2
   0.000 MODE: WORKER
   0.001 function:swift:constants
   0.001 function:swift:constants
   0.001 enter function: main
   0.002 allocated u:b=<1> u:v=<2>
   0.002 allocated file: u:data=<3>
   0.002 store: <3>=path input.data
CAUGHT ERROR:
can't read "src": no such variable
    while executing
"retrieve_decr_file $src "
    (procedure "blob_read_body" line 2)
    invoked from within
"blob_read_body 1 {file 3 is_mapped 0}"
Turbine worker task error in: blob_read_body 1 {file 3 is_mapped 0}
    invoked from within
"c::worker_loop $WORK_TYPE(WORK)"
    (procedure "worker" line 28)
    invoked from within
"worker $rules $startup_cmd"
    (procedure "enter_mode_unchecked" line 5)
    invoked from within
"enter_mode_unchecked $rules $startup_cmd"
    (procedure "enter_mode" line 5)
    invoked from within
"enter_mode $rules $startup_cmd "
CALLING adlb::abort
ADLB_Abort(1)
MPI_Abort(1)
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

It appears that this is being triggered by the line:
printf("size(v) = %i", size(v));

Any thoughts on what might be going on?

I’m testing this on a Mac OS X setup.

Jonathan

On Sep 22, 2014, at 10:20 PM, Ozik, Jonathan <jozik at anl.gov<mailto:jozik at anl.gov>> wrote:

> Thank you Justin. That should work great.
>
> Jonathan
>
> On Sep 22, 2014, at 8:05 PM, Justin M Wozniak <wozniak at mcs.anl.gov<mailto:wozniak at mcs.anl.gov>> wrote:
>
>> Hi
>>   You can do this based on the instructions for C binary data:
>>
>> http://swift-lang.org/Swift-T/leaf.html#_complete_example_3_c_function_with_binary_data
>>
>> The Swift blob is a void* - you can cast to whatever you want.  You can cast it to a double* that is actually a 2D array (following BLAS conventions).
>>
>>   Justin
>>
>> On 9/22/2014 6:32 PM, Ozik, Jonathan wrote:
>>> Hello all,
>>>
>>> I’m curious about how people generally go about passing (and returning) two (or n-) dimensional arrays to leaf functions. Are there simple examples using SWIG and Tcl? Or are file based approaches used via app fuctions?
>>>
>>> Any thoughts would be appreciated,
>>>
>>> Jonathan
>>>
>>>
>>>
>>> _______________________________________________
>>> ExM-user mailing list
>>> ExM-user at lists.mcs.anl.gov<mailto:ExM-user at lists.mcs.anl.gov>
>>> https://lists.mcs.anl.gov/mailman/listinfo/exm-user
>>
>> --
>> Justin M Wozniak
>>
>
> _______________________________________________
> ExM-user mailing list
> ExM-user at lists.mcs.anl.gov<mailto:ExM-user at lists.mcs.anl.gov>
> https://lists.mcs.anl.gov/mailman/listinfo/exm-user

_______________________________________________
ExM-user mailing list
ExM-user at lists.mcs.anl.gov<mailto:ExM-user at lists.mcs.anl.gov>
https://lists.mcs.anl.gov/mailman/listinfo/exm-user


_______________________________________________
ExM-user mailing list
ExM-user at lists.mcs.anl.gov<mailto:ExM-user at lists.mcs.anl.gov>
https://lists.mcs.anl.gov/mailman/listinfo/exm-user


<turbine-0.5.1-pointer-fix.patch>


<turbine-0.5.1-pointer-fix-v2.patch>

_______________________________________________
ExM-user mailing list
ExM-user at lists.mcs.anl.gov<mailto:ExM-user at lists.mcs.anl.gov>
https://lists.mcs.anl.gov/mailman/listinfo/exm-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/exm-user/attachments/20140924/9d2aac3d/attachment-0001.html>

More information about the ExM-user mailing list