[Swift-devel] CDM Tests

Mon Nov 14 13:44:16 CST 2011

Ketan, if you can add these tests to the test suite in tests/cdm (or 
tests/cdm/absolute) that will be the fastest way to precisely determine 
what you are running and make these tests reproducible by the group.  I 
should be able to take a look at them later today.
 	Thanks
 	Justin

On Mon, 14 Nov 2011, Michael Wilde wrote:

> Even though the tests worked in each of these 4 cases, the question remains of "did the app see the same path names" and "did copies get done"?
>
> So I think you need to dig deeper and explain the difference between what CDM does for the direct case, and how (if at all) the wrapper mode affects things.
>
> Its equally possible that CDM overrode the wrapper mode arg.  Just specifying the wrapper mode arg will not, I think, achieve the elimination of the "copy to shared workflow directory" that "direct mode" accomplishes.
>
> - Mike
>
> ----- Original Message -----
>> From: "Ketan Maheshwari" <ketancmaheshwari at gmail.com>
>> To: "Michael Wilde" <wilde at mcs.anl.gov>
>> Cc: "Swift Devel" <swift-devel at ci.uchicago.edu>, "Justin M Wozniak" <wozniak at mcs.anl.gov>
>> Sent: Monday, November 14, 2011 11:29:36 AM
>> Subject: Re: [Swift-devel] CDM Tests
>> Mike,
>>
>>
>> In my initial email I said:
>>
>>
>> ... when specifying the relative option on the config for
>> wrapper.invocation.mode property the script works regardless of the
>> paths of input/output.
>>
>>
>> Which made me conclude that the option specified in the config for
>> wrapper.invocation.mode property overrides the CDM policies.
>>
>>
>> On Mon, Nov 14, 2011 at 1:21 PM, Michael Wilde < wilde at mcs.anl.gov >
>> wrote:
>>
>>
>> Ketan,
>>
>> I thought your initial email stated that the 4 cases of relative path
>> names "worked". But this latter comment indicates that there is some
>> kind of problem even in the relative cases.
>>
>> Can you clarify?
>>
>>
>> - Mike
>>
>>
>> ----- Original Message -----
>>> From: "Ketan Maheshwari" < ketancmaheshwari at gmail.com >
>>
>>
>>
>>> To: "Justin M Wozniak" < wozniak at mcs.anl.gov >
>>> Cc: "Michael Wilde" < wilde at mcs.anl.gov >, "Swift Devel" <
>>> swift-devel at ci.uchicago.edu >
>>> Sent: Monday, November 14, 2011 11:01:35 AM
>>> Subject: Re: [Swift-devel] CDM Tests
>>> Given the CDM's support for relative pathnames, I would expect the
>>> case of relative inputs, relative outputs would succeed irrespective
>>> of config. However, from the tests, it seems that the config option
>>> is
>>> overriding CDM directives.
>>>
>>>
>>> On Mon, Nov 14, 2011 at 12:44 PM, Justin M Wozniak <
>>> wozniak at mcs.anl.gov > wrote:
>>>
>>>
>>>
>>> CDM DIRECT does not currently support absolute path names. I think
>>> we
>>> could provide something simple like "" for the field, as you
>>> suggest.
>>>
>>>
>>>
>>>
>>> On Mon, 14 Nov 2011, Michael Wilde wrote:
>>>
>>>> I would expect that for absolute path mappings, the user would
>>>> specify
>>>> either :: or "/" as the path in the CDM DIRECT pattern. In other
>>>> words,
>>>> the user is saying "I have mapped the object to an absolute path
>>>> name,
>>>> and I want the app to get exactly this name". Hence I think that
>>>> the
>>>> best "location" field for a DIRECT rule is null ("").
>>>>
>>>> Justin, can you clarify what the code expects for this case?
>>>>
>>>> - Mike
>>>>
>>>> ----- Original Message -----
>>>>> From: "Jonathan Monette" < jonmon at mcs.anl.gov >
>>>>> To: "Michael Wilde" < wilde at mcs.anl.gov >
>>>>> Cc: "Ketan Maheshwari" < ketancmaheshwari at gmail.com >, "Swift
>>>>> Devel" < swift-devel at ci.uchicago.edu >
>>>>> Sent: Monday, November 14, 2011 9:13:01 AM
>>>>> Subject: Re: [Swift-devel] CDM Tests
>>>>> I couldn't tell from the excel(using my phone) but what was
>>>>> absolute?
>>>>> If I am not mistaken what CDM does is takes the path that is
>>>>> specified
>>>>> in the CDM file(which should be absolute) and appends the file
>>>>> path
>>>>> that is used in the swift script to it(which should be relative).
>>>>> So
>>>>> if they are both absolute I would expect it to fail.
>>>>>
>>>>>
>>>>>
>>>>> On Nov 14, 2011, at 10:56 AM, Michael Wilde < wilde at mcs.anl.gov >
>>>>> wrote:
>>>>>
>>>>>> Thanks, Ketan - very nice tests and summary document.
>>>>>>
>>>>>> I should point out for those that didn't open the doc: the 4
>>>>>> tests
>>>>>> with absolute pathnames are failing.
>>>>>>
>>>>>> Ketan, can you work with Justin to see if this is a bug, or if
>>>>>> the
>>>>>> CDM directive needs to be coded differently for absolute paths?
>>>>>> Then
>>>>>> please test a fix, and as we discussed adapt the tests with
>>>>>> annotations to enhance the User Guide section on CDM.
>>>>>>
>>>>>> - Mike
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Ketan Maheshwari" < ketancmaheshwari at gmail.com >
>>>>>>> To: "Swift Devel" < swift-devel at ci.uchicago.edu >
>>>>>>> Sent: Monday, November 14, 2011 8:13:48 AM
>>>>>>> Subject: [Swift-devel] CDM Tests
>>>>>>> Hello,
>>>>>>>
>>>>>>>
>>>>>>> I had a discussion with Mike about testing the CDM behavior for
>>>>>>> the
>>>>>>> following cases:
>>>>>>> 1. full versus relative paths for input
>>>>>>> 2. full versus relative paths for output
>>>>>>> 3. relative versus absolute option in config property:
>>>>>>> wrapper.invocation.mode
>>>>>>>
>>>>>>>
>>>>>>> In this regard, I made 8 tests for all the above combinations.
>>>>>>> I
>>>>>>> used
>>>>>>> simple local provider in this first set of tests.
>>>>>>>
>>>>>>>
>>>>>>> From the tests it seems that when specifying the relative
>>>>>>> option
>>>>>>> on
>>>>>>> the config for wrapper.invocation.mode property the script
>>>>>>> works
>>>>>>> regardless of the paths of input/output.
>>>>>>>
>>>>>>>
>>>>>>> A detailed result with stdout, and paths to logs can be found
>>>>>>> here:
>>>>>>> https://docs.google.com/spreadsheet/ccc?key=0AmvYSwENKFY9dG44V2VjRXJlUmZLNG9saERFeWZDcUE
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The tests are in my CI dir: /home/ketan/cdm_tests
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards, --
>>>>>>> Ketan
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Swift-devel mailing list
>>>>>>> Swift-devel at ci.uchicago.edu
>>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>>>>
>>>>>> --
>>>>>> Michael Wilde
>>>>>> Computation Institute, University of Chicago
>>>>>> Mathematics and Computer Science Division
>>>>>> Argonne National Laboratory
>>>>>>
>>>>>> _______________________________________________
>>>>>> Swift-devel mailing list
>>>>>> Swift-devel at ci.uchicago.edu
>>>>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>>
>>>>
>>>
>>> --
>>> Justin M Wozniak
>>>
>>>
>>>
>>> _______________________________________________
>>> Swift-devel mailing list
>>> Swift-devel at ci.uchicago.edu
>>> https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-devel
>>>
>>>
>>>
>>>
>>> --
>>> Ketan
>>
>> --
>> Michael Wilde
>> Computation Institute, University of Chicago
>> Mathematics and Computer Science Division
>> Argonne National Laboratory
>>
>>
>>
>>
>>
>> --
>> Ketan
>
>

-- 
Justin M Wozniak