[Swift-devel] [Bug 209] New: 1MB file corrupted when transferred using Coaster-Data-Provider

[bugzilla-daemon at mcs.anl.gov]

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=209

           Summary: 1MB file corrupted when transferred using
                    Coaster-Data-Provider
           Product: Swift
           Version: unspecified
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: General
        AssignedTo: hategan at mcs.anl.gov
        ReportedBy: zhaozhang at uchicago.edu


[zzhang at communicado 1KB]$ cat data-test.swift
type Mol {}
type DOCKOut {}

app (DOCKOut ofile) cat (Mol molfile)
{
  cat @filename(molfile) stdout=@filename(ofile);
}

Mol texts[] <filesys_mapper;location="input/", suffix=".mol2">;

doall(Mol texts[])
{
 foreach p in texts {
   DOCKOut r <regexp_mapper;             source=@p,            
match="input/(.*)\\.mol2",             transform=@strcat("result/", "\\1",
".out")    >;
     (r) = cat(p);
 }
}

// Main
doall(texts);


It copies a file "D00001.mol2", then redirects the file to "D00001.out",
"D00001.out" is taken as the output file.

I have two test cases on CI netowrk,
/home/zzhang/swift_coaster/cog/modules/swift/tests/sites/swift_data_test/1KB
and
/home/zzhang/swift_coaster/cog/modules/swift/tests/sites/swift_data_test/1MB
If you want to repeat the test, you will also need
/home/zzhang/swift_coaster/cog/modules/swift/tests/sites/tc.data and
/home/zzhang/swift_coaster/cog/modules/swift/tests/sites/coaster_test/AGLT2.xml
[zzhang at communicado 1MB]$ cat ../../coaster_test/AGLT2.xml
<config>
 <!-- AGLT2 -->
 <pool handle="AGLT2" >
   <filesystem provider="coaster" url="gt2://gate01.aglt2.org" />
   <execution  provider="coaster" url="gate01.aglt2.org"
jobManager="gt2:gt2:condor"/>
   <workdirectory >/atlas/data08/OSG/DATA/osg/tmp/AGLT2</workdirectory>
 </pool>
</config>


So, I ran the test twice with those two cases.
[zzhang at communicado 1KB]$ swift -sites.file ../../coaster_test/AGLT2.xml
-tc.file ../../tc.data data-test.swift
Swift svn swift-r2896 cog-r2392

RunID: 20090518-1355-dksv64l5
Progress:
Progress:  Stage in:1
Progress:  Submitting:1
Progress:  Submitted:1
Progress:  Submitted:1
Progress:  Submitted:1
Progress:  Active:1
Final status:  Finished successfully:1
Cleaning up...
Shutting down service at https://192.41.230.11:41507
Got channel MetaChannel: 7132795 -> GSSSChannel-null(1)
- Done
[zzhang at communicado input]$ cksum D00001.mol2
2023409842 1054 D00001.mol2
[zzhang at communicado input]$ cksum ../result/D00001.out
2023409842 1054 ../result/D00001.out

The 1KB test case is fine. Then I tried 1MB test case

[zzhang at communicado 1MB]$ swift -sites.file ../../coaster_test/AGLT2.xml
-tc.file ../../tc.data data-test.swift
Swift svn swift-r2896 cog-r2392
RunID: 20090518-1357-ad024vu1
Progress:
Progress:  Stage in:1
Progress:  Submitting:1
Progress:  Submitted:1
Progress:  Active:1
Progress:  Finished successfully:1
Final status:  Finished successfully:1
Cleaning up...
Shutting down service at https://192.41.230.11:47431
Got channel MetaChannel: 8676252 -> GSSSChannel-null(1)
- Done
[zzhang at communicado 1MB]$ cksum input/D00001.mol2
1748450002 1000000 input/D00001.mol2
[zzhang at communicado 1MB]$ cksum result/D00001.out
4159559983 1000000 result/D00001.out

as you could tell from the check sum, the input and output are not the same. I
repeated the 1MB test case, the same
thing happened
[zzhang at communicado 1MB]$ swift -sites.file ../../coaster_test/AGLT2.xml
-tc.file ../../tc.data data-test.swift
Swift svn swift-r2896 cog-r2392

RunID: 20090518-1629-1gef4tm8
Progress:
Progress:  Stage in:1
Progress:  Submitting:1
Progress:  Submitted:1
Progress:  Active:1
Progress:  Finished successfully:1
Final status:  Finished successfully:1
Cleaning up...
Shutting down service at https://192.41.230.11:43400
Got channel MetaChannel: 31518612 -> GSSSChannel-null(1)
- Done
[zzhang at communicado 1MB]$ cksum input/D00001.mol2
1748450002 1000000 input/D00001.mol2
[zzhang at communicado 1MB]$ cksum result/D00001.out
2414944118 1000000 result/D00001.out

-- 
Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.