[Swift-devel] How does swift know if a task is successful

[Zhao Zhang]

Hi, Mihael

I commented the following lines
  /*dir:make(ldir)
     restartOnError(".*", 2
     task:transfer(srchost=host, srcfile=bname,
     srcdir=rdir, destdir=ldir, desthost=dhost, destprovider=provider)
  )*/

Then I modified wrapper.sh to not to copy output file back, but I still 
got an error.
The log file is at 
http://www.ci.uchicago.edu/~zzhang/first-20090317-1327-oqgttus8.log
Thanks

zhao

zzhang at login6.surveyor:~/new_dock6> ./run_swift_ssh.sh 1117 64 first.swift
waiting for at least 64 nodes to register before submitting workload...
waiting to find at least 1 services in file 
/home/falkon/users/zzhang/1117/config/Client-service-URIs.config...
all done, file has found at least 1 services
found at least 64 registered, submitting workload...
Swift svn swift-r2676 (swift modified locally) cog-r2305

RunID: 20090317-1327-oqgttus8
Progress:
Progress:  Selecting site:1 Stage in:1
Progress:  Submitting:1 Submitted:1
Progress:  Submitted:1 Failed but can retry:1
Failed to transfer wrapper log from 
first-20090317-1327-oqgttus8/info/b/n/bgp000
Progress:  Submitted:1 Active:1
Failed to transfer wrapper log from 
first-20090317-1327-oqgttus8/info/e/n/bgp000
Progress:  Submitted:1 Active:1
Failed to transfer wrapper log from 
first-20090317-1327-oqgttus8/info/g/n/bgp000
Execution failed:
        Exception in echo:
Arguments: [Hello, world!]
Host: bgp000
Directory: first-20090317-1327-oqgttus8/jobs/g/n/echo-gnlq238j
stderr.txt:

stdout.txt:

----

Caused by:
        Cannot transfer 
"/tmp/first-20090317-1327-oqgttus8/shared/hello.txt" to 
"/gpfs/home/zzhang/new_dock6/./hello.txt"
Caused by:
        No such file


Mihael Hategan wrote:
> On Tue, 2009-03-17 at 12:23 -0500, Zhao Zhang wrote:
>   
>> Hi, Mihael
>>
>> yes, can I do that?
>>     
>
> You should know this by now:
> in vdl-int.k, in doStageout, comment out the task:transfer invocation
> (and dir:make).
>
>   
>> zhao
>>
>> Mihael Hategan wrote:
>>     
>>> On Tue, 2009-03-17 at 12:14 -0500, Zhao Zhang wrote:
>>>   
>>>       
>>>> Here comes another question, is there any place that I could set to 
>>>> disable swift's waiting for data feature?
>>>>     
>>>>         
>>> Do you mean disable the stage-outs?
>>>
>>>   
>>>       
>>>> Or is there any way for me to cheat swift that the data is already 
>>>> there? thanks.
>>>>
>>>> zhao
>>>>
>>>> Mihael Hategan wrote:
>>>>     
>>>>         
>>>>> On Fri, 2009-03-13 at 17:07 -0500, Zhao Zhang wrote:
>>>>>   
>>>>>       
>>>>>           
>>>>>> Hi, All
>>>>>>
>>>>>> I have a question on how swift knows if a task is successful.
>>>>>> In my case, I am using a status notification instead of a status file.
>>>>>>
>>>>>> So my question is is this status notification the only thing swift is
>>>>>> waiting for, or is swift also waiting for the output data to appear to
>>>>>> say that a job is successful?
>>>>>>     
>>>>>>         
>>>>>>             
>>>>> Once the job is done, swift will attempt to stage out all the files that
>>>>> it expects the job to have produced.
>>>>>
>>>>> Should one of those files not be there, there will be failures.
>>>>>
>>>>>
>>>>>
>>>>>   
>>>>>       
>>>>>           
>>>   
>>>       
>
>
>