[Swift-devel] How does swift know if a task is successful

[Mihael Hategan]

On Tue, 2009-03-17 at 13:36 -0500, Zhao Zhang wrote:
> Hi, Mihael
> 
> I commented the following lines
>   /*dir:make(ldir)
>      restartOnError(".*", 2
>      task:transfer(srchost=host, srcfile=bname,
>      srcdir=rdir, destdir=ldir, desthost=dhost, destprovider=provider)
>   )*/
> 

Did you modify this file in dist/?/libexec? If not, did you re-compile
swift after the modification?

Put an echo or a log message in place, to see if your change is picked
up by swift next time.

> Then I modified wrapper.sh to not to copy output file back, but I still 
> got an error.
> The log file is at 
> http://www.ci.uchicago.edu/~zzhang/first-20090317-1327-oqgttus8.log
> Thanks
> 
> zhao
> 
> zzhang at login6.surveyor:~/new_dock6> ./run_swift_ssh.sh 1117 64 first.swift
> waiting for at least 64 nodes to register before submitting workload...
> waiting to find at least 1 services in file 
> /home/falkon/users/zzhang/1117/config/Client-service-URIs.config...
> all done, file has found at least 1 services
> found at least 64 registered, submitting workload...
> Swift svn swift-r2676 (swift modified locally) cog-r2305
> 
> RunID: 20090317-1327-oqgttus8
> Progress:
> Progress:  Selecting site:1 Stage in:1
> Progress:  Submitting:1 Submitted:1
> Progress:  Submitted:1 Failed but can retry:1
> Failed to transfer wrapper log from 
> first-20090317-1327-oqgttus8/info/b/n/bgp000
> Progress:  Submitted:1 Active:1
> Failed to transfer wrapper log from 
> first-20090317-1327-oqgttus8/info/e/n/bgp000
> Progress:  Submitted:1 Active:1
> Failed to transfer wrapper log from 
> first-20090317-1327-oqgttus8/info/g/n/bgp000
> Execution failed:
>         Exception in echo:
> Arguments: [Hello, world!]
> Host: bgp000
> Directory: first-20090317-1327-oqgttus8/jobs/g/n/echo-gnlq238j
> stderr.txt:
> 
> stdout.txt:
> 
> ----
> 
> Caused by:
>         Cannot transfer 
> "/tmp/first-20090317-1327-oqgttus8/shared/hello.txt" to 
> "/gpfs/home/zzhang/new_dock6/./hello.txt"
> Caused by:
>         No such file
> 
> 
> Mihael Hategan wrote:
> > On Tue, 2009-03-17 at 12:23 -0500, Zhao Zhang wrote:
> >   
> >> Hi, Mihael
> >>
> >> yes, can I do that?
> >>     
> >
> > You should know this by now:
> > in vdl-int.k, in doStageout, comment out the task:transfer invocation
> > (and dir:make).
> >
> >   
> >> zhao
> >>
> >> Mihael Hategan wrote:
> >>     
> >>> On Tue, 2009-03-17 at 12:14 -0500, Zhao Zhang wrote:
> >>>   
> >>>       
> >>>> Here comes another question, is there any place that I could set to 
> >>>> disable swift's waiting for data feature?
> >>>>     
> >>>>         
> >>> Do you mean disable the stage-outs?
> >>>
> >>>   
> >>>       
> >>>> Or is there any way for me to cheat swift that the data is already 
> >>>> there? thanks.
> >>>>
> >>>> zhao
> >>>>
> >>>> Mihael Hategan wrote:
> >>>>     
> >>>>         
> >>>>> On Fri, 2009-03-13 at 17:07 -0500, Zhao Zhang wrote:
> >>>>>   
> >>>>>       
> >>>>>           
> >>>>>> Hi, All
> >>>>>>
> >>>>>> I have a question on how swift knows if a task is successful.
> >>>>>> In my case, I am using a status notification instead of a status file.
> >>>>>>
> >>>>>> So my question is is this status notification the only thing swift is
> >>>>>> waiting for, or is swift also waiting for the output data to appear to
> >>>>>> say that a job is successful?
> >>>>>>     
> >>>>>>         
> >>>>>>             
> >>>>> Once the job is done, swift will attempt to stage out all the files that
> >>>>> it expects the job to have produced.
> >>>>>
> >>>>> Should one of those files not be there, there will be failures.
> >>>>>
> >>>>>
> >>>>>
> >>>>>   
> >>>>>       
> >>>>>           
> >>>   
> >>>       
> >
> >
> >