[Swift-user] persitent service + manual workers

[wilde at mcs.anl.gov]

Allan, I think the worker should connect on port 50000 and Swift should connect on port 1984.

I further think that if you are running the service and the workers manually, you probably want to set the sites.xml entry for Swift to coasters-persistent and passive mode, otherwise the service seems to enforce come kind of state machine whereby it expects Swift to operate in automatic mode.

Ive started trying to document this on page 1 of the attached document but this needs more work.

What I found works for me is to run a dummy swift job set to persistent+passive mode, as there is no way to force the standalone service to passive move by command line option. My R scripts in ~wilde/SwiftR/swift/exec/start-swift-workers do this but are a bit complex and in the process of cleanup.  If I get something from the R scripts that you can use I'll send asap.

- Mike


----- "Allan Espinosa" <aespinosa at cs.uchicago.edu> wrote:

> Hi,
> 
> I'm having trouble with the workers registering with the coaster
> service:
> 
> $ ./trunk/bin/coaster-service -nosec
> Local contacts: [http://128.135.125.18:50000]
> Started local service: http://128.135.125.18:50000
> Started coaster service: http://128.135.125.18:1984
> Started coaster service: http://128.135.125.18:1984
> SC-null: Disabling heartbeats (config is null)
> Multiplexer 0 started
> (0) Scheduling SC-null for addition
> nullChannel started
> Multiplexer 1 started
> Unknown handler: REGISTER. Available handlers: {CHMOD=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.ChmodHandler,
> ISDIR=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.IsDirectoryHandler,
> LIST=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.ListHandler,
> SUBMITJOB=class
> org.globus.cog.abstraction.coaster.service.SubmitJobHandler,
> MKDIR=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.MkdirHandler,
> PUT=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.PutFileHandler,
> DEL=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.DeleteHandler,
> HEARTBEAT=class
> org.globus.cog.karajan.workflow.service.handlers.HeartBeatHandler,
> CONFIGSERVICE=class
> org.globus.cog.abstraction.coaster.service.ServiceConfigurationHandler,
> FILEINFO=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.FileInfoHandler,
> SHUTDOWNSERVICE=class
> org.globus.cog.abstraction.coaster.service.ServiceShutdownHandler,
> SHUTDOWN=class
> org.globus.cog.karajan.workflow.service.handlers.ShutdownHandler,
> EXISTS=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.ExistsHandler,
> CHANNELCONFIG=class
> org.globus.cog.karajan.workflow.service.handlers.ChannelConfigurationHandler,
> RMDIR=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.RmdirHandler,
> RENAME=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.RenameHandler,
> VERSION=class
> org.globus.cog.karajan.workflow.service.handlers.VersionHandler,
> WORKERSHELLCMD=class
> org.globus.cog.abstraction.coaster.service.WorkerShellHandler,
> GET=class
> org.globus.cog.abstraction.impl.file.coaster.handlers.GetFileHandler}
> 
> 
> Worker connection invocation:
> $
> /home/aespinosa/swift/cogkit/modules/provider-coaster/resources/worker.pl
> http://128.135.125.18:1984  foo /home/aespinosa/tmp
> Failed to process data: Failed to register (service returned error:
> Unknown command: REGISTER) at
> /home/aespinosa/swift/cogkit/modules/provider-coaster/resources/worker.pl
> line 676.
> 
> 
> 
> Invocation through the other port:
> $
> /home/aespinosa/swift/cogkit/modules/provider-coaster/resources/worker.pl
> http://128.135.125.18:50000  foo /home/aespinosa/tmp
> Failed to process data: Failed to register (service returned error:
> java.lang.NullPointerException) at
> /home/aespinosa/swift/cogkit/modules/provider-coaster/resources/worker.pl
> line 676.
> 
> service dump:
> $ ./trunk/bin/coaster-service -nosec
> Local contacts: [http://128.135.125.18:50000]
> Started local service: http://128.135.125.18:50000
> Started coaster service: http://128.135.125.18:1984
> Started coaster service: http://128.135.125.18:1984
> SC-null: Disabling heartbeats (config is null)
> Multiplexer 0 started
> (0) Scheduling SC-null for addition
> nullChannel started
> Multiplexer 1 started
> Received registration: blockid = foo, url =
> Avg stream buf: 0
> Avg stream buf: 0
> Avg stream buf: 0
> 
> 
> I'm using the latest trunk code from swift and cog
> 
> 
> 
> 
> Just to confirm the "local service" is where swift submits the jobs
> and "coaster service" is the one the workers connect to, correct?
> 
> Thanks,
> -Allan
> 
> -- 
> Allan M. Espinosa <http://amespinosa.wordpress.com>
> PhD student, Computer Science
> University of Chicago <http://people.cs.uchicago.edu/~aespinosa>
> _______________________________________________
> Swift-user mailing list
> Swift-user at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-user

-- 
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division
Argonne National Laboratory

-------------- next part --------------
A non-text attachment was scrubbed...
Name: SwiftConfigurations.odg
Type: application/vnd.oasis.opendocument.graphics
Size: 21664 bytes
Desc: not available
URL: <http://lists.mcs.anl.gov/pipermail/swift-user/attachments/20100928/d8b47534/attachment.odg>