Services State Summary

[Ivan R. Judson]

Goal
----

The goal of this task is to support dynamically executable external services
that can register and unregister with the venue server to provide
extensibility, options for administrators, and long term transitional paths
through various pieces of research technology, while at the same time
accomodating the stable operation of venue servers.

The services that are in use today are the event service, the text service
and the data service. For simplicity, and because the data service has not
been considered rigorously for 3.0, I'll only discuss the event and text
services.

Requirements
------------

- Support multiple services of each type
- Support variation among venue clients for service types
- Make services optional (the benefits being thus a choice of the client)
- Design using at least two examples (text: jabber, binary custom, event:
binary custom, ?)
	- Perhaps Binary Custom should be based on BEEP
(http://www.beepcore.org/)
- Allow dynamic execution of services
- Allow externally executed services and in process executed services

Design Overview
---------------

There are three entities involved in the services work, they need very few
interfaces or data to support this functionality, but the devil is in the
details. Here is the list of entities, annotated with what attributes and
methods they need:

Service
	update(<event>, <connection id> : none)

	create_channel(<channel id>)
	destroy_channel(<channel id>)

	create_connection(<channel id>, <connection id>)
	destroy_connection(<channel id>, <connection id>)

	get_description() => <service description>
	get_conn_status(<channel id>, <connection id>)

Venue
+ subscription list { conn, event_description(s) }*

	subscribe(<connection id>, <event description>)
	unsubscribe(<connection id>, <event description>)

	get_event_descriptions(string: query) => <event description>*
	set_status(<service id>, <channel id>, <connection id>, <status>)

VenueServer
+ subscription list {service, events(s)}*

	register(<service description>, <service reference>)
	unregister(<service id>)

	subscribe(<service id>, <event>)
	unsubscribe(<service id>, <event>)

	distribute_event(<event>, <channel>=None, <connection>=None)

	get_services(string: query) => <service description>*

All of the things in <>'s are data that are passed into or out of methods.
Here is the definition of those data objects. These are not complete, but
need to be finished next before much validation of the design can be done.

Service Description:
	Name: [string]
	Description: [string]
	ID: [string: GUID]
	Type: [string: mime type]
	Location: [network location | URL]

Service Rerence: ***
	Instance
	URL => wsdl2python gets a IW-like object

Channel
	ID: [string: GUID]
	Capabilities?
	Connections?

Connection
	ID: [string: GUID]
	Client?

[ I'm not entirely sure separating Event and Event Description makes any
sense, these need to be sorted out still. ]
	
Event:
Event Description:

Status: [string]

*** The magic of using services in process vs out of process is hidden by
enabling the Service Reference object to dispatch based on it's internal
data. If what it holds internally is an instance then it calls directly on
it, but if it's holding a string/url it creates a ServiceClient(url) as it's
internal instance -- from there on out it can call it like a local object.
This could be done in the constructor, like this:

def __init__(self, ref=None):
	if ref is None:
		raise Exception, "Can't initialize without a reference to a
service."
      elif isinstance(ref, types.ObjectType):
		self._instance = ref
	elif isinstance(ref, StringType):
		self._instance = ServiceClient(ref)
      else:
		raise Exception, "Initialization Error."

# I'm not sure of this syntax, but the idea is you overload __call__
# to invoke the method on self._instance and return the result
def __call__(self, *args, **kwargs):
	return self._instance(*args, **kwargs)

Dual Service Implementations
----------------------------
In order to ensure a good robust design, two implementations should be done
of a singe service from end to end to shake out design assumptions that
might be otherwise hidden. The way to do this efficiently is to focus on the
text service. We already have the LBL folks working on jabber infrastructure
so we have a jabber implementation that needs to be refined into this new
model. We can tailor the below "custom protocol" to do text fairly
trivially. Between these two services we should be able to verify the design
of the service structure and then implement either a custom event service
(based on the below protocol) or leverage an existing service.


Current Status
--------------

Currently all services have been removed from the venueserver, client and
other areas. The previous Globus based services are in cvs, but are not
being used. We should consider removing them from CVS and replacing them
with versions based on the below protocol. The above interfaces have not yet
been put into the VenueServer or Venue WSDL, I will do that while I finish
up the ZSI stuff and make sure the interface layer is working end to end.
The data objects need to be verified and schemas written and put into the
wsdl dir. I will leave that up to whoever picks up the work, but I'll be
happy to help out and answer questions to ease the transition.


--Ivan


Here is the packet structure for a binary (packed) service, that could be
used for either a custom text or custom event service, to verify the
implementation. This is based on discussions of the short comings of XML and
other data payloads.

packet:
	HEADER
	
	size: N
	channel: id
	from: conn_id
	flags: <message flags>
	n_recv: N
	recv_1: conn_id
	recv_2: conn_id
	recv_3: conn_id
	recv_n: conn_id
	
	DATA
	
Message Flags:
	SERVER_SUBSCRIBE
	SERVER_UNSUBSCRIBE
	PRIVATE_MESSAGE