[AG-TECH] Struggling with starting of producer service and encapsulated executable

[Brian Corrie]

Hi Tom,

Thanks for the info. That clarifies some things for me... A couple more 
questions below.

Thomas D. Uram wrote:
> Hi Brian, Michael:
> 
> I'm happy that you're working on node services.  You've gleaned alot of 
> the details correctly; we'll be refining this in the next release, 
> during which time your experiences will be helpful.
> 
> The intention with the node services is this:  When a user enters a 
> venue, their capabilities (as derived from services in the node) are 
> presented to the venue to determine which streams are appropriate for 
> that user.  For each type of "producer" capability, the venue allocates 
> a stream (multicast address) if one does not exist.  "Consumer" 
> capabilities are matched against existing streams, and the resulting 
> streams are returned; streams are never allocated for consumers.  In the 
> case of the audio and video services, when the streams are set, the 
> services restart, using the new stream information.

The stream allocation is a multicast address that is generated by the 
venue server I believe. Is that correct? If so, is it possible for the 
producer service to "reconfigure the stream" to a fixed IP number. In 
the multicast case this might be a fixed multicast IP/port pair that the 
service uses. This would allow the service to manage the addresses 
instead of the venue (this is required if the service can not use 
arbitrary multicast addresses).

In our case, we have a client server application that we want to 
integrate, and therefore we want to assign a unicast address to the 
stream. Again, the service would manage the addresses provided. Does 
this break the model you have (does it depend on multicast necessarily)?

Also, is there any reason that a NodeService should NOT refer to the 
Venue data store like some of the SharedApps do?

> While the Capability class defines the video, audio, and text types, the 
> "type" is intended to be arbitrary.  This is so that additional node 
> services can be developed with a capability of a different type, and the 
> venue will allocate a stream (mcast address) for them.  So, you could 
> define the type as "vr_hlrs", for example, and have the service work 
> correctly.   I'll look into why this isn't working for the 'text' type, 
> and resolve that for the 2.2 release.

That is what I thought, but because the text thing didn't appear to work 
and video did, it appeared that I "didn't understand" 8-)

> I hope this helps to clarify some of the issues.  After talking with 
> Ivan today, I'm planning to write up details of the workings of node 
> services and their development, and will let you know when that's 
> available (though at the moment the release certainly takes precedence  ;-)

That would be great!!!

Cheers,

Brian

> Brian Corrie wrote:
> 
>> Hi Michael,
>>
>> Michael Braitmaier wrote:
>>
>>> Hello Brian!
>>>
>>> Thanks so far for the information. At least I got it working so my 
>>> small app is called from within the producer service. Addditionally 
>>> now wrote a consumer service. My main problem now is how to get these 
>>> two services set up correctly. Up to now I still only integrated the 
>>> producer service into the node services associated with a venue 
>>> client. Its still unclear to me how I can tell the venue server to 
>>> load my producer service, so when I connect with a venue client to 
>>> the ag node running the venue server, I get a. connection betwen the 
>>> app of the consumer service and the app of the producer service.
>>
>>
>>
>> Ideally I think what you want to be able to do here (at least what we 
>> would like to do here) is to be able to have the producer configure 
>> the "StreamDescription" that is being used for that stream. There may 
>> be a way to do that but I haven't figured it out yet. Thus when the 
>> producer would start it would configure the stream appropriately. When 
>> the consumer starts it just looks at the stream and connects based on 
>> the information in that stream.
>>
>> The way we get around this is to set the host configuration info as a 
>> service parameter that you can change when the node service is loaded 
>> or configured. Of course, this is not satisfactory (AKA a hack) 
>> because this has to be done manually...
>>
>> It is also worth noting that this mechanism does not associate the 
>> services with particular venues. Thus whenever you enter a venue on 
>> the machine that has these services installed you will get the 
>> applications appearing. What we want to be able to do is to associate 
>> this with a specific venue so that the service only starts up in that 
>> venue. This is what the Shared App model gives you. Unfortunately 
>> there is no producer consumer model in the shared app framework so a 
>> client/server type of model doesn't seem to fit all that well.
>>
>> I have been looking at the RasMol shared app (see the AG Wiki) and it 
>> uses the Venue infrastructure in some interesting ways. There are lots 
>> of things you can do in the AG environment, the big problem is 
>> figuring out what capabilities are available. The RasMol app was 
>> written by Bob Olson and Tom Uram, the AG developers. They know what 
>> the useful hooks are so it is a good place to look for those that are 
>> working on shared apps 8-)
>>
>>> I can confirm that using TEXT as role for services does't work the 
>>> way I would have expected. Though the service is started, the 
>>> application belonging to the service is not. If I change the role to 
>>> VIDEO, it works fine. Generally I wonder what influence the role 
>>> parameter of the capapilities class has, besides restricting call of 
>>> executeables?
>>
>>
>>
>> I am not sure on that one. My inclination is that it specifies the 
>> type of stream that is created. It may be that audio and video are 
>> special in some way so that their streams are created auto-magically 
>> in each venue. There may be something one has to do to create a TEXT 
>> stream for example???  Without the stream creation the application 
>> would probably not start... Any comments from the AG team?
>>
>> Cheers,
>>
>> Brian
>>
>>> Brian Corrie wrote:
>>>
>>>> Hi Ivan, Michael,
>>>>
>>>> We have gone down this path as well, but have had a bit more success 
>>>> with the NodeService side of things. There are really two issues 
>>>> here. One is getting the node service to work. The other is on which 
>>>> way to integrate the application...
>>>>
>>>> Integrating Node Services:
>>>>
>>>> We have a very early alpha visualization service integrated into AG 
>>>> as a NodeService. We do not use the streamDescription.location.host 
>>>> and port in our service. I have not found a way of actually 
>>>> assigning any info to the streamDescription though. For the service 
>>>> we use we are currently using the service configuration stuff to 
>>>> manage this. This is a hack for now until we can find another 
>>>> mechanism to configure the stream description directly... I haven't 
>>>> experimented with that alot.
>>>>
>>>> One question I have for you is what capabilities does your service 
>>>> use. I was unable to get our service to work unless I used 
>>>> Capability.PRODUCER and Capability.VIDEO (AUDIO would probably work 
>>>> as well). Capability.TEXT did not work and I didn't see an easy way 
>>>> to use anything else cleanly in the same model... So again, a hack 
>>>> to make it work was to use VIDEO and PRODUCER.
>>>>
>>>> You have to create a zip file and a service description file (svc 
>>>> file) and stick them in the C:\Documents and Settings\All 
>>>> Users\Application Data\AccessGrid\services folder (on Windows of 
>>>> course). Once I did, this appears in my list of services in the 
>>>> NodeManager, I can add it, and when I change venues my visualization 
>>>> application starts up with the correct connection information... 
>>>> Unfortunately the connection info is not communicated as part of the 
>>>> stream description...
>>>>
>>>> WRT node services this raises a question in my mind as to what it 
>>>> means to be a producer and a consumer in the AG model. Ivan (or 
>>>> anyone else) can you comment? I would have thought that a producer 
>>>> would create a stream if it did not already exist and if it already 
>>>> existed it would do nothing. It also seems to make sense that a 
>>>> consumer would only start up if the appropriate stream existed and 
>>>> it would not if there were no producers in the environment. This 
>>>> does not seem to be the case at the moment.
>>>>
>>>> This is the model that I would have liked for our service and I 
>>>> think what Michael was leaning toward. My node creates a 
>>>> VizProducerService. This checks to see if a stream exists of this 
>>>> type (Capability.VIZ). If not it creates a stream with the correct 
>>>> info. In the video case this would be the assignment of a stream 
>>>> with a multicast address. In a client server model it might create a 
>>>> stream that was a unicast address to connect to. In the client 
>>>> server model it probably only makes sense to have one Producer. When 
>>>> a consumer joins it checks to see if a stream of the correct type 
>>>> exists and if so it joins it. The stream info tells it how to 
>>>> connect to the server...
>>>>
>>>> This is kind of what I thought the model was for Node Services. I 
>>>> don't think this is correct but I am not sure.
>>>>
>>>> There are some problems with this model of course, which brings us 
>>>> to the second part of the question:
>>>>
>>>> How to integrate applications:
>>>>
>>>> The problem with the above model is that my visualization 
>>>> application starts up whenever I enter a venue (any venue). What I 
>>>> really want is to be able to associate a visualization application 
>>>> with a specific venue. This is what the Shared Application model 
>>>> that Ivan talked about does for you. Michael, I think we are doing 
>>>> exactly the same thing that you are. Trying to determine how best to 
>>>> integrate applications into AG (NodeService versus SharedApp). I am 
>>>> currently working on the same visualization service but instead 
>>>> works under the SharedApp model. This is fairly nice from what I can 
>>>> tell, as you can publish information into the venue and other 
>>>> clients can consume that information (connection info, etc). In the 
>>>> AG 2.2 code there is a VNC client/server Shared App that is very 
>>>> good for basing such an App on (we are having some problems with 
>>>> AG2.2 - anyone else been brave and able to get AG 2.2 alpha up and 
>>>> running?). I don't have this working yet, but hopefully once we get 
>>>> AG 2.2 up and running we will have some success...
>>>>
>>>> The problem with this model is that I would like to have the shared 
>>>> application start up automatically when a venue is entered (like 
>>>> audio and video). Ultimately it would be preferable to have the 
>>>> option to either start automatically or have to start it directly 
>>>> (both are important)... Is there a way to do that with SharedApps?
>>>>
>>>> Best of luck getting the node service going... If you have more 
>>>> questions give me a shout.
>>>>
>>>> Cheers,
>>>>
>>>> Brian
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Michael Braitmaier wrote:
>>>>
>>>>> In fact we aim for integrating VR-collaboration into AccessGrid. At 
>>>>> the moment I am evaluating the different methods for integration. 
>>>>> We already did a small shared application coupling the UI part of 
>>>>> the VR-controll software with AccessGrid. Adiditionally I am 
>>>>> working on a method of allowing our render software to have a 
>>>>> connection to access grid. As I said this just experimental at the 
>>>>> moment. But I think it might be a good idea to consider our 
>>>>> VR-rendering software as a kind of a capability of a node, or in 
>>>>> more detail have some kind of daemon or server which provides a 
>>>>> interface to the VR environment for other access grid participants. 
>>>>> This server would then also only be started if you enter a venue or 
>>>>> leave it, providing a continues stream of 3D data. Initially I 
>>>>> first want to check out the complexity of integration of our 
>>>>> application as a service before I decide on one way of 
>>>>> implementation. Therefore I wrote a small client server app just 
>>>>> sending information using sockets. I now want to incorporate this 
>>>>> app into a producer service python script so there is alaways a 
>>>>> server app running on the node and of course get a client instance 
>>>>> of the app running when entering a venu on the server.
>>>>> But as described I still struggle with invoking my server app. 
>>>>> Furthermore I don' t  know at which point to integrate the client 
>>>>> side application call.
>>>>>
>>>>> Thanks
>>>>> Michael
>>>>>
>>>>> Ivan R. Judson wrote:
>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> I'm sorry I haven't replied to your email yet, can you tell me 
>>>>>> what the
>>>>>> application you are trying to incorporate does? It might be better
>>>>>> integrated as a shared application rather than a node service, the
>>>>>> difference being that shared applications are what users interact 
>>>>>> with, like
>>>>>> a shared editor or shared rasmol application, whereas a node 
>>>>>> service is
>>>>>> providing some underlying capability, like shared video or shared 
>>>>>> audio.
>>>>>> Once configured the users doesn't start and stop the node services
>>>>>> typically, there are automatically started and stopped when they 
>>>>>> enter a
>>>>>> venue.
>>>>>>
>>>>>> I hope that helps a bit, please let me know what I can do to help.
>>>>>>
>>>>>> --Ivan
>>>>>>  
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: owner-ag-tech at mcs.anl.gov 
>>>>>>> [mailto:owner-ag-tech at mcs.anl.gov] On Behalf Of Michael Braitmaier
>>>>>>> Sent: Thursday, May 06, 2004 5:09 AM
>>>>>>> To: ag-tech at mcs.anl.gov
>>>>>>> Subject: [AG-TECH] Struggling with starting of producer service 
>>>>>>> and encapsulated executable
>>>>>>>
>>>>>>> Currently I am trying to incorporate a simple executeable into my 
>>>>>>> VideoProducerService-derived own ProducerService., but I can't 
>>>>>>> figure out some aspects of how the producer service works.
>>>>>>> Especially I am stuck with the start method of the service. As 
>>>>>>> far as I understand it, for example the Videoproducer service 
>>>>>>> does nothing more than calling vis with appropriate parameters 
>>>>>>> passed in the options array. Therefore I wanted to replace vic by 
>>>>>>> my own executeable and accordingly change the options passed to 
>>>>>>> the called AGService._Start method. But when I remove the options 
>>>>>>> self.streamDescription.location.host and 
>>>>>>> self.streamDescription.location.port my producer service can't be 
>>>>>>> added to my service manager because the service manager reports 
>>>>>>> that the service is unreachable. I thought these values are only 
>>>>>>> needed for vic to be able to broadcast its video stream and 
>>>>>>> receive video streams, so I should be able to pass my own server 
>>>>>>> information format at this point which is understandable by my 
>>>>>>> executeable. So what do I have to pay attention to when replacing 
>>>>>>> vic with my own executeable concerning the start method of the 
>>>>>>> producer service, to avoid my service being unreachable ?
>>>>>>>
>>>>>>> Any help would be greatly appreciated.
>>>>>>> Michael
>>>>>>>
>>>>>>> -- 
>>>>>>> Michael Braitmaier
>>>>>>> HLRS
>>>>>>> University of Stuttgart
>>>>>>> Germany
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>   
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>