[AG-DEV] Continuation of AG services - Update 2
Martin Turner
Martin.Turner at manchester.ac.uk
Wed May 2 08:47:31 CDT 2012
From the UK's perspective we have, like most countries, an independent
network (in our case mixing AGTkit, IOCOM (IG2 and visimeet) and EVO)
but will like to cross-collaborate post June.
Three things that need to be acted upon centrally:
- Establish Google code or Source-Forge repository needed; source code
and packages
- Mailing Lists: listserv or equivalent accounts are essential
- Name/ownership of website accessgrid.org
Extra point:
- Central list of the bridge registry file locations. This could be held
as a list at accessgrid.org.
(UK bridge registry
http://www.ja.net/services/video/agsc/AGSCHome/ag3peers.txt)
- Central list of countries venue server list
(UK venue server https://sam.ag.manchester.ac.uk:8000/Venues/default)
Less important and can be regional:
- Jabber: most countries appear to use their own jabber server -
including now one in the UK.
- Certificates - we have been testing and believe self-certification is
acceptable and note this is only really required when building a new
server or if used for client-server authentication. Also servers run
nicely even when a certificate expires.
- Final issue is to check on who owns which multicast addresses.
I will be at the worldwide meeting on Monday.
yours Martin (ag-collaboration at listserv.manchester.ac.uk
<mailto:ag-collaboration at listserv.manchester.ac.uk>)
http://wiki.rcs.manchester.ac.uk/community/VTAS
On 16/04/2012 14:55, Christoph Willing wrote:
> Comments below:
>
> On 16/04/2012, at 11:59 AM, Jason Bell wrote:
>
>> Colleagues (resent – I haven’t received any feedback from this recent
>> update)
>>
>> I have started to draft feedback (apologies for the roughness) from
>> everyone and collated a few ideas on ways to move the AG services
>> forward. I have attempted to group all comments and added a few of
>> my own, but please feel free to comment, even editorial feedback
>> welcomed!
>>
>> It is hoped that this email is only the beginning and additional
>> content will be provide once new feedback has been obtained,
>> particularly to the table below.
>>
>> Please note, that though a number of suggestions were made on how the
>> Access Grid software and services can be improved, I believe it is
>> important to focus on the continuation of current services, before
>> additional code development or the likes can be considered.
>>
>> One idea that has been floated around, is to setup
>> a/many Virtual Machine/s to run many of the services on.
>
> Virtual or physical machine is a site dependent implementation detail
> we shouldn't need to worry about here.
>
>
>> There are many options with this idea, either use one of the many
>> global hosting services, or to use an institutional system (thanks to
>> all those institutional groups who have already offered their
>> services!). Some things to consider though:
>>
>> Commercial hosting service
>>
>> · Someone or group has to finance the ongoing costs
>> o Even if multiple groups contribute, a board or person will have
>> to manage payment, etc.
>> · Commercial grade network (will large downloads and site
>> visits be slow or incur large costs)
>> · Will the “hosting” be reliable?
>> · Given the many hosting services that are starting and
>> stopping, will a new host be needed every couple of years?
>> · Not tied to any particular group or organisation, therefore
>> if funding stops from one location, another can take over or replace
>> easily, without the need to move the services to another location.
>> · Not being tied to anyone particular, this can be seen as a
>> more open source product and community.
>>
>> Institutional hosting
>>
>> · Will the services need to be transferred again, after
>> support runs out?
>> · Will network charges be incurred?
>> · Most intuitional organisations are funded on a “term”
>> basis, therefore, will services need to be moved regularly once
>> support runs out?
>> o For example, ANL will no longer fund/support the AG projects,
>> therefore services need to be moved. Will this be something that
>> occurs every few years?
>> o How reliable can we count on the support?
>> · Institutions generally have very large and fast networks,
>> thus large data traffic should cause any performance issues.
>
> Answers to just about all these questions is pretty obviously yes.
> Basically any support system that anyone sets up will occasionally
> need renewal. I guess its best to minimise the frequency of that;
> we've had a good run with ANL - over 10 years.
>
>
>>
>> What do people think of the two options listed? Any advantages or
>> disadvantages to add? Any preference?
>
> Its probably harder with a commercial option esp finding ongoing
> funding. Doing it institutionally, we can take advantage of people's
> spare of after hours cycles. Of course that obviates performance
> guarantees (but we've never had them previously anyway).
>
>
>>
>> Another question that I guess needs to be answered is domain names.
>> For example, some of the default aspects of the Venue Server and
>> Venue Client code points to ANL addresses. For example,
>> “vv3.mcs.anl.gov”, I am wondering if it be possible to have rules in
>> place that forward this address (DNS redirection) to another location
>> (one to be determined of course).
>
> According to whois, accessgrid.org belongs to University of Chicago.
> They would have to (as mentioned below) either:
> 1. transfer ownership to the new "owners" (yet to be determined)
> or 2. point DNS to new owners IP addresses
>
>
>> This will prevent the need to immediately modify source code and
>> roll out updates to support new internet addresses.
>>
>> I have attempted to table
>
> Sorry my email client has lost the table layout - hope the comments
> make sense.
>
>
>> the various Access Grids Services and Systems that need to be
>> transitioned, provided some general comments and possible options.
>> If anyone have anything else to add or modify, please do contribute
>> to this conversation.
>>
>> Access Grid Services and Systems
>> Comments
>> Priority
>> Option 1
>> Option 2
>> Option 3
>> Recommendation
>> Additional Comments
>> www.accessgrid.orgdomain name
>> · Can the domain name be transferred?
>>
>> · Who will own (be responsible) for maintaining this address
>>
>> · Do we want to obtain a new address?
>>
>>
>> Nice to have
>> Transfer domain
>> Use a new address
>> ???
>>
>>
>> vv3.mcs.anl.gov and other anl.gov domain names
>> · Can all these address be forward to new location.
>>
>> · Thinking out loud, but standardising to a new address, such
>> aswww.accessgrid.org would be useful in the long term.
>>
>>
>> Nice to have
>> Forward to new domain names
>> Update Venue Client and Venue Server code and roll up new versions
>> with these updates
>
> Packaging new versions for Linux after code updates is pretty easy.
>
> Mac packaging has been (I think) exclusively an ANL activity.
>
> Windows packages have been based on ANL packaging too haven't they.
>
> That probably means that those at ANL who have been making the Mac &
> Windows packages will need to keep making them, or others will have to
> figure out how to do it (hopefully with ANL packagers' help).
>
> Who's volunteering to do the Mac & Windows builds? Hard to continue
> without them.
>
>
>> ???
>>
>>
>> Access Grid Source Code
>> · A system needs to be implemented that allows the storage
>> and Access to the Access Grid Source Code.
>>
>> · It is expected that an international group will oversee
>> updates, etc.
>>
>> · This is currently in a SVN repository.
>>
>> · Only one repository required.
>> Must to have
>> Using Source Forge
>>
>>
>> Google ????
>>
>> Nothing further investigated?
>>
>> Using a hosted service (Commercial or Institutional) and implement a
>> SVN repository
>>
>> Source Forge
>> Instructions:http://sourceforge.net/apps/trac/sourceforge/wiki/SVN%20adminrepo#ImportingfromotherreposincludingotherSCMs
>>
>> ANL Jabber Server
>> (could be made a regional service)
>> · Currently, any newly configured Venue Server will use the
>> ANL Jabber Server.
>
> This is not exactly correct; while the ANL jabber server is the
> default, this is actually set in the VenueServer.cfg file so is a
> configurable setting. The catch is that this file is generated for
> you, if it doesn't already exist, from defaults in the VenueServer
> library. Therefore if you don't have a VenueServer.cfg already, create
> a minimal one with (for instance)
>
> # AGTk 3.2
> [VenueServer]
> textHost = jabberserver.somewhere.org
>
> - the remaining entries will be created for you when you start the
> server.
>
> If you already have a VenueServer.cfg, edit the textHost entry to
> whatever you like and restart the server.
>
>
>
>>
>> · Questions:
>> o Do we want to use a different technology than jabber?
>
> There are some good reasons to use something different. A recently
> discovered good reason to keep it is that its logfile provides an
> excellent record of venue server usage. See:
> http://www.rcc.uq.edu.au/accessgrid/usage/
> for a display of the APAG Venue Server usage, derived wholly from the
> logfile of the Jabber server we run here.
>
>
>> o Should a number of regional jabber servers be configured and a
>> Venue Server config option be implemented.
>
> As above, thats configurable in the VenueServer.cfg file right now.
> There'd be no problem with adding a specific option but since its
> already configurable by other means, needn't be a high priority.
>
>
>> o Should each Venue Server run their own Jabber server, therefore
>> removing a single point of failure?
>
> While its pretty easy to run a venue server, its much, much harder to
> set up a compatible jabber server (one of the good reasons to use
> something other than jabber as the text chat protocol).
>
>>
>> · Should a Global Jabber server be implemented as a stop gap
>> measure?
>
> I'm happy for the APAG jabber server to be used globally as a more or
> less indefinite stop gap measure.
>
>
>> Critical
>> Use a hosted service (Commercial or Institutional) and implement a
>> Global Jabber server
>
> From the above, I'm not sure its actually critical (at least not
> urgent) to to implement a new jabber server. In any case it may be a
> somewhat wasted effort if we end up using something other than jabber.
>
>
>>
>> Note: The Venue Server source code will have to be modified to point
>> to new server, if DNS redirection cannot be implemented.
>> Use a different technology than Jabber.
>>
>> One technology that has been mentioned is MQTT (Message Queue
>> Telemetry Protocol).
>
> I've suggested this in the past because I've used it myself in other
> projects - other people will have their own favourites. MQTT already
> has a python binding and I found it was pretty easy to use and think
> it should be relatively easy to incorporate into the AG code. However
> there would be a reasonable effort to actually do it.
>
>
>>
>> Each host running their own jabber server?
>>
>> · Is this Feasible or even possible?
>
> Setup complexity make this possibility doubtful.
>
>
>> · Is the development work require too much before the
>> required deadline for the shutdown?
>>
>>
>> ANL Bridge Registry
>> (could be made a regional service)
>> It is already possible to run your own Bridge Registry. The question
>> is more, currently Venue Clients default to using “vv3.mcs.anl.gov:8030”
>
> A second default (www.ap-accessgrid.org:8030) has been included as a
> default for quite a while (years?) now. Others can be easily added but
> would need to be part of a new package release for them to be
> available by default. Otherwise they can be added by hand via
> VenueClient gui (hardly automatic though).
>
>
>>
>> Should another global “bridge registry” be implemented, or should a
>> number of “regional bridge registries” be implemented that these be
>> added to the source code?
>>
>> Critical
>> Use a hosted service (Commercial or Institutional) and implement a
>> Global Bridge Registry
>>
>> Note: The Venue Client source code will have to be modified to point
>> to new server, if DNS redirection cannot be implemented.
>> Roll out a set of regional registries, which is documented on the AG
>> website and provide documentation to users for adding new registries.
>>
>> · Additionally, update source code to contain new registries.
>> ???
>>
>>
>> AG-Dev Certificate Authority
>> Currently, whenever new Venue Servers are implemented, they must
>> request and install an identity certificate.
>
> An identity will work but are pretty inconvenient to use from boot
> time scripts; better to use service certificates.
>
>
>> Critical
>> Centralised certificates?
>>
>> Use a hosted service (Commercial or Institutional) and implement a
>> Global Jabber server
>>
>> Note: The Venue Server source code will have to be modified to point
>> to new server, if DNS redirection cannot be implemented.
>> Self-signed certificates
>> · lessens the security features available
>>
>> · Does anyone care about this?
>
> Yes, I care about it and have started looking at how to set up a
> certificate issuing infrastructure based on the ANL model. Its
> incomplete - waiting for more time to spend on it. Needs some changes
> to current AG code which points requests to particular servers running
> particulars cgi scripts.
>
> I'd recommend anyone running AG services to obtain new certificates
> before June deadline so they have a year before certificate system
> needs to be fully functional.
>
>
>
>>
>> ???
>>
>>
>> The accessgrid.org web site
>> The www.accessgrid.org website is a very useful resource that
>> contained things like:
>>
>> · Global Node listing;
>> · QA information and results;
>> · AG Projects (http://www.accessgrid.org/projects)
>> · Software downloads;
>> · Documentation;
>> · Hardware information;
>> · Community information; and
>> · New and events
>>
>> Things to note:
>> · Website is based on Drupal 4.7 and is extremely long
>> overdue for a software upgrade. Current version is 7.12 (as of
>> 10/4/2012)
>> · Newer versions may cause issues with some aspects of the
>> current content, particularly Node listing and QA;
>>
>> Must to have
>> Use a hosted service (Commercial or Institutional) and implement a
>> Global Bridge Registry
>>
>> Simply move current content to new location.
>>
>> · Current version of drupal out-dated;
>> · Content will be the same as current website.
>> · Assumed less work required
>> Use a hosted service (Commercial or Institutional) and implement a
>> Global Bridge Registry
>>
>> Install newer version of drupal and port content over.
>>
>> · Significant amount of work required;
>> · Some aspects, particularly Node listing and QA may have to
>> be completely redeveloped;
>> Start a fresh, possibly look towards using a different CMS system?
>>
>> It is assumed that the domain name www.accessgrid.org can still be used.
>> AG-related mailing lists
>> There are many community mailing list that are actively used. These
>> being:
>>
>> ag-tech at mcs.anl.gov
>> ag-users at mcs.anl.gov
>> ag-announce at mcs.anl.gov
>> ag-dev at mcs.anl.gov
>> ag-cvs at mcs.anl.gov
>>
>> · Are all these mailing list still required?
>>
>> · Are there any other ones that should be created?
>>
>> · Can the history be ported?
>>
>>
>> Must to have
>> Use a hosted service (Commercial or Institutional) and implement an
>> Access Grid mailing list service.
>>
>> Use Source Forge. If using Source Forge is decided as the best
>> option for housing the source code, they have additional services
>> such as mailing lists that could be used.
>>
>>
>>
>> ANL Venue Server
>> It is already possible to run your own Venue Server. The question is
>> more, currently Venue Clients default to using “vv3.mcs.anl.gov:8000”
>>
>> Should another global “Venue Server” be implemented, or should a
>> number of “regional Venue Server” be implemented that these be added
>> to the source code?
>> · Which ones become default?
>>
>>
>>
>> Nice to have
>> Use a hosted service (Commercial or Institutional) and implement a
>> Global Venue Server
>>
>> Note: The Venue Client source code will have to be modified to point
>> to new server, if DNS redirection cannot be implemented.
>> Roll out / utilise more a set of regional Venue Servers, which is
>> documented on the AG website and provide documentation to users for
>> adding Venue Server addresses.
>> Set the default Venue Server to one currently listed
>> athttp://www.accessgrid.org/communityor another one???
>>
>> Any volunteers?
>
> We'll continue to run APAG venue server from UQ.
>
> We could easily run a more generic vv3.accessgrid.org server too but
> maybe people would prefer that to be hosted more centrally (US, Europe).
>
>
>
>>
>>
>> NCSA AG Scheduler
>> NCSA are interested in finding someone to take on the support of
>> AGSchedule
>> Critical ???
>> Use a hosted service (Commercial or Institutional) and implement and
>> support AGSchedule on a new server.
>>
>>
>> Multicast Beacon
>> Is a global or regional solution required?
>
> It would be good to have some widely used system again.
>
> We've been using dbeacon for a while in Asia Pacific region and it
> seems pretty good. It has no real central server - any participating
> site can host web server for its display e.g. display nominally hosted
> at:
> http://syd-a-ext1.aarnet.net.au/matrix/ (AARNet)
> is also available at:
> http://ag0-video.vislab.uq.edu.au:8888/matrix/ (UQ RCC)
>
>
>>
>> I would propose that we should try to break down some of the current
>> AG services into 2 categories. These being Global and Regional
>> Services.
>>
>> · Global Services - are services that only requires a single
>> instance or implementation;
>>
>> · Regional Services – are services that can have multiple
>> instances or implementations of;
>
> I'm not necessarily against it but I'm not sure this is a particularly
> important distinction. Any AG service can be run by as many different
> sites as want to run them. Whether you label them global or regional
> doesn't really mean much. For instance the APAG venue server has lots
> of venues related to the asia pacific region but that hasn't prevented
> sites from elsewhere in the world also using it (and they've been
> welcome to do so).
>
>
>>
>> Once the various services are determined to be Global or Regional, a
>> total list of global resources can be identified. For example, if
>> throughout this process, it is determined that a regional list of
>> bridge registries is the best option, then a global service is no
>> longer needed. Thereby no resources will be required in running this
>> as a global service, as it will be expected various “regional” group
>> will run their own implementation.
>
> Further to above, the concept of various groups running "their own
> implementation" has been a core feature of the AG as I understand it -
> it was designed to work in precisely that way. Lots of "regional"
> services is the way to go, I think. No harm in having some "model
> service site" as ANL has been over the years, but not sure where that
> should be.
>
>
> chris
>
>
>>
>> Many thanks for your time and I look forward to everyone’s feedback.
>>
>> Many regards,
>> Jason.
>>
>> -------------------------------------------------------------------------
>>
>> Jason Bell, B.I.T. (Honours)
>>
>> Video Collaboration Champion
>> Queensland Cyber Infrastructure Foundation
>> http://www.qcif.edu.au/
>>
>> ARCS eResearch Collaborative Services
>> http://www.arcs.org.au/
>>
>> Senior Research Technologies Officer
>> Information Technology Directorate
>> CQ University Australia
>> http://www.cqu.edu.au/
>>
>> E-mail : j.bell at cqu.edu.au
>> j.bell at qcif.edu.au
>> jason.bell at arcs.org.au
>> Work : +61 7 4930 9229
>> Mobile : 0409 630897
>> Postal : Building 19, Room 1.55
>> Central Queensland University
>> Bruce Highway
>> Rockhampton, Queensland, Australia, 4702
>> -------------------------------------------------------------------------
>>
>> Patience is a virtue.
>>
>> But if I wanted Patience,
>> I would have become a Doctor.
>> -------------------------------------------------------------------------
>>
>>
>
> Christoph Willing +61 7 3365 8316
> Research Computing Centre
> University of Queensland
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.mcs.anl.gov/pipermail/ag-dev/attachments/20120502/6069afba/attachment-0001.htm>
More information about the ag-dev
mailing list