[AG-DEV] Re-using node listing data

[Andrew Rowley]

Hi,

Mirroring could be complicated as it will probably only allow updating of this from the US site - this would stop our day-to-day activities if it went down and so would be unacceptable for us to rely on without a guarantee of service.

Does the site use some sort of SQL database backend?  If so, couldn't you trawl the info from various SQL databases, rather than just one?  As well as this, you could have the information trawled stored in the central accessgrid.org database in case the other sites were down...

Andrew :)

============================================
Access Grid Support Centre,
RSS Group,
Manchester Computing,
Kilburn Building,
University of Manchester,
Oxford Road,
Manchester,
M13 9PL,
UK
Tel: +44(0)161-275 0685
Email: Andrew.Rowley at manchester.ac.uk 
 

-----Original Message-----
From: Jason Bell [mailto:j.bell at cqu.edu.au] 
Sent: 28 May 2007 00:29
To: Andrew.Rowley at manchester.ac.uk; Brian Corrie
Cc: AG-DEV
Subject: RE: [AG-DEV] Re-using node listing data

G'day All

Well, I must say that it appears I may have opened a can of worms here
:-)

Anyway, I don't know if the idea of the AG website sourcing data from
other sites is practical or what.  But here are some comments:

*	Given that a lot of the data will be static, you would only need
to update once or twice a day at a maximum.

*	Instead of having a single repository for the data, I am
wondering if it would be practical or viable to have mirrors of the
data.  If we had mirrors in the US, UK and possibly in AUS, than the
three major times zones would be covered.

Now, as part of the discussion at the retreat, I think it was mentioned
that it would be important to decide on a "Schema" on the data, so that
communication between the various "parties" could be established.  I
think it was also mentioned able possibly using "OpenID" for
authorization, but having never heard of it personally, I cannot
comment.

Anyway, I having said my 2 cents worth, I would like to hear what others
have to say and whether anyone can provide assistance to make it a
practical reality.

Cheers,
Jason.
    

-----Original Message-----
From: Andrew Rowley [mailto:Andrew.Rowley at manchester.ac.uk] 
Sent: Friday, 25 May 2007 6:20 PM
To: Brian Corrie; Jason Bell
Cc: AG-DEV; qa at accessgrid.org
Subject: RE: [AG-DEV] Re-using node listing data

Hi,

I very much like the idea of having the accessgrid.org site extract the
information from our local repositories.  This means that we are not
creating a single point of failure.  The AGSC have been caught in the
past with relying on services in the US - this means that if it goes
down in our morning, it may not be back until our afternoon!

Andrew :)

============================================
Access Grid Support Centre,
RSS Group,
Manchester Computing,
Kilburn Building,
University of Manchester,
Oxford Road,
Manchester,
M13 9PL,
UK
Tel: +44(0)161-275 0685
Email: Andrew.Rowley at manchester.ac.uk 
 

-----Original Message-----
From: owner-ag-dev at mcs.anl.gov [mailto:owner-ag-dev at mcs.anl.gov] On
Behalf Of Brian Corrie
Sent: 23 May 2007 20:51
To: Jason Bell
Cc: AG-DEV
Subject: Re: [AG-DEV] Re-using node listing data

Hi Jason,

I think this is a great idea, if we can pull it off. I think Todd
probably mentioned some of our efforts here to keep track of nodes that
we collaborate with and that are within our consortium. With a national
collaboration infrastructure on the horizon within Canada, this is only
going to increase.

I think the trick here is to create a data description/schema that
describes nodes that we can all use for a baseline. It is likely that we
will all have ancillary information that we need for individual
purposes, but if there is a single schema where this baseline
information is defined then storage of the information itself is a
secondary question.

The data could be centrally stored and other groups (the Aussies, AG
Schedule, AGSC, Compute Canada) can pull this information for the nodes
of interest as required. Of course another alternative is to have each
of the groups maintain this information locally (probably makes more
sense), and we have a mechanism/filter that extracts the "common schema"
from known regionally authoritative databases for the purposes of
centralizing this information. That is, AGSC, Compute Canada, Pacific
Rim AG, Main AG site, might store regional information and we extract
that information for central purposes (AG Schedule and the main AG site
knows about all nodes) as required. This implies there is a home data
repository for each site somewhere and information gets stored there and
is used by other resources. We would need a catch all site that can
handle those sites that don't have a regional data repository.

Anyway, just some thoughts off the top of my head... Good idea in
general, how do we make it work???

Brian


Jason Bell wrote:
> G'day All
>
> I thought I would get the ball rolling after our interesting
discussion
> at the AG-Retreat Reception.
>
> An interesting topic was raised and one I would like to discuss within
> this forum, is how we can use the information stored within the AG
Node
> listing for other programs.
>
> Initially, it was conceived that it would be beneficial for other
group
> to be able to access the data so they could simply display
"Australian"
> nodes at a local website, etc.  From this, other interesting concepts
> arose.  One such example was "wouldn't it be good if AG-Schedule could
> use the data within the node listing", thereby negating the
requirement
> to have to register in multiple locations. From this, a concept of a
> single database, or at least mirroring of information was suggested.
>
> Now, I know there were a lot of other issues raised, but whilst the
idea
> is still in my mind and hopefully our enthusiasm created from the
> Retreat is still fresh, I would like to see what people think of this
> idea and how we could actually go about implementing it.
>
> I would like to hear what others have to say.
>
> Cheers,
> Jason.
>
> --------------------------------------------
>
> Jason Bell, B.I.T.
> B. Info. Tech. (Honours) Student
>
> Network Engineer
> Information Technology Division
> Central Queensland University
>
> High Performance Computing Support Officer
> Central Queensland University
>
> E-mail : j.bell at cqu.edu.au
> Phone : 07 4930 9229
>
> --------------------------------------------
>
> Patience is a virtue.
>
> But if I wanted Patience,
> I would have become a Doctor.
>
> --------------------------------------------
>