[AG-DEV] [AG-TECH] New accessgrid.org Venue Server

Wed May 30 20:41:38 CDT 2012

Debian & Ubuntu support multiple python versions coexisting in the  
system at the same time. Any given system could have just one or a  
number of python versions installed. To support all cases, the  
packaged files are actually in /usr/share/pyshared/AccessGrid3 and  
when the package is installed, the system makes the appropriate  
symlinks based on the python versions installed at the time - as  
Robert has noted.

It may be possible to just generate new packages which include the  
necessary name changes - its probably easier for sites to install new  
packages than modify existing installations (for Debian & Ubuntu it  
would just be a matter of running apt-get update and apt-get upgrade).  
I'll have a look at that this afternoon.

chris

On 31/05/2012, at 12:58 AM, Robert Frank wrote:

> Hi Salim,
>
> I can confirm this for Debian.
> I've tried your script on Squeeze and it doesn't work.
> The AccessGrid libs are installed in /usr/share/pyshared/ 
> AccessGrid3, get_python_lib() returns /usr/lib/python2.6/dist- 
> packages.
> I have 2 versions of python installed (2.5 and 2.6) and the package  
> creates symlinks to the installed AG lib files in the lib trees of  
> both python versions:
> /usr/lib/pymodules/python2.5/AccessGrid3
> /usr/lib/pymodules/python2.6/AccessGrid3
>
> I searched for a way of figuring out where the libs are and the best  
> I could come up with was to import the library and let python tell  
> me its path:
>
> import AccessGrid3
> site_packages_dir = os.path.join(AccessGrid3.__path__[0],  
> "AccessGrid")
>
> In case of the "AccessGrid" package name the above would have to be  
> (not tested):
>
> import AccessGrid
> site_packages_dir = AccessGrid3.__path__[0]
>
> I wonder if this approach would work on all systems.
>
> Regards,
>
> Robert
> JANET AG Collaborations (UK)
>
> On 30/05/12 15:41, Salim Haniff wrote:
>> Hi everyone,
>>
>> I just installed the AGTk on my co-worker's Ubuntu (10.04.4) box. The
>> AGTk python module are located in
>> /usr/lib/pymodules/python2.6/AccessGrid3/AccessGrid . Is this a  
>> common
>> area for other versions of Debain and Ubuntu? The user of that system
>> has tinkered around with the system so I cannot confirm if this is a
>> normal installation area.
>>
>> Best regards,
>> Salim
>>
>>
>> Quoting James S Perrin <james.s.perrin at manchester.ac.uk>:
>>
>>> Hi Tim, Salim,
>>>
>>> I quickly ran Tim's script on my Windows XP 32 machine using python
>>> 2.4 (as installed by AGTK-3).
>>>
>>> 1. Added code for WindowsXP*Updaters
>>> 2. needed a 'self.' in front of the root_path_list in the
>>> Windows*AGUpdaters.
>>>
>>> As there is only one AGtk-3 package for all windows machine the
>>> location will either be:
>>>
>>> C:\Program Files\AGTk-3 on 32bit or
>>> C:\Program Files (x86)\AGTk-3 on 64bit
>>>
>>> So XP, Vista and 7 can be combined if you wish.
>>>
>>> 3. I also tried it on a Mac OSX 10.8 'Darwin' and it seemed to  
>>> work as
>>> expected.
>>>
>>> Regards
>>> James
>>> JANET AG Collaborations (UK)
>>>
>>>
>>> On 30/05/2012 13:28, Tim Salmon wrote:
>>>> Dear Salim, I ran the script on a Windows 7 64 but if returned only
>>>> that the AccessGrid libs could not be found. on inspection I found
>>>> that the path (default for the installer i used i assume) is
>>>> ......\AccessGrid3 not ....\AccessGrid. rather than being a fault
>>>> with your script i think that this may reflect my concern that  
>>>> there
>>>> is a great diversity of installs out in the world that may have
>>>> arisen by a variety of factors including platform, build process  
>>>> and
>>>> usage patterns. I also have noted the occurrence of a number of
>>>> places outside the python libs and userspaces that the AccessGrid
>>>> files are installed. Others will know more than me about what role
>>>> these play but it is probably necessary to ensure consistency.
>>>>
>>>> I have had a go at approaching this more generally but I fear that
>>>> there may be little chance of getting away from a lot of testing.
>>>>
>>>> At this stage I have only started on the "system" files. To
>>>> generalise the process I use a class of file walkers that traverse
>>>> specified subdirectories of the filesystem, identify filesnames  
>>>> that
>>>> match what we care about (unfortunately more reliable methods of
>>>> ignoring binary files is not implemented in python 2.4/2.5) and
>>>> perform the substitutions by calling a chunk of your script.
>>>>
>>>> you will see at the bottom a chunk of slightly over-engineered
>>>> inheritence statements and the start of some logic for different
>>>> platforms. you can see the included examples of some system file
>>>> areas that are particular to different platforms and are outside  
>>>> the
>>>> python library space.
>>>>
>>>> I have not yet actually set the script to make the alterations:  
>>>> just
>>>> find the targets but that should be a simple change of function  
>>>> call
>>>> once we feel it is ready. i have also not yet incorporated
>>>> operations that work in the user space: it occurred to me that it
>>>> might be best to approach that in a separate script: one for
>>>> sudo/admin users to carefully apply to their system, and one for
>>>> users to blithely run without bringing the whole thing down in a
>>>> screaming heap.
>>>>
>>>> by all means send back edits, suggestions or point out how my
>>>> approach is doomed to failure. in particular, suggestions of a more
>>>> pythonic approach are welcome: those that know java will  
>>>> recognise my
>>>> object-heavy paradigm. i will definitely boost the comments and  
>>>> look
>>>> at where checks are needed too.
>>>>
>>>> pasted below and attached
>>>>
>>>> Tim Salmon IT Technical Officer Faculty of Science UNSW
>>>>
>>>> import fileinput import os import platform import re import shutil
>>>> import sys import tempfile from distutils.sysconfig import
>>>> get_python_lib
>>>>
>>>> ## RewriteWalker performs a os.path.walk of base_dir class
>>>> RewriteWalker: def __init__(self, base_dir, file_re, find_str,
>>>> replace_str): self.base_dir = base_dir self.file_filter =
>>>> re.compile(file_re) self.find_str = find_str self.replace_str =
>>>> replace_str
>>>>
>>>> def walk_files(self): for root, directories, filenames in
>>>> os.walk(self.base_dir): for filename in filenames: if
>>>> (self.file_filter.match(filename)): filepath = os.path.join(root,
>>>> filename) self.find_lines(filepath) #rewrite(filepath)
>>>>
>>>> def find_lines(self, srcfile): line_count = 0; src = open(srcfile,
>>>> 'r') for line in src: line_count += 1 if self.find_str in line:  
>>>> print
>>>> srcfile print str(line_count) + ": " + line
>>>>
>>>> def rewrite(srcfile, searchPattern, replaceStr): dstfile =  
>>>> srcfile +
>>>> '.bak'
>>>>
>>>> #Checks to see if the file has been backed up if not then create a
>>>> backup of the file. if not os.path.isfile(dstfile):
>>>> shutil.copy(srcfile,dstfile)
>>>>
>>>> #creates a temporary file in the same directory that the  
>>>> "executable"
>>>> was launched from tmpfilename = str(os.getpid()) tfile =
>>>> open(tmpfilename,'w') src = open(srcfile,'r')
>>>>
>>>> #loops through the entire file changing text for line in src: if
>>>> searchPattern in line:
>>>> tfile.write(line.replace(self.find_str,self.replace_str)) else:
>>>> tfile.write(line) tfile.close() src.close()
>>>>
>>>> #overwrites the srcfile and deletes the temp file
>>>> shutil.copy2(tmpfilename,srcfile) os.unlink(tmpfilename)
>>>>
>>>> class AGUpdater: def __init__(self):
>>>>
>>>> # define list of root paths and populate with those in python_lib
>>>> self.root_path_list = [] ag_python_lib_folder_list = ["AccessGrid",
>>>> "AccessGrid3"] python_lib_dir = get_python_lib() for folder_name in
>>>> ag_python_lib_folder_list: folder_path =  
>>>> os.path.join(python_lib_dir,
>>>> folder_name) if os.path.exists(folder_path):
>>>> self.root_path_list.append(folder_path)
>>>>
>>>> # define map of substitutions that can be applied [globally]
>>>> self.subs_map = {} ##self.subs_map['bugzilla.mcs.anl.gov'] =
>>>> 'bugzilla.accessgrid.org' self.subs_map['mcs.anl.gov'] =
>>>> 'accessgrid.org' ##self.subs_map['140.221.6.95'] =
>>>> 'www.accessgrid.org' ##self.subs_map['140.221.8.33'] =
>>>> 'vv3.accessgrid.org'
>>>>
>>>> # define list of filename_re that identify which files will be
>>>> modified # please note that this uses regular expressions not just
>>>> simple expansions self.filename_re_list = []
>>>> self.filename_re_list.append('.*\.py$')
>>>> self.filename_re_list.append('.*\.txt$')
>>>>
>>>> # iterate throught root paths, filename_res and substitutions and
>>>> create all combinations def generate_walkers(self): self.walkers=[]
>>>> for root_folder in self.root_path_list: for filename_re in
>>>> self.filename_re_list: for subs_key, subs_val in
>>>> self.subs_map.iteritems(): walker = RewriteWalker(root_folder,
>>>> filename_re, subs_key, subs_val) self.walkers.append(walker)
>>>>
>>>> def perform_update(self): for walker in self.walkers:
>>>> walker.walk_files()
>>>>
>>>> class WindowsAGUpdater(AGUpdater): def __init__(self):
>>>> AGUpdater.__init__(self)
>>>>
>>>> class WindowsXPAGUpdater(WindowsAGUpdater): pass
>>>>
>>>> class WindowsXP_32AGUpdater(WindowsXPAGUpdater): pass
>>>>
>>>> class WindowsXP_64AGUpdater(WindowsXPAGUpdater): pass
>>>>
>>>> class Windows7AGUpdater(WindowsAGUpdater): def __init__(self):
>>>> WindowsAGUpdater.__init__(self)
>>>>
>>>> class Windows7_32AGUpdater(Windows7AGUpdater): def __init__(self):
>>>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
>>>> Files\\AGTk3\\")
>>>>
>>>> class Windows7_64AGUpdater(Windows7AGUpdater): def __init__(self):
>>>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
>>>> Files (x86)\\AGTk3\\")
>>>>
>>>> class PosixAGUpdater(AGUpdater): def __init__(self):
>>>> AGUpdater.__init__(self)
>>>>
>>>> class LinuxAGUpdater(PosixAGUpdater): def __init__(self):
>>>> PosixAGUpdater.__init__(self)
>>>>
>>>> class RedHatLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>>>> LinuxAGUpdater.__init__(self)
>>>>
>>>> class RHELLinuxAGUpdater(RedHatLinuxAGUpdater): def __init__(self):
>>>> RedHatLinuxAGUpdater.__init__(self)
>>>> self.root_path_list.append('/usr/share/doc/AccessGrid-3.2/')
>>>> self.filename_re_list.append('^Install\.LINUX')
>>>> self.filename_re_list.append('^README')
>>>> self.filename_re_list.append('^VERSION')
>>>>
>>>> class DebianLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>>>> LinuxAGUpdater.__init__(self)
>>>>
>>>> class UbuntuLinuxAGUpdater(DebianLinuxAGUpdater): def
>>>> __init__(self): DebianLinuxAGUpdater.__init__(self)
>>>>
>>>> class OSXAGUpdater(PosixAGUpdater): def __init__(self):
>>>> PosixAGUpdater.__init__(self)
>>>>
>>>> if __name__ == "__main__": platform_str = platform.platform()
>>>>
>>>> updater = AGUpdater() if "Windows" in platform_str: if "7" in
>>>> platform_str: if "64" in platform_str: updater =
>>>> Windows7_64AGUpdater() elif "Linux" in platform_str: if "redhat" in
>>>> platform_str: if "6.2" in platform_str: updater =
>>>> RHELLinuxAGUpdater()
>>>>
>>>> updater.generate_walkers() updater.perform_update()
>>>>
>>>> ________________________________________ From:
>>>> ag-dev-bounces at lists.mcs.anl.gov [ag-dev-bounces at lists.mcs.anl.gov]
>>>> on behalf of Salim Haniff [Salim.Haniff at uta.fi] Sent: Thursday, May
>>>> 24, 2012 4:18 PM To: ag-tech at lists.mcs.anl.gov; ag-dev Subject: Re:
>>>> [AG-DEV] [AG-TECH] New accessgrid.org Venue Server
>>>>
>>>> Hi,
>>>>
>>>> The script for renaming urls and disabling venueclient's submit
>>>> report bug is currently located at
>>>>
>>>> http://code.google.com/p/salims-scratch-box/source/browse/AGpatch/trunk/src/AGpatch.py
>>>>
>>>>
>>>> I will move it later this week to the sourceforge AccessGrid
>>>> location. Sorry if the script is a bit rough I'm a novice at python
>>>> but feel free to suggest improvements :)
>>>>
>>>> The script was developed and tested on a Fedora box and a  
>>>> Scientific
>>>> linux box as root, since root is the owner of the site-packages
>>>> directory in python. I need to make a modification later today to
>>>> allow the user to execute the python app to rename the urls in  
>>>> their
>>>> ~/.AccessGrid3/Config alone (probably takes about 5 minutes).  
>>>> Windows
>>>> and Mac still needs to be tested.
>>>>
>>>> At this time, disabling multicast by default for Windows 7 users is
>>>> still outstanding. If anyone knows how to do that feel free to
>>>> email me.
>>>>
>>>> Best, Salim
>>>>
>>>>
>>>> Quoting Christoph Willing<c.willing at uq.edu.au>:
>>>>
>>>>> The machine which will become vv3.accessgrid.org has now been
>>>>> populated with all the venues& layout from vv3.mcs.anl.gov (which
>>>>> will close down on 1st June).
>>>>>
>>>>> Could anyone who has a few minutes to spare please try out the new
>>>>> server and report back any problems. Feel free to give it a good
>>>>> workout. It would be great to have any issues with it sorted out
>>>>> by the time its supposed to be on line for real - 1st June is just
>>>>> about a week away.
>>>>>
>>>>> Until the DNS is arranged, access to the venue server is via:
>>>>> https://130.102.78.188:8000/Venues/default
>>>>>
>>>>>
>>>>>
>>>>> chris
>>>>>
>>>>>
>>>>>
>>>>> Christoph Willing +61 7 3365 8316 Research Computing
>>>>> Centre University of Queensland
>>>>
>>>>
>>>> -- Salim Haniff (salim.haniff at uta.fi) School of Information
>>>> Sciences 33014 University of Tampere, Finland
>>>>
>>>
>>> --
>>> ------------------------------------------------------------------------
>>> James S. Perrin
>>> AccessGrid/MAGIC/Visualization
>>>
>>> Research Computing Services
>>> Devonshire House, University Precinct
>>> The University of Manchester
>>> Oxford Road, Manchester, M13 9PL
>>>
>>> t: +44 (0) 161 275 6945
>>> e: james.perrin at manchester.ac.uk
>>> w: www.manchester.ac.uk/researchcomputing
>>> ------------------------------------------------------------------------
>>> "The test of intellect is the refusal to belabour the obvious"
>>> - Alfred Bester
>>> ------------------------------------------------------------------------
>>
>>
>>

Christoph Willing              +61 7 3365 8316
Research Computing Centre
University of Queensland