[AG-TECH] New accessgrid packages with mcs.anl.gov name changes

Christoph Willing c.willing at uq.edu.au
Thu May 31 00:54:35 CDT 2012


I've made a new package for Debian squeeze, patched to change most of  
the mcs.anl.gov references to accessgrid.org. Its sitting in the  
normal repo, but not actually part of it yet (waiting for a bit of  
testing) i.e. normal apt-get commands won't work for now.

Please download and try it (install with dpkg) and report - its at:
     http://www.vislab.uq.edu.au/debuntu/squeeze/accessgrid3.2_3.3-0r68351_all.deb

If testing is favourable I'll put it in the repo properly.

A patched Ubuntu package is in the oven right now - for 12.04. If  
anyone wants to test on a different Ubuntu version, let me know and  
I'll make one.


chris


On 31/05/2012, at 11:41 AM, Christoph Willing wrote:

> Debian & Ubuntu support multiple python versions coexisting in the  
> system at the same time. Any given system could have just one or a  
> number of python versions installed. To support all cases, the  
> packaged files are actually in /usr/share/pyshared/AccessGrid3 and  
> when the package is installed, the system makes the appropriate  
> symlinks based on the python versions installed at the time - as  
> Robert has noted.
>
> It may be possible to just generate new packages which include the  
> necessary name changes - its probably easier for sites to install  
> new packages than modify existing installations (for Debian & Ubuntu  
> it would just be a matter of running apt-get update and apt-get  
> upgrade). I'll have a look at that this afternoon.
>
>
> chris
>
>
>
> On 31/05/2012, at 12:58 AM, Robert Frank wrote:
>
>> Hi Salim,
>>
>> I can confirm this for Debian.
>> I've tried your script on Squeeze and it doesn't work.
>> The AccessGrid libs are installed in /usr/share/pyshared/ 
>> AccessGrid3, get_python_lib() returns /usr/lib/python2.6/dist- 
>> packages.
>> I have 2 versions of python installed (2.5 and 2.6) and the package  
>> creates symlinks to the installed AG lib files in the lib trees of  
>> both python versions:
>> /usr/lib/pymodules/python2.5/AccessGrid3
>> /usr/lib/pymodules/python2.6/AccessGrid3
>>
>> I searched for a way of figuring out where the libs are and the  
>> best I could come up with was to import the library and let python  
>> tell me its path:
>>
>> import AccessGrid3
>> site_packages_dir = os.path.join(AccessGrid3.__path__[0],  
>> "AccessGrid")
>>
>> In case of the "AccessGrid" package name the above would have to be  
>> (not tested):
>>
>> import AccessGrid
>> site_packages_dir = AccessGrid3.__path__[0]
>>
>> I wonder if this approach would work on all systems.
>>
>> Regards,
>>
>> Robert
>> JANET AG Collaborations (UK)
>>
>> On 30/05/12 15:41, Salim Haniff wrote:
>>> Hi everyone,
>>>
>>> I just installed the AGTk on my co-worker's Ubuntu (10.04.4) box.  
>>> The
>>> AGTk python module are located in
>>> /usr/lib/pymodules/python2.6/AccessGrid3/AccessGrid . Is this a  
>>> common
>>> area for other versions of Debain and Ubuntu? The user of that  
>>> system
>>> has tinkered around with the system so I cannot confirm if this is a
>>> normal installation area.
>>>
>>> Best regards,
>>> Salim
>>>
>>>
>>> Quoting James S Perrin <james.s.perrin at manchester.ac.uk>:
>>>
>>>> Hi Tim, Salim,
>>>>
>>>> I quickly ran Tim's script on my Windows XP 32 machine using python
>>>> 2.4 (as installed by AGTK-3).
>>>>
>>>> 1. Added code for WindowsXP*Updaters
>>>> 2. needed a 'self.' in front of the root_path_list in the
>>>> Windows*AGUpdaters.
>>>>
>>>> As there is only one AGtk-3 package for all windows machine the
>>>> location will either be:
>>>>
>>>> C:\Program Files\AGTk-3 on 32bit or
>>>> C:\Program Files (x86)\AGTk-3 on 64bit
>>>>
>>>> So XP, Vista and 7 can be combined if you wish.
>>>>
>>>> 3. I also tried it on a Mac OSX 10.8 'Darwin' and it seemed to  
>>>> work as
>>>> expected.
>>>>
>>>> Regards
>>>> James
>>>> JANET AG Collaborations (UK)
>>>>
>>>>
>>>> On 30/05/2012 13:28, Tim Salmon wrote:
>>>>> Dear Salim, I ran the script on a Windows 7 64 but if returned  
>>>>> only
>>>>> that the AccessGrid libs could not be found. on inspection I found
>>>>> that the path (default for the installer i used i assume) is
>>>>> ......\AccessGrid3 not ....\AccessGrid. rather than being a fault
>>>>> with your script i think that this may reflect my concern that  
>>>>> there
>>>>> is a great diversity of installs out in the world that may have
>>>>> arisen by a variety of factors including platform, build process  
>>>>> and
>>>>> usage patterns. I also have noted the occurrence of a number of
>>>>> places outside the python libs and userspaces that the AccessGrid
>>>>> files are installed. Others will know more than me about what role
>>>>> these play but it is probably necessary to ensure consistency.
>>>>>
>>>>> I have had a go at approaching this more generally but I fear that
>>>>> there may be little chance of getting away from a lot of testing.
>>>>>
>>>>> At this stage I have only started on the "system" files. To
>>>>> generalise the process I use a class of file walkers that traverse
>>>>> specified subdirectories of the filesystem, identify filesnames  
>>>>> that
>>>>> match what we care about (unfortunately more reliable methods of
>>>>> ignoring binary files is not implemented in python 2.4/2.5) and
>>>>> perform the substitutions by calling a chunk of your script.
>>>>>
>>>>> you will see at the bottom a chunk of slightly over-engineered
>>>>> inheritence statements and the start of some logic for different
>>>>> platforms. you can see the included examples of some system file
>>>>> areas that are particular to different platforms and are outside  
>>>>> the
>>>>> python library space.
>>>>>
>>>>> I have not yet actually set the script to make the alterations:  
>>>>> just
>>>>> find the targets but that should be a simple change of function  
>>>>> call
>>>>> once we feel it is ready. i have also not yet incorporated
>>>>> operations that work in the user space: it occurred to me that it
>>>>> might be best to approach that in a separate script: one for
>>>>> sudo/admin users to carefully apply to their system, and one for
>>>>> users to blithely run without bringing the whole thing down in a
>>>>> screaming heap.
>>>>>
>>>>> by all means send back edits, suggestions or point out how my
>>>>> approach is doomed to failure. in particular, suggestions of a  
>>>>> more
>>>>> pythonic approach are welcome: those that know java will  
>>>>> recognise my
>>>>> object-heavy paradigm. i will definitely boost the comments and  
>>>>> look
>>>>> at where checks are needed too.
>>>>>
>>>>> pasted below and attached
>>>>>
>>>>> Tim Salmon IT Technical Officer Faculty of Science UNSW
>>>>>
>>>>> import fileinput import os import platform import re import shutil
>>>>> import sys import tempfile from distutils.sysconfig import
>>>>> get_python_lib
>>>>>
>>>>> ## RewriteWalker performs a os.path.walk of base_dir class
>>>>> RewriteWalker: def __init__(self, base_dir, file_re, find_str,
>>>>> replace_str): self.base_dir = base_dir self.file_filter =
>>>>> re.compile(file_re) self.find_str = find_str self.replace_str =
>>>>> replace_str
>>>>>
>>>>> def walk_files(self): for root, directories, filenames in
>>>>> os.walk(self.base_dir): for filename in filenames: if
>>>>> (self.file_filter.match(filename)): filepath = os.path.join(root,
>>>>> filename) self.find_lines(filepath) #rewrite(filepath)
>>>>>
>>>>> def find_lines(self, srcfile): line_count = 0; src = open(srcfile,
>>>>> 'r') for line in src: line_count += 1 if self.find_str in line:  
>>>>> print
>>>>> srcfile print str(line_count) + ": " + line
>>>>>
>>>>> def rewrite(srcfile, searchPattern, replaceStr): dstfile =  
>>>>> srcfile +
>>>>> '.bak'
>>>>>
>>>>> #Checks to see if the file has been backed up if not then create a
>>>>> backup of the file. if not os.path.isfile(dstfile):
>>>>> shutil.copy(srcfile,dstfile)
>>>>>
>>>>> #creates a temporary file in the same directory that the  
>>>>> "executable"
>>>>> was launched from tmpfilename = str(os.getpid()) tfile =
>>>>> open(tmpfilename,'w') src = open(srcfile,'r')
>>>>>
>>>>> #loops through the entire file changing text for line in src: if
>>>>> searchPattern in line:
>>>>> tfile.write(line.replace(self.find_str,self.replace_str)) else:
>>>>> tfile.write(line) tfile.close() src.close()
>>>>>
>>>>> #overwrites the srcfile and deletes the temp file
>>>>> shutil.copy2(tmpfilename,srcfile) os.unlink(tmpfilename)
>>>>>
>>>>> class AGUpdater: def __init__(self):
>>>>>
>>>>> # define list of root paths and populate with those in python_lib
>>>>> self.root_path_list = [] ag_python_lib_folder_list =  
>>>>> ["AccessGrid",
>>>>> "AccessGrid3"] python_lib_dir = get_python_lib() for folder_name  
>>>>> in
>>>>> ag_python_lib_folder_list: folder_path =  
>>>>> os.path.join(python_lib_dir,
>>>>> folder_name) if os.path.exists(folder_path):
>>>>> self.root_path_list.append(folder_path)
>>>>>
>>>>> # define map of substitutions that can be applied [globally]
>>>>> self.subs_map = {} ##self.subs_map['bugzilla.mcs.anl.gov'] =
>>>>> 'bugzilla.accessgrid.org' self.subs_map['mcs.anl.gov'] =
>>>>> 'accessgrid.org' ##self.subs_map['140.221.6.95'] =
>>>>> 'www.accessgrid.org' ##self.subs_map['140.221.8.33'] =
>>>>> 'vv3.accessgrid.org'
>>>>>
>>>>> # define list of filename_re that identify which files will be
>>>>> modified # please note that this uses regular expressions not just
>>>>> simple expansions self.filename_re_list = []
>>>>> self.filename_re_list.append('.*\.py$')
>>>>> self.filename_re_list.append('.*\.txt$')
>>>>>
>>>>> # iterate throught root paths, filename_res and substitutions and
>>>>> create all combinations def generate_walkers(self):  
>>>>> self.walkers=[]
>>>>> for root_folder in self.root_path_list: for filename_re in
>>>>> self.filename_re_list: for subs_key, subs_val in
>>>>> self.subs_map.iteritems(): walker = RewriteWalker(root_folder,
>>>>> filename_re, subs_key, subs_val) self.walkers.append(walker)
>>>>>
>>>>> def perform_update(self): for walker in self.walkers:
>>>>> walker.walk_files()
>>>>>
>>>>> class WindowsAGUpdater(AGUpdater): def __init__(self):
>>>>> AGUpdater.__init__(self)
>>>>>
>>>>> class WindowsXPAGUpdater(WindowsAGUpdater): pass
>>>>>
>>>>> class WindowsXP_32AGUpdater(WindowsXPAGUpdater): pass
>>>>>
>>>>> class WindowsXP_64AGUpdater(WindowsXPAGUpdater): pass
>>>>>
>>>>> class Windows7AGUpdater(WindowsAGUpdater): def __init__(self):
>>>>> WindowsAGUpdater.__init__(self)
>>>>>
>>>>> class Windows7_32AGUpdater(Windows7AGUpdater): def __init__(self):
>>>>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\ 
>>>>> \Program
>>>>> Files\\AGTk3\\")
>>>>>
>>>>> class Windows7_64AGUpdater(Windows7AGUpdater): def __init__(self):
>>>>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\ 
>>>>> \Program
>>>>> Files (x86)\\AGTk3\\")
>>>>>
>>>>> class PosixAGUpdater(AGUpdater): def __init__(self):
>>>>> AGUpdater.__init__(self)
>>>>>
>>>>> class LinuxAGUpdater(PosixAGUpdater): def __init__(self):
>>>>> PosixAGUpdater.__init__(self)
>>>>>
>>>>> class RedHatLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>>>>> LinuxAGUpdater.__init__(self)
>>>>>
>>>>> class RHELLinuxAGUpdater(RedHatLinuxAGUpdater): def  
>>>>> __init__(self):
>>>>> RedHatLinuxAGUpdater.__init__(self)
>>>>> self.root_path_list.append('/usr/share/doc/AccessGrid-3.2/')
>>>>> self.filename_re_list.append('^Install\.LINUX')
>>>>> self.filename_re_list.append('^README')
>>>>> self.filename_re_list.append('^VERSION')
>>>>>
>>>>> class DebianLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>>>>> LinuxAGUpdater.__init__(self)
>>>>>
>>>>> class UbuntuLinuxAGUpdater(DebianLinuxAGUpdater): def
>>>>> __init__(self): DebianLinuxAGUpdater.__init__(self)
>>>>>
>>>>> class OSXAGUpdater(PosixAGUpdater): def __init__(self):
>>>>> PosixAGUpdater.__init__(self)
>>>>>
>>>>> if __name__ == "__main__": platform_str = platform.platform()
>>>>>
>>>>> updater = AGUpdater() if "Windows" in platform_str: if "7" in
>>>>> platform_str: if "64" in platform_str: updater =
>>>>> Windows7_64AGUpdater() elif "Linux" in platform_str: if "redhat"  
>>>>> in
>>>>> platform_str: if "6.2" in platform_str: updater =
>>>>> RHELLinuxAGUpdater()
>>>>>
>>>>> updater.generate_walkers() updater.perform_update()
>>>>>
>>>>> ________________________________________ From:
>>>>> ag-dev-bounces at lists.mcs.anl.gov [ag-dev- 
>>>>> bounces at lists.mcs.anl.gov]
>>>>> on behalf of Salim Haniff [Salim.Haniff at uta.fi] Sent: Thursday,  
>>>>> May
>>>>> 24, 2012 4:18 PM To: ag-tech at lists.mcs.anl.gov; ag-dev Subject:  
>>>>> Re:
>>>>> [AG-DEV] [AG-TECH] New accessgrid.org Venue Server
>>>>>
>>>>> Hi,
>>>>>
>>>>> The script for renaming urls and disabling venueclient's submit
>>>>> report bug is currently located at
>>>>>
>>>>> http://code.google.com/p/salims-scratch-box/source/browse/AGpatch/trunk/src/AGpatch.py
>>>>>
>>>>>
>>>>> I will move it later this week to the sourceforge AccessGrid
>>>>> location. Sorry if the script is a bit rough I'm a novice at  
>>>>> python
>>>>> but feel free to suggest improvements :)
>>>>>
>>>>> The script was developed and tested on a Fedora box and a  
>>>>> Scientific
>>>>> linux box as root, since root is the owner of the site-packages
>>>>> directory in python. I need to make a modification later today to
>>>>> allow the user to execute the python app to rename the urls in  
>>>>> their
>>>>> ~/.AccessGrid3/Config alone (probably takes about 5 minutes).  
>>>>> Windows
>>>>> and Mac still needs to be tested.
>>>>>
>>>>> At this time, disabling multicast by default for Windows 7 users  
>>>>> is
>>>>> still outstanding. If anyone knows how to do that feel free to
>>>>> email me.
>>>>>
>>>>> Best, Salim
>>>>>
>>>>>
>>>>> Quoting Christoph Willing<c.willing at uq.edu.au>:
>>>>>
>>>>>> The machine which will become vv3.accessgrid.org has now been
>>>>>> populated with all the venues& layout from vv3.mcs.anl.gov (which
>>>>>> will close down on 1st June).
>>>>>>
>>>>>> Could anyone who has a few minutes to spare please try out the  
>>>>>> new
>>>>>> server and report back any problems. Feel free to give it a good
>>>>>> workout. It would be great to have any issues with it sorted out
>>>>>> by the time its supposed to be on line for real - 1st June is  
>>>>>> just
>>>>>> about a week away.
>>>>>>
>>>>>> Until the DNS is arranged, access to the venue server is via:
>>>>>> https://130.102.78.188:8000/Venues/default
>>>>>>
>>>>>>
>>>>>>
>>>>>> chris
>>>>>>
>>>>>>
>>>>>>
>>>>>> Christoph Willing +61 7 3365 8316 Research Computing
>>>>>> Centre University of Queensland
>>>>>
>>>>>
>>>>> -- Salim Haniff (salim.haniff at uta.fi) School of Information
>>>>> Sciences 33014 University of Tampere, Finland
>>>>>
>>>>
>>>> --
>>>> ------------------------------------------------------------------------
>>>> James S. Perrin
>>>> AccessGrid/MAGIC/Visualization
>>>>
>>>> Research Computing Services
>>>> Devonshire House, University Precinct
>>>> The University of Manchester
>>>> Oxford Road, Manchester, M13 9PL
>>>>
>>>> t: +44 (0) 161 275 6945
>>>> e: james.perrin at manchester.ac.uk
>>>> w: www.manchester.ac.uk/researchcomputing
>>>> ------------------------------------------------------------------------
>>>> "The test of intellect is the refusal to belabour the obvious"
>>>> - Alfred Bester
>>>> ------------------------------------------------------------------------
>>>
>>>
>>>
>
> Christoph Willing              +61 7 3365 8316
> Research Computing Centre
> University of Queensland
>
>
>

Christoph Willing              +61 7 3365 8316
Research Computing Centre
University of Queensland





More information about the ag-tech mailing list