[AG-DEV] [AG-TECH] New accessgrid.org Venue Server

[Robert Frank]

Hi Salim,

I can confirm this for Debian.
I've tried your script on Squeeze and it doesn't work.
The AccessGrid libs are installed in /usr/share/pyshared/AccessGrid3, 
get_python_lib() returns /usr/lib/python2.6/dist-packages.
I have 2 versions of python installed (2.5 and 2.6) and the package 
creates symlinks to the installed AG lib files in the lib trees of both 
python versions:
/usr/lib/pymodules/python2.5/AccessGrid3
/usr/lib/pymodules/python2.6/AccessGrid3

I searched for a way of figuring out where the libs are and the best I 
could come up with was to import the library and let python tell me its 
path:

import AccessGrid3
site_packages_dir = os.path.join(AccessGrid3.__path__[0], "AccessGrid")

In case of the "AccessGrid" package name the above would have to be (not 
tested):

import AccessGrid
site_packages_dir = AccessGrid3.__path__[0]

I wonder if this approach would work on all systems.

Regards,

Robert
JANET AG Collaborations (UK)

On 30/05/12 15:41, Salim Haniff wrote:
> Hi everyone,
>
> I just installed the AGTk on my co-worker's Ubuntu (10.04.4) box. The
> AGTk python module are located in
> /usr/lib/pymodules/python2.6/AccessGrid3/AccessGrid . Is this a common
> area for other versions of Debain and Ubuntu? The user of that system
> has tinkered around with the system so I cannot confirm if this is a
> normal installation area.
>
> Best regards,
> Salim
>
>
> Quoting James S Perrin <james.s.perrin at manchester.ac.uk>:
>
>> Hi Tim, Salim,
>>
>> I quickly ran Tim's script on my Windows XP 32 machine using python
>> 2.4 (as installed by AGTK-3).
>>
>> 1. Added code for WindowsXP*Updaters
>> 2. needed a 'self.' in front of the root_path_list in the
>> Windows*AGUpdaters.
>>
>> As there is only one AGtk-3 package for all windows machine the
>> location will either be:
>>
>> C:\Program Files\AGTk-3 on 32bit or
>> C:\Program Files (x86)\AGTk-3 on 64bit
>>
>> So XP, Vista and 7 can be combined if you wish.
>>
>> 3. I also tried it on a Mac OSX 10.8 'Darwin' and it seemed to work as
>> expected.
>>
>> Regards
>> James
>> JANET AG Collaborations (UK)
>>
>>
>> On 30/05/2012 13:28, Tim Salmon wrote:
>>> Dear Salim, I ran the script on a Windows 7 64 but if returned only
>>> that the AccessGrid libs could not be found. on inspection I found
>>> that the path (default for the installer i used i assume) is
>>> ......\AccessGrid3 not ....\AccessGrid. rather than being a fault
>>> with your script i think that this may reflect my concern that there
>>> is a great diversity of installs out in the world that may have
>>> arisen by a variety of factors including platform, build process and
>>> usage patterns. I also have noted the occurrence of a number of
>>> places outside the python libs and userspaces that the AccessGrid
>>> files are installed. Others will know more than me about what role
>>> these play but it is probably necessary to ensure consistency.
>>>
>>> I have had a go at approaching this more generally but I fear that
>>> there may be little chance of getting away from a lot of testing.
>>>
>>> At this stage I have only started on the "system" files. To
>>> generalise the process I use a class of file walkers that traverse
>>> specified subdirectories of the filesystem, identify filesnames that
>>> match what we care about (unfortunately more reliable methods of
>>> ignoring binary files is not implemented in python 2.4/2.5) and
>>> perform the substitutions by calling a chunk of your script.
>>>
>>> you will see at the bottom a chunk of slightly over-engineered
>>> inheritence statements and the start of some logic for different
>>> platforms. you can see the included examples of some system file
>>> areas that are particular to different platforms and are outside the
>>> python library space.
>>>
>>> I have not yet actually set the script to make the alterations: just
>>> find the targets but that should be a simple change of function call
>>> once we feel it is ready. i have also not yet incorporated
>>> operations that work in the user space: it occurred to me that it
>>> might be best to approach that in a separate script: one for
>>> sudo/admin users to carefully apply to their system, and one for
>>> users to blithely run without bringing the whole thing down in a
>>> screaming heap.
>>>
>>> by all means send back edits, suggestions or point out how my
>>> approach is doomed to failure. in particular, suggestions of a more
>>> pythonic approach are welcome: those that know java will recognise my
>>> object-heavy paradigm. i will definitely boost the comments and look
>>> at where checks are needed too.
>>>
>>> pasted below and attached
>>>
>>> Tim Salmon IT Technical Officer Faculty of Science UNSW
>>>
>>> import fileinput import os import platform import re import shutil
>>> import sys import tempfile from distutils.sysconfig import
>>> get_python_lib
>>>
>>> ## RewriteWalker performs a os.path.walk of base_dir class
>>> RewriteWalker: def __init__(self, base_dir, file_re, find_str,
>>> replace_str): self.base_dir = base_dir self.file_filter =
>>> re.compile(file_re) self.find_str = find_str self.replace_str =
>>> replace_str
>>>
>>> def walk_files(self): for root, directories, filenames in
>>> os.walk(self.base_dir): for filename in filenames: if
>>> (self.file_filter.match(filename)): filepath = os.path.join(root,
>>> filename) self.find_lines(filepath) #rewrite(filepath)
>>>
>>> def find_lines(self, srcfile): line_count = 0; src = open(srcfile,
>>> 'r') for line in src: line_count += 1 if self.find_str in line: print
>>> srcfile print str(line_count) + ": " + line
>>>
>>> def rewrite(srcfile, searchPattern, replaceStr): dstfile = srcfile +
>>> '.bak'
>>>
>>> #Checks to see if the file has been backed up if not then create a
>>> backup of the file. if not os.path.isfile(dstfile):
>>> shutil.copy(srcfile,dstfile)
>>>
>>> #creates a temporary file in the same directory that the "executable"
>>> was launched from tmpfilename = str(os.getpid()) tfile =
>>> open(tmpfilename,'w') src = open(srcfile,'r')
>>>
>>> #loops through the entire file changing text for line in src: if
>>> searchPattern in line:
>>> tfile.write(line.replace(self.find_str,self.replace_str)) else:
>>> tfile.write(line) tfile.close() src.close()
>>>
>>> #overwrites the srcfile and deletes the temp file
>>> shutil.copy2(tmpfilename,srcfile) os.unlink(tmpfilename)
>>>
>>> class AGUpdater: def __init__(self):
>>>
>>> # define list of root paths and populate with those in python_lib
>>> self.root_path_list = [] ag_python_lib_folder_list = ["AccessGrid",
>>> "AccessGrid3"] python_lib_dir = get_python_lib() for folder_name in
>>> ag_python_lib_folder_list: folder_path = os.path.join(python_lib_dir,
>>> folder_name) if os.path.exists(folder_path):
>>> self.root_path_list.append(folder_path)
>>>
>>> # define map of substitutions that can be applied [globally]
>>> self.subs_map = {} ##self.subs_map['bugzilla.mcs.anl.gov'] =
>>> 'bugzilla.accessgrid.org' self.subs_map['mcs.anl.gov'] =
>>> 'accessgrid.org' ##self.subs_map['140.221.6.95'] =
>>> 'www.accessgrid.org' ##self.subs_map['140.221.8.33'] =
>>> 'vv3.accessgrid.org'
>>>
>>> # define list of filename_re that identify which files will be
>>> modified # please note that this uses regular expressions not just
>>> simple expansions self.filename_re_list = []
>>> self.filename_re_list.append('.*\.py$')
>>> self.filename_re_list.append('.*\.txt$')
>>>
>>> # iterate throught root paths, filename_res and substitutions and
>>> create all combinations def generate_walkers(self): self.walkers=[]
>>> for root_folder in self.root_path_list: for filename_re in
>>> self.filename_re_list: for subs_key, subs_val in
>>> self.subs_map.iteritems(): walker = RewriteWalker(root_folder,
>>> filename_re, subs_key, subs_val) self.walkers.append(walker)
>>>
>>> def perform_update(self): for walker in self.walkers:
>>> walker.walk_files()
>>>
>>> class WindowsAGUpdater(AGUpdater): def __init__(self):
>>> AGUpdater.__init__(self)
>>>
>>> class WindowsXPAGUpdater(WindowsAGUpdater): pass
>>>
>>> class WindowsXP_32AGUpdater(WindowsXPAGUpdater): pass
>>>
>>> class WindowsXP_64AGUpdater(WindowsXPAGUpdater): pass
>>>
>>> class Windows7AGUpdater(WindowsAGUpdater): def __init__(self):
>>> WindowsAGUpdater.__init__(self)
>>>
>>> class Windows7_32AGUpdater(Windows7AGUpdater): def __init__(self):
>>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
>>> Files\\AGTk3\\")
>>>
>>> class Windows7_64AGUpdater(Windows7AGUpdater): def __init__(self):
>>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
>>> Files (x86)\\AGTk3\\")
>>>
>>> class PosixAGUpdater(AGUpdater): def __init__(self):
>>> AGUpdater.__init__(self)
>>>
>>> class LinuxAGUpdater(PosixAGUpdater): def __init__(self):
>>> PosixAGUpdater.__init__(self)
>>>
>>> class RedHatLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>>> LinuxAGUpdater.__init__(self)
>>>
>>> class RHELLinuxAGUpdater(RedHatLinuxAGUpdater): def __init__(self):
>>> RedHatLinuxAGUpdater.__init__(self)
>>> self.root_path_list.append('/usr/share/doc/AccessGrid-3.2/')
>>> self.filename_re_list.append('^Install\.LINUX')
>>> self.filename_re_list.append('^README')
>>> self.filename_re_list.append('^VERSION')
>>>
>>> class DebianLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>>> LinuxAGUpdater.__init__(self)
>>>
>>> class UbuntuLinuxAGUpdater(DebianLinuxAGUpdater): def
>>> __init__(self): DebianLinuxAGUpdater.__init__(self)
>>>
>>> class OSXAGUpdater(PosixAGUpdater): def __init__(self):
>>> PosixAGUpdater.__init__(self)
>>>
>>> if __name__ == "__main__": platform_str = platform.platform()
>>>
>>> updater = AGUpdater() if "Windows" in platform_str: if "7" in
>>> platform_str: if "64" in platform_str: updater =
>>> Windows7_64AGUpdater() elif "Linux" in platform_str: if "redhat" in
>>> platform_str: if "6.2" in platform_str: updater =
>>> RHELLinuxAGUpdater()
>>>
>>> updater.generate_walkers() updater.perform_update()
>>>
>>> ________________________________________ From:
>>> ag-dev-bounces at lists.mcs.anl.gov [ag-dev-bounces at lists.mcs.anl.gov]
>>> on behalf of Salim Haniff [Salim.Haniff at uta.fi] Sent: Thursday, May
>>> 24, 2012 4:18 PM To: ag-tech at lists.mcs.anl.gov; ag-dev Subject: Re:
>>> [AG-DEV] [AG-TECH] New accessgrid.org Venue Server
>>>
>>> Hi,
>>>
>>> The script for renaming urls and disabling venueclient's submit
>>> report bug is currently located at
>>>
>>> http://code.google.com/p/salims-scratch-box/source/browse/AGpatch/trunk/src/AGpatch.py
>>>
>>>
>>> I will move it later this week to the sourceforge AccessGrid
>>> location. Sorry if the script is a bit rough I'm a novice at python
>>> but feel free to suggest improvements :)
>>>
>>> The script was developed and tested on a Fedora box and a Scientific
>>> linux box as root, since root is the owner of the site-packages
>>> directory in python. I need to make a modification later today to
>>> allow the user to execute the python app to rename the urls in their
>>> ~/.AccessGrid3/Config alone (probably takes about 5 minutes). Windows
>>> and Mac still needs to be tested.
>>>
>>> At this time, disabling multicast by default for Windows 7 users is
>>> still outstanding. If anyone knows how to do that feel free to
>>> email me.
>>>
>>> Best, Salim
>>>
>>>
>>> Quoting Christoph Willing<c.willing at uq.edu.au>:
>>>
>>>> The machine which will become vv3.accessgrid.org has now been
>>>> populated with all the venues& layout from vv3.mcs.anl.gov (which
>>>> will close down on 1st June).
>>>>
>>>> Could anyone who has a few minutes to spare please try out the new
>>>> server and report back any problems. Feel free to give it a good
>>>> workout. It would be great to have any issues with it sorted out
>>>> by the time its supposed to be on line for real - 1st June is just
>>>> about a week away.
>>>>
>>>> Until the DNS is arranged, access to the venue server is via:
>>>> https://130.102.78.188:8000/Venues/default
>>>>
>>>>
>>>>
>>>> chris
>>>>
>>>>
>>>>
>>>> Christoph Willing +61 7 3365 8316 Research Computing
>>>> Centre University of Queensland
>>>
>>>
>>> -- Salim Haniff (salim.haniff at uta.fi) School of Information
>>> Sciences 33014 University of Tampere, Finland
>>>
>>
>> --
>> ------------------------------------------------------------------------
>> James S. Perrin
>> AccessGrid/MAGIC/Visualization
>>
>> Research Computing Services
>> Devonshire House, University Precinct
>> The University of Manchester
>> Oxford Road, Manchester, M13 9PL
>>
>> t: +44 (0) 161 275 6945
>> e: james.perrin at manchester.ac.uk
>> w: www.manchester.ac.uk/researchcomputing
>> ------------------------------------------------------------------------
>> "The test of intellect is the refusal to belabour the obvious"
>> - Alfred Bester
>> ------------------------------------------------------------------------
>
>
>