[AG-TECH] [AG-DEV] New accessgrid.org Venue Server

Salim Haniff Salim.Haniff at uta.fi
Wed May 30 09:41:10 CDT 2012


Hi everyone,

I just installed the AGTk on my co-worker's Ubuntu (10.04.4) box. The  
AGTk python module are located in  
/usr/lib/pymodules/python2.6/AccessGrid3/AccessGrid .  Is this a  
common area for other versions of Debain and Ubuntu?  The user of that  
system has tinkered around with the system so I cannot confirm if this  
is a normal installation area.

Best regards,
Salim


Quoting James S Perrin <james.s.perrin at manchester.ac.uk>:

> Hi Tim, Salim,
>
> 	I quickly ran Tim's script on my Windows XP 32 machine using python  
> 2.4 (as installed by AGTK-3).
>
> 1. Added code for WindowsXP*Updaters
> 2. needed a 'self.' in front of the root_path_list in the Windows*AGUpdaters.
>
> 	As there is only one AGtk-3 package for all windows machine the  
> location will either be:
>
> C:\Program Files\AGTk-3 on 32bit or
> C:\Program Files (x86)\AGTk-3 on 64bit
>
> So XP, Vista and 7 can be combined if you wish.
>
> 3. I also tried it on a Mac OSX 10.8 'Darwin' and it seemed to work  
> as expected.
>
> Regards
> James
> JANET AG Collaborations (UK)
>
>
> On 30/05/2012 13:28, Tim Salmon wrote:
>> Dear Salim, I ran the script on a Windows 7 64 but if returned only
>> that the AccessGrid libs could not be found.  on inspection I found
>> that the path (default for the installer i used i assume) is
>> ......\AccessGrid3 not  ....\AccessGrid.  rather than being a fault
>> with your script i think that this may reflect my concern that there
>> is a great diversity of installs out in the world that may have
>> arisen by a variety of factors including platform, build process and
>> usage patterns. I also have noted the occurrence of a number of
>> places outside the python libs and userspaces that the AccessGrid
>> files are installed.  Others will know more than me about what role
>> these play but it is probably necessary to ensure consistency.
>>
>> I have had a go at approaching this more generally but I fear that
>> there may be little chance of getting away from a lot of testing.
>>
>> At this stage I have only started on the "system" files.  To
>> generalise the process I use a class of file walkers that traverse
>> specified subdirectories of the filesystem, identify filesnames that
>> match what we care about (unfortunately more reliable methods of
>> ignoring binary files is not implemented in python 2.4/2.5) and
>> perform the substitutions by calling a chunk of your script.
>>
>> you will see at the bottom a chunk of slightly over-engineered
>> inheritence statements and the start of some logic for different
>> platforms.  you can see the included examples of some system file
>> areas that are particular to different platforms and are outside the
>> python library space.
>>
>> I have not yet actually set the script to make the alterations: just
>> find the targets but that should be a simple change of function call
>> once we feel it is ready.  i have also not yet incorporated
>> operations that work in the user space: it occurred to me that it
>> might be best to approach that in a separate script: one for
>> sudo/admin users to carefully apply to their system, and one for
>> users to blithely run without bringing the whole thing down in a
>> screaming heap.
>>
>> by all means send back edits, suggestions or point out how my
>> approach is doomed to failure.  in particular, suggestions of a more
>> pythonic approach are welcome: those that know java will recognise my
>> object-heavy paradigm.  i will definitely boost the comments and look
>> at where checks are needed too.
>>
>> pasted below and attached
>>
>> Tim Salmon IT Technical Officer Faculty of Science UNSW
>>
>> import fileinput import os import platform import re import shutil
>> import sys import tempfile from distutils.sysconfig import
>> get_python_lib
>>
>> ## RewriteWalker performs a os.path.walk of base_dir class
>> RewriteWalker: def __init__(self, base_dir, file_re, find_str,
>> replace_str): self.base_dir = base_dir self.file_filter =
>> re.compile(file_re) self.find_str = find_str self.replace_str =
>> replace_str
>>
>> def walk_files(self): for root, directories, filenames in
>> os.walk(self.base_dir): for filename in filenames: if
>> (self.file_filter.match(filename)): filepath = os.path.join(root,
>> filename) self.find_lines(filepath) #rewrite(filepath)
>>
>> def find_lines(self, srcfile): line_count = 0; src = open(srcfile,
>> 'r') for line in src: line_count += 1 if self.find_str in line: print
>> srcfile print str(line_count) + ": " + line
>>
>> def rewrite(srcfile, searchPattern, replaceStr): dstfile = srcfile +
>> '.bak'
>>
>> #Checks to see if the file has been backed up if not then create a
>> backup of the file. if not os.path.isfile(dstfile):
>> shutil.copy(srcfile,dstfile)
>>
>> #creates a temporary file in the same directory that the "executable"
>> was launched from tmpfilename = str(os.getpid()) tfile =
>> open(tmpfilename,'w') src = open(srcfile,'r')
>>
>> #loops through the entire file changing text for line in src: if
>> searchPattern in line:
>> tfile.write(line.replace(self.find_str,self.replace_str)) else:
>> tfile.write(line) tfile.close() src.close()
>>
>> #overwrites the srcfile and deletes the temp file
>> shutil.copy2(tmpfilename,srcfile) os.unlink(tmpfilename)
>>
>> class AGUpdater: def __init__(self):
>>
>> # define list of root paths and populate with those in python_lib
>> self.root_path_list = [] ag_python_lib_folder_list = ["AccessGrid",
>> "AccessGrid3"] python_lib_dir = get_python_lib() for folder_name in
>> ag_python_lib_folder_list: folder_path = os.path.join(python_lib_dir,
>> folder_name) if os.path.exists(folder_path):
>> self.root_path_list.append(folder_path)
>>
>> # define map of substitutions that can be applied [globally]
>> self.subs_map = {} ##self.subs_map['bugzilla.mcs.anl.gov'] =
>> 'bugzilla.accessgrid.org' self.subs_map['mcs.anl.gov'] =
>> 'accessgrid.org' ##self.subs_map['140.221.6.95'] =
>> 'www.accessgrid.org' ##self.subs_map['140.221.8.33'] =
>> 'vv3.accessgrid.org'
>>
>> # define list of filename_re that identify which files will be
>> modified # please note that this uses regular expressions not just
>> simple expansions self.filename_re_list = []
>> self.filename_re_list.append('.*\.py$')
>> self.filename_re_list.append('.*\.txt$')
>>
>> # iterate throught root paths, filename_res and substitutions and
>> create all combinations def generate_walkers(self): self.walkers=[]
>> for root_folder in self.root_path_list: for filename_re in
>> self.filename_re_list: for subs_key, subs_val in
>> self.subs_map.iteritems(): walker = RewriteWalker(root_folder,
>> filename_re, subs_key, subs_val) self.walkers.append(walker)
>>
>> def perform_update(self): for walker in self.walkers:
>> walker.walk_files()
>>
>> class WindowsAGUpdater(AGUpdater): def __init__(self):
>> AGUpdater.__init__(self)
>>
>> class WindowsXPAGUpdater(WindowsAGUpdater): pass
>>
>> class WindowsXP_32AGUpdater(WindowsXPAGUpdater): pass
>>
>> class WindowsXP_64AGUpdater(WindowsXPAGUpdater): pass
>>
>> class Windows7AGUpdater(WindowsAGUpdater): def __init__(self):
>> WindowsAGUpdater.__init__(self)
>>
>> class Windows7_32AGUpdater(Windows7AGUpdater): def __init__(self):
>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
>> Files\\AGTk3\\")
>>
>> class Windows7_64AGUpdater(Windows7AGUpdater): def __init__(self):
>> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
>> Files (x86)\\AGTk3\\")
>>
>> class PosixAGUpdater(AGUpdater): def __init__(self):
>> AGUpdater.__init__(self)
>>
>> class LinuxAGUpdater(PosixAGUpdater): def __init__(self):
>> PosixAGUpdater.__init__(self)
>>
>> class RedHatLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>> LinuxAGUpdater.__init__(self)
>>
>> class RHELLinuxAGUpdater(RedHatLinuxAGUpdater): def __init__(self):
>> RedHatLinuxAGUpdater.__init__(self)
>> self.root_path_list.append('/usr/share/doc/AccessGrid-3.2/')
>> self.filename_re_list.append('^Install\.LINUX')
>> self.filename_re_list.append('^README')
>> self.filename_re_list.append('^VERSION')
>>
>> class DebianLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
>> LinuxAGUpdater.__init__(self)
>>
>> class UbuntuLinuxAGUpdater(DebianLinuxAGUpdater): def
>> __init__(self): DebianLinuxAGUpdater.__init__(self)
>>
>> class OSXAGUpdater(PosixAGUpdater): def __init__(self):
>> PosixAGUpdater.__init__(self)
>>
>> if __name__ == "__main__": platform_str = platform.platform()
>>
>> updater = AGUpdater() if "Windows" in platform_str: if "7" in
>> platform_str: if "64" in platform_str: updater =
>> Windows7_64AGUpdater() elif "Linux" in platform_str: if "redhat" in
>> platform_str: if "6.2" in platform_str: updater =
>> RHELLinuxAGUpdater()
>>
>> updater.generate_walkers() updater.perform_update()
>>
>> ________________________________________ From:
>> ag-dev-bounces at lists.mcs.anl.gov [ag-dev-bounces at lists.mcs.anl.gov]
>> on behalf of Salim Haniff [Salim.Haniff at uta.fi] Sent: Thursday, May
>> 24, 2012 4:18 PM To: ag-tech at lists.mcs.anl.gov; ag-dev Subject: Re:
>> [AG-DEV] [AG-TECH] New accessgrid.org Venue Server
>>
>> Hi,
>>
>> The script for renaming urls and disabling venueclient's submit
>> report bug is currently located at
>>
>> http://code.google.com/p/salims-scratch-box/source/browse/AGpatch/trunk/src/AGpatch.py
>>
>> I will move it later this week to the sourceforge AccessGrid
>> location. Sorry if the script is a bit rough I'm a novice at python
>> but feel free to suggest improvements :)
>>
>> The script was developed and tested on a Fedora box and a Scientific
>> linux box as root, since root is the owner of the site-packages
>> directory in python. I need to make a modification later today to
>> allow the user to execute the python app to rename the urls in their
>> ~/.AccessGrid3/Config alone (probably takes about 5 minutes). Windows
>> and Mac still needs to be tested.
>>
>> At this time, disabling multicast by default for Windows 7 users is
>> still outstanding.  If anyone knows how to do that feel free to
>> email me.
>>
>> Best, Salim
>>
>>
>> Quoting Christoph Willing<c.willing at uq.edu.au>:
>>
>>> The machine which will become vv3.accessgrid.org has now been
>>> populated with all the venues&  layout from vv3.mcs.anl.gov (which
>>> will close down on 1st June).
>>>
>>> Could anyone who has a few minutes to spare please try out the new
>>> server and report back any problems. Feel free to give it a good
>>> workout. It would be great to have any issues with it sorted out
>>> by the time its supposed to be on line for real - 1st June is just
>>> about a week away.
>>>
>>> Until the DNS is arranged, access to the venue server is via:
>>> https://130.102.78.188:8000/Venues/default
>>>
>>>
>>>
>>> chris
>>>
>>>
>>>
>>> Christoph Willing              +61 7 3365 8316 Research Computing
>>> Centre University of Queensland
>>
>>
>> -- Salim Haniff  (salim.haniff at uta.fi) School of Information
>> Sciences 33014 University of Tampere, Finland
>>
>
> -- 
> ------------------------------------------------------------------------
>   James S. Perrin
>   AccessGrid/MAGIC/Visualization
>
>   Research Computing Services
>   Devonshire House, University Precinct
>   The University of Manchester
>   Oxford Road, Manchester, M13 9PL
>
>   t: +44 (0) 161 275 6945
>   e: james.perrin at manchester.ac.uk
>   w: www.manchester.ac.uk/researchcomputing
> ------------------------------------------------------------------------
>  "The test of intellect is the refusal to belabour the obvious"
>  - Alfred Bester
> ------------------------------------------------------------------------





More information about the ag-tech mailing list