[AG-DEV] [AG-TECH] New accessgrid.org Venue Server
James S Perrin
james.s.perrin at manchester.ac.uk
Wed May 30 09:11:05 CDT 2012
Hi Tim, Salim,
I quickly ran Tim's script on my Windows XP 32 machine using python 2.4
(as installed by AGTK-3).
1. Added code for WindowsXP*Updaters
2. needed a 'self.' in front of the root_path_list in the
Windows*AGUpdaters.
As there is only one AGtk-3 package for all windows machine the
location will either be:
C:\Program Files\AGTk-3 on 32bit or
C:\Program Files (x86)\AGTk-3 on 64bit
So XP, Vista and 7 can be combined if you wish.
3. I also tried it on a Mac OSX 10.8 'Darwin' and it seemed to work as
expected.
Regards
James
JANET AG Collaborations (UK)
On 30/05/2012 13:28, Tim Salmon wrote:
> Dear Salim, I ran the script on a Windows 7 64 but if returned only
> that the AccessGrid libs could not be found. on inspection I found
> that the path (default for the installer i used i assume) is
> ......\AccessGrid3 not ....\AccessGrid. rather than being a fault
> with your script i think that this may reflect my concern that there
> is a great diversity of installs out in the world that may have
> arisen by a variety of factors including platform, build process and
> usage patterns. I also have noted the occurrence of a number of
> places outside the python libs and userspaces that the AccessGrid
> files are installed. Others will know more than me about what role
> these play but it is probably necessary to ensure consistency.
>
> I have had a go at approaching this more generally but I fear that
> there may be little chance of getting away from a lot of testing.
>
> At this stage I have only started on the "system" files. To
> generalise the process I use a class of file walkers that traverse
> specified subdirectories of the filesystem, identify filesnames that
> match what we care about (unfortunately more reliable methods of
> ignoring binary files is not implemented in python 2.4/2.5) and
> perform the substitutions by calling a chunk of your script.
>
> you will see at the bottom a chunk of slightly over-engineered
> inheritence statements and the start of some logic for different
> platforms. you can see the included examples of some system file
> areas that are particular to different platforms and are outside the
> python library space.
>
> I have not yet actually set the script to make the alterations: just
> find the targets but that should be a simple change of function call
> once we feel it is ready. i have also not yet incorporated
> operations that work in the user space: it occurred to me that it
> might be best to approach that in a separate script: one for
> sudo/admin users to carefully apply to their system, and one for
> users to blithely run without bringing the whole thing down in a
> screaming heap.
>
> by all means send back edits, suggestions or point out how my
> approach is doomed to failure. in particular, suggestions of a more
> pythonic approach are welcome: those that know java will recognise my
> object-heavy paradigm. i will definitely boost the comments and look
> at where checks are needed too.
>
> pasted below and attached
>
> Tim Salmon IT Technical Officer Faculty of Science UNSW
>
> import fileinput import os import platform import re import shutil
> import sys import tempfile from distutils.sysconfig import
> get_python_lib
>
> ## RewriteWalker performs a os.path.walk of base_dir class
> RewriteWalker: def __init__(self, base_dir, file_re, find_str,
> replace_str): self.base_dir = base_dir self.file_filter =
> re.compile(file_re) self.find_str = find_str self.replace_str =
> replace_str
>
> def walk_files(self): for root, directories, filenames in
> os.walk(self.base_dir): for filename in filenames: if
> (self.file_filter.match(filename)): filepath = os.path.join(root,
> filename) self.find_lines(filepath) #rewrite(filepath)
>
> def find_lines(self, srcfile): line_count = 0; src = open(srcfile,
> 'r') for line in src: line_count += 1 if self.find_str in line: print
> srcfile print str(line_count) + ": " + line
>
> def rewrite(srcfile, searchPattern, replaceStr): dstfile = srcfile +
> '.bak'
>
> #Checks to see if the file has been backed up if not then create a
> backup of the file. if not os.path.isfile(dstfile):
> shutil.copy(srcfile,dstfile)
>
> #creates a temporary file in the same directory that the "executable"
> was launched from tmpfilename = str(os.getpid()) tfile =
> open(tmpfilename,'w') src = open(srcfile,'r')
>
> #loops through the entire file changing text for line in src: if
> searchPattern in line:
> tfile.write(line.replace(self.find_str,self.replace_str)) else:
> tfile.write(line) tfile.close() src.close()
>
> #overwrites the srcfile and deletes the temp file
> shutil.copy2(tmpfilename,srcfile) os.unlink(tmpfilename)
>
> class AGUpdater: def __init__(self):
>
> # define list of root paths and populate with those in python_lib
> self.root_path_list = [] ag_python_lib_folder_list = ["AccessGrid",
> "AccessGrid3"] python_lib_dir = get_python_lib() for folder_name in
> ag_python_lib_folder_list: folder_path = os.path.join(python_lib_dir,
> folder_name) if os.path.exists(folder_path):
> self.root_path_list.append(folder_path)
>
> # define map of substitutions that can be applied [globally]
> self.subs_map = {} ##self.subs_map['bugzilla.mcs.anl.gov'] =
> 'bugzilla.accessgrid.org' self.subs_map['mcs.anl.gov'] =
> 'accessgrid.org' ##self.subs_map['140.221.6.95'] =
> 'www.accessgrid.org' ##self.subs_map['140.221.8.33'] =
> 'vv3.accessgrid.org'
>
> # define list of filename_re that identify which files will be
> modified # please note that this uses regular expressions not just
> simple expansions self.filename_re_list = []
> self.filename_re_list.append('.*\.py$')
> self.filename_re_list.append('.*\.txt$')
>
> # iterate throught root paths, filename_res and substitutions and
> create all combinations def generate_walkers(self): self.walkers=[]
> for root_folder in self.root_path_list: for filename_re in
> self.filename_re_list: for subs_key, subs_val in
> self.subs_map.iteritems(): walker = RewriteWalker(root_folder,
> filename_re, subs_key, subs_val) self.walkers.append(walker)
>
> def perform_update(self): for walker in self.walkers:
> walker.walk_files()
>
> class WindowsAGUpdater(AGUpdater): def __init__(self):
> AGUpdater.__init__(self)
>
> class WindowsXPAGUpdater(WindowsAGUpdater): pass
>
> class WindowsXP_32AGUpdater(WindowsXPAGUpdater): pass
>
> class WindowsXP_64AGUpdater(WindowsXPAGUpdater): pass
>
> class Windows7AGUpdater(WindowsAGUpdater): def __init__(self):
> WindowsAGUpdater.__init__(self)
>
> class Windows7_32AGUpdater(Windows7AGUpdater): def __init__(self):
> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
> Files\\AGTk3\\")
>
> class Windows7_64AGUpdater(Windows7AGUpdater): def __init__(self):
> Windows7AGUpdater.__init__(self) root_path_list.append("C:\\Program
> Files (x86)\\AGTk3\\")
>
> class PosixAGUpdater(AGUpdater): def __init__(self):
> AGUpdater.__init__(self)
>
> class LinuxAGUpdater(PosixAGUpdater): def __init__(self):
> PosixAGUpdater.__init__(self)
>
> class RedHatLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
> LinuxAGUpdater.__init__(self)
>
> class RHELLinuxAGUpdater(RedHatLinuxAGUpdater): def __init__(self):
> RedHatLinuxAGUpdater.__init__(self)
> self.root_path_list.append('/usr/share/doc/AccessGrid-3.2/')
> self.filename_re_list.append('^Install\.LINUX')
> self.filename_re_list.append('^README')
> self.filename_re_list.append('^VERSION')
>
> class DebianLinuxAGUpdater(LinuxAGUpdater): def __init__(self):
> LinuxAGUpdater.__init__(self)
>
> class UbuntuLinuxAGUpdater(DebianLinuxAGUpdater): def
> __init__(self): DebianLinuxAGUpdater.__init__(self)
>
> class OSXAGUpdater(PosixAGUpdater): def __init__(self):
> PosixAGUpdater.__init__(self)
>
> if __name__ == "__main__": platform_str = platform.platform()
>
> updater = AGUpdater() if "Windows" in platform_str: if "7" in
> platform_str: if "64" in platform_str: updater =
> Windows7_64AGUpdater() elif "Linux" in platform_str: if "redhat" in
> platform_str: if "6.2" in platform_str: updater =
> RHELLinuxAGUpdater()
>
> updater.generate_walkers() updater.perform_update()
>
> ________________________________________ From:
> ag-dev-bounces at lists.mcs.anl.gov [ag-dev-bounces at lists.mcs.anl.gov]
> on behalf of Salim Haniff [Salim.Haniff at uta.fi] Sent: Thursday, May
> 24, 2012 4:18 PM To: ag-tech at lists.mcs.anl.gov; ag-dev Subject: Re:
> [AG-DEV] [AG-TECH] New accessgrid.org Venue Server
>
> Hi,
>
> The script for renaming urls and disabling venueclient's submit
> report bug is currently located at
>
> http://code.google.com/p/salims-scratch-box/source/browse/AGpatch/trunk/src/AGpatch.py
>
> I will move it later this week to the sourceforge AccessGrid
> location. Sorry if the script is a bit rough I'm a novice at python
> but feel free to suggest improvements :)
>
> The script was developed and tested on a Fedora box and a Scientific
> linux box as root, since root is the owner of the site-packages
> directory in python. I need to make a modification later today to
> allow the user to execute the python app to rename the urls in their
> ~/.AccessGrid3/Config alone (probably takes about 5 minutes). Windows
> and Mac still needs to be tested.
>
> At this time, disabling multicast by default for Windows 7 users is
> still outstanding. If anyone knows how to do that feel free to
> email me.
>
> Best, Salim
>
>
> Quoting Christoph Willing<c.willing at uq.edu.au>:
>
>> The machine which will become vv3.accessgrid.org has now been
>> populated with all the venues& layout from vv3.mcs.anl.gov (which
>> will close down on 1st June).
>>
>> Could anyone who has a few minutes to spare please try out the new
>> server and report back any problems. Feel free to give it a good
>> workout. It would be great to have any issues with it sorted out
>> by the time its supposed to be on line for real - 1st June is just
>> about a week away.
>>
>> Until the DNS is arranged, access to the venue server is via:
>> https://130.102.78.188:8000/Venues/default
>>
>>
>>
>> chris
>>
>>
>>
>> Christoph Willing +61 7 3365 8316 Research Computing
>> Centre University of Queensland
>
>
> -- Salim Haniff (salim.haniff at uta.fi) School of Information
> Sciences 33014 University of Tampere, Finland
>
--
------------------------------------------------------------------------
James S. Perrin
AccessGrid/MAGIC/Visualization
Research Computing Services
Devonshire House, University Precinct
The University of Manchester
Oxford Road, Manchester, M13 9PL
t: +44 (0) 161 275 6945
e: james.perrin at manchester.ac.uk
w: www.manchester.ac.uk/researchcomputing
------------------------------------------------------------------------
"The test of intellect is the refusal to belabour the obvious"
- Alfred Bester
------------------------------------------------------------------------
-------------- next part --------------
import fileinput
import os
import platform
import re
import shutil
import sys
import tempfile
from distutils.sysconfig import get_python_lib
## RewriteWalker performs a os.path.walk of base_dir
class RewriteWalker:
def __init__(self, base_dir, file_re, find_str, replace_str):
self.base_dir = base_dir
self.file_filter = re.compile(file_re)
self.find_str = find_str
self.replace_str = replace_str
def walk_files(self):
for root, directories, filenames in os.walk(self.base_dir):
for filename in filenames:
if (self.file_filter.match(filename)):
filepath = os.path.join(root, filename)
self.find_lines(filepath)
#rewrite(filepath)
def find_lines(self, srcfile):
line_count = 0;
src = open(srcfile, 'r')
for line in src:
line_count += 1
if self.find_str in line:
print srcfile
print str(line_count) + ": " + line
def rewrite(srcfile, searchPattern, replaceStr):
dstfile = srcfile + '.bak'
#Checks to see if the file has been backed up if not then create a backup of the file.
if not os.path.isfile(dstfile):
shutil.copy(srcfile,dstfile)
#creates a temporary file in the same directory that the "executable" was launched from
tmpfilename = str(os.getpid())
tfile = open(tmpfilename,'w')
src = open(srcfile,'r')
#loops through the entire file changing text
for line in src:
if searchPattern in line:
tfile.write(line.replace(self.find_str,self.replace_str))
else:
tfile.write(line)
tfile.close()
src.close()
#overwrites the srcfile and deletes the temp file
shutil.copy2(tmpfilename,srcfile)
os.unlink(tmpfilename)
class AGUpdater:
def __init__(self):
# define list of root paths and populate with those in python_lib
self.root_path_list = []
ag_python_lib_folder_list = ["AccessGrid", "AccessGrid3"]
python_lib_dir = get_python_lib()
for folder_name in ag_python_lib_folder_list:
folder_path = os.path.join(python_lib_dir, folder_name)
if os.path.exists(folder_path):
self.root_path_list.append(folder_path)
# define map of substitutions that can be applied [globally]
self.subs_map = {}
##self.subs_map['bugzilla.mcs.anl.gov'] = 'bugzilla.accessgrid.org'
self.subs_map['mcs.anl.gov'] = 'accessgrid.org'
##self.subs_map['140.221.6.95'] = 'www.accessgrid.org'
##self.subs_map['140.221.8.33'] = 'vv3.accessgrid.org'
# define list of filename_re that identify which files will be modified
# please note that this uses regular expressions not just simple expansions
self.filename_re_list = []
self.filename_re_list.append('.*\.py$')
self.filename_re_list.append('.*\.txt$')
# iterate throught root paths, filename_res and substitutions and create all combinations
def generate_walkers(self):
self.walkers=[]
for root_folder in self.root_path_list:
for filename_re in self.filename_re_list:
for subs_key, subs_val in self.subs_map.iteritems():
walker = RewriteWalker(root_folder, filename_re, subs_key, subs_val)
self.walkers.append(walker)
def perform_update(self):
for walker in self.walkers:
walker.walk_files()
class WindowsAGUpdater(AGUpdater):
def __init__(self):
AGUpdater.__init__(self)
class WindowsXPAGUpdater(WindowsAGUpdater):
def __init__(self):
WindowsAGUpdater.__init__(self)
class WindowsXP_32AGUpdater(WindowsXPAGUpdater):
def __init__(self):
WindowsXPAGUpdater.__init__(self)
self.root_path_list.append("C:\\Program Files\\AGTk3\\")
class WindowsXP_64AGUpdater(WindowsXPAGUpdater):
def __init__(self):
WindowsXPAGUpdater.__init__(self)
self.root_path_list.append("C:\\Program Files (x86)\\AGTk3\\")
class Windows7AGUpdater(WindowsAGUpdater):
def __init__(self):
WindowsAGUpdater.__init__(self)
class Windows7_32AGUpdater(Windows7AGUpdater):
def __init__(self):
Windows7AGUpdater.__init__(self)
self.root_path_list.append("C:\\Program Files\\AGTk3\\")
class Windows7_64AGUpdater(Windows7AGUpdater):
def __init__(self):
Windows7AGUpdater.__init__(self)
self.root_path_list.append("C:\\Program Files (x86)\\AGTk3\\")
class PosixAGUpdater(AGUpdater):
def __init__(self):
AGUpdater.__init__(self)
class LinuxAGUpdater(PosixAGUpdater):
def __init__(self):
PosixAGUpdater.__init__(self)
class RedHatLinuxAGUpdater(LinuxAGUpdater):
def __init__(self):
LinuxAGUpdater.__init__(self)
class RHELLinuxAGUpdater(RedHatLinuxAGUpdater):
def __init__(self):
RedHatLinuxAGUpdater.__init__(self)
self.root_path_list.append('/usr/share/doc/AccessGrid-3.2/')
self.filename_re_list.append('^Install\.LINUX')
self.filename_re_list.append('^README')
self.filename_re_list.append('^VERSION')
class DebianLinuxAGUpdater(LinuxAGUpdater):
def __init__(self):
LinuxAGUpdater.__init__(self)
class UbuntuLinuxAGUpdater(DebianLinuxAGUpdater):
def __init__(self):
DebianLinuxAGUpdater.__init__(self)
class OSXAGUpdater(PosixAGUpdater):
def __init__(self):
PosixAGUpdater.__init__(self)
self.root_path_list.append('/Applications/AccessGridToolKit3.app/Contents/Resources')
if __name__ == "__main__":
platform_str = platform.platform()
updater = AGUpdater()
if "Windows" in platform_str:
if "7" in platform_str:
if "64" in platform_str:
updater = Windows7_64AGUpdater()
else:
updater = Windows7_32AGUpdater()
elif "XP" in platform_str:
if "64" in platform_str:
updater = WindowsXP_64AGUpdater()
else:
updater = WindowsXP_32AGUpdater()
elif "Linux" in platform_str:
if "redhat" in platform_str:
if "6.2" in platform_str:
updater = RHELLinuxAGUpdater()
elif "Darwin" in platform_str:
updater = OSXAGUpdater()
updater.generate_walkers()
updater.perform_update()
More information about the ag-dev
mailing list