[AG-DEV] [AG-TECH] New accessgrid.org Venue Server
Tim Salmon
tim at unsw.edu.au
Wed May 30 07:28:16 CDT 2012
Dear Salim,
I ran the script on a Windows 7 64 but if returned only that the AccessGrid libs could not be found. on inspection I found that the path (default for the installer i used i assume) is ......\AccessGrid3 not ....\AccessGrid. rather than being a fault with your script i think that this may reflect my concern that there is a great diversity of installs out in the world that may have arisen by a variety of factors including platform, build process and usage patterns.
I also have noted the occurrence of a number of places outside the python libs and userspaces that the AccessGrid files are installed. Others will know more than me about what role these play but it is probably necessary to ensure consistency.
I have had a go at approaching this more generally but I fear that there may be little chance of getting away from a lot of testing.
At this stage I have only started on the "system" files. To generalise the process I use a class of file walkers that traverse specified subdirectories of the filesystem, identify filesnames that match what we care about (unfortunately more reliable methods of ignoring binary files is not implemented in python 2.4/2.5) and perform the substitutions by calling a chunk of your script.
you will see at the bottom a chunk of slightly over-engineered inheritence statements and the start of some logic for different platforms. you can see the included examples of some system file areas that are particular to different platforms and are outside the python library space.
I have not yet actually set the script to make the alterations: just find the targets but that should be a simple change of function call once we feel it is ready. i have also not yet incorporated operations that work in the user space: it occurred to me that it might be best to approach that in a separate script: one for sudo/admin users to carefully apply to their system, and one for users to blithely run without bringing the whole thing down in a screaming heap.
by all means send back edits, suggestions or point out how my approach is doomed to failure. in particular, suggestions of a more pythonic approach are welcome: those that know java will recognise my object-heavy paradigm. i will definitely boost the comments and look at where checks are needed too.
pasted below and attached
Tim Salmon
IT Technical Officer
Faculty of Science
UNSW
import fileinput
import os
import platform
import re
import shutil
import sys
import tempfile
from distutils.sysconfig import get_python_lib
## RewriteWalker performs a os.path.walk of base_dir
class RewriteWalker:
def __init__(self, base_dir, file_re, find_str, replace_str):
self.base_dir = base_dir
self.file_filter = re.compile(file_re)
self.find_str = find_str
self.replace_str = replace_str
def walk_files(self):
for root, directories, filenames in os.walk(self.base_dir):
for filename in filenames:
if (self.file_filter.match(filename)):
filepath = os.path.join(root, filename)
self.find_lines(filepath)
#rewrite(filepath)
def find_lines(self, srcfile):
line_count = 0;
src = open(srcfile, 'r')
for line in src:
line_count += 1
if self.find_str in line:
print srcfile
print str(line_count) + ": " + line
def rewrite(srcfile, searchPattern, replaceStr):
dstfile = srcfile + '.bak'
#Checks to see if the file has been backed up if not then create a backup of the file.
if not os.path.isfile(dstfile):
shutil.copy(srcfile,dstfile)
#creates a temporary file in the same directory that the "executable" was launched from
tmpfilename = str(os.getpid())
tfile = open(tmpfilename,'w')
src = open(srcfile,'r')
#loops through the entire file changing text
for line in src:
if searchPattern in line:
tfile.write(line.replace(self.find_str,self.replace_str))
else:
tfile.write(line)
tfile.close()
src.close()
#overwrites the srcfile and deletes the temp file
shutil.copy2(tmpfilename,srcfile)
os.unlink(tmpfilename)
class AGUpdater:
def __init__(self):
# define list of root paths and populate with those in python_lib
self.root_path_list = []
ag_python_lib_folder_list = ["AccessGrid", "AccessGrid3"]
python_lib_dir = get_python_lib()
for folder_name in ag_python_lib_folder_list:
folder_path = os.path.join(python_lib_dir, folder_name)
if os.path.exists(folder_path):
self.root_path_list.append(folder_path)
# define map of substitutions that can be applied [globally]
self.subs_map = {}
##self.subs_map['bugzilla.mcs.anl.gov'] = 'bugzilla.accessgrid.org'
self.subs_map['mcs.anl.gov'] = 'accessgrid.org'
##self.subs_map['140.221.6.95'] = 'www.accessgrid.org'
##self.subs_map['140.221.8.33'] = 'vv3.accessgrid.org'
# define list of filename_re that identify which files will be modified
# please note that this uses regular expressions not just simple expansions
self.filename_re_list = []
self.filename_re_list.append('.*\.py$')
self.filename_re_list.append('.*\.txt$')
# iterate throught root paths, filename_res and substitutions and create all combinations
def generate_walkers(self):
self.walkers=[]
for root_folder in self.root_path_list:
for filename_re in self.filename_re_list:
for subs_key, subs_val in self.subs_map.iteritems():
walker = RewriteWalker(root_folder, filename_re, subs_key, subs_val)
self.walkers.append(walker)
def perform_update(self):
for walker in self.walkers:
walker.walk_files()
class WindowsAGUpdater(AGUpdater):
def __init__(self):
AGUpdater.__init__(self)
class WindowsXPAGUpdater(WindowsAGUpdater):
pass
class WindowsXP_32AGUpdater(WindowsXPAGUpdater):
pass
class WindowsXP_64AGUpdater(WindowsXPAGUpdater):
pass
class Windows7AGUpdater(WindowsAGUpdater):
def __init__(self):
WindowsAGUpdater.__init__(self)
class Windows7_32AGUpdater(Windows7AGUpdater):
def __init__(self):
Windows7AGUpdater.__init__(self)
root_path_list.append("C:\\Program Files\\AGTk3\\")
class Windows7_64AGUpdater(Windows7AGUpdater):
def __init__(self):
Windows7AGUpdater.__init__(self)
root_path_list.append("C:\\Program Files (x86)\\AGTk3\\")
class PosixAGUpdater(AGUpdater):
def __init__(self):
AGUpdater.__init__(self)
class LinuxAGUpdater(PosixAGUpdater):
def __init__(self):
PosixAGUpdater.__init__(self)
class RedHatLinuxAGUpdater(LinuxAGUpdater):
def __init__(self):
LinuxAGUpdater.__init__(self)
class RHELLinuxAGUpdater(RedHatLinuxAGUpdater):
def __init__(self):
RedHatLinuxAGUpdater.__init__(self)
self.root_path_list.append('/usr/share/doc/AccessGrid-3.2/')
self.filename_re_list.append('^Install\.LINUX')
self.filename_re_list.append('^README')
self.filename_re_list.append('^VERSION')
class DebianLinuxAGUpdater(LinuxAGUpdater):
def __init__(self):
LinuxAGUpdater.__init__(self)
class UbuntuLinuxAGUpdater(DebianLinuxAGUpdater):
def __init__(self):
DebianLinuxAGUpdater.__init__(self)
class OSXAGUpdater(PosixAGUpdater):
def __init__(self):
PosixAGUpdater.__init__(self)
if __name__ == "__main__":
platform_str = platform.platform()
updater = AGUpdater()
if "Windows" in platform_str:
if "7" in platform_str:
if "64" in platform_str:
updater = Windows7_64AGUpdater()
elif "Linux" in platform_str:
if "redhat" in platform_str:
if "6.2" in platform_str:
updater = RHELLinuxAGUpdater()
updater.generate_walkers()
updater.perform_update()
________________________________________
From: ag-dev-bounces at lists.mcs.anl.gov [ag-dev-bounces at lists.mcs.anl.gov] on behalf of Salim Haniff [Salim.Haniff at uta.fi]
Sent: Thursday, May 24, 2012 4:18 PM
To: ag-tech at lists.mcs.anl.gov; ag-dev
Subject: Re: [AG-DEV] [AG-TECH] New accessgrid.org Venue Server
Hi,
The script for renaming urls and disabling venueclient's submit report
bug is currently located at
http://code.google.com/p/salims-scratch-box/source/browse/AGpatch/trunk/src/AGpatch.py
I will move it later this week to the sourceforge AccessGrid location.
Sorry if the script is a bit rough I'm a novice at python but feel
free to suggest improvements :)
The script was developed and tested on a Fedora box and a Scientific
linux box as root, since root is the owner of the site-packages
directory in python. I need to make a modification later today to
allow the user to execute the python app to rename the urls in their
~/.AccessGrid3/Config alone (probably takes about 5 minutes).
Windows and Mac still needs to be tested.
At this time, disabling multicast by default for Windows 7 users is
still outstanding. If anyone knows how to do that feel free to email
me.
Best,
Salim
Quoting Christoph Willing <c.willing at uq.edu.au>:
> The machine which will become vv3.accessgrid.org has now been
> populated with all the venues & layout from vv3.mcs.anl.gov (which
> will close down on 1st June).
>
> Could anyone who has a few minutes to spare please try out the new
> server and report back any problems. Feel free to give it a good
> workout. It would be great to have any issues with it sorted out by
> the time its supposed to be on line for real - 1st June is just
> about a week away.
>
> Until the DNS is arranged, access to the venue server is via:
> https://130.102.78.188:8000/Venues/default
>
>
>
> chris
>
>
>
> Christoph Willing +61 7 3365 8316
> Research Computing Centre
> University of Queensland
--
Salim Haniff (salim.haniff at uta.fi)
School of Information Sciences
33014 University of Tampere, Finland
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AGUpdate.py
Type: text/x-python
Size: 6205 bytes
Desc: AGUpdate.py
URL: <http://lists.mcs.anl.gov/pipermail/ag-dev/attachments/20120530/55bded1e/attachment.py>
More information about the ag-dev
mailing list