[Swift-devel] Re: What is best OSG site test suite ?
Zhao Zhang
zhaozhang at uchicago.edu
Fri May 8 11:47:10 CDT 2009
Hi, Jing
I tried the test_osg_ce tool out. I made two runs there,
/home/zzhang/test_osg_ce/200958-10:39:37 is the latter one.
For this test, no site showed in ce.pass. I am trying to figure out the
error for each site, could you help me point out the
reason for error? I mean how to interpret the result file. Also, I am
attaching part of the ce.fail file of the first site.
best wishes
zhao
----------------------- AGLT2: gate01.aglt2.org -----------------------
Checking for a running gatekeeper:
<RESULTS> 1; port 2119
<STATUS> 1
Checking remote host uptime:
<RESULTS> 1
11:39:50 up 4 days, 12:51, 1 user, load average: 1.18, 1.21, 1.22
<STATUS> 1
Checking for GLOBUS_LOCATION:
<RESULTS> /opt/OSG100/globus
<STATUS> 1
Checking expiration date of remote host certificate:
<RESULTS> Mar 24 17:17:48 2010 GMT
<STATUS> 1
Checking for gatekeeper configuration file:
<RESULTS> 1
/opt/OSG100/globus/etc/globus-gatekeeper.conf
<STATUS> 1
Checking users in grid-mapfile, if none must be using Prima:
<RESULTS> usatlas1,usatlas3,usatlas4,usatlas5
<STATUS> 1
Checking for remote globus-sh-tools-vars.sh:
<RESULTS> 1
<STATUS> 1
Checking configured grid services:
<RESULTS> PASS
jobmanager,jobmanager-condor,jobmanager-fork,jobmanager-managedfork
<STATUS> 1
Checking for OSG osg-attributes.conf:
<RESULTS> 1
<STATUS> 1
Checking scheduler types associated with remote jobmanagers:
<RESULTS> 1
jobmanager is of type managedfork
jobmanager-condor is of type condor
jobmanager-fork is of type managedfork
jobmanager-managedfork is of type managedfork
<STATUS> 1
Checking for paths to binaries of remote schedulers:
<RESULTS> 1
Path to condor binaries is /opt/condor/bin
Path to managedfork binaries is .
<STATUS> 1
Checking remote scheduler status:
<RESULTS> 1
condor : 14 jobs running, 0 jobs idle/pending
<STATUS> 1
Checking if Globus is deployed from the VDT:
<RESULTS> 1; version 1.10.1j
<STATUS> 1
Checking for OSG version:
<RESULTS> 1; version 1.0.0
<STATUS> 1
Checking for OSG grid3-user-vo-map.txt:
<RESULTS> 1
usatlas users: usatlas1,usatlas2,usatlas3,usatlas4
gridex users: gridex
ops users: ops
mis users: mis
osg users: osg
<STATUS> 1
Checking for OSG site name:
<RESULTS> AGLT2
<STATUS> 1
Checking for OSG $GRID3 definition:
<RESULTS> /opt/OSG100
<STATUS> 1
Checking for OSG $OSG_GRID definition:
<RESULTS> /afs/atlas.umich.edu/OSGWN
<STATUS> 1
Checking for OSG $APP definition:
<RESULTS> /atlas/data08/OSG/APP
<STATUS> 1
Checking for OSG $DATA definition:
<RESULTS> /atlas/data08/OSG/DATA
<STATUS> 1
Checking for OSG $TMP definition:
<RESULTS> /atlas/data08/OSG/DATA
<STATUS> 1
Checking for OSG $WNTMP definition:
<RESULTS> /tmp
<STATUS> 1
Checking for OSG $APP available space:
<RESULTS> 2032.963 GB
<STATUS> 1
Checking for OSG $DATA available space:
<RESULTS> 2032.963 GB
<STATUS> 1
Checking for OSG $TMP available space:
<RESULTS> 2032.963 GB
<STATUS> 1
Checking for OSG additional site-specific variable definitions:
<RESULTS> 1
<No Location List Name>
ATLAS_APP prod /atlas/data08/OSG/APP/atlas_app
ATLAS_DATA prod /atlas/data08/OSG/DATA/atlas_data
...
...
<STATUS> 1
Checking for OSG execution jobmanager(s):
<RESULTS> gate01.aglt2.org/jobmanager-condor
<STATUS> 1
Checking for OSG utility jobmanager(s):
<RESULTS> gate01.aglt2.org/jobmanager
<STATUS> 1
Checking for OSG sponsoring VO:
<RESULTS> usatlas:80 local:20
<STATUS> 1
Checking for OSG policy expression:
<RESULTS> NONE
<STATUS> 1
Checking for OSG setup.sh:
<RESULTS> 1
<STATUS> 1
Checking for OSG $Monalisa_HOME definition:
<RESULTS> /opt/osg-ce-1.0.0-r2/MonaLisa
<STATUS> 1
Checking for MonALISA configuration:
<RESULTS> 0
Can't obtain ml_env
<STATUS> 0
Checking for a running MonALISA:
<RESULTS> 0
MonALISA does not appear to be running
<STATUS> 1
Checking for a running GANGLIA gmond daemon:
<RESULTS> 1 (pid 8095 ...)
/usr/sbin/gmond
name = "swap_free"
owner = "University of Michigan"
url = "https://hep.pa.msu.edu/twiki/bin/view/AGLT2"
<STATUS> 1
Checking for a running GANGLIA gmetad daemon:
<RESULTS> 0
gmetad does not appear to be running
<STATUS> 1
Checking for a running gsiftp server:
<RESULTS> 1; port 2811
<STATUS> 1
Jing Tie wrote:
> On Fri, May 8, 2009 at 6:21 AM, Michael Wilde <wilde at mcs.anl.gov> wrote:
>
>> Jing, Ben, thanks very much for the tips.
>>
>> Jing, was site_verify.pr the older test suite was done by Jens as part of
>> VDS?
>>
>
> I think site_verify scanner is a part of OSG MIS, and the author is
> Craig Prescott.
>
> VORS (http://scan.grid.iu.edu/) runs the script in MIS VO and shows
> the testing results. But VORS has problems since if a site doesn't
> support MIS VO, the site cannot be seen on the VORS. So GOC is
> replacing VORS with RSV.
>
>
>> Also, how would you compare test_osg_ce with osg-vo-test?
>>
>
> osg-vo-test contains one more thing than test_osg_ce: resource
> selection info such as free slot and cpu type. But it selects sites
> from dynamic VORS which has the problem described above, while
> test_osg_ce selects from a static sites list.
>
> RSV starts recently, so I am not familiar with it. But I think RSV
> contains the best list of sites in a chosen VO.
>
> Best,
> Jing
>
>
>> Mats, do you have anything for such site testing?
>>
>> Zhao, I suggest you start with test_osg_ce.
>>
>> - Mike
>>
>> On 5/8/09 12:43 AM, Jing Tie wrote:
>>
>>> Hi Mike,
>>>
>>> I think others might have better tools, but I can list three here as
>>> possible options.
>>>
>>> The first one is "osg-vo-test" developed by Chris Green. It lists
>>> probing results for all the sites in a certain VO (e.g.
>>>
>>> http://www.ci.uchicago.edu/~jtie/osg-vo-test/osg_summary/2008727-19:20:43.html).
>>> I attached his announcement about this tool in the end of the email.
>>>
>>> The second one "test_osg_ce" is used by Zhengxiong, Xi and me a lot.
>>> It's a simple package that I developed based on site_verify.pl. It
>>> runs the script (check authentication, globus-job-run, gridftp, user
>>> directory...) for each site in a user configuration file, and print
>>> out the good ones. I have attached it in the email.
>>>
>>> The last one is RSV developed by GOC. It runs similar scripts as
>>> site_verify.pl by system administrators, and reports to a centralized
>>> server periodically. You can find it on MyOSG
>>> (http://myosg.grid.iu.edu/about). I think it won't be hard to setup an
>>> independent version for user.
>>>
>>> Hope it helps,
>>> Jing
>>>
>>>
>>> ------------------------------------------------------
>>> Email from Chris Green:
>>>
>>> Hi,
>>>
>>> I am happy to announce the first general release of an extensible
>>> VO-centric site testing kit, "osg-vo-test".
>>>
>>> To explain why you might be interested in this package, I quote from the
>>> overview
>>>
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests#Overview>
>>> on the package's home TWiki page
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests>:
>>>
>>> ------------------------------------------------------------------------
>>>
>>> This package is an attempt to allow /application owners/ (by which I
>>> mean people responsible for running an application on OSG) to
>>> characterize OSG sites from the point of view of being able to run your
>>> application. Questions can be asked of each site in multiple ways, for
>>> instance:
>>>
>>> * Command line, eg:
>>>
>>> ping my-ce.my-domain
>>>
>>> * Fork job, eg:
>>>
>>> globus-job-run my-ce.my-domain /usr/bin/printenv
>>>
>>> * Batch job via CondorG.
>>>
>>> * ReSS, the *Re*source *S*election *S*ystem.
>>>
>>> * VORS, the *VO* *R*esource *S*elector.
>>>
>>> The results are presented primarily in the form of an HTML table
>>>
>>> <http://user-support.opensciencegrid.org/osg-vo-tests/site-test-examples/fermilab/20070927_004422/fermilab_summary.html>
>>> with results columns (possibly multiple columns per test), with a link
>>> to more detailed information for the test.
>>>
>>> In addition, the summary results are available in .CSV format
>>>
>>> <http://user-support.opensciencegrid.org/osg-vo-tests/site-test-examples/fermilab/20070927_004422/fermilab_summary.csv>
>>> for machine readability; a true XML format may be forthcoming if there
>>> is enough demand.
>>>
>>> The application owner can write new test modules inheriting from the
>>> old; for more details, see Making your own module
>>>
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests#Making_your_own_test_module>.
>>> In addition, existing tests are highly configurable and allow for the
>>> addition of new results columns with minimal effort; for a quick
>>> example, see the Getting Started
>>>
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests#Getting_Started>
>>> section; and also the detail
>>>
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests#More_Details>
>>> section.
>>>
>>> ------------------------------------------------------------------------
>>>
>>> It is extremely straightforward, for example, to test basic
>>> authorization at a site from the point of view of your own voms-proxy;
>>> and other tests are showcased in example control scripts; or provided as
>>> standalone test modules. The example summary page
>>>
>>> <http://user-support.opensciencegrid.org/osg-vo-tests/site-test-examples/fermilab/20070927_004422/fermilab_summary.html>
>>> shows a wide range of tests of which this extensible system is capable.
>>>
>>> Please, visit the package's home TWiki page
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests>,
>>> download
>>>
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests#Download_Source>
>>> the source; use
>>>
>>> <https://twiki.grid.iu.edu/twiki/bin/view/UserGroup/OsgVoSiteTests#Getting_Started>;
>>> and give feedback <mailto:osg-user-group?subject=osg-vo-tests>. The aim
>>> is to make it easy for application owners to put together their own
>>> suite or suites of tests to analyze sites across the OSG from the
>>> perspective of the needs of their own application(s) without having to
>>> re-invent the wheel to interrogate all the different sources of
>>> information about OSG sites.
>>>
>>>
>>> On Thu, May 7, 2009 at 2:47 PM, Michael Wilde <wilde at mcs.anl.gov> wrote:
>>>
>>>> Hi All,
>>>>
>>>> Many of you have developed, or use, various test suites to probe the
>>>> sites
>>>> of an OSG VO to see if basic authentication, job execution, and data
>>>> transfer work for a given user (cert).
>>>>
>>>> Can you let us know, on swift-devel, what test suite you use, and what
>>>> you
>>>> suggest that the Swift team use, as part of a Swift verification test
>>>> suite
>>>> that reports what sites Swift does (and does not) work on?
>>>>
>>>> In other words, once a user probes OSG to create a Swift sites file, it
>>>> would be useful to test if the basic services are working for the test
>>>> user,
>>>> before testing at a higher level through Swift.
>>>>
>>>> Thanks,
>>>>
>>>> Mike
>>>>
>>>>
>>>>
> _______________________________________________
> Swift-devel mailing list
> Swift-devel at ci.uchicago.edu
> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>
>
More information about the Swift-devel
mailing list