[Swift-devel] Q about MolDyn

Veronika Nefedova nefedova at mcs.anl.gov
Mon Aug 6 21:59:52 CDT 2007


Well, there are some discrepancies:

nefedova at viper:~/alamines> grep "Completed job" MolDyn-244-loops- 
zhgo6be8tjhi1.log | wc
    7959  244749 3241072
nefedova at viper:~/alamines> grep "Running job" MolDyn-244-loops- 
zhgo6be8tjhi1.log | wc
   17207  564648 7949388
nefedova at viper:~/alamines>

I.e. almost half of the jobs haven't finished (according to swift)

Nika

On Aug 6, 2007, at 9:51 PM, Ioan Raicu wrote:

> I have 7959 jobs completed with an exit code of 0, no failed jobs!   
> All the Falkon logs point to the same 7959 number of jobs, and when  
> they were all completed, no new jobs came in from Swift...
>
> How many jobs do you see submitted, and how many have been  
> completed in the Swift logs?
>
> Everything looks 100% normal on Falkon's end.
>
> Ioan
>
> Veronika Nefedova wrote:
>> Whats up now? Everything has stopped, no errors on swift site...
>> Do you have any errors now?
>>
>> Nika
>>
>> On Aug 6, 2007, at 6:04 PM, Ioan Raicu wrote:
>>
>>> OK, I restarted Falkon as well as there were 12K jobs trying to  
>>> go through, and keeping the entire ANL/UC site busy, although  
>>> there was no Swift on the other end to pick up the notifications...
>>>
>>> here is the new info:
>>>
>>> Falkon Factory Service: http://tg-viz-login2:50020/wsrf/services/ 
>>> GenericPortal/core/WS/GPFactoryService
>>> Web server: http://tg-viz-login2.uc.teragrid.org:51000/index.htm
>>>
>>> Note that I changed the port #, its now 50020, so don't forget to  
>>> change that before you start Swift...
>>>
>>> Ioan
>>>
>>> Veronika Nefedova wrote:
>>>> OK. I accidentally closed viper window where I started the  
>>>> workflow. The workflow was started with & so it was supposed to  
>>>> stay up even if I exited the shell. But apparently it didn't!
>>>>
>>>> This is the last entry in the log:
>>>>
>>>> 2007-08-06 17:16:59,483 INFO  ResourcePool Destroying remote  
>>>> service instance... dummy function, this doesn't really do  
>>>> anything...
>>>>
>>>> (and it doesn't change ever since).
>>>>
>>>> What went wrong ? Why closing the shell actually killed the job?  
>>>> (ps shows no swift job)
>>>> I checked 'history' and in fact the job was started with &:
>>>>
>>>>   999  swift -tc.file tc-uc.data -sites.file sites-uc-64.xml - 
>>>> debug MolDyn-244-loops.swift &
>>>>
>>>> I'll restart the workflow in 30 mins or so (from home) again.
>>>>
>>>> Sigh...
>>>>
>>>> Nika
>>>>
>>>>
>>>> On Aug 6, 2007, at 4:29 PM, Veronika Nefedova wrote:
>>>>
>>>>> Ioan, its all was due to NFS problems, I am convinced now...
>>>>>
>>>>> I restarted the run, the log is ~nefedova/alamines/MolDyn-244- 
>>>>> loops-hxl1glhtqsag0.log
>>>>>
>>>>> Nika
>>>>>
>>>>> On Aug 6, 2007, at 4:20 PM, Ioan Raicu wrote:
>>>>>
>>>>>> Just to debug further.... I picked out 1 task at random from  
>>>>>> the Swift log...
>>>>>> iraicu at viper:/home/nefedova/alamines> cat MolDyn-244-loops- 
>>>>>> dbui34oxjr4j2.log | grep "urn:0-1-62-0-1186429258791"
>>>>>> 2007-08-06 14:47:03,281 DEBUG TaskImpl Task(type=2,  
>>>>>> identity=urn:0-1-62-0-1186429258791) setting status to Submitted
>>>>>> 2007-08-06 14:47:03,281 DEBUG TaskImpl Task(type=2,  
>>>>>> identity=urn:0-1-62-0-1186429258791) setting status to Active
>>>>>> 2007-08-06 14:47:03,704 DEBUG TaskImpl Task(type=2,  
>>>>>> identity=urn:0-1-62-0-1186429258791) setting status to Failed  
>>>>>> Exception in getFile
>>>>>>
>>>>>> but in my log, it is nowhere to be found...
>>>>>> iraicu at tg-viz-login2:~/java/Falkon_v0.8.1/service/logs> cat  
>>>>>> GenericPortalWS_taskPerf.txt | grep "urn:0-1-62-0-1186429258791"
>>>>>>
>>>>>> What does "setting status to Failed Exception in getFile"  
>>>>>> mean?  Could this mean that it failed on the data staging  
>>>>>> part, and that it never made it to Falkon?
>>>>>>
>>>>>> BTW, it lloks as if there were really 539 jobs submitted...
>>>>>>
>>>>>> iraicu at viper:/home/nefedova/alamines> grep "Submitted"  
>>>>>> MolDyn-244-loops-dbui34oxjr4j2.log | wc
>>>>>>    539    5390   62835
>>>>>>
>>>>>> but again, only 57 made it to Falkon, and there were no  
>>>>>> exceptions thrown anywhere to indicate that something unusual  
>>>>>> happened.
>>>>>>
>>>>>> Ioan
>>>>>>
>>>>>> Ioan Raicu wrote:
>>>>>>> Falkon only has 57 tasks received, here they are:
>>>>>>> tg-viz-login.uc.teragrid.org:/home/iraicu/java/Falkon_v0.8.1/ 
>>>>>>> service/logs/GenericPortalWS.txt.0.summary
>>>>>>>
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh pre_ch-vsk58efi stdout.txt stderr.txt  .  ./ 
>>>>>>> m179.mol2 ./m050.mol2 m179_am1 m050_am1  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/pre-antch.pl
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-xsk58efi stdout.txt stderr.txt   m179_am1  
>>>>>>> m179_am1.rtf m179_am1.crd m179_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m179_am1 -fi mol2 - 
>>>>>>> rn m179 -o m179_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-ysk58efi stdout.txt stderr.txt   m050_am1  
>>>>>>> m050_am1.rtf m050_am1.crd m050_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m050_am1 -fi mol2 - 
>>>>>>> rn m050 -o m050_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh chrm-0tk58efi equil_solv.out_m050 stderr.txt  
>>>>>>> equil_solv.inp  parm03_gaff_all.rtf parm03_gaffnb_all.prm  
>>>>>>> equil_solv.inp m050_am1.rtf m050_am1.prm m050_am1.crd  
>>>>>>> water_400.crd equil_solv.out_m050 solv_m050.psf  
>>>>>>> solv_m050_eq.crd solv_m050.rst solv_m050.trj  
>>>>>>> solv_m050_min.crd  /disks/scratchgpfs1/iraicu/ModLyn/bin/ 
>>>>>>> charmm.sh system:solv_m050 title:solv stitle:m050  
>>>>>>> rtffile:parm03_gaff_all.rtf paramfile:parm03_gaffnb_all.prm  
>>>>>>> gaff:m050_am1 nwater:400 ligcrd:lyz rforce:0 iseed:3131887  
>>>>>>> rwater:15 nstep:10000 minstep:100 skipstep:100 startstep:10000
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh chrm-zsk58efi equil_solv.out_m179 stderr.txt  
>>>>>>> equil_solv.inp  parm03_gaff_all.rtf parm03_gaffnb_all.prm  
>>>>>>> equil_solv.inp m179_am1.rtf m179_am1.prm m179_am1.crd  
>>>>>>> water_400.crd equil_solv.out_m179 solv_m179.psf  
>>>>>>> solv_m179_eq.crd solv_m179.rst solv_m179.trj  
>>>>>>> solv_m179_min.crd  /disks/scratchgpfs1/iraicu/ModLyn/bin/ 
>>>>>>> charmm.sh system:solv_m179 title:solv stitle:m179  
>>>>>>> rtffile:parm03_gaff_all.rtf paramfile:parm03_gaffnb_all.prm  
>>>>>>> gaff:m179_am1 nwater:400 ligcrd:lyz rforce:0 iseed:3131887  
>>>>>>> rwater:15 nstep:10000 minstep:100 skipstep:100 startstep:10000
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh pre_ch-38lc8efi stdout.txt stderr.txt  .  ./ 
>>>>>>> m197.mol2 ./m129.mol2 ./m069.mol2 ./m163.mol2 ./m128.mol2 ./ 
>>>>>>> m035.mol2 ./m070.mol2 ./m221.mol2 ./m162.mol2 ./m198.mol2 ./ 
>>>>>>> m034.mol2 ./m001.mol2 ./m220.mol2 ./m033.mol2 ./m161.mol2 ./ 
>>>>>>> m032.mol2 ./m160.mol2 ./m130.mol2 ./m071.mol2 ./m002.mol2 ./ 
>>>>>>> m199.mol2 ./m175.mol2 ./m234.mol2 ./m048.mol2 ./m107.mol2 ./ 
>>>>>>> m047.mol2 ./m106.mol2 ./m124.mol2 ./m193.mol2 ./m225.mol2 ./ 
>>>>>>> m066.mol2 ./m125.mol2 ./m176.mol2 ./m194.mol2 ./m224.mol2 ./ 
>>>>>>> m235.mol2 ./m067.mol2 ./m165.mol2 ./m049.mol2 ./m126.mol2 ./ 
>>>>>>> m166.mol2 ./m108.mol2 ./m195.mol2 ./m038.mol2 ./m059.mol2 ./ 
>>>>>>> m036.mol2 ./m186.mol2 ./m164.mol2 ./m117.mol2 ./m223.mol2 ./ 
>>>>>>> m058.mol2 ./m037.mol2 ./m188.mol2 ./m068.mol2 ./m119.mol2 ./ 
>>>>>>> m187.mol2 ./m196.mol2 ./m118.mol2 ./m127.mol2 ./m222.mol2 ./ 
>>>>>>> m189.mol2 ./m060.mol2 ./m236.mol2 ./m109.mol2 ./m177.mol2 ./ 
>>>>>>> m050.mol2 ./m179.mol2 ./m178.mol2 ./m123.mol2 ./m237.mol2 ./ 
>>>>>>> m110.mol2 ./m191.mol2 ./m100.mol2 ./m064.mol2 ./m041.mol2 ./ 
>>>>>>> m238.mol2 ./m063.mol2 ./m228.mol2 ./m051.mol2 ./m122.mol2 ./ 
>>>>>>> m169.mol2 ./m121.mol2 ./m190.mol2 ./m120.mol2 ./m062.mol2 ./ 
>>>>>>> m065.mol2 ./m039.mol2 ./m192.mol2 ./m167.mol2 ./m227.mol2 ./ 
>>>>>>> m040.mol2 ./m226.mol2 ./m168.mol2 ./m239.mol2 ./m052.mol2 ./ 
>>>>>>> m111.mol2 ./m180.mol2 ./m053.mol2 ./m112.mol2 ./m181.mol2 ./ 
>>>>>>> m240.mol2 ./m054.mol2 ./m044.mol2 ./m113.mol2 ./m230.mol2 ./ 
>>>>>>> m103.mol2 ./m229.mol2 ./m061.mol2 ./m042.mol2 ./m101.mol2 ./ 
>>>>>>> m170.mol2 ./m043.mol2 ./m102.mol2 ./m171.mol2 ./m151.mol2 ./ 
>>>>>>> m083.mol2 ./m210.mol2 ./m014.mol2 ./m023.mol2 ./m200.mol2 ./ 
>>>>>>> m092.mol2 ./m091.mol2 ./m150.mol2 ./m209.mol2 ./m022.mol2 ./ 
>>>>>>> m024.mol2 ./m093.mol2 ./m015.mol2 ./m084.mol2 ./m142.mol2 ./ 
>>>>>>> m201.mol2 ./m016.mol2 ./m085.mol2 ./m143.mol2 ./m202.mol2 ./ 
>>>>>>> m010.mol2 ./m212.mol2 ./m138.mol2 ./m026.mol2 ./m011.mol2 ./ 
>>>>>>> m095.mol2 ./m139.mol2 ./m154.mol2 ./m211.mol2 ./m025.mol2 ./ 
>>>>>>> m094.mol2 ./m153.mol2 ./m213.mol2 ./m080.mol2 ./m012.mol2 ./ 
>>>>>>> m152.mol2 ./m081.mol2 ./m140.mol2 ./m013.mol2 ./m082.mol2 ./ 
>>>>>>> m141.mol2 ./m028.mol2 ./m097.mol2 ./m155.mol2 ./m008.mol2 ./ 
>>>>>>> m214.mol2 ./m135.mol2 ./m029.mol2 ./m076.mol2 ./m098.mol2 ./ 
>>>>>>> m007.mol2 ./m156.mol2 ./m134.mol2 ./m215.mol2 ./m137.mol2 ./ 
>>>>>>> m079.mol2 ./m009.mol2 ./m078.mol2 ./m077.mol2 ./m096.mol2 ./ 
>>>>>>> m136.mol2 ./m027.mol2 ./m132.mol2 ./m158.mol2 ./m073.mol2 ./ 
>>>>>>> m217.mol2 ./m030.mol2 ./m159.mol2 ./m072.mol2 ./m218.mol2 ./ 
>>>>>>> m003.mol2 ./m031.mol2 ./m004.mol2 ./m219.mol2 ./m131.mol2 ./ 
>>>>>>> m074.mol2 ./m133.mol2 ./m006.mol2 ./m075.mol2 ./m157.mol2 ./ 
>>>>>>> m099.mol2 ./m005.mol2 ./m216.mol2 ./m090.mol2 ./m021.mol2 ./ 
>>>>>>> m208.mol2 ./m149.mol2 ./m020.mol2 ./m207.mol2 ./m148.mol2 ./ 
>>>>>>> m088.mol2 ./m089.mol2 ./m206.mol2 ./m147.mol2 ./m019.mol2 ./ 
>>>>>>> m205.mol2 ./m146.mol2 ./m087.mol2 ./m018.mol2 ./m204.mol2 ./ 
>>>>>>> m145.mol2 ./m086.mol2 ./m017.mol2 ./m144.mol2 ./m203.mol2 ./ 
>>>>>>> m057.mol2 ./m116.mol2 ./m232.mol2 ./m173.mol2 ./m105.mol2 ./ 
>>>>>>> m046.mol2 ./m231.mol2 ./m172.mol2 ./m104.mol2 ./m045.mol2 ./ 
>>>>>>> m174.mol2 ./m233.mol2 ./m244.mol2 ./m185.mol2 ./m182.mol2 ./ 
>>>>>>> m243.mol2 ./m055.mol2 ./m241.mol2 ./m183.mol2 ./m114.mol2 ./ 
>>>>>>> m056.mol2 ./m242.mol2 ./m184.mol2 ./m115.mol2 m197_am1  
>>>>>>> m129_am1 m069_am1 m163_am1 m128_am1 m035_am1 m070_am1  
>>>>>>> m221_am1 m162_am1 m198_am1 m034_am1 m001_am1 m220_am1  
>>>>>>> m033_am1 m161_am1 m032_am1 m160_am1 m130_am1 m071_am1  
>>>>>>> m002_am1 m199_am1 m175_am1 m234_am1 m048_am1 m107_am1  
>>>>>>> m047_am1 m106_am1 m124_am1 m193_am1 m225_am1 m066_am1  
>>>>>>> m125_am1 m176_am1 m194_am1 m224_am1 m235_am1 m067_am1  
>>>>>>> m165_am1 m049_am1 m126_am1 m166_am1 m108_am1 m195_am1  
>>>>>>> m038_am1 m059_am1 m036_am1 m186_am1 m164_am1 m223_am1  
>>>>>>> m117_am1 m037_am1 m058_am1 m068_am1 m188_am1 m119_am1  
>>>>>>> m196_am1 m187_am1 m222_am1 m127_am1 m118_am1 m189_am1  
>>>>>>> m060_am1 m236_am1 m109_am1 m177_am1 m050_am1 m179_am1  
>>>>>>> m123_am1 m178_am1 m237_am1 m100_am1 m191_am1 m110_am1  
>>>>>>> m041_am1 m064_am1 m228_am1 m063_am1 m238_am1 m169_am1  
>>>>>>> m122_am1 m051_am1 m121_am1 m190_am1 m120_am1 m062_am1  
>>>>>>> m039_am1 m065_am1 m167_am1 m192_am1 m227_am1 m040_am1  
>>>>>>> m226_am1 m168_am1 m239_am1 m052_am1 m111_am1 m180_am1  
>>>>>>> m053_am1 m112_am1 m181_am1 m240_am1 m054_am1 m044_am1  
>>>>>>> m113_am1 m230_am1 m103_am1 m229_am1 m061_am1 m042_am1  
>>>>>>> m101_am1 m170_am1 m043_am1 m102_am1 m171_am1 m151_am1  
>>>>>>> m083_am1 m210_am1 m014_am1 m023_am1 m200_am1 m092_am1  
>>>>>>> m091_am1 m150_am1 m209_am1 m022_am1 m024_am1 m093_am1  
>>>>>>> m015_am1 m084_am1 m142_am1 m201_am1 m016_am1 m085_am1  
>>>>>>> m143_am1 m202_am1 m010_am1 m212_am1 m138_am1 m026_am1  
>>>>>>> m011_am1 m095_am1 m139_am1 m154_am1 m211_am1 m025_am1  
>>>>>>> m094_am1 m153_am1 m213_am1 m080_am1 m012_am1 m152_am1  
>>>>>>> m081_am1 m140_am1 m013_am1 m082_am1 m141_am1 m028_am1  
>>>>>>> m097_am1 m155_am1 m008_am1 m214_am1 m135_am1 m029_am1  
>>>>>>> m076_am1 m098_am1 m007_am1 m156_am1 m134_am1 m215_am1  
>>>>>>> m137_am1 m079_am1 m009_am1 m078_am1 m077_am1 m096_am1  
>>>>>>> m136_am1 m027_am1 m132_am1 m158_am1 m073_am1 m217_am1  
>>>>>>> m030_am1 m159_am1 m072_am1 m218_am1 m003_am1 m031_am1  
>>>>>>> m004_am1 m219_am1 m131_am1 m074_am1 m133_am1 m006_am1  
>>>>>>> m075_am1 m157_am1 m099_am1 m216_am1 m005_am1 m090_am1  
>>>>>>> m021_am1 m208_am1 m149_am1 m020_am1 m207_am1 m148_am1  
>>>>>>> m089_am1 m088_am1 m206_am1 m147_am1 m019_am1 m205_am1  
>>>>>>> m146_am1 m087_am1 m018_am1 m204_am1 m145_am1 m086_am1  
>>>>>>> m017_am1 m144_am1 m203_am1 m057_am1 m116_am1 m232_am1  
>>>>>>> m173_am1 m105_am1 m046_am1 m231_am1 m172_am1 m104_am1  
>>>>>>> m045_am1 m174_am1 m233_am1 m244_am1 m185_am1 m182_am1  
>>>>>>> m243_am1 m055_am1 m241_am1 m183_am1 m114_am1 m056_am1  
>>>>>>> m242_am1 m184_am1 m115_am1  /disks/scratchgpfs1/iraicu/ModLyn/ 
>>>>>>> bin/pre-antch.pl
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-58lc8efi stdout.txt stderr.txt   m197_am1  
>>>>>>> m197_am1.rtf m197_am1.crd m197_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m197_am1 -fi mol2 - 
>>>>>>> rn m197 -o m197_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-48lc8efi stdout.txt stderr.txt   m129_am1  
>>>>>>> m129_am1.rtf m129_am1.crd m129_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m129_am1 -fi mol2 - 
>>>>>>> rn m129 -o m129_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-68lc8efi stdout.txt stderr.txt   m069_am1  
>>>>>>> m069_am1.rtf m069_am1.crd m069_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m069_am1 -fi mol2 - 
>>>>>>> rn m069 -o m069_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-88lc8efi stdout.txt stderr.txt   m163_am1  
>>>>>>> m163_am1.rtf m163_am1.crd m163_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m163_am1 -fi mol2 - 
>>>>>>> rn m163 -o m163_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-78lc8efi stdout.txt stderr.txt   m128_am1  
>>>>>>> m128_am1.rtf m128_am1.crd m128_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m128_am1 -fi mol2 - 
>>>>>>> rn m128 -o m128_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-98lc8efi stdout.txt stderr.txt   m035_am1  
>>>>>>> m035_am1.rtf m035_am1.crd m035_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m035_am1 -fi mol2 - 
>>>>>>> rn m035 -o m035_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-a8lc8efi stdout.txt stderr.txt   m070_am1  
>>>>>>> m070_am1.rtf m070_am1.crd m070_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m070_am1 -fi mol2 - 
>>>>>>> rn m070 -o m070_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-b8lc8efi stdout.txt stderr.txt   m221_am1  
>>>>>>> m221_am1.rtf m221_am1.crd m221_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m221_am1 -fi mol2 - 
>>>>>>> rn m221 -o m221_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-c8lc8efi stdout.txt stderr.txt   m162_am1  
>>>>>>> m162_am1.rtf m162_am1.crd m162_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m162_am1 -fi mol2 - 
>>>>>>> rn m162 -o m162_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-d8lc8efi stdout.txt stderr.txt   m198_am1  
>>>>>>> m198_am1.rtf m198_am1.crd m198_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m198_am1 -fi mol2 - 
>>>>>>> rn m198 -o m198_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-e8lc8efi stdout.txt stderr.txt   m034_am1  
>>>>>>> m034_am1.rtf m034_am1.crd m034_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m034_am1 -fi mol2 - 
>>>>>>> rn m034 -o m034_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-f8lc8efi stdout.txt stderr.txt   m001_am1  
>>>>>>> m001_am1.rtf m001_am1.crd m001_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m001_am1 -fi mol2 - 
>>>>>>> rn m001 -o m001_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-h8lc8efi stdout.txt stderr.txt   m033_am1  
>>>>>>> m033_am1.rtf m033_am1.crd m033_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m033_am1 -fi mol2 - 
>>>>>>> rn m033 -o m033_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-g8lc8efi stdout.txt stderr.txt   m220_am1  
>>>>>>> m220_am1.rtf m220_am1.crd m220_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m220_am1 -fi mol2 - 
>>>>>>> rn m220 -o m220_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-i8lc8efi stdout.txt stderr.txt   m161_am1  
>>>>>>> m161_am1.rtf m161_am1.crd m161_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m161_am1 -fi mol2 - 
>>>>>>> rn m161 -o m161_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-j8lc8efi stdout.txt stderr.txt   m032_am1  
>>>>>>> m032_am1.rtf m032_am1.crd m032_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m032_am1 -fi mol2 - 
>>>>>>> rn m032 -o m032_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-k8lc8efi stdout.txt stderr.txt   m160_am1  
>>>>>>> m160_am1.rtf m160_am1.crd m160_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m160_am1 -fi mol2 - 
>>>>>>> rn m160 -o m160_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-l8lc8efi stdout.txt stderr.txt   m130_am1  
>>>>>>> m130_am1.rtf m130_am1.crd m130_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m130_am1 -fi mol2 - 
>>>>>>> rn m130 -o m130_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-m8lc8efi stdout.txt stderr.txt   m071_am1  
>>>>>>> m071_am1.rtf m071_am1.crd m071_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m071_am1 -fi mol2 - 
>>>>>>> rn m071 -o m071_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-o8lc8efi stdout.txt stderr.txt   m199_am1  
>>>>>>> m199_am1.rtf m199_am1.crd m199_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m199_am1 -fi mol2 - 
>>>>>>> rn m199 -o m199_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-n8lc8efi stdout.txt stderr.txt   m002_am1  
>>>>>>> m002_am1.rtf m002_am1.crd m002_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m002_am1 -fi mol2 - 
>>>>>>> rn m002 -o m002_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-p8lc8efi stdout.txt stderr.txt   m175_am1  
>>>>>>> m175_am1.rtf m175_am1.crd m175_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m175_am1 -fi mol2 - 
>>>>>>> rn m175 -o m175_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-q8lc8efi stdout.txt stderr.txt   m234_am1  
>>>>>>> m234_am1.rtf m234_am1.crd m234_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m234_am1 -fi mol2 - 
>>>>>>> rn m234 -o m234_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-s8lc8efi stdout.txt stderr.txt   m107_am1  
>>>>>>> m107_am1.rtf m107_am1.crd m107_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m107_am1 -fi mol2 - 
>>>>>>> rn m107 -o m107_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-r8lc8efi stdout.txt stderr.txt   m048_am1  
>>>>>>> m048_am1.rtf m048_am1.crd m048_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m048_am1 -fi mol2 - 
>>>>>>> rn m048 -o m048_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-v8lc8efi stdout.txt stderr.txt   m124_am1  
>>>>>>> m124_am1.rtf m124_am1.crd m124_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m124_am1 -fi mol2 - 
>>>>>>> rn m124 -o m124_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-t8lc8efi stdout.txt stderr.txt   m047_am1  
>>>>>>> m047_am1.rtf m047_am1.crd m047_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m047_am1 -fi mol2 - 
>>>>>>> rn m047 -o m047_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-u8lc8efi stdout.txt stderr.txt   m106_am1  
>>>>>>> m106_am1.rtf m106_am1.crd m106_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m106_am1 -fi mol2 - 
>>>>>>> rn m106 -o m106_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-x8lc8efi stdout.txt stderr.txt   m193_am1  
>>>>>>> m193_am1.rtf m193_am1.crd m193_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m193_am1 -fi mol2 - 
>>>>>>> rn m193 -o m193_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-y8lc8efi stdout.txt stderr.txt   m225_am1  
>>>>>>> m225_am1.rtf m225_am1.crd m225_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m225_am1 -fi mol2 - 
>>>>>>> rn m225 -o m225_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-z8lc8efi stdout.txt stderr.txt   m066_am1  
>>>>>>> m066_am1.rtf m066_am1.crd m066_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m066_am1 -fi mol2 - 
>>>>>>> rn m066 -o m066_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-09lc8efi stdout.txt stderr.txt   m125_am1  
>>>>>>> m125_am1.rtf m125_am1.crd m125_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m125_am1 -fi mol2 - 
>>>>>>> rn m125 -o m125_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-29lc8efi stdout.txt stderr.txt   m194_am1  
>>>>>>> m194_am1.rtf m194_am1.crd m194_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m194_am1 -fi mol2 - 
>>>>>>> rn m194 -o m194_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-19lc8efi stdout.txt stderr.txt   m176_am1  
>>>>>>> m176_am1.rtf m176_am1.crd m176_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m176_am1 -fi mol2 - 
>>>>>>> rn m176 -o m176_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-39lc8efi stdout.txt stderr.txt   m224_am1  
>>>>>>> m224_am1.rtf m224_am1.crd m224_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m224_am1 -fi mol2 - 
>>>>>>> rn m224 -o m224_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-49lc8efi stdout.txt stderr.txt   m235_am1  
>>>>>>> m235_am1.rtf m235_am1.crd m235_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m235_am1 -fi mol2 - 
>>>>>>> rn m235 -o m235_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-69lc8efi stdout.txt stderr.txt   m165_am1  
>>>>>>> m165_am1.rtf m165_am1.crd m165_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m165_am1 -fi mol2 - 
>>>>>>> rn m165 -o m165_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-59lc8efi stdout.txt stderr.txt   m067_am1  
>>>>>>> m067_am1.rtf m067_am1.crd m067_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m067_am1 -fi mol2 - 
>>>>>>> rn m067 -o m067_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-79lc8efi stdout.txt stderr.txt   m049_am1  
>>>>>>> m049_am1.rtf m049_am1.crd m049_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m049_am1 -fi mol2 - 
>>>>>>> rn m049 -o m049_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-89lc8efi stdout.txt stderr.txt   m126_am1  
>>>>>>> m126_am1.rtf m126_am1.crd m126_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m126_am1 -fi mol2 - 
>>>>>>> rn m126 -o m126_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-99lc8efi stdout.txt stderr.txt   m166_am1  
>>>>>>> m166_am1.rtf m166_am1.crd m166_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m166_am1 -fi mol2 - 
>>>>>>> rn m166 -o m166_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-a9lc8efi stdout.txt stderr.txt   m108_am1  
>>>>>>> m108_am1.rtf m108_am1.crd m108_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m108_am1 -fi mol2 - 
>>>>>>> rn m108 -o m108_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-b9lc8efi stdout.txt stderr.txt   m195_am1  
>>>>>>> m195_am1.rtf m195_am1.crd m195_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m195_am1 -fi mol2 - 
>>>>>>> rn m195 -o m195_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-d9lc8efi stdout.txt stderr.txt   m038_am1  
>>>>>>> m038_am1.rtf m038_am1.crd m038_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m038_am1 -fi mol2 - 
>>>>>>> rn m038 -o m038_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-c9lc8efi stdout.txt stderr.txt   m059_am1  
>>>>>>> m059_am1.rtf m059_am1.crd m059_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m059_am1 -fi mol2 - 
>>>>>>> rn m059 -o m059_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-e9lc8efi stdout.txt stderr.txt   m186_am1  
>>>>>>> m186_am1.rtf m186_am1.crd m186_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m186_am1 -fi mol2 - 
>>>>>>> rn m186 -o m186_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-f9lc8efi stdout.txt stderr.txt   m164_am1  
>>>>>>> m164_am1.rtf m164_am1.crd m164_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m164_am1 -fi mol2 - 
>>>>>>> rn m164 -o m164_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-h9lc8efi stdout.txt stderr.txt   m036_am1  
>>>>>>> m036_am1.rtf m036_am1.crd m036_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m036_am1 -fi mol2 - 
>>>>>>> rn m036 -o m036_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-g9lc8efi stdout.txt stderr.txt   m223_am1  
>>>>>>> m223_am1.rtf m223_am1.crd m223_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m223_am1 -fi mol2 - 
>>>>>>> rn m223 -o m223_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-j9lc8efi stdout.txt stderr.txt   m058_am1  
>>>>>>> m058_am1.rtf m058_am1.crd m058_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m058_am1 -fi mol2 - 
>>>>>>> rn m058 -o m058_am1 -fo charmm -c bcc
>>>>>>> 128.135.160.234 : EXECUTABLE /bin/sh ARGUEMENTS shared/ 
>>>>>>> wrapper.sh antch-k9lc8efi stdout.txt stderr.txt   m037_am1  
>>>>>>> m037_am1.rtf m037_am1.crd m037_am1.prm  /disks/scratchgpfs1/ 
>>>>>>> iraicu/ModLyn/bin/antechamber.sh -s 2 -i m037_am1 -fi mol2 - 
>>>>>>> rn m037 -o m037_am1 -fo charmm -c bcc
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Veronika Nefedova wrote:
>>>>>>>> Swift thinks that it sent 248 jobs.
>>>>>>>>
>>>>>>>> nefedova at viper:~/alamines> grep "Running job " MolDyn-244- 
>>>>>>>> loops-dbui34oxjr4j2.log | wc
>>>>>>>>     248    6931   56718
>>>>>>>> nefedova at viper:~/alamines>
>>>>>>>>
>>>>>>>> On Aug 6, 2007, at 3:27 PM, Ioan Raicu wrote:
>>>>>>>>
>>>>>>>>> Everything is idle, there is no work to be done...
>>>>>>>>>
>>>>>>>>> iraicu at tg-viz-login2:~/java/Falkon_v0.8.1/service/logs>  
>>>>>>>>> tail GenericPortalWS_perf_per_sec.txt
>>>>>>>>> 3510.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3511.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3512.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3513.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3514.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3515.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3516.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3517.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3518.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>> 3519.997 0 2 41 24 24 0 0 0.0 0.0 0.0 0.0 57.0 0.0
>>>>>>>>>
>>>>>>>>> 24 workers are registered but idle.... queue length 0, 57  
>>>>>>>>> jobs completed.
>>>>>>>>>
>>>>>>>>> Also, see below all 57 jobs, they all finished with an exit  
>>>>>>>>> code of 0, in other words succesfully!  How many jobs does  
>>>>>>>>> Swift think it sent?
>>>>>>>>>
>>>>>>>>> Ioan
>>>>>>>>>
>>>>>>>>> iraicu at tg-viz-login2:~/java/Falkon_v0.8.1/service/logs> cat  
>>>>>>>>> GenericPortalWS_taskPerf.txt
>>>>>>>>> //taskNum taskID workerID startTimeStamp execTimeStamp  
>>>>>>>>> resultsQueueTimeStamp endTimeStamp waitQueueTime ex
>>>>>>>>> ecTime resultsQueueTime totalTime exitCode
>>>>>>>>> 1 urn:0-0-1186428880921 192.5.198.70:50100 510496 560276  
>>>>>>>>> 560614 560629 49780 338 15 50133 0
>>>>>>>>> 2 urn:0-1-1-0-1186428880939 192.5.198.70:50101 560984  
>>>>>>>>> 561200 561899 561909 216 699 10 925 0
>>>>>>>>> 3 urn:0-1-2-0-1186428880941 192.5.198.70:50100 560991  
>>>>>>>>> 561373 562150 562159 382 777 9 1168 0
>>>>>>>>> 4 urn:0-0-1186429254652 192.5.198.71:50100 972312 1034716  
>>>>>>>>> 1044916 1044926 62404 10200 10 72614 0
>>>>>>>>> 5 urn:0-1-2-0-1186429255467 192.5.198.71:50101 1046318  
>>>>>>>>> 1046453 1047038 1047067 135 585 29 749 0
>>>>>>>>> 6 urn:0-1-1-0-1186429255461 192.5.198.71:50100 1046315  
>>>>>>>>> 1046429 1053072 1053080 114 6643 8 6765 0
>>>>>>>>> 7 urn:0-1-3-0-1186429255469 192.5.198.71:50101 1046320  
>>>>>>>>> 1047051 1054256 1054290 731 7205 34 7970 0
>>>>>>>>> 8 urn:0-1-5-0-1186429255481 192.5.198.71:50101 1046324  
>>>>>>>>> 1054267 1054570 1054579 7943 303 9 8255 0
>>>>>>>>> 9 urn:0-1-4-0-1186429255479 192.5.198.71:50100 1046322  
>>>>>>>>> 1053087 1056811 1056819 6765 3724 8 10497 0
>>>>>>>>> 10 urn:0-1-6-0-1186429255484 192.5.198.71:50101 1046326  
>>>>>>>>> 1054583 1058691 1058719 8257 4108 28 12393 0
>>>>>>>>> 11 urn:0-1-8-0-1186429255495 192.5.198.71:50101 1046331  
>>>>>>>>> 1058704 1059363 1059385 12373 659 22 13054 0
>>>>>>>>> 12 urn:0-1-7-0-1186429255486 192.5.198.71:50100 1046329  
>>>>>>>>> 1056826 1060315 1060323 10497 3489 8 13994 0
>>>>>>>>> 13 urn:0-1-9-0-1186429255502 192.5.198.71:50101 1046333  
>>>>>>>>> 1059375 1060589 1060596 13042 1214 7 14263 0
>>>>>>>>> 14 urn:0-1-11-0-1186429255514 192.5.198.71:50101 1046338  
>>>>>>>>> 1060603 1060954 1061054 14265 351 100 14716 0
>>>>>>>>> 15 urn:0-1-10-0-1186429255511 192.5.198.71:50100 1046336  
>>>>>>>>> 1060329 1061094 1061126 13993 765 32 14790 0
>>>>>>>>> 16 urn:0-1-14-0-1186429255533 192.5.198.71:50100 1046691  
>>>>>>>>> 1061105 1065608 1065617 14414 4503 9 18926 0
>>>>>>>>> 17 urn:0-1-13-0-1186429255535 192.5.198.71:50100 1046693  
>>>>>>>>> 1065622 1066307 1066315 18929 685 8 19622 0
>>>>>>>>> 18 urn:0-1-12-0-1186429255524 192.5.198.71:50101 1046689  
>>>>>>>>> 1061045 1067540 1067563 14356 6495 23 20874 0
>>>>>>>>> 19 urn:0-1-15-0-1186429255539 192.5.198.71:50100 1046695  
>>>>>>>>> 1066320 1069262 1069271 19625 2942 9 22576 0
>>>>>>>>> 20 urn:0-1-16-0-1186429255543 192.5.198.71:50101 1046697  
>>>>>>>>> 1067551 1071003 1071011 20854 3452 8 24314 0
>>>>>>>>> 21 urn:0-1-18-0-1186429255559 192.5.198.71:50101 1046700  
>>>>>>>>> 1071016 1071664 1071671 24316 648 7 24971 0
>>>>>>>>> 22 urn:0-1-17-0-1186429255557 192.5.198.71:50100 1046698  
>>>>>>>>> 1069275 1071679 1071692 22577 2404 13 24994 0
>>>>>>>>> 23 urn:0-1-19-0-1186429255565 192.5.198.71:50101 1046702  
>>>>>>>>> 1071687 1073978 1073988 24985 2291 10 27286 0
>>>>>>>>> 24 urn:0-1-20-0-1186429255572 192.5.198.71:50101 1046706  
>>>>>>>>> 1073992 1075959 1075969 27286 1967 10 29263 0
>>>>>>>>> 25 urn:0-1-21-0-1186429255567 192.5.198.71:50100 1046704  
>>>>>>>>> 1071699 1076704 1076713 24995 5005 9 30009 0
>>>>>>>>> 26 urn:0-1-22-0-1186429255587 192.5.198.71:50101 1046708  
>>>>>>>>> 1075972 1077451 1077459 29264 1479 8 30751 0
>>>>>>>>> 27 urn:0-1-23-0-1186429255595 192.5.198.71:50100 1046710  
>>>>>>>>> 1076717 1080157 1080165 30007 3440 8 33455 0
>>>>>>>>> 28 urn:0-1-25-0-1186429255599 192.5.198.71:50101 1046712  
>>>>>>>>> 1077464 1080270 1080286 30752 2806 16 33574 0
>>>>>>>>> 29 urn:0-1-24-0-1186429255601 192.5.198.71:50100 1046713  
>>>>>>>>> 1080170 1080611 1080619 33457 441 8 33906 0
>>>>>>>>> 30 urn:0-1-26-0-1186429255613 192.5.198.71:50100 1046717  
>>>>>>>>> 1080624 1080973 1080983 33907 349 10 34266 0
>>>>>>>>> 31 urn:0-1-28-0-1186429255611 192.5.198.71:50101 1046715  
>>>>>>>>> 1080281 1081405 1081413 33566 1124 8 34698 0
>>>>>>>>> 32 urn:0-1-27-0-1186429255616 192.5.198.71:50100 1046719  
>>>>>>>>> 1080986 1082989 1082996 34267 2003 7 36277 0
>>>>>>>>> 33 urn:0-1-30-0-1186429255635 192.5.198.71:50100 1046723  
>>>>>>>>> 1083002 1083370 1083378 36279 368 8 36655 0
>>>>>>>>> 34 urn:0-1-29-0-1186429255622 192.5.198.71:50101 1046721  
>>>>>>>>> 1081417 1084830 1084837 34696 3413 7 38116 0
>>>>>>>>> 35 urn:0-1-32-0-1186429255652 192.5.198.71:50101 1047082  
>>>>>>>>> 1084843 1085854 1085879 37761 1011 25 38797 0
>>>>>>>>> 36 urn:0-1-34-0-1186429255654 192.5.198.71:50101 1047085  
>>>>>>>>> 1085865 1089502 1089511 38780 3637 9 42426 0
>>>>>>>>> 37 urn:0-1-33-0-1186429255656 192.5.198.71:50101 1047087  
>>>>>>>>> 1089515 1089966 1089974 42428 451 8 42887 0
>>>>>>>>> 38 urn:0-1-31-0-1186429255642 192.5.198.71:50100 1046725  
>>>>>>>>> 1083383 1091316 1091324 36658 7933 8 44599 0
>>>>>>>>> 39 urn:0-1-36-0-1186429255664 192.5.198.71:50100 1047092  
>>>>>>>>> 1091329 1092042 1092049 44237 713 7 44957 0
>>>>>>>>> 40 urn:0-1-38-0-1186429255673 192.5.198.71:50100 1047095  
>>>>>>>>> 1092055 1094242 1094249 44960 2187 7 47154 0
>>>>>>>>> 41 urn:0-1-35-0-1186429255658 192.5.198.71:50101 1047090  
>>>>>>>>> 1089979 1094418 1094428 42889 4439 10 47338 0
>>>>>>>>> 42 urn:0-1-40-0-1186429255696 192.5.198.71:50101 1047102  
>>>>>>>>> 1094433 1095082 1095089 47331 649 7 47987 0
>>>>>>>>> 43 urn:0-1-41-0-1186429255692 192.5.198.71:50101 1047104  
>>>>>>>>> 1095095 1096846 1096853 47991 1751 7 49749 0
>>>>>>>>> 44 urn:0-1-39-0-1186429255686 192.5.198.71:50100 1047100  
>>>>>>>>> 1094256 1098214 1098221 47156 3958 7 51121 0
>>>>>>>>> 45 urn:0-1-42-0-1186429255700 192.5.198.71:50101 1047107  
>>>>>>>>> 1096859 1098627 1098637 49752 1768 10 51530 0
>>>>>>>>> 46 urn:0-1-37-0-1186429255681 192.5.198.67:50100 1047097  
>>>>>>>>> 1094037 1098903 1098910 46940 4866 7 51813 0
>>>>>>>>> 47 urn:0-1-50-0-1186429255749 192.5.198.67:50101 1047121  
>>>>>>>>> 1099192 1100210 1100246 52071 1018 36 53125 0
>>>>>>>>> 48 urn:0-1-44-0-1186429255720 192.5.198.57:50101 1047111  
>>>>>>>>> 1097371 1100555 1100562 50260 3184 7 53451 0
>>>>>>>>> 49 urn:0-1-43-0-1186429255705 192.5.198.66:50100 1047109  
>>>>>>>>> 1097135 1100896 1100904 50026 3761 8 53795 0
>>>>>>>>> 50 urn:0-1-48-0-1186429255737 192.5.198.71:50101 1047117  
>>>>>>>>> 1098640 1101106 1101127 51523 2466 21 54010 0
>>>>>>>>> 51 urn:0-1-51-0-1186429255755 192.5.198.55:50100 1047123  
>>>>>>>>> 1099965 1101217 1101224 52842 1252 7 54101 0
>>>>>>>>> 52 urn:0-1-47-0-1186429255731 192.5.198.71:50100 1047115  
>>>>>>>>> 1098227 1101820 1101828 51112 3593 8 54713 0
>>>>>>>>> 53 urn:0-1-45-0-1186429255723 192.5.198.57:50100 1047113  
>>>>>>>>> 1097375 1104132 1104139 50262 6757 7 57026 0
>>>>>>>>> 54 urn:0-1-52-0-1186429255764 192.5.198.67:50101 1047125  
>>>>>>>>> 1100221 1106449 1106458 53096 6228 9 59333 0
>>>>>>>>> 55 urn:0-1-46-0-1186429255743 192.5.198.67:50100 1047119  
>>>>>>>>> 1098916 1106473 1106481 51797 7557 8 59362 0
>>>>>>>>> 56 urn:0-1-2-1-1186428881026 192.5.198.70:50101 563313  
>>>>>>>>> 563384 1207793 1207801 71 644409 8 644488 0
>>>>>>>>> 57 urn:0-1-1-1-1186428881028 192.5.198.70:50100 563315  
>>>>>>>>> 563413 1216404 1216425 98 652991 21 653110 0
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Veronika Nefedova wrote:
>>>>>>>>>> OK. There is something weird happening. I've got several  
>>>>>>>>>> such entries in my swift log:
>>>>>>>>>>
>>>>>>>>>> 2007-08-06 14:46:58,565 DEBUG vdl:execute2 Application  
>>>>>>>>>> exception: Task failed
>>>>>>>>>>         task:execute @ vdl-int.k, line: 332
>>>>>>>>>>         vdl:execute2 @ execute-default.k, line: 22
>>>>>>>>>>         vdl:execute @ MolDyn-244-loops.kml, line: 20
>>>>>>>>>>         antchmbr @ MolDyn-244-loops.kml, line: 2845
>>>>>>>>>>         vdl:mains @ MolDyn-244-loops.kml, line: 2267
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Looks like antechamber has failed (?). And the failure is  
>>>>>>>>>> only on a swfit side, it never made it across to Falcon  
>>>>>>>>>> (there are no remote directories created). But I see some  
>>>>>>>>>> of antechamber jobs have finished (in shared).
>>>>>>>>>>
>>>>>>>>>> Yuqing -- could the changes you've made be responsible for  
>>>>>>>>>> these failures (I do not see how it could though) ?
>>>>>>>>>>
>>>>>>>>>> Ioan, what do you see in your logs ion these tasks:
>>>>>>>>>>
>>>>>>>>>> 2007-08-06 14:46:58,555 DEBUG TaskImpl Task(type=1,  
>>>>>>>>>> identity=urn:0-1-56-0-1186429255786) setting status to Failed
>>>>>>>>>> 2007-08-06 14:46:58,556 DEBUG TaskImpl Task(type=1,  
>>>>>>>>>> identity=urn:0-1-57-0-1186429255798) setting status to Failed
>>>>>>>>>> 2007-08-06 14:46:58,558 DEBUG TaskImpl Task(type=1,  
>>>>>>>>>> identity=urn:0-1-59-0-1186429255800) setting status to Failed
>>>>>>>>>> 2007-08-06 14:46:58,558 DEBUG TaskImpl Task(type=1,  
>>>>>>>>>> identity=urn:0-1-60-0-1186429255805) setting status to Failed
>>>>>>>>>> 2007-08-06 14:46:58,558 DEBUG TaskImpl Task(type=1,  
>>>>>>>>>> identity=urn:0-1-61-0-1186429255811) setting status to Failed
>>>>>>>>>> 2007-08-06 14:46:58,558 DEBUG TaskImpl Task(type=1,  
>>>>>>>>>> identity=urn:0-1-58-0-1186429255814) setting status to Failed
>>>>>>>>>>
>>>>>>>>>> Nika
>>>>>>>>>>
>>>>>>>>>> On Aug 6, 2007, at 2:29 PM, Ioan Raicu wrote:
>>>>>>>>>>
>>>>>>>>>>> OK!
>>>>>>>>>>> Why don't we do one last run from my allocation, as  
>>>>>>>>>>> everything is set up already and ready to go!  Make sure  
>>>>>>>>>>> to enable all debug logging.  Falkon is up and running  
>>>>>>>>>>> with all debug enabled!
>>>>>>>>>>>
>>>>>>>>>>> Falkon location is unchanged from the last experiment.
>>>>>>>>>>> Falkon Factory Service: http://tg-viz-login2:50010/wsrf/ 
>>>>>>>>>>> services/GenericPortal/core/WS/GPFactoryService
>>>>>>>>>>> Web Server (graphs): http://tg-viz-login2.uc.teragrid.org: 
>>>>>>>>>>> 51000/index.htm
>>>>>>>>>>>
>>>>>>>>>>> ANL/UC is not quite so idle as it was earlier, but I bet  
>>>>>>>>>>> we could still get 150~200 processors!
>>>>>>>>>>>
>>>>>>>>>>> Ioan
>>>>>>>>>>>
>>>>>>>>>>> Veronika Nefedova wrote:
>>>>>>>>>>>> m050 and m179 finished just fine now via GRAM (thanks to  
>>>>>>>>>>>> Yuqing who fixed the m179 just in time!). We could start  
>>>>>>>>>>>> again the 244- molecule run to verify that nothing is  
>>>>>>>>>>>> wrong with the whole system.
>>>>>>>>>>>>
>>>>>>>>>>>> Nika
>>>>>>>>>>>>
>>>>>>>>>>>> On Aug 6, 2007, at 12:20 PM, Veronika Nefedova wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Aug 6, 2007, at 11:51 AM, Ioan Raicu wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I started those 2 molecules via GRAM. I have no trust  
>>>>>>>>>>>>> in m179 finishing completely since I didn't change  
>>>>>>>>>>>>> anything. I hope for m050 to finish though...
>>>>>>>>>>>>> You can watch the swift log on viper in ~nefedova/ 
>>>>>>>>>>>>> alamines/MolDyn-2-loops-be9484k93kk21.log
>>>>>>>>>>>>>
>>>>>>>>>>>>> Nika
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Then, let's try another run with 244 molecules soon,  
>>>>>>>>>>>>>> as most of ANL/UC is free!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ioan
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Swift-devel mailing list
>>>>>>> Swift-devel at ci.uchicago.edu
>>>>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Swift-devel mailing list
>>>>> Swift-devel at ci.uchicago.edu
>>>>> http://mail.ci.uchicago.edu/mailman/listinfo/swift-devel
>>>>>
>>>>
>>>>
>>>
>>
>>
>




More information about the Swift-devel mailing list