<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
Hi again,
<div><br>
</div>
<div>I get this on the local:</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">/dev/sda1 on / type ext3 (rw,noatime,nodiratime) none on /proc type proc (rw,nosuid) none on /sys type sysfs (rw) none on /dev/pts type devpts (rw,gid=5,mode=620) /dev/sda2 on /var type ext3 (rw,noatime,nodiratime)
none on /dev/shm type tmpfs (rw) none on /tmp type tmpfs (rw,nodev,noatime,nodiratime,size=32g) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw) nfs:/exports/cmshared on /cm/shared type nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,addr=10.143.0.14)
nfs:/exports/home on /home type nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,acdirmin=0,acdirmax=0,acregmax=10,noac,addr=10.143.0.14) nfs:/exports/home0 on /imports/home0 type nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,acdirmin=0,acdirmax=0,acregmax=10,noac,addr=10.143.0.14)
nfs:/exports/home1 on /imports/home1 type nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,acdirmin=0,acdirmax=0,acregmax=10,noac,addr=10.143.0.14) nfs:/exports/homeL on /imports/homeL type nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,acdirmin=0,acdirmax=0,acregmax=10,noac,addr=10.143.0.14)
nfs:/exports/software on /shared type nfs (ro,rsize=32768,wsize=32768,hard,intr,addr=10.143.0.14) nfs:/exports/sge on /cm/shared/apps/sge type nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,addr=10.143.0.14) nfs:/exports/lcgsoft on /imports/lcgsoft type
nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,addr=10.143.0.14) nfs:/exports/deptapps on /imports/deptapps type nfs (rw,noatime,rsize=32768,wsize=32768,hard,intr,acdirmin=0,acdirmax=0,acregmax=10,noac,addr=10.143.0.14) nfsd on /proc/fs/nfsd type nfsd (rw)
none on /dev/cpuset type cpuset (rw) 10.143.0.127@tcp:10.143.0.126@tcp:/scratch on /scratch type lustre (rw,_netdev,noatime,nodiratime,flock)</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Don’t really see the output on the compute node.</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;">Mark</div>
<div style="margin: 0px; font-size: 11px; font-family: Menlo;"><br>
</div>
<div>
<div>On May 3, 2015, at 9:30 PM, Mihael Hategan <<a href="mailto:hategan@mcs.anl.gov">hategan@mcs.anl.gov</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
Hi,<br>
<br>
That's actually good since we eliminated lots of moving parts.<br>
<br>
~/Scratch seems to be the right spot according to<br>
<a href="https://wiki.rc.ucl.ac.uk/wiki/Managing_Data_on_Legion">https://wiki.rc.ucl.ac.uk/wiki/Managing_Data_on_Legion</a><br>
<br>
What I suspect might be happening is that the mountpoints are different<br>
between login nodes and compute nodes.<br>
<br>
Can you try running these on both the login node and a compute node:<br>
<br>
mount (or df)<br>
ls -al $HOME/Scratch<br>
<br>
and then pasting the outputs back in an email.<br>
<br>
Mihael<br>
<br>
On Sun, 2015-05-03 at 20:18 +0000, Altaweel, Mark wrote:<br>
<blockquote type="cite">If I do a qsub on the script I get the same error message:<br>
<br>
job_number: 6597054<br>
exec_file: job_scripts/6597054<br>
submission_time: Sun May 3 21:15:23 2015<br>
owner: tcrnma3<br>
uid: 147447<br>
group: users<br>
gid: 1002<br>
sge_o_home: /home/tcrnma3/<br>
sge_o_log_name: tcrnma3<br>
sge_o_path: /shared/ucl/apps/mrxvt/0.5.4/bin:/shared/ucl/apps/nedit/5.6/bin:/shared/ucl/apps/gerun/i:/usr/mpi/qlogic//sbin:/usr/mpi/qlogic//bin:/usr/lib64/qt-3.3/bin:/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/sbin:/usr/sbin:/shared/ucl/apps/bin:/cm/shared/apps/intel/toolkit/Compiler/11.1/072//bin:/cm/shared/apps/intel/toolkit/Compiler/11.1/072//bin/intel64:/cm/shared/apps/sge/6.2u3/bin/lx26-amd64:/home/tcrnma3//bin<br>
sge_o_shell: /bin/bash<br>
sge_o_workdir: /imports/home1/tcrnma3/Scratch/UrbanModel/run005/scripts<br>
sge_o_host: login06<br>
account: ucl_jsv4h;S=0;T=1.0;W=1.0;X=1.0;Y=1.0;V=0;Z=1.0;U=1.0<br>
stderr_path_list: NONE:NONE:/imports/home1/tcrnma3/Scratch/UrbanModel/run005/scripts/SGE7948718974736431209.submit.stderr<br>
hard resource_list: batch=true,bonus=0,h_rt=540,jcs=0,jct=1,jcu=1,jcv=0,jcw=1,jcx=1,jcy=1,jcz=1,maxversion=2,memory=1M,penalty=604801,s_rt=530<br>
mail_list: <a href="mailto:tcrnma3@login06.data.legion.ucl.ac.uk">tcrnma3@login06.data.legion.ucl.ac.uk</a><<a href="mailto:tcrnma3@login06.data.legion.ucl.ac.uk">mailto:tcrnma3@login06.data.legion.ucl.ac.uk</a>><br>
notify: FALSE<br>
job_name: B0503-3707460-0<br>
stdout_path_list: NONE:NONE:/imports/home1/tcrnma3/Scratch/UrbanModel/run005/scripts/SGE7948718974736431209.submit.stdout<br>
jobshare: 0<br>
restart: n<br>
shell_list: NONE:/bin/ksh<br>
env_list: WORKER_LOGGING_LEVEL=NONE,XAUTHORITY=/scratch/scratch/tcrnma3/.Xauthority,PAID=0,GPU=0,OMP_NUM_THREADS=1,MICCOUNT=0,SCRATCH_SPACE=10737418240,MEMPERSLOT=1048576,SGE_SHARENODE=1,IFS=<br>
script_file: SGE7948718974736431209.submit<br>
project: AllUsers<br>
error reason 1: 05/03/2015 21:15:57 [147447:18805]: error: can't open output file "/imports/home1/tcrnma3/Scratch/Ur<br>
scheduling info: (Collecting of scheduler job information is turned off)<br>
<br>
Mark<br>
<br>
On May 3, 2015, at 9:06 PM, Mihael Hategan <<a href="mailto:hategan@mcs.anl.gov">hategan@mcs.anl.gov</a><<a href="mailto:hategan@mcs.anl.gov">mailto:hategan@mcs.anl.gov</a>>> wrote:<br>
<br>
It seems that it is more likely that the error message gets truncated<br>
rather than the path itself. After all, stdout_path_list does contain<br>
what seems to be the correct path.<br>
<br>
There should be a<br>
script: /imports/home1/tcrnma3/Scratch/UrbanModel/run005/scripts/SGE7948718974736431209.submit<br>
(or similar) that should be available while a swift run is in progress.<br>
<br>
I think one way to troubleshoot things would be to copy that script and<br>
submit it manually.<br>
<br>
Mihael<br>
<br>
On Sun, 2015-05-03 at 19:20 +0000, Altaweel, Mark wrote:<br>
Yes so I do import swift in the shell script that gets distributed. However, same conclusion it seems. I don’t understand why it truncates the path, unless it is there but only writes a certain number of the characters.<br>
<br>
This is added to the script:<br>
<br>
export PATH=$PATH:~/Scratch/swift-0.96-sge-mod/bin<br>
module load java/1.7.0_45<br>
<br>
So java is included. If I remove it same thing happens though.<br>
<br>
Mark<br>
<br>
<br>
<br>
On May 3, 2015, at 8:07 PM, Mihael Hategan <<a href="mailto:hategan@mcs.anl.gov">hategan@mcs.anl.gov</a><<a href="mailto:hategan@mcs.anl.gov">mailto:hategan@mcs.anl.gov</a>><<a href="mailto:hategan@mcs.anl.gov">mailto:hategan@mcs.anl.gov</a>>> wrote:<br>
<br>
On Sun, 2015-05-03 at 18:43 +0000, Altaweel, Mark wrote:<br>
error reason 1: 05/03/2015 19:38:15 [147447:22761]: error: can't open output file "/imports/home1/tcrnma3/Scratch/Ur<br>
<br>
... aaand my PE suggestion had little to do with the problem.<br>
<br>
Is /imports mounted on compute nodes?<br>
<br>
Mihael<br>
<br>
_______________________________________________<br>
Swift-user mailing list<br>
<a href="mailto:Swift-user@ci.uchicago.edu">Swift-user@ci.uchicago.edu</a><<a href="mailto:Swift-user@ci.uchicago.edu">mailto:Swift-user@ci.uchicago.edu</a>><<a href="mailto:Swift-user@ci.uchicago.edu">mailto:Swift-user@ci.uchicago.edu</a>><br>
<a href="https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user">https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user</a></blockquote>
</div>
</blockquote>
</div>
<br>
</div>
</body>
</html>