[opa-nightly-tests] jam OPA_Daily_Tests_0423Mon_FAILED!!!
Neil Fortner
nfortne2 at hdfgroup.org
Mon Apr 23 14:38:53 CDT 2012
I added some barriers to the ABA tests (which is where the actual
failure was). I ran it 100 times afterwards with no failures. The
warnings from the stack tests happen when a pusher thread tries 100000
times to push its object on the stack, and fails because it is already
on the stack. I suspect OPA_TEST_YIELD() doesn't do much (if anything)
on POWER7, which is why it is more likely to show up there. I do not
think it is a major issue.
Of course feel free to review my work. There could still be missing
barriers in the stack test as well.
-Neil
On 04/23/2012 09:12 AM, Dave Goodell wrote:
> Looks like a failure on pcc64. Maybe some missing memory barriers in the LL/SC stack test?
>
> -Dave
>
> On Apr 23, 2012, at 12:15 AM CDT, HDF Tester wrote:
>
>> *** OPA Tests on 0423Mon ***
>> =============================
>> Tests Summary
>> =============================
>> ****FAILED ostrich: standard****
>>
>> PASSED freedom: standard
>> PASSED jam: standard
>> PASSED koala: standard
>> PASSED linew: standard
>> PASSED loyalty: standard
>>
>> =============================
>> Tests Time Summary
>> =============================
>> koala: Ran 1(1/0/0) tests, Grand total test time = 0m 44s
>> jam: Ran 1(1/0/0) tests, Grand total test time = 1m 21s
>> freedom: Ran 1(1/0/0) tests, Grand total test time = 1m 24s
>> loyalty: Ran 1(1/0/0) tests, Grand total test time = 1m 26s
>> ostrich: Ran 1(0/1/0) tests, Grand total test time = 2m 36s
>> linew: Ran 1(1/0/0) tests, Grand total test time = 3m 47s
>> jam: Ran 6(0/0/0) hosts, Grand total test time = 4m 47s
>>
>>
>> =============================
>> Timekeeper log
>> =============================
>> Timekeeper started at Mon Apr 23 00:10:46 CDT 2012
>> Timekeeper sleeping for 1800 seconds
>>
>>
>> =============================
>> Tests Failures
>> =============================
>> =========================
>> Dumping logfile of ostrich: standard
>> Last 50 lines of /mnt/scr1/SnapTest/snapshots-opa/log/ostrich_0423Mon_0011
>> =========================
>> Testing integer swap with 100 threads PASSED
>> Testing pointer swap with 100 threads PASSED
>> Testing integer LL/SC stack with 100 threads *WARNING*
>> Only 969996/970000 pushes succeeded. There may be an issue with fairness.
>> Testing pointer LL/SC stack with 100 threads *WARNING*
>> Only 969996/970000 pushes succeeded. There may be an issue with fairness.
>> ***** 1 PRIMITIVES TEST FAILED! *****
>> FAIL: test_primitives
>> Testing memory barrier sanity PASSED
>> Testing memory barriers with linear array with 2 threads PASSED
>> Testing memory barriers with local variables with 2 threads PASSED
>> Testing memory barriers with scattered array with 2 threads PASSED
>> Testing memory barriers with linear array with 4 threads PASSED
>> Testing memory barriers with local variables with 4 threads PASSED
>> Testing memory barriers with scattered array with 4 threads PASSED
>> Testing memory barriers with linear array with 10 threads PASSED
>> Testing memory barriers with local variables with 10 threads PASSED
>> Testing memory barriers with scattered array with 10 threads PASSED
>> Testing memory barriers with linear array with 100 threads PASSED
>> Testing memory barriers with local variables with 100 threads PASSED
>> Testing memory barriers with scattered array with 100 threads PASSED
>> All barriers tests passed.
>> PASS: test_barriers
>> Testing queue sanity PASSED
>> Testing multithreaded queue with 2 threads PASSED
>> Testing multithreaded queue (empty queue) with 2 threads PASSED
>> Testing multithreaded queue (full queue) with 2 threads PASSED
>> Testing multithreaded queue with 4 threads PASSED
>> Testing multithreaded queue (empty queue) with 4 threads PASSED
>> Testing multithreaded queue (full queue) with 4 threads PASSED
>> Testing multithreaded queue with 10 threads PASSED
>> Testing multithreaded queue (empty queue) with 10 threads PASSED
>> Testing multithreaded queue (full queue) with 10 threads PASSED
>> Testing multithreaded queue with 100 threads PASSED
>> Testing multithreaded queue (empty queue) with 100 threads PASSED
>> Testing multithreaded queue (full queue) with 100 threads PASSED
>> All queue tests passed.
>> PASS: test_queue
>> ===================================================================
>> 1 of 4 tests failed
>> Please report to https://trac.mcs.anl.gov/projects/openpa/newticket
>> ===================================================================
>> gmake[2]: *** [check-TESTS] Error 1
>> gmake[2]: Leaving directory `/scr/hdftest/snapshots-opa/TestDir/ostrich/test'
>> gmake[1]: *** [check-am] Error 2
>> gmake[1]: Leaving directory `/scr/hdftest/snapshots-opa/TestDir/ostrich/test'
>> gmake: *** [check-recursive] Error 1
>> Failed running make check
>> ===== Exit bin/snapshot with status=2: Mon Apr 23 00:13:44 CDT 2012 =====
>> Mon Apr 23 00:13:44 CDT 2012
>> =========================
>> Dumping done
>> =========================
>>
>> Runtest did not exit normally.
>>
>> =============================
>> Watchers List
>> =============================
>> OPA Daily test features/platforms watchers and procedure
>> ---------------------------------------------------------
>>
>> Procedure:
>> The watcher will investigate and report the cause of failure by 11am.
>> The developer who checked in the error code may report so by then too.
>> The watcher or the developer should get the failure fixed and report it
>> by 3pm.
>>
>>
>> Watcher for OPA: Neil
>>
>>
>> ---
>> updated: 2009/05/05
>>
>> =============================
>> Tests Details
>> =============================
>> 00:10:21 up 32 days, 15:18, 96 users, load average: 2.15, 1.79, 1.40
>> Filesystem 1K-blocks Used Available Use% Mounted on
>> /dev/sda3 31738420 4014012 26086180 14% /
>> /dev/sda1 101086 34394 61473 36% /boot
>> /dev/sda2 31738420 180648 29919544 1% /tmp
>> /dev/sda6 31738392 3983132 26117036 14% /var
>> /dev/sda7 31738392 26976908 3123260 90% /usr
>> /dev/sda8 124991068 343896 118195472 1% /var/tmp
>> /dev/mapper/VolGroup00-home
>> 198351840 14951516 173161964 8% /home
>> /dev/sde1 565688764 429320064 107169764 81% /scr
>> /dev/sdc1 961432072 678358756 234235316 75% /mnt/scr1
>> /dev/sdd1 961432072 784958516 127635556 87% /mnt/hdf
>> tmpfs 8313848 0 8313848 0% /dev/shm
>> gumund:/data/ftp 480719104 330136576 126163328 73% /mnt/ftp
>> gumund:/data/web 480719104 330136576 126163328 73% /mnt/web
>> STANDARD_OPT=op-configure --prefix=${PWD}/opainstall
>> TEST_TYPES=standard
>>
>> Running source repository checkout with output saved in
>> /mnt/scr1/SnapTest/snapshots-opa/log/REPO_LOG_0423Mon
>> Checking MANIFEST file ...
>> cat: /mnt/scr1/SnapTest/snapshots-opa/log/#runtest.0423Mon.27083: No such file or directory
>> rm: cannot remove `/mnt/scr1/SnapTest/snapshots-opa/log/#runtest.0423Mon.27083': No such file or directory
>>
>> Mon Apr 23 00:10:46 CDT 2012
>> *** launching tests from jam ***
>>
>> TESTHOST is linew
>> jam
>> koala
>> ostrich
>> loyalty
>> freedom
>> Fork off timekeeper 30
>> ==============
>> Testing linew
>> ==============
>> ssh linew -n cd /home/hdftest/snapshots-opa;/mnt/scr1/SnapTest/snapshots-opa/bin/runtest -nodiff -norepo -configname linew
>> 12:10am up 3 day(s), 12:10, 0 users, load average: 0.02, 0.02, 0.03
>> / (/dev/dsk/c1t0d0s0 ):50754410 blocks 8221246 files
>> /devices (/devices ): 0 blocks 0 files
>> /system/contract (ctfs ): 0 blocks 2147483605 files
>> /proc (proc ): 0 blocks 29945 files
>> /etc/mnttab (mnttab ): 0 blocks 0 files
>> /etc/svc/volatile (swap ):21102816 blocks 1180602 files
>> /system/object (objfs ): 0 blocks 2147483490 files
>> /etc/dfs/sharetab (sharefs ): 0 blocks 2147483646 files
>> /platform/sun4u-us3/lib/libc_psr.so.1(/platform/sun4u-us3/lib/libc_psr/libc_psr_hwcap1.so.1):50754410 blocks 8221246 files
>> /platform/sun4u-us3/lib/sparcv9/libc_psr.so.1(/platform/sun4u-us3/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1):50754410 blocks 8221246 files
>> /dev/fd (fd ): 0 blocks 0 files
>> /tmp (swap ):21102512 blocks 1180602 files
>> /var/run (swap ):21102512 blocks 1180602 files
>> /scr (/dev/dsk/c1t1d0s0 ):861210130 blocks 226682796 files
>> /home (jam:/home ):366800648 blocks 50905395 files
>> /mnt/hdf (jam:/mnt/hdf ):352947112 blocks 113564784 files
>> /mnt/scr1 (jam:/mnt/scr1 ):566144752 blocks 118638406 files
>> /mnt/web (gumund:/mnt/web ):301164808 blocks 60877640 files
>> /mnt/ftp (gumund:/mnt/ftp ):301164808 blocks 60877640 files
>> STANDARD_OPT=op-configure --prefix=${PWD}/opainstall
>> TEST_TYPES=standard
>>
>> Mon Apr 23 00:11:02 CDT 2012
>> *** starting standard tests in linew ***
>> Uname -a: SunOS linew 5.10 Generic_147440-15 sun4u sparc SUNW,A70
>> Running snapshot with output saved in
>> /mnt/scr1/SnapTest/snapshots-opa/log/linew_0423Mon_0011
>> PASSED linew: standard
>> *** finished standard tests for linew ***
>> Mon Apr 23 00:14:48 CDT 2012
>> Total time = 3m 46s
>>
>> *** finished tests in linew ***
>> Mon Apr 23 00:14:48 CDT 2012
>> linew: Ran 1(1/0/0) tests, Grand total test time = 3m 47s
>>
>>
>> ==============
>> Testing jam
>> ==============
>> ssh jam -n cd /home/hdftest/snapshots-opa;/mnt/scr1/SnapTest/snapshots-opa/bin/runtest -nodiff -norepo -configname jam
>> 00:10:53 up 32 days, 15:18, 96 users, load average: 2.30, 1.85, 1.43
>> Filesystem 1K-blocks Used Available Use% Mounted on
>> /dev/sda3 31738420 4014016 26086176 14% /
>> /dev/sda1 101086 34394 61473 36% /boot
>> /dev/sda2 31738420 180768 29919424 1% /tmp
>> /dev/sda6 31738392 3983136 26117032 14% /var
>> /dev/sda7 31738392 26976908 3123260 90% /usr
>> /dev/sda8 124991068 343896 118195472 1% /var/tmp
>> /dev/mapper/VolGroup00-home
>> 198351840 14951516 173161964 8% /home
>> /dev/sde1 565688764 429320064 107169764 81% /scr
>> /dev/sdc1 961432072 678359720 234234352 75% /mnt/scr1
>> /dev/sdd1 961432072 784958516 127635556 87% /mnt/hdf
>> tmpfs 8313848 0 8313848 0% /dev/shm
>> gumund:/data/ftp 480719104 330136576 126163328 73% /mnt/ftp
>> gumund:/data/web 480719104 330136576 126163328 73% /mnt/web
>> STANDARD_OPT=op-configure --prefix=${PWD}/opainstall
>> TEST_TYPES=standard
>>
>> Mon Apr 23 00:11:03 CDT 2012
>> *** starting standard tests in jam ***
>> Uname -a: Linux jam 2.6.18-194.3.1.el5PAE #1 SMP Thu May 13 13:48:44 EDT 2010 i686 i686 i386 GNU/Linux
>> Running snapshot with output saved in
>> /mnt/scr1/SnapTest/snapshots-opa/log/jam_0423Mon_0011
>> PASSED jam: standard
>> *** finished standard tests for jam ***
>> Mon Apr 23 00:12:24 CDT 2012
>> Total time = 1m 21s
>>
>> *** finished tests in jam ***
>> Mon Apr 23 00:12:24 CDT 2012
>> jam: Ran 1(1/0/0) tests, Grand total test time = 1m 21s
>>
>>
>> ==============
>> Testing koala
>> ==============
>> ssh koala -n cd /home/hdftest/snapshots-opa;/mnt/scr1/SnapTest/snapshots-opa/bin/runtest -nodiff -norepo -configname koala
>> 00:10:55 up 2 days, 11:17, 2 users, load average: 0.00, 0.00, 0.00
>> Filesystem 1K-blocks Used Available Use% Mounted on
>> /dev/sda7 22292052 17537616 3603784 83% /
>> /dev/sda5 12192608 498988 11064276 5% /tmp
>> /dev/sda3 12192636 1293772 10269520 12% /var
>> /dev/sda1 101086 37243 58624 39% /boot
>> tmpfs 6149560 0 6149560 0% /dev/shm
>> /dev/sdd1 41279536 22244100 16938556 57% /usr
>> /dev/md0 48091528 37300400 8348200 82% /scr
>> jam:/home 198351840 14951488 173161984 8% /home
>> jam:/mnt/hdf 961432096 784958528 127635584 87% /mnt/hdf
>> jam:/mnt/scr1 961432096 678359872 234234240 75% /mnt/scr1
>> gumund:/data/ftp 480719104 330136576 126163328 73% /mnt/ftp
>> gumund:/data/web 480719104 330136576 126163328 73% /mnt/web
>> STANDARD_OPT=op-configure --prefix=${PWD}/opainstall
>> TEST_TYPES=standard
>>
>> Mon Apr 23 00:11:05 CDT 2012
>> *** starting standard tests in koala ***
>> Uname -a: Linux koala 2.6.18-308.4.1.el5 #1 SMP Tue Apr 17 17:08:00 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
>> Running snapshot with output saved in
>> /mnt/scr1/SnapTest/snapshots-opa/log/koala_0423Mon_0011
>> PASSED koala: standard
>> *** finished standard tests for koala ***
>> Mon Apr 23 00:11:49 CDT 2012
>> Total time = 0m 44s
>>
>> *** finished tests in koala ***
>> Mon Apr 23 00:11:49 CDT 2012
>> koala: Ran 1(1/0/0) tests, Grand total test time = 0m 44s
>>
>>
>> ==============
>> Testing ostrich
>> ==============
>> ssh ostrich -n cd /home/hdftest/snapshots-opa;/mnt/scr1/SnapTest/snapshots-opa/bin/runtest -nodiff -norepo -configname ostrich
>> 00:10:58 up 45 days, 12:09, 2 users, load average: 0.00, 0.00, 0.00
>> Filesystem 1K-blocks Used Available Use% Mounted on
>> /dev/mapper/vg_ostrich-lv_root
>> 140610456 9054328 124417084 7% /
>> tmpfs 7821760 0 7821760 0% /dev/shm
>> /dev/sda2 495844 130766 339478 28% /boot
>> /dev/sdb2 124608436 1490984 116787700 2% /scr
>> jam:/home 198351840 14951488 173161984 8% /home
>> jam:/mnt/hdf 961432096 784958528 127635584 87% /mnt/hdf
>> jam:/mnt/scr1 961432096 678359904 234234208 75% /mnt/scr1
>> gumund:/data/ftp 480719072 330136640 126163232 73% /mnt/ftp
>> gumund:/data/web 480719072 330136640 126163232 73% /mnt/web
>> STANDARD_OPT=op-configure --prefix=${PWD}/opainstall
>> TEST_TYPES=standard
>>
>> Mon Apr 23 00:11:08 CDT 2012
>> *** starting standard tests in ostrich ***
>> Uname -a: Linux ostrich 2.6.32-220.7.1.el6.ppc64 #1 SMP Fri Feb 10 15:24:28 EST 2012 ppc64 ppc64 ppc64 GNU/Linux
>> Running snapshot with output saved in
>> /mnt/scr1/SnapTest/snapshots-opa/log/ostrich_0423Mon_0011
>> *************************************
>> Mon Apr 23 00:13:44 CDT 2012
>> ****FAILED ostrich: standard****
>> *************************************
>> *** finished standard tests for ostrich ***
>> Mon Apr 23 00:13:44 CDT 2012
>> Total time = 2m 36s
>>
>> *** finished tests in ostrich ***
>> Mon Apr 23 00:13:44 CDT 2012
>> ostrich: Ran 1(0/1/0) tests, Grand total test time = 2m 36s
>>
>> ****SYSTEM ERROR ostrich: Abnormal exit from runtest ****
>>
>> *************************************
>> Mon Apr 23 00:15:18 CDT 2012
>> ****SYSTEM ERROR ostrich: runtest command failed ****
>> *************************************
>>
>> ==============
>> Testing loyalty
>> ==============
>> ssh loyalty -n cd /home/hdftest/snapshots-opa;/mnt/scr1/SnapTest/snapshots-opa/bin/runtest -nodiff -norepo -configname loyalty
>> grep: /proc/cpuinfo: No such file or directory
>> 12:11AM up 45 days, 10:25, 0 users, load averages: 0.00, 0.00, 0.00
>> Filesystem 1K-blocks Used Avail Capacity Mounted on
>> /dev/da0s1a 507630 103648 363372 22% /
>> devfs 1 1 0 100% /dev
>> /dev/da0s1g 58896528 31170704 23014102 58% /local_home
>> /dev/da0s1f 16244334 4862524 10082264 33% /usr
>> /dev/da0s1d 1012974 203388 728550 22% /var
>> /dev/da0s1e 507630 16732 450288 4% /var/tmp
>> procfs 4 4 0 100% /proc
>> jam:/home 198351840 14951516 173161964 8% /home
>> jam:/mnt/hdf 961432072 784958516 127635556 86% /mnt/hdf
>> jam:/mnt/scr1 961432072 678359968 234234104 74% /mnt/scr1
>> gumund:/data/web 480719056 330136652 126163204 72% /mnt/web
>> gumund:/data/ftp 480719056 330136652 126163204 72% /mnt/ftp
>> devfs 1 1 0 100% /var/named/dev
>> /dev/md0 126702 26 116540 0% /tmp
>> STANDARD_OPT=op-configure --prefix=${PWD}/opainstall
>> TEST_TYPES=standard
>>
>> Mon Apr 23 00:11:11 CDT 2012
>> *** starting standard tests in loyalty ***
>> Uname -a: FreeBSD loyalty.hdfgroup.uiuc.edu 8.2-STABLE FreeBSD 8.2-STABLE #0: Tue May 3 22:23:36 CDT 2011 sukoziol at loyalty.hdfgroup.uiuc.edu:/usr/obj/usr/src/sys/LOYALTY i386
>> Running snapshot with output saved in
>> /mnt/scr1/SnapTest/snapshots-opa/log/loyalty_0423Mon_0011
>> PASSED loyalty: standard
>> *** finished standard tests for loyalty ***
>> Mon Apr 23 00:12:37 CDT 2012
>> Total time = 1m 26s
>>
>> *** finished tests in loyalty ***
>> Mon Apr 23 00:12:37 CDT 2012
>> loyalty: Ran 1(1/0/0) tests, Grand total test time = 1m 26s
>>
>>
>> ==============
>> Testing freedom
>> ==============
>> ssh freedom -n cd /home/hdftest/snapshots-opa;/mnt/scr1/SnapTest/snapshots-opa/bin/runtest -nodiff -norepo -configname freedom
>> grep: /proc/cpuinfo: No such file or directory
>> 12:11AM up 45 days, 10:25, 2 users, load averages: 0.00, 0.00, 0.00
>> Filesystem 1K-blocks Used Avail Capacity Mounted on
>> /dev/da0s1a 507630 116218 350802 25% /
>> devfs 1 1 0 100% /dev
>> /dev/da0s1g 45184754 28993502 12576472 70% /local_home
>> /dev/da0s1f 24372460 5411794 17010870 24% /usr
>> /dev/da0s1d 2026030 201314 1662634 11% /var
>> /dev/da0s1e 1012974 542144 389794 58% /var/tmp
>> procfs 4 4 0 100% /proc
>> jam:/home 198351840 14951516 173161964 8% /usr/home
>> jam:/mnt/hdf 961432072 784958516 127635556 86% /mnt/hdf
>> jam:/mnt/scr1 961432072 678360072 234234000 74% /mnt/scr1
>> gumund:/data/web 480719056 330136652 126163204 72% /mnt/web
>> gumund:/data/ftp 480719056 330136652 126163204 72% /mnt/ftp
>> devfs 1 1 0 100% /var/named/dev
>> /dev/md0 126702 136 116430 0% /tmp
>> STANDARD_OPT=op-configure --prefix=${PWD}/opainstall
>> TEST_TYPES=standard
>>
>> Mon Apr 23 00:11:13 CDT 2012
>> *** starting standard tests in freedom ***
>> Uname -a: FreeBSD freedom.hdfgroup.uiuc.edu 8.2-STABLE FreeBSD 8.2-STABLE #0: Tue May 3 22:26:37 CDT 2011 sukoziol at freedom.hdfgroup.uiuc.edu:/usr/obj/usr/src/sys/FREEDOM amd64
>> Running snapshot with output saved in
>> /mnt/scr1/SnapTest/snapshots-opa/log/freedom_0423Mon_0011
>> PASSED freedom: standard
>> *** finished standard tests for freedom ***
>> Mon Apr 23 00:12:37 CDT 2012
>> Total time = 1m 24s
>>
>> *** finished tests in freedom ***
>> Mon Apr 23 00:12:37 CDT 2012
>> freedom: Ran 1(1/0/0) tests, Grand total test time = 1m 24s
>>
>>
>> *** finished tests in jam ***
>> Mon Apr 23 00:15:18 CDT 2012
>> jam: Ran 6(0/0/0) hosts, Grand total test time = 4m 47s
>>
>> _______________________________________________
>> opa-nightly-tests mailing list
>> opa-nightly-tests at lists.mcs.anl.gov
>> https://lists.mcs.anl.gov/mailman/listinfo/opa-nightly-tests
> _______________________________________________
> opa-nightly-tests mailing list
> opa-nightly-tests at lists.mcs.anl.gov
> https://lists.mcs.anl.gov/mailman/listinfo/opa-nightly-tests
>
More information about the opa-nightly-tests
mailing list