Current status: merged into GHC HEAD!

I’ve thought it might be a good idea to post some information about how is it going with the project.
So yes, thanks to help provided by David Terei and Manuel M T Chakravarty our project results were merged into GHC HEAD. Last commit (so far!) went in during August 20/21 2011. If you do have some ARM system, then please give it a try! You will need your own build of LLVM, which is described here. If you are curious and would just like to see tests results, then look here:

OVERALL SUMMARY for test run started at Tue Aug 23 22:59:36 CEST 2011
    2927 total tests, which gave rise to
    7123 test cases, of which
       1 caused framework failures
    2646 were skipped

    4260 expected passes
     148 expected failures
       0 unexpected passes
      68 unexpected failures

Unexpected failures:
   ../../libraries/random/tests  rangeTest [bad exit code] (normal,threaded1,threaded2,optllvm)
   annotations/should_run        annrun01 [exit code non-0] (normal,threaded1,threaded2,optllvm)
   cabal                         ghcpkg05 [bad stderr] (normal)
   cabal/cabal04                 cabal04 [bad exit code] (normal)
   codeGen/should_compile        jmp_tbl [exit code non-0] (normal)
   codeGen/should_compile        massive_array [exit code non-0] (normal)
   dph/dotp                      dph-dotp-fast [exit code non-0] (normal,threaded1,threaded2)
   dph/dotp                      dph-dotp-opt [exit code non-0] (normal,threaded1,threaded2)
   dph/primespj                  dph-primespj-fast [exit code non-0] (normal,threaded1,threaded2)
   dph/primespj                  dph-primespj-opt [exit code non-0] (normal,threaded1,threaded2)
   dph/quickhull                 dph-quickhull-fast [exit code non-0] (normal,threaded1,threaded2)
   dph/quickhull                 dph-quickhull-opt [exit code non-0] (normal,threaded1,threaded2)
   dph/sumnats                   dph-sumnats [exit code non-0] (normal,threaded1,threaded2)
   dph/words                     dph-words-fast [exit code non-0] (normal)
   dph/words                     dph-words-opt [exit code non-0] (normal)
   driver                        5313 [exit code non-0] (normal,threaded1,threaded2,optllvm)
   driver/recomp009              recomp009 [bad exit code] (normal)
   dynlibs                       T3807 [bad exit code] (normal)
   ghc-api/T4891                 T4891 [bad exit code] (normal)
   ghc-api/apirecomp001          apirecomp001 [bad exit code] (normal)
   ghci/linking                  ghcilink001 [bad exit code] (normal)
   ghci/linking                  ghcilink002 [bad exit code] (normal)
   ghci/linking                  ghcilink003 [bad exit code] (normal)
   ghci/linking                  ghcilink004 [bad exit code] (normal)
   ghci/linking                  ghcilink005 [bad exit code] (normal)
   ghci/linking                  ghcilink006 [bad exit code] (normal)
   ghci/scripts                  ghci024 [bad exit code] (normal)
   perf/compiler                 T1969 [stat not good enough] (normal)
   perf/compiler                 T3064 [stat not good enough] (normal)
   perf/compiler                 T5030 [stat not good enough] (normal)
   quasiquotation/qq007          qq007 [exit code non-0] (normal)
   quasiquotation/qq008          qq008 [exit code non-0] (normal)
   rts                           T2615 [exit code non-0] (normal,threaded1,threaded2,optllvm)
   rts                           derefnull [bad exit code] (threaded2)
   rts                           testblockalloc [bad exit code] (normal,threaded1)
   safeHaskell/flags             Flags02 [exit code non-0] (normal)
   simplCore/should_compile      T3016 [exit code non-0] (normal)
   typecheck/should_run          T4809 [exit code non-0] (normal,threaded1,threaded2,optllvm)

Majority of the failures are caused by missing GHCi support, which is also my next item on the project’s TODO list.

LLVM on ARM testing

In my LLVM on ARM post I’ve claimed that the best GCC to compile LLVM on ARM natively (no cross compiling from x86/amd64 to ARM!) is GCC 4.4.1. Last week I kept all my available hardware running to test LLVM 2.9 and LLVM HEAD more thoroughly so I’ll have some proof that this claim is actually true. Well, it’s indeed true at least so far, but there are also other GCC versions able to get to the same level of quality as 4.4.1 while compiling LLVM source base on ARM.
Anyway, I’ve tested 6 GCC versions on both LLVM 2.9 and LLVM HEAD as of following commit

commit ca42299619cf47371a42c2bda87d067e003657ea
Author: Eric Christopher <echristo@apple.com>
Date:   Wed Jun 29 17:53:29 2011 +0000

    Move XCore from getRegClassForInlineAsmConstraint to
    getRegForInlineAsmConstraint.
    
    Part of rdar://9643582
    
    
    git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@134080 91177308-0d34-0410-b5e6-96231b3b80

I’ll list results in two tables, one for LLVM 2.9 and one for LLVM HEAD. Compilation parameters were -O0, -O1, -O2, and default which is -O3. I’ve run only LLVM tests which are distributed with LLVM itself (in the same package, not separate huge LLVM testsuite!) and which are invoked by make check. The tables lists tested GCC versions, compile parameters and a number of unexpected failures which I got from this combination. Interested reader might click on the number to download output of the make check command to see which tests exactly failed and why. I hope someone from LLVM community and also from Linaro community will actually do, I’ve done this testing as a service to both communities… Below the tables you can also find gcc -v output for all GCC version involved for the idea how compilers where configured and built.

LLVM 2.9 results:

	-O0	-O1	-O2	default
GCC 4.3.4 (1)	1	1	45	39
GCC 4.4.1 (2)	1	1	45	39
GCC 4.4.5 (3)	54	54	98	92
GCC 4.5.2 (4)	54	54	112	112
GCC 4.6.1/2011.05 (5)	1	1	59	59
GCC 4.6.1/2011.06 (6)	1	1	59	59

LLVM HEAD results:

	-O0	-O1	-O2	default
GCC 4.3.4 (1)	29	29	82	82
GCC 4.4.1 (2)	29	29	82	82
GCC 4.4.5 (3)	29	29	82	82
GCC 4.5.2 (4)	29	29	100	100
GCC 4.6.1/2011.05 (5)	29	29	100	100
GCC 4.6.1/2011.06 (6)	29	29	100	100

Interesting is number of regression presented in LLVM HEAD in comparison with LLVM 2.9. Compilers were configured as:

(1) GCC 4.3.4: run on Ubuntu 10.04.2 LTS on i.MX53 Quick Start Board together with binutils 2.20.1-system.20100303

Using built-in specs.
Target: arm-linux-gnueabi
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.3.4-10ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --disable-sjlj-exceptions --with-arch=armv6 --with-tune=cortex-a8 --with-float=softfp --with-fpu=vfp --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.3.4 (Ubuntu 4.3.4-10ubuntu1) 

(2) GCC 4.4.1: run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
Target: arm-linux-gnueabi
Configured with: /export/home/karel/src/gcc-4.4.1-t2/gcc-4.4-4.4.1/src/configure -v --with-pkgversion='Ubuntu 4.4.1-4ubuntu9' --enable-languages=c,c++ --prefix=/export/home/karel/arm-sfw/gcc-4.4.1-4ubuntu9 --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --without-included-gettext --enable-threads=posix --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --disable-sjlj-exceptions --with-arch=armv6 --with-tune=cortex-a8 --with-float=softfp --with-fpu=vfp --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9) 

(3) GCC 4.4.5: run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
Target: arm-linux-gnueabi
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.4.5-15ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.4 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib/arm-linux-gnueabi --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib/arm-linux-gnueabi --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3-d16 --with-mode=thumb --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.4.5 (Ubuntu/Linaro 4.4.5-15ubuntu1) 

(4) GCC 4.5.2: run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
COLLECT_GCC=gcc-4.5
COLLECT_LTO_WRAPPER=/usr/lib/arm-linux-gnueabi/gcc/arm-linux-gnueabi/4.5.2/lto-wrapper
Target: arm-linux-gnueabi
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.5.2-8ubuntu4' --with-bugurl=file:///usr/share/doc/gcc-4.5/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.5 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib/arm-linux-gnueabi --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.5 --libdir=/usr/lib/arm-linux-gnueabi --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --enable-objc-gc --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3-d16 --with-mode=thumb --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.5.2 (Ubuntu/Linaro 4.5.2-8ubuntu4) 

(5) GCC 4.6.1 Linaro 2011.05: compiler compiled by me from Linaro 2011.05 source distribution. Run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
COLLECT_GCC=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.05-0/bin/gcc
COLLECT_LTO_WRAPPER=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.05-0/libexec/gcc/arm-linux-gnueabi/4.6.1/lto-wrapper
Target: arm-linux-gnueabi
Configured with: ../gcc-linaro-4.6-2011.05-0/configure -v --enable-languages=c,c++ --prefix=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.05-0 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --without-included-gettext --enable-threads=posix --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3 --with-mode=arm --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.6.1 20110506 (prerelease) (Linaro GCC 4.6-2011.05-0) 

(6) GCC 4.6.1 Linaro 2011.06: compiler compiled by me from Linaro 2011.06 source distribution. Run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
COLLECT_GCC=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.06-0/bin/gcc
COLLECT_LTO_WRAPPER=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.06-0/libexec/gcc/arm-linux-gnueabi/4.6.1/lto-wrapper
Target: arm-linux-gnueabi
Configured with: ../gcc-linaro-4.6-2011.06-0/configure -v --enable-languages=c,c++ --prefix=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.06-0 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --without-included-gettext --enable-threads=posix --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3 --with-mode=arm --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.6.1 20110526 (prerelease) (Linaro GCC 4.6-2011.06-0) 

I hope someone will find those results useful. If anything here is wrong or require correction or if someone needs to perform some additional test, please ask in comments.