LLVM on ARM testing

In my LLVM on ARM post I’ve claimed that the best GCC to compile LLVM on ARM natively (no cross compiling from x86/amd64 to ARM!) is GCC 4.4.1. Last week I kept all my available hardware running to test LLVM 2.9 and LLVM HEAD more thoroughly so I’ll have some proof that this claim is actually true. Well, it’s indeed true at least so far, but there are also other GCC versions able to get to the same level of quality as 4.4.1 while compiling LLVM source base on ARM.
Anyway, I’ve tested 6 GCC versions on both LLVM 2.9 and LLVM HEAD as of following commit

commit ca42299619cf47371a42c2bda87d067e003657ea
Author: Eric Christopher <echristo@apple.com>
Date:   Wed Jun 29 17:53:29 2011 +0000

    Move XCore from getRegClassForInlineAsmConstraint to
    getRegForInlineAsmConstraint.
    
    Part of rdar://9643582
    
    
    git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@134080 91177308-0d34-0410-b5e6-96231b3b80

I’ll list results in two tables, one for LLVM 2.9 and one for LLVM HEAD. Compilation parameters were -O0, -O1, -O2, and default which is -O3. I’ve run only LLVM tests which are distributed with LLVM itself (in the same package, not separate huge LLVM testsuite!) and which are invoked by make check. The tables lists tested GCC versions, compile parameters and a number of unexpected failures which I got from this combination. Interested reader might click on the number to download output of the make check command to see which tests exactly failed and why. I hope someone from LLVM community and also from Linaro community will actually do, I’ve done this testing as a service to both communities… Below the tables you can also find gcc -v output for all GCC version involved for the idea how compilers where configured and built.

LLVM 2.9 results:

-O0 -O1 -O2 default
GCC 4.3.4 (1) 1 1 45 39
GCC 4.4.1 (2) 1 1 45 39
GCC 4.4.5 (3) 54 54 98 92
GCC 4.5.2 (4) 54 54 112 112
GCC 4.6.1/2011.05 (5) 1 1 59 59
GCC 4.6.1/2011.06 (6) 1 1 59 59

LLVM HEAD results:

-O0 -O1 -O2 default
GCC 4.3.4 (1) 29 29 82 82
GCC 4.4.1 (2) 29 29 82 82
GCC 4.4.5 (3) 29 29 82 82
GCC 4.5.2 (4) 29 29 100 100
GCC 4.6.1/2011.05 (5) 29 29 100 100
GCC 4.6.1/2011.06 (6) 29 29 100 100

Interesting is number of regression presented in LLVM HEAD in comparison with LLVM 2.9. Compilers were configured as:

(1) GCC 4.3.4: run on Ubuntu 10.04.2 LTS on i.MX53 Quick Start Board together with binutils 2.20.1-system.20100303

Using built-in specs.
Target: arm-linux-gnueabi
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.3.4-10ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --disable-sjlj-exceptions --with-arch=armv6 --with-tune=cortex-a8 --with-float=softfp --with-fpu=vfp --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.3.4 (Ubuntu 4.3.4-10ubuntu1) 

(2) GCC 4.4.1: run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
Target: arm-linux-gnueabi
Configured with: /export/home/karel/src/gcc-4.4.1-t2/gcc-4.4-4.4.1/src/configure -v --with-pkgversion='Ubuntu 4.4.1-4ubuntu9' --enable-languages=c,c++ --prefix=/export/home/karel/arm-sfw/gcc-4.4.1-4ubuntu9 --enable-shared --enable-multiarch --enable-linker-build-id --with-system-zlib --without-included-gettext --enable-threads=posix --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --disable-sjlj-exceptions --with-arch=armv6 --with-tune=cortex-a8 --with-float=softfp --with-fpu=vfp --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.4.1 (Ubuntu 4.4.1-4ubuntu9) 

(3) GCC 4.4.5: run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
Target: arm-linux-gnueabi
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.4.5-15ubuntu1' --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.4 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib/arm-linux-gnueabi --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4 --libdir=/usr/lib/arm-linux-gnueabi --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3-d16 --with-mode=thumb --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.4.5 (Ubuntu/Linaro 4.4.5-15ubuntu1) 

(4) GCC 4.5.2: run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
COLLECT_GCC=gcc-4.5
COLLECT_LTO_WRAPPER=/usr/lib/arm-linux-gnueabi/gcc/arm-linux-gnueabi/4.5.2/lto-wrapper
Target: arm-linux-gnueabi
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.5.2-8ubuntu4' --with-bugurl=file:///usr/share/doc/gcc-4.5/README.Bugs --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.5 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --libexecdir=/usr/lib/arm-linux-gnueabi --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.5 --libdir=/usr/lib/arm-linux-gnueabi --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --enable-objc-gc --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3-d16 --with-mode=thumb --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.5.2 (Ubuntu/Linaro 4.5.2-8ubuntu4) 

(5) GCC 4.6.1 Linaro 2011.05: compiler compiled by me from Linaro 2011.05 source distribution. Run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
COLLECT_GCC=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.05-0/bin/gcc
COLLECT_LTO_WRAPPER=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.05-0/libexec/gcc/arm-linux-gnueabi/4.6.1/lto-wrapper
Target: arm-linux-gnueabi
Configured with: ../gcc-linaro-4.6-2011.05-0/configure -v --enable-languages=c,c++ --prefix=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.05-0 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --without-included-gettext --enable-threads=posix --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3 --with-mode=arm --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.6.1 20110506 (prerelease) (Linaro GCC 4.6-2011.05-0) 

(6) GCC 4.6.1 Linaro 2011.06: compiler compiled by me from Linaro 2011.06 source distribution. Run on Ubuntu 11.04 on Pandaboard together with binutils 2.21.0.20110327

Using built-in specs.
COLLECT_GCC=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.06-0/bin/gcc
COLLECT_LTO_WRAPPER=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.06-0/libexec/gcc/arm-linux-gnueabi/4.6.1/lto-wrapper
Target: arm-linux-gnueabi
Configured with: ../gcc-linaro-4.6-2011.06-0/configure -v --enable-languages=c,c++ --prefix=/export/home/karel/arm-sfw/gcc-4.6-linaro-2011.06-0 --enable-shared --enable-multiarch --with-multiarch-defaults=arm-linux-gnueabi --enable-linker-build-id --with-system-zlib --without-included-gettext --enable-threads=posix --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-plugin --enable-gold --enable-ld=default --with-plugin-ld=ld.gold --disable-sjlj-exceptions --with-arch=armv7-a --with-float=softfp --with-fpu=vfpv3 --with-mode=arm --disable-werror --enable-checking=release --build=arm-linux-gnueabi --host=arm-linux-gnueabi --target=arm-linux-gnueabi
Thread model: posix
gcc version 4.6.1 20110526 (prerelease) (Linaro GCC 4.6-2011.06-0) 

I hope someone will find those results useful. If anything here is wrong or require correction or if someone needs to perform some additional test, please ask in comments.

Advertisements

5 thoughts on “LLVM on ARM testing

  1. Hi I am running the armv5 llvm buildbot that can be observed at:
    http://google1.osuosl.org:8011/builders/llvm-arm-linux

    This builder currently passes the llvm tests on HEAD with 0 unexpected failures.

    OS Ubuntu Jaunty
    GCC gcc (Ubuntu 4.3.3-5ubuntu4) 4.3.3
    Target: arm-linux-gnueabi
    Configured with: ../src/configure -v –with-pkgversion=’Ubuntu 4.3.3-5ubuntu4′ –with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs –enable-languages=c,c++,fortran,objc,obj-c++ –prefix=/usr –enable-shared –with-system-zlib –libexecdir=/usr/lib –without-included-gettext –enable-threads=posix –enable-nls –with-gxx-include-dir=/usr/include/c++/4.3 –program-suffix=-4.3 –enable-clocale=gnu –enable-libstdcxx-debug –enable-objc-gc –enable-mpfr –disable-libssp –disable-sjlj-exceptions –with-arch=armv5t –with-tune=cortex-a8 –enable-checking=release –build=arm-linux-gnueabi –host=arm-linux-gnueabi –target=arm-linux-gnueabi
    Thread model: posix
    BINUTILS (GNU Binutils for Ubuntu) 2.19.1

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s