Unregisterised GHC HEAD build for ARM64 (ARMv8/AArch64) platform.

I’ve thought I’ll have some ARM/GHC fun again after a while and thought to give a try to ARM64 port. I mean AArch64 mode of ARMv8 platform as the ARM64 of course.
GHC Wiki contains nice page describing porting GHC using LLVM backend. I’ll point here right now that I’ve decided to jump directly to (4) and attempt to build GHC compiler using LLVM backend. As you probably know, the first thing you need to do while porting GHC to other/unsupported platform is so called unregisterised build of GHC for the platform. Thanks to a lot of improvements in GHC 7.6/HEAD in cross-compilation it is even possible to build GHC unregisterised cross-compiler. That’s exactly what I did. I’ve not expected this to work smoothly, but at the end was quite surprised how easily I got stage1 cross-compiler for ARM64. Well, there were few issues, but solvable as you will see.
If you are going to follow my path on this just to have some fun (not that many people compile Haskell for ARM64 these days. :-), let’s prepare some Ubuntu 13.10 amd64 box for it. I used virtualbox image for that. Into it, you can install Ubuntu 13.10 core running on ARM64 emulator. Concretely speaking, you will have Ubuntu 13.10 core running on ARM Foundation Model, which is free (as a beer) tool to let you run system(s) on ARM64 platform. Ubuntu is really helpful here, just follow their wiki page and you should have it running quickly.
Also Ubuntu 13.10 is really nice platform for ARM64 cross-compiling since it offers ARM64 GNU C/C++ cross-compilers as its native package. So everything you need to do is just

$ sudo apt-get install gcc-aarch64-linux-gnu

After this you should have GNU C for ARM64 cross-compiler ready:

$ aarch64-linux-gnu-gcc -v
Using built-in specs.
Target: aarch64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 4.8.1-10ubuntu7' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,d,fortran --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/aarch64-linux-gnu/include/c++/4.8.1 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libssp --disable-libmudflap --disable-libitm --disable-libsanitizer --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-arm64-cross/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-arm64-cross --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-arm64-cross --with-arch-directory=arm64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libgcj --enable-multiarch --disable-werror --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=aarch64-linux-gnu --program-prefix=aarch64-linux-gnu- --includedir=/usr/aarch64-linux-gnu/include
Thread model: posix
gcc version 4.8.1 (Ubuntu/Linaro 4.8.1-10ubuntu7)

You will also need to build LLVM HEAD which contains revision r199265. This is a fix for LLVM bug observed by me while building GHC cross-compiler for ARM64. I’ve reported the issue to llvm-dev mailing list and Tim Northover was so kind and fix that really quickly.
So, obtain LLVM HEAD and compile on your Ubuntu box. Now, let’s assume you do have GNU C aarch64 cross-compiler, you do have Ubuntu 13.10 core running on ARM Foundation Model and you also do have LLVM HEAD compiled. Is that enough for building GHC cross-compiler? Not yet unfortunately! As long as GHC dev folks does not fix #8864 issue, you will need to update GHC’s libffi manually. I’ve tested 3.0.13 and it works fine. The reason is obvious, GHC bundled libffi 3.0.11’s configure script does not recognize aarch64 platform. Anyway, this is quite easy to fix. Just delete libffi-3.0.11.tar.gz in GHC’s libffi-tarballs subdirectory and put libffi-3.0.13.tar.gz there.
Now, you are nearly ready. Nearly, because GHC as a part of building process also builds some of its libraries and here terminfo library will make us a problem. The problem is that the library needs to have libncurses installed on the target platform as a development package. That means not only shared library libncurses needs to be installed but also its C header files, e.g. ncurses.h. Now, the problem is that GNU C cross-compiler comes only with really just core libraries for cross-compilation, i.e. libc. Nobody counted with the fact that you will need libncurses for cross-compiling. The issue is solvable by using different cross-compiler sysroot. Explanation: GNU C cross-compiler looks into a certain path to find its target libraries and head files for compilation. This certain path is known as sysroot. Fortunately for us, Ubuntu’s distributed aarch64 cross-compiler supports an option for setting the sysroot to some user defined directory. Now, you may ask where you will get all the required target libraries/header files besides ncurses which you need for terminfo library? Well, you already do have them in a form of ubuntu-core-13.10-core-arm64.tar.gz file which you needed to download for installation of Ubuntu core on ARM64/Foundation Model. Just unpack the file somewhere and let’s call its parent directory for example sysroot. If you search a little bit directory structure resulting from unpacking this tarbal you will find out quickly that even there there is no libncurses development package installed. There is however one hackish way how to get libncurses-dev there. You already do have Ubuntu 13.10/ARM64 running on ARM Foundation Model right? So start it, give it a time to boot and then install ssh there since you will need it to transfer files between your Ubuntu host and ARM64 target — well, if you don’t use for example NFS for the same purpose. And now, Debian/Ubuntu’s apt-get supports also an option to download just the package without actually installing it. You will need to download those two packages:

# apt-get download libncurses5-dev
# apt-get download libtinfo-dev

The libtinfo-dev package is a dependency of libncurses5-dev. Now, move those two packages to your Ubuntu host. If you have not done that before, please install mc package there to obtain nice midnight commander application. It’s very usable for example for browsing content of packages. Just start it by typing “mc” in your shell, navigate to the directory which holds two packages above and press enter on one of the package files. Midnight command will allow you to see content of the selected package and you will also be able to copy its files to your sysroot directory. Do this with both packages.
The last step before configuring GHC cross-compiler is creation of custom C compiler script which will invoke aarch64 C cross-compiler with a required –sysroot parameter. For my application here I’ve named it aarch64-linux-gnu-gccsysroot with following content:

/usr/bin/aarch64-linux-gnu-gcc --sysroot=/home/karel/arm64/sysroot $@

Just modify the sysroot parameter value. I guess you don’t have your ARM64 sysroot located in the same directory like me. :-) Of course make this file executable too!

And, now, time comes to configure GHC finally:

$ perl boot
$ ./configure --target=aarch64-linux-gnu --with-gcc=/home/karel/bin/aarch64-linux-gnu-gccsysroot --enable-unregisterised --with-llc=/export/home/karel/vcs/llvm-head/Release+Asserts/bin/llc --with-opt=/export/home/karel/vcs/llvm-head/Release+Asserts/bin/opt

Again, please modify your paths to the cross-compiler script, LLVM’s llc and LLVM’s opt to your desired host values.
For compilation, you will need to modify build.mk makefile so please do:

$ cd mk
$ cp build.mk.sample build.mk

and edit build.mk file in your preferred editor. You will need to uncomment quick-cross build flavour line to enable cross-compiler build:

# Fast build configured for a cross compiler
BuildFlavour = quick-cross

Now, if you type

$ make

then after some time you will get your ghc-stage1 cross-compiler for ARM64 platform! Have fun with it! At least you can use GHC’s HelloWorld example to test it:

$ cd bindisttest/
$ ../inplace/bin/ghc-stage1 --make HelloWorld.lhs
[1 of 1] Compiling Main ( HelloWorld.lhs, HelloWorld.o )
Linking HelloWorld ...
$ file HelloWorld
HelloWorld: ELF 64-bit LSB executable, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 3.7.0, BuildID[sha1]=0xaafd52827fa12be43b564b11b96d9131d8f6a498, not stripped

6 thoughts on “Unregisterised GHC HEAD build for ARM64 (ARMv8/AArch64) platform.

  1. you don’t need to work in an extra chroot, but you can use the multiarch approch to cross-building.

    – dpkg –add-architecture arm64

    – edit /etc/apt/sources.list to something like
    deb [arch=amd64] http://archive.ubuntu.com/ubuntu trusty main universe
    deb [arch=arm64] http://ports.ubuntu.com/ubuntu-ports trusty main universe
    deb-src http://archive.ubuntu.com/ubuntu trusty main universe

    – then install your build dependencies for the target:
    apt-get install libncurses5-dev:arm64

    This works as long as all your needed target libraries are prepared for multiarch (apt-cache show should show the Multi-Arch: same attribute). If some of your target libraries are not ready, please file a bug report, or even better attach a patch to add multiarch support.

  2. At least some patches that aren’t even in ghc.git master seem to be needed to get this far; GHC_CONVERT_CPU and FPTOOLS_SET_HASKELL_PLATFORM_VARS both need to be patched (fairly obviously if going for ArchUnknown, less so if your plan is to fold this into ArchARM somehow). Based on that I suspect that more might be missing. Do you have a stash of additional patches needed for this? Thanks!

    • Well, indeed, I’ve tested HEAD of that time and in fact started working on GHC registerised build support. So for your unregisterised first version you will probably not need all the bits of the changes…
      The problem is that I naturally prefer native compiles as this adds support for running GHC testsuite so basically my effort is delayed while waiting for some ARMv8 board. Ehm, no, X-Gene is out of my financial reach I’m waiting for something cheaper. Pity that GCC compile farm does not have any X-Gene available yet…
      … or is there any developer account ARMv8 machine already available? If so, please send me an email! You can find it on about page which I already edited to provide necessary information…
      Ah, patch, well it’s here: https://app.box.com/s/8x3wjeoppr63lgf7cbue

  3. Unfortunately I can’t find contact details for the author of this blog, kgardas. (About page is useless.)

  4. Guys,
    looks like Ubuntu is interested in ARMv8/GHC. Good! Anyway, one advice, please do not use LLVM way described above in my post. Rather use good-old -fvia-C way, that means use simple unregisterised build and do not set additional options for LLVM. This way you will rely only on GNU C and not use LLVM. I’ve used this way to compile non-crashing ghc-stage2 which you will probably need in order you do have ARMv8 hardware access for moving to the box and compiling GHC natively…
    So my experience is LLVM -> crashing ghc-stage2, usable only ghc-stage1 (so just for cross-compiling), GCC -> ok ghc-stage2.
    I hope it’s more clear now. And if you are curious why I’ve attempted LLVM way, then that is to get more close to registerised GHC/ARMv8 which is my target goal. Unfortunately as mentioned above, this is delayed till I do have hardware access or accurate and fast (read faster than ARM’s Foundation model) emulator/simulator access….

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s