Recently, I decided I’d tinker with writing an operating system. I’ve been wanting to do this for decades, but always became fed-up with the revolting x86 architecture and would always abandon the project in disgust. Also, the internet wasn’t as big back when I first dove into the endeavor and information was harder to come by. Now, things have been cleaned up a bit, the x86 architecture isn’t as messy and there are other platforms to work on (Raspberry Pi, Arduino). The Linux kernel and *BSD kernels are free to study for issues in dealing with some hardware problems. I also have a lot more programming experience.
Another development that has brought about this renewed interest is my discouragement with the current state of operating system development. Linux has become a mess. All of the major distributions have lost their minds and moved to systemd, which is complete garbage (a rant for another time). Microsoft recently reorganized their development and appear to be moving toward focusing more on mobile devices. Apple did that a long time ago. I still use a MacBook Pro (pre-touch bar) but, despite the excellent build quality and stable, FreeBSD-based operating system with a pleasant GUI, they’ve always lagged behind the leading edge in hardware and are overpriced. They’ve also, in my opinion, lessened their GUI by making it more mobile-like.
Another thing that has always bothered me about Linux is the reliance on GNU software. Some of it is really well-written, but I think it is dated and some of the more important software is junk (GCC, GLibC). Initially, I had thought about building a Linux distribution, but using a different collection of base system software, some of which I imagined I’d write myself (the fun part of such a project). However, the state of compiling the Linux kernel without the GNU tools isn’t quite mature enough and that’s not a project on which I care to spend time. Lastly, Linus Torvalds himself has lamented the growing bloat of the kernel.
And that’s how I ended up here. I’m going to try my hand at building a small operating system, with no legacy cruft, using anything but GNU tools. That brings me to UEFI.
Modern systems have replaced the old system BIOS with the Unified Extensible Firmware Interface. This is almost an operating system in itself and makes some of the vagaries of bringing up a system a little more clean than in the past. Several well-defined software interfaces are provided for mapping hardware components, figuring out memory layout, loading kernels, showing startup screens and some other boring junk operating systems have to do to survive.
UEFI has actually been around a while, but the platforms I’ve seen still seem a little immature. A couple of the better ones are the implementations on my Asus Zenbook Pro and the two Intel NUCs I use for development (in fact, I bought the NUCs specifically for this project). Having downloaded a copy of the UEFI specification, I set out to build a Hello World UEFI application.
That’s when things got a little messy. I found out there are basically three options when it comes to building UEFI applications: Tianocore, which is Intel’s UEFI build environment, gnu-efi, and FASM which can natively produce EFI application binaries (crazily enough).
I tried Tianocore first—straight from the horse’s mouth as it were. The application built fine and it worked instantly. The problem is the labyrinthine build environment. They actually recommend sticking your code somewhere in the Tianocore build tree and modifying the build files to include your application code. Sorry, Intel, but I’m not going to change the layout of my source tree to what you think it should be. Actually, with the instructions I’ll provide shortly, you should be able to use Tianocore’s headers and libraries and dump their build system without too much trouble. If you care about correctness (which is generally probably good) and/or aren’t as mental about your build system as I am, then I would recommend going with that option. Also, Tianocore’s github repo has thorough instructions on setting it up on various platforms.
Disillusioned (more like repulsed) with Tianocore’s offering, I decided to hold my nose and try the gnu-efi way of doing things. There was a package available for it in the Arch repository, so getting it setup was easy enough. It also took minimal effort to build a working application. I was excited it was so much more straight-forward than the Tianocore mess, but I was still holding my nose. It wasn’t long before I realized gnu-efi’s build instructions had dumped an unreasonably large binary in my build directory. I couldn’t imagine what all those bytes were doing in a little Hello World EFI application. Later, I discovered the problem: the gnu-efi application provides binary startup objects and a library. I wasn’t about to build my application like that, using kludgey functions that end up in the code for relocation and dynamic resolving of relocatable symbols.
I decided I’d go with my own UEFI build environment using Clang and LLD as the compiler and linker and using my own code instead of Tianocore or gnu-efi project libraries and headers. As it turns out, I had to do considerable research before I could put all the bits together to make it work.
The first thing you need is a set of header files. I built my own, using the spec. I had a creepy feeling Tianocore and GNU would use their own versions of what a header should be, and I preferred to use something custom. My application would have no startup code or link-libraries. Everything used would be defined in the project itself and linked in if it’s used and rejected if not.
Next, I installed the build tools themselves: clang and lld. It was an easy install on Arch Linux and I had everything up and going without incident. Then it got hard. Clang had to produce Microsoft object files to be compatible with the way UEFI does things. I had to search down the options and fiddle with them until I got them working properly. LLVM itself has to be called as though it were Microsoft’s LINK. After working through all of these problems, I discovered that the output binary still isn’t a UEFI format binary. I had to write a utility to change the value of one byte in the EFI header to make everything work. Once I got everything together, I captured it in git where others could use it. Take special care to examine the clang, lld and petouefi commands in the makefile.
Notice the code I make available doesn’t use a runtime library or startup files. The loader will read it into memory and execute it directly. I prefer to access the various functions and structures in their raw form, without a library hiding the details from me. This gives me a more intimate knowledge of the library that I find useful. As I said earlier, this setup can be modified to use the official Tianocore headers. Indeed, the headers provided here are incomplete.
Finally, the goal with your c compiler is to build an MS object format output file for each of your compiled modules. Those object files will be linked with Clang’s linker in Microsoft LINK mode to generate a PE64 executable.
I’ve used this to successfully build on FASM without using FASM’s built-in PE format generator. I still haven’t quite gotten NASM to work. The binary is recognized and loads fine and, if it does nothing but terminate, it works great! I think there are some position independent code related issues. I investigated it briefly, but didn’t find a solution and had more pressing things to work on. I think NASM is very close to working and I may look into it in the future. For now, I am content to use FASM when necessary. If anyone reading this figures out the secret, I’d definitely be interested in knowing.
I hope this post will find others looking into getting into UEFI and offer them some help in how it fits together. Maybe it will even give enough information to help building UEFI apps on other systems or with other compilers. Good Luck!