While on a hike with colleagues, one asked a thought provoking question: Should the United States Post Service (USPS) operate as an Internet Service Provider (ISP)? This post is an opinion piece playing with that idea; here are my thoughts. It’s far from a hot take, and has been discussed before.

Article 1 Section 8 of the United States Constitution states: “The Congress shall have Power…To establish Post Offices and post Roads;”

If you’re new to the Arm ecosystem, consider this a quick primer on terms you likely have seen before but might have questions about.

The Arm architecture is a family of Reduced Instruction Set Architectures (RISC) with simple addressing modes. Data processing is done on register operands otherwise relying on loads and stores to move data into and out of registers.

Arm Limited, the British company, stewards the Arm architecture.

ARM is a legacy acronym for Acorn RISC Machine, then Advanced RISC Machines. As we’ll see, with new advancements in the architecture, previous terms for things sometimes get renamed.

The Open Source software development model is a powerful approach to collaboratively iterating on a shared solution to a common problem. Rather than competitors duplicating effort to come up with multiple suboptimal solutions, they can collaborate together to build something greater.

Forking is the term used to denote making a copy of a codebase. The copy, known as a “fork” is used as a starting point for work that may be contributed back to the original maintainers; the canonical “upstream” from which we derive our “downstream” clone. These copies share a similar origin point; think of all of the things in life that share a similar origin, but due to various factors have diverged slightly over time in assorted measures.

A maximal length sequence of branch-free code that terminates with a branch or jump is referred to as a basic block.

A basic block that branches to another forms an edge in the Control Flow Graph (CFG).

The initial basic block starting an edge is the predecessor; it precedes and is succeeded by the successor basic block.

An edge between basic blocks is considered a critical edge if the predecessor has multiple successors, and the successor has multiple predecessors.

Unless work is done per architecture to implement HAVE_ARCH_VMAP_STACK / CONFIG_VMAP_STACK, the Linux kernel defaults to two pages worth of stack per thread.

Note: on many contemporary systems the page size is 4KiB, but this is actually configurable for many architectures. The trade offs probably require a separate post. If you see code that checks for alignment via bitwise tricks like addr & 4095 == 0 without checking sysconf(_SC_PAGESIZE) it is perhaps a red flag for code that might be to reused on different systems.

“War stories” in programming are entertaining tales of truly evil bugs that kept you up at night. Inspired by posts like My Hardest Bug Ever, Debugging an evil Go runtime bug, and others from /r/TalesFromDebugging, I wanted to share with you one of my favorites from recent memory. Recent work has given me much fulfilment and a long list of truly awful bugs to recount. My blog has been quieter than I would have liked; hopefully I can find more time to document some of these, maybe in series form. May I present to you episode I; “Off by Two.”

TL;DR

Prefer f(void) in C to potentially save a 2B instruction per function call when targeting x86_64 as a micro-optimization. -Wstrict-prototypes can help. Doesn’t matter for C++.

The Problem

While messing around with some C code in godbolt Compiler Explorer, I kept noticing a particular funny case. It seemed with my small test cases that sometimes function calls would zero out the return register before calling a function that took no arguments, but other times not. Upon closer inspection, it seemed like a difference between function definitions, particularly f() vs f(void). For example, the following C code:

Support for a long awaited GNU C extension, asm goto, is in the midst of landing in Clang and LLVM. We want to make sure that we release a high quality implementation, so it’s important to test the new patches on real code and not just small test cases. When we hit compiler bugs in large source files, it can be tricky to find exactly what part of potentially large translation units are problematic. In this post, we’ll take a look at using C-Reduce, a multithreaded code bisection utility for C/C++, to help narrow done a reproducer for a real compiler bug (potentially; in a patch that was posted, and will be fixed before it can ship in production) from a real code base (the Linux kernel). It’s mostly a post to myself in the future, so that I can remind myself how to run C-reduce on the Linux kernel again, since this is now the third real compiler bug it’s helped me track down.

Typically, when we modify a program, we’d like to run it to verify our changes. Before booting a compiled Linux kernel image on actual hardware, it can save us time and potential headache to do a quick boot in a virtual machine like QEMU as a sanity check. If your kernel boots in QEMU, it’s not a guarantee it will boot on metal, but it is a quick assurance that the kernel image is not completely busted. Since I finally got this working, I figured I’d post the built up command line arguments (and error messages observed) for future travelers. Also, QEMU has more flags than virtually any other binary I’ve ever seen (other than a google3 binary; shots fired), and simply getting it to print to the terminal is 3/4 the battle. If I don’t write it down now, or lose my shell history, I’ll probably forget how to do this.

ccache, the compiler cache, is a fantastic way to speed up build times for C and C++ code that I previously recommended. Recently, I was playing around with trying to get it to speed up my Linux kernel builds, but wasn’t seeing any benefit. Usually when this happens with ccache, there’s something non-deterministic about the builds that prevents cache hits.

Turns out someone asked this exact question on the ccache mailing list back in 2014, and a teammate from my Android days supposed a timestamp was the culprit. That, and this LKML post from the KBUILD maintainer in 2011 about determinism helped me find commit 87c94bfb8ad35 (“kbuild: override build timestamp & version”) that introduced manually overriding part of the version string that contains the build timestamp that can be seen from: