Blog

See selected articles, grouped by topic. Feeds: Atom, RSS.

Async hazard: mmap is secretly blocking IO
21 Aug 2024
Memory mapping a file for reading sounds nice: turn inconvenient read calls and manual buffering into just simple indexing of a memory… but it does blocking IO under the hood, turn a &[u8] byte arrays into an async hazard and making “concurrent” async code actually run sequentially!
Keep reading
GitHub tip-hub: habitual permalinks
24 Jul 2024
GitHub offers permalinks to versions of files and lines, within a repository. They’re easy to create (y keyboard shortcut) and have some nifty affordances like displaying a preview, plus they don’t become invalid as code changes. Use them!
Keep reading
10 > 64, in QR codes
31 Mar 2024
Encoding data in decimal requires many more characters than the same data encoded in base64—06513249 vs YWJj—but using decimal is better when stored in a QR code. The magic of QR modes means all those extra digits are stored efficiently, almost as though there was no encoding at all. Decimal encoding makes for QR codes that store more data, or are easier to scan.
Keep reading
Mechanical sympathy for QR codes: making NSW check-in better
13 Oct 2021
Governments here in Australia have been telling us to keep distance from each other. Surprisingly, the same government has simultaneously put out posters that required people to get close, unnecessarily. They contain QR codes for contact-tracing check-ins that are small and dense, meaning they’re hard to scan. How could they be better?
Keep reading
QR error correction helps and hinders scanning
28 Sep 2021
Error correction sounds good. It means fewer errors, right? When it comes to QR codes, that’ll mean easier scanning for people, surely? It seems like that’s not the whole story.
Keep reading
The joy of cooking (an app)
30 Aug 2021
Taking shortcuts? Leaving edge-cases unconsidered and unhandled? Is this engineering?! My approach to programming and software engineering has been shaped by years of building open source compilers and libraries, where those edge cases matter, reliability is crucial and flexibility is important. It’s been a breath of fresh air to take a step back and stop thinking about every detail, and instead write code that works for a specific set of people: me and my wife.
Keep reading
Git worktrees and pyenv: developing Python libraries faster
15 Apr 2020
It’s glorious to work on one task at a time, to be able to take it from start to finish completely, and then move onto the next one. Sounds great, but reality is messier than that and context switches are common. Doing so can be annoying and inefficient, but not switching means tasks and colleagues are delayed. I combine git worktrees and pyenv (and plugins) to reduce the overhead of a context switch and keep development and collaboration as smooth as possible when working on a Python library.
Keep reading
Myths and Legends about Integer Overflow in Rust
29 Apr 2016
The primitive integer types supported by CPUs are finite approximations to the infinite set of integers we’re all used to. This approximation breaks down and some computations will give results that don’t match real integers, like 255_u8 + 1 == 0. Often, this mismatch is something the programmer didn’t think about, and thus can easily result in bugs.
Keep reading
Memory Leaks are Memory Safe
04 Apr 2016
Memory unsafety and memory leaks are arguably the two categories of bugs that have received the most attention for prevention and mitigation. As their names suggest, they are in the same part of “bug space”, however they are in some ways diametric opposites, and solving one does not solve the other. The widespread use of memory-safe managed languages hammers this point home: they avoid some memory unsafety by presenting a “leak everything” model to programmers.
Keep reading
Rreverrse Debugging
27 Oct 2015
Imagine being able to step forward and backwards as code runs in your debugger. Imagine being able to do an test run multiple times with exactly the same sequence of instructions and values, right down to memory addresses and IO. Imagine being able to run an executable thousands of times and then do all that in the one execution that triggers the rare bug that’s draining you of life…
Keep reading
simple_parallel 0.3: Revisiting k-NN
24 Oct 2015
I recently released version 0.3 of my simple_parallel crate, which builds on Aaron Turon’s crossbeam to resolve the stability and safety difficulties: the crate now works with Rust 1.3.0 stable, and offers safe data-parallel for loops and maps.
Keep reading
SIMD in Rust
24 Aug 2015
A new scheme for SIMD in Rust is available in the latest nightly compilers, fresh off the builders (get it while it’s hot!).
Keep reading
What is SIMD?
10 Jul 2015
I’m currently in San Francisco doing an internship at Mozilla Research, working on creating functionality for SIMD in the Rust programming language.
Keep reading
travis-cargo 0.1.3: --no-sudo
30 Jun 2015
I just pushed travis-cargo version 0.1.3, which adds a --no-sudo command to the coveralls and coverage subcommands to allow recording/uploading test coverage without needing sudo.
Keep reading
Announcing Primal: Putting Raw Power Into Prime Numbers
08 Jun 2015
Over the past few weeks I’ve been working on improving my slow_primes library, culminating in needing rename and the 0.2 release of primal, a Rust crate for computing properties of prime numbers with state-of-the-art algorithms, while still maintaining an idiomatic and easy-to-use interface.
Keep reading
Defaulting to Thread-Safety: Closures and Concurrency
26 May 2015
Rust has some powerful tricks to model properties of aggregate types via unsafe traits with default and negative implementations. These features motivated by offering flexible concurrency/parallelism, and allow powerful closure-based APIs without losing any thread-safety (or memory-safety) guarantees at all.
Keep reading
Rust 1.0 in Numbers
20 May 2015
Rust 1.0 was made with⁰…
1. I’ve tried to get the numbers that apply leading up to ↩
Keep reading
Finding Closure in Rust
08 May 2015
Have you ever used an iterator adapter in Rust? Called a method on Option? Spawned a thread? You’ve almost certainly used a closure. The design in Rust may seem a little complicated, but it slides right into Rust’s normal ownership model so let’s reinvent it from scratch.
Keep reading
Where Self Meets Sized: Revisiting Object Safety
06 May 2015
The concept of object safety in Rust was recently refined to be more flexible in an important way: the checks can be disabled for specific methods by using where clauses to restrict them to only work when Self: Sized.
Keep reading
Travis on the train, part 2
01 May 2015
After announcing travis-cargo a few days ago in Helping Travis catch the rustc train, I got some great hints/contributions from Jan Segre and had a fun little time automating code coverage collection via coveralls.io. Unfortunately, this is a breaking change for existing users of travis-cargo, but the migration is easy.
Keep reading
Helping Travis catch the rustc train
28 Apr 2015
I’ve been putting off configuring my continuous integration settings to match the Rust train model: it involves non-trivial branching on the configuration, and duplicating that over a pile of repos is not something I looked forward to. So, instead, I wrote travis-cargo to make things easier.
Keep reading
Little libraries
27 Apr 2015
I’ve been having a lot of fun recently solving “little” problems in Rust. I have a long term project to make something for displaying my (GPS-tagged) photos nicely and, along the way, I’ve discovered and filled in a few gaps by creating focused crates for small tasks.
Keep reading
Rust infrastructure can be your infrastructure
17 Mar 2015
Rust is a reasonably large project: the compiler and standard libraries are over 350kloc, built across nearly 40000 commits by the hands of around 900 contributors. Not only that: there are more than 30 other repositories in the rust-lang GitHub organisation that shouldn’t fall by the wayside, and, for rust-lang/rust alone, there are often more than 100 pull requests landing and a dozen new contributors in a single week.
Keep reading
Some notes on Send and Sync
20 Feb 2015
If you’ve been in the #rust-internals IRC channel recently, you may’ve caught a madman raving about how much they like Rust:
Keep reading
Rust Sydney's first meetup: trip report
14 Feb 2015
Rust Sydney had its first event last Monday! The first Rust gathering I know of in Sydney, in Australia, or in the whole southern hemisphere.
Keep reading
Object Safety
13 Jan 2015
A trait object in Rust⁰ can only be constructed out of traits that satisfy certain restrictions, which are collectively called “object safety”. This object safety can appear to be a needless restriction at first, I’ll try to give a deeper understanding into why it exists and related compiler behaviour.
1. [As usual][previouspost], this post is designed to reflect ↩
Keep reading
The Sized Trait
12 Jan 2015
An important piece in my story about trait objects in Rust⁰ is the Sized trait, so I’m slotting in this short post between my discussion of low-level details and the post on “object safety”.
1. Per the previous post, this post is ↩
Keep reading
Peeking inside Trait Objects
10 Jan 2015
One of the most powerful parts of the Rust programming language⁰ is the trait system. They form the basis of the generic system and polymorphic functions and types. There’s an interesting use of traits, as so-called “trait objects”, that allows for dynamic polymorphism and heterogeneous uses of types, which I’m going to look at in more detail over a short series of posts.
1. It’s generally good practice for Rust posts to mention ↩
Keep reading
crates.io crate graph
04 Jan 2015
Rust is a systems programming language that comes with an awesome package manager Cargo, which hooks into the crates.io registry as one of its possible sources of packages. The packages can have dependency relationships between each other, making the database into a natural directed graph.
Keep reading
What does Rust's “unsafe” mean?
24 Jul 2014
Rust is an in-development⁰ systems programming language with a strong focus on no-overhead memory safety. This is achieved through a powerful type system (with similarities to Haskell), and careful tracking of ownership and pointers, guaranteeing safety. However, this is too restrictive for a low-level systems language, an escape hatch is occasionally required. Enter the unsafe keyword.
1. The code in this post compiles with rustc 0.12.0-pre-nightly (aa0e35bc6 2014-07-22 00:26:21 +0000). ↩
Keep reading
Error handling in Rust: a k-NN case study
11 Jun 2014
After posting a Rust translation of some k-nearest neighbour code, I got a few comments asking “how would you handle errors if you wanted to?”. This is the perfect chance to briefly demonstrate a few idioms.
Keep reading
Comparing k-NN in Rust
10 Jun 2014
In my voyages around the internet, I came across a pair of blog posts which compare the implementation of a k-nearest neighbour (k-NN) classifier in F# and OCaml. I couldn’t resist writing the code into Rust to see how it fared.
Keep reading