This... is really helpful. I've been trying to contribute some drivers for Pine64 devices lately and am trying to get up to speed on Linux Kernel development. There's... a lot.
Does anyone know if this is up to date with current Linux best practices?
According to the github readme, the guide has been around since 2001 but it seems fairly up to date..
There are numerous discussion going on the issue tracker, about adding rust chapter, targeting linux 6.1, etc.. Very cool to see github issues and pull request actively used to keep a guide up to date.
The guide will walk you through the basic of a kernel module. Exposing virtual files under /proc or /sys or messing around with system calls. It will fall short of helping you with the nitty gritty of writing an hardware driver though.
The problem with writing drivers is that you are interfacing with a real-world machine for which you will have very little information about.
Pick a device similar to the one you want to support, but that is already supported in linux. Go look at its source code. This will tell you far more about the work involved. You might realize that there is simply not enough information available to support the driver.
No, it's not uptodate. But it's probably good enough to learn about basic concepts. For the devices you're talking about, you'll need to learn about device trees (didn't exist in 2001 at all in Linux), power management, etc.
I've been reading up on DTS stuff already, there's some resources out there, but all of the "kernel developer guides" seem to predate that by a fair bit. I suppose I'll have to familiarize myself with U-Boot as well...
U-Boot has a sort of baby device model implementation. :) Simpler and perhaps easier to grok than Linux, but using the identical DT based HW representation Linux uses later on during boot (it may even modify DTB a bit in memory to pass some information to kernel).
OTOH, depending on platform it may mix this device model with some ad-hoc/hardcoded knowledge about the platform and ifdef mess[1], that is otherwise properly described by DT in Linux. So it can be a mixed bag.
Android Binder Driver: This project is an effort to rewrite Android's Binder kernel driver in Rust.
Motivation: Binder is one of the most security and performance critical components of Android. Android isolates apps from each other and the system by assigning each app a unique user ID (UID). This is called "application sandboxing", and is a fundamental tenet of the Android Platform Security Model.
The majority of inter-process communication (IPC) on Android goes through Binder. Thus, memory unsafety vulnerabilities are especially critical when they happen in the Binder driver
[...] We also need designs that allow code in the two languages to interact with each other: we're particularly interested in safe, zero-cost abstractions that allow Rust code to use kernel functionality written in C, and how to implement functionality in idiomatic Rust that can be called seamlessly from the C portions of the kernel.
Since Rust is a new language for the kernel, we also have the opportunity to enforce best practices in terms of documentation and uniformity. For example, we have specific machine-checked requirements around the usage of unsafe code: for every unsafe function, the developer must document the requirements that need to be satisfied by callers to ensure that its usage is safe; additionally, for every call to unsafe functions (or usage of unsafe constructs like dereferencing a raw pointer), the developer must document the justification for why it is safe to do so.
We'll now show how such a driver would be implemented in Rust, contrasting it with a C implementation. [...]
There aren't really any non-trivial mainline modules, since the Rust support is so new. There's the non-mainline Asahi M1 GPU driver though! It will eventually be mainlined, but IIRC some more Rust support code needs to be mainlined first.
On Android the Linux kernel is its own thing, and after Project Treble, it follows a microkernel like approach to drivers, where standard Linux drivers are considered "legacy" since Android 8.
In your opinion, do you think that the microkernel approach is more secure? (Should processes run as separate users with separate SELinux contexts like Android 4.4+)
Why do you think that the Android binder module rust implementation is listed as an example of a Rust for Linux kernel module on the site?
The Android open-source project "AOSP" with its latest code is very close to being able to boot off the mainline Linux kernel when assuming the device drivers are all upstream.
Of course microkernel approach is more secure, if a driver gets p0wned, corrupts data structures, or plain keeps crashing, it doesn't take the whole kernel with it.
Naturally the issue might be as bad that the whole stack can't recover from, but still much better than corrupting the kernel.
One of the SecDevOps guidelines when hardening servers is that every process should have its own user, yes.
Kmods and akmods run in kernel memory space and aren't ABI stable.
Binder is not a HAL, since binder is how HALs communicate with each other. It's an actual proper Linux driver. The C version is in the upstream kernel as a module you can enable when building the kernel.
IIRC, when I tried to install LEGO Boost app (before I found pybricks and wokwi simulator) on Waydroid, I had trouble patching Bluetooth BLE on the host through to the waydroid container; due to device virtualization less than modules fwiu
I fear "Project Treble" may reduce incentives for vendors to upstream their drivers. Are there efforts to mainline it or the drivers currently using it?
Followed by GSI (Generic System Images), where AOSP can be plugged alongside modular component packages, thus allowing OEMs a more Lego like experience to creating custom Android builds
That's pretty tidy. Is there any noticeable overhead in listening to all kernel sockets and hooking in your code when it matches a packet filter, compared to just building it into the kernel?
It's pretty damm fast. Everything is JITed and there's codegen for the "trampoline" logic so that there aren't even indirect branches to get to the program code.
The "overhead" comes from the fact that writing nontrivial BPF programs is quite hard. Getting programs to pass the verifier can become a bit of an art. This does tend to get easier if your production environment has few and recent kernel versions.
And then of course this "overhead" totally pays for itself in terms of reliability. It's really hard to accidentally break machines with BPF, while a kernel module in your prod environment is a time bomb :D
(Deliberately breaking machines with BPF is very easy though. This sometimes seems to surprise people but if your attacker gets CAP_BPF it's probably game over).
Nevertheless, the OP is a classic guide and worth a read.
For anyone curious why a book published in 2023 is being called a "classic", it appears to be a republication (possible update) of https://tldp.org/LDP/lkmpg/2.6/lkmpg.pdf from 2001
There are 2 new authors listed so I'm assuming the content has been refreshed
I've probably written or dabbled with kernel modules three times in my life: first in college (~2010), second for a take-home interview question (~2015), and third as part of my job duties (~2020), and every time I do so I find that the APIs/best practices have changed. So I really hope there's a definitive guide that stays up to date until at least ~2025 when I next need it.
This... is really helpful. I've been trying to contribute some drivers for Pine64 devices lately and am trying to get up to speed on Linux Kernel development. There's... a lot.
Does anyone know if this is up to date with current Linux best practices?
According to the github readme, the guide has been around since 2001 but it seems fairly up to date..
There are numerous discussion going on the issue tracker, about adding rust chapter, targeting linux 6.1, etc.. Very cool to see github issues and pull request actively used to keep a guide up to date.
The guide will walk you through the basic of a kernel module. Exposing virtual files under /proc or /sys or messing around with system calls. It will fall short of helping you with the nitty gritty of writing an hardware driver though.
The problem with writing drivers is that you are interfacing with a real-world machine for which you will have very little information about.
Pick a device similar to the one you want to support, but that is already supported in linux. Go look at its source code. This will tell you far more about the work involved. You might realize that there is simply not enough information available to support the driver.
No, it's not uptodate. But it's probably good enough to learn about basic concepts. For the devices you're talking about, you'll need to learn about device trees (didn't exist in 2001 at all in Linux), power management, etc.
I've been reading up on DTS stuff already, there's some resources out there, but all of the "kernel developer guides" seem to predate that by a fair bit. I suppose I'll have to familiarize myself with U-Boot as well...
U-Boot has a sort of baby device model implementation. :) Simpler and perhaps easier to grok than Linux, but using the identical DT based HW representation Linux uses later on during boot (it may even modify DTB a bit in memory to pass some information to kernel).
OTOH, depending on platform it may mix this device model with some ad-hoc/hardcoded knowledge about the platform and ifdef mess[1], that is otherwise properly described by DT in Linux. So it can be a mixed bag.
[1] eg. https://xff.cz/git/u-boot/tree/board/sunxi/board.c
Fantastic resource. Thanks for sharing.
I always liked the Hitchhikers Guide to Kernel module programming, with DONT PANIC on the cover.
Previous discussions:
https://news.ycombinator.com/item?id=28036619 (154 points | Aug 2, 2021 | 30 comments)
https://news.ycombinator.com/item?id=28283030 (339 points | Aug 23, 2021 | 42 comments)
See also: https://0xax.gitbooks.io/linux-insides/content/ (A book-in-progress about the linux kernel and its insides)
Ctrl-F "rust"
https://rust-for-linux.com/ links to LWN articles at https://lwn.net/Kernel/Index/#Development_tools-Rust that suggest that only basic modules are yet possible with the rust support in Linux kernels 6.2 and 6.3.
Rust-for-linux links to the Android binder module though: https://rust-for-linux.com/Android-Binder-Driver.html :
... "Rust in the Linux kernel" (2021) https://security.googleblog.com/2021/04/rust-in-linux-kernel... :
Is this the source for the rust port of the Android binder kernel module?: https://android.googlesource.com/platform/frameworks/native/...
This guide with unsafe rust that calls into the C, and then with next gen much safer rust right next to it would be a helpful resource too.
What of the post-docker container support (with userspaces also written in go) should be cloned to rust first?
What are some good examples of non-trivial Linux kernel modules written in Rust?
There aren't really any non-trivial mainline modules, since the Rust support is so new. There's the non-mainline Asahi M1 GPU driver though! It will eventually be mainlined, but IIRC some more Rust support code needs to be mainlined first.
https://github.com/AsahiLinux/linux/tree/asahi/drivers/gpu/d...
On Android the Linux kernel is its own thing, and after Project Treble, it follows a microkernel like approach to drivers, where standard Linux drivers are considered "legacy" since Android 8.
https://source.android.com/docs/core/architecture/hal
Don't use how Linux kernel does things on Android as understanding from how upstream works.
In your opinion, do you think that the microkernel approach is more secure? (Should processes run as separate users with separate SELinux contexts like Android 4.4+)
Why do you think that the Android binder module rust implementation is listed as an example of a Rust for Linux kernel module on the site?
"Android AOSP Can Boot Off Mainline Linux 5.9 With Just One Patch" (2020) https://www.phoronix.com/news/Android-AOSP-Close-Linux-5.9 :
Other distros support kmods and akmods: https://www.reddit.com/r/PINE64official/comments/ijfbgl/comm... :
Of course microkernel approach is more secure, if a driver gets p0wned, corrupts data structures, or plain keeps crashing, it doesn't take the whole kernel with it.
Naturally the issue might be as bad that the whole stack can't recover from, but still much better than corrupting the kernel.
One of the SecDevOps guidelines when hardening servers is that every process should have its own user, yes.
Kmods and akmods run in kernel memory space and aren't ABI stable.
Binder is not a HAL, since binder is how HALs communicate with each other. It's an actual proper Linux driver. The C version is in the upstream kernel as a module you can enable when building the kernel.
It doesn't change the fact that since Project Treble introduction, and Android 8, modern drivers live on their own processes.
Waydroid (Android in containers) requires binder and optionally ashmem, though ashmem is not required anymore because memfd works with vanilla kernel: https://wiki.archlinux.org/title/Waydroid#Kernel_Modules
There is a Google Play Certification process for waydroid devices: https://docs.waydro.id/faq/google-play-certification
(That manual provisioning step is not necessary for e.g. optional widevine DRM on other OSes)
IIRC, when I tried to install LEGO Boost app (before I found pybricks and wokwi simulator) on Waydroid, I had trouble patching Bluetooth BLE on the host through to the waydroid container; due to device virtualization less than modules fwiu
I fear "Project Treble" may reduce incentives for vendors to upstream their drivers. Are there efforts to mainline it or the drivers currently using it?
The goal was to provide a stable ABI like non Linux OSes, with all the plus and minus it entails, not to upstream stuff.
Project Treble was only the begining, besides none of the PlayStore acceptance requirements forces the OEMs to update the drives.
Project Mainline followed Project Treble, where Android was further modularized.
https://source.android.com/docs/core/ota/modular-system
Followed by GSI (Generic System Images), where AOSP can be plugged alongside modular component packages, thus allowing OEMs a more Lego like experience to creating custom Android builds
https://source.android.com/docs/setup/create/gsi
Many things that you may have wanted to do with a kernel module can now be done more easily and safely with eBPF.
https://ebpf.io/get-started/
Nevertheless, the OP is a classic guide and worth a read.
That's pretty tidy. Is there any noticeable overhead in listening to all kernel sockets and hooking in your code when it matches a packet filter, compared to just building it into the kernel?
It's pretty damm fast. Everything is JITed and there's codegen for the "trampoline" logic so that there aren't even indirect branches to get to the program code.
The "overhead" comes from the fact that writing nontrivial BPF programs is quite hard. Getting programs to pass the verifier can become a bit of an art. This does tend to get easier if your production environment has few and recent kernel versions.
And then of course this "overhead" totally pays for itself in terms of reliability. It's really hard to accidentally break machines with BPF, while a kernel module in your prod environment is a time bomb :D
(Deliberately breaking machines with BPF is very easy though. This sometimes seems to surprise people but if your attacker gets CAP_BPF it's probably game over).
For anyone curious why a book published in 2023 is being called a "classic", it appears to be a republication (possible update) of https://tldp.org/LDP/lkmpg/2.6/lkmpg.pdf from 2001
There are 2 new authors listed so I'm assuming the content has been refreshed
I've probably written or dabbled with kernel modules three times in my life: first in college (~2010), second for a take-home interview question (~2015), and third as part of my job duties (~2020), and every time I do so I find that the APIs/best practices have changed. So I really hope there's a definitive guide that stays up to date until at least ~2025 when I next need it.
This one is also good.
https://linux-kernel-labs.github.io/refs/heads/master/so2/in...
Neat! will this finally replace the outdated Linux Device Drivers Third Edition as the de-facto standard reference book for building kernel modules?
https://lwn.net/Kernel/LDD3/
This is really helpful. Beyond that its open LaTeX format can be used for documentation, which I will do for my own work.