It sounds like a cool project and I hope it continues development, but there is such a huge graveyard of such projects that have never gone anywhere that I struggle to get excited about them anymore. Replacing Linux is really hard, even for specific uses like cloud.
You think BSDs are next-in-line? Why is that?
I have the distinct impression BSDs will remain niche. Something radical (like MotorOS?) seems more likely.
If I'm in the mood for being a little glib... where do you think that Linux distribution that SteamOS is based on got its user-space (or more seriously, its drivers?)
Also: the SteamDeck is by any reasonable standard a console. It just happens to run a windowed environment out of the box. Don't be that guy. If you want to pump Valve, focus instead on their contributions to the Wine project.
No idea why the personal involvement with valve pumping has come from. Atari "VCS All-In Bundle" is debian based, too. They are not playstation popular, of course. Like mentioned "no love for consoles", dont own one, not interested, either.
And thanks to the BSD license, the project is getting zero back from Sony, while those Playstation profits get a big higher thanks to less R&D costs spent on OS code.
Same applies to clang/LLVM port to the Playstation, regarding everything that would expose console implementation details without an NDA.
It is probably for another product but there is at least one Sony email address, and a number email addresses from different corporate entities, on the FreeBSD contributors list. All 3 majors BSD OSes also list donators (can be financially or hardware).
So saying corps that use BSD code never give anything back because of license is not true. And an awful lot don't do any more or even hide their use of gpl licensed code anyway.
When, say, IBM contributed a lot of stuff into the Linux kernel in early 2000s, that stuff became immediately available to anyone. Whatever cool stuff Nintendo or Sony may introduce in their versions of BSD kernels, we don't even know, let alone seeing them contribute it back.
GPL works similarly to a patent pool: every participant sees that openly contributing to the pool is more profitable than being a renegade, as long as everyone else plays by the rules, too. MIT/BSD, while as open as possible, can easily promote a trade-secret type of environment, where any enhancements are never heard of, except under an NDA, and perish if their creators go under or lose interest.
but with a huge chunk of user space borrowed from FreeBSD
Any sources on this, and on what parts were borrowed specifically?
I was under the impression that Nintendo did away with most of the Unix layers we know and love and went all-in on custom code and APIs, is that not the case?
This is pretty much a myth. Both run a different kernel. macOS used be known to have a network stack (and maybe some stuff like a virtual file system) from freebsd but I am pretty sure most of the code has been replaced by now.
Having some BSD userland binaries doesn't make your OS a BSD. Otherwise Windows is just a fork of curl.
Last I checked, Darwin sources for tcp still look a lot like FreeBSD circa 2000 plus some Apple patches (MPTCP). No syncookies in 2024, because FreeBSD added those months after Apple forked the stack.
I argue that having a BSD license (in Darwin), BSD heritage (NeXTSTEP, FreeBSD, briefly NetBSD), and a mostly BSD userland 20+ years into the project makes this OS a BSD.
What does "Rust-first" mean here? It means not only that both the (micro) kernel and the drivers are implemented in Rust, but also that Rust is the first (and only, at the moment) language that userspace programs can be written in.
Although technically one can reverse-engineer the Rust-based ABI and the provided Rust toolchain to write apps for Motor OS in e.g. C, that is some work. But standard Rust programs (using standard Rust library, without FFI) will just compile and run - see e.g. https://github.com/moturus/motor-os/tree/main/src/bin/httpd.
This Rust-first approach is rather unique, as e.g. Redox uses relibc and C-based kernel ABI as the glue...
Unstable, undocumented, compiler-(version-)dependent ABIs are the reason Haiku for x86-32 still has to use GCC 2.95.
I’m never not going to welcome a hobby OS, especially in an interesting language (still have fond memories of House). By all means, go for it. Just... it’s important to be aware of the prior art here.
Haiku was stuck on old GCC only because they wanted to support old BeOS binaries? But binary backwards/forwards compatibility is not requirement for many use-cases.
I was skeptical at first- the healthy approach to any new tech. But thinking again, the efficiency and security gains from stripping away layers of cruft after rather compelling
One thing I keep hoping to see in all of these kernels in Rust is an async first kernel. Is there something that makes this particularly difficult or do folks not see the value in it? I know from following along with Phil Oppermann’s OS in Rust series that is definitely possible, but these last few OS’ in Rust seem to not be attempting this, https://os.phil-opp.com/async-await/
It would help if async Rust was actually a fully done feature, instead of half way there.
As Niko Matsakis puts it, async/await is Rust in hard mode, you don't need that when having to also worry about everything writing an OS from scratch entails.
Yes, there might be some significant gaps that make kernel development much more difficult. Off the top of my head, the current state of async traits is still being fleshed out, but it’s coming. The async_trait macros rely on boxing, so I could see that as a big downside in the kernel space. And I know that we don’t yet have an async streaming interface stabilized in the std lib.
I’m not sure if these are blocking issues or not, thus my question.
async trait macros are no longer needed for the majority of use cases as of 1.75. additionally, I doubt a hobby OS will hit bottlenecks where boxing like this is a problem. Async streaming not being standardized as an interface would only matter if the kernel wanted to use libraries, and I would assume most kernels will be light on external dependencies so it shouldn't matter.
If you are targeting a posix syscall interface, a lot of that doesn't really require async to pull off. You need to switch threads whenever an operation is blocked rather than run unrelated work on that thread. I suppose you can still model that with async, but it certainly doesn't feel as helpful.
In theory, it would simplify concurrency primitives in the kernel space, especially on the main thread, which is something I’ve notice is not an area that a lot of kernels explore.
Maybe I'm misunderstanding your proposal, but it seems to undermine process isolation. If you trust processes to yield under a co-operative multitasking system, why not trust them to leave each other's memory alone?
I was thinking of the internal Kernel tasks that might block, where it might be possible to have things happen in parallel, like bringing various hardware components online. See the sibling discussion on 200ms startup times.
Linux is definitely complex. Linux's ABI stability (good!) also means we cannot really blame C / glibc so much, it has internalized that problem.
I agree the goal posts make much of this complexity inevitable, but the solution is to stop trying to intersect everyone's objectives into a much narrower and daunting goal.
The big galaxy-brained goal should be:
1. Make more of Linux Rust.
2. Leverage Rust's superior expressive power to switch from a "configuration" mindset (Linux's current gazillion options on single giant code base) to a "composition mindset": we want to use types to combine various subsystems in various flavors.
3. Something like this and Linux proper can share lots of library code / subsystems. We can blur the line between "separate kernels" and "separate 'distrobutions' mixing and matching stand-alone Linux components".
I think this is the only sustainable way to reign in the accidental complexity.
Sounds interesting, but it also reminds me of what Linus once said when asked about fearing competition.
From my memory his answer was something like: I really like writing device drivers. Few people like that and until someone young and hungry comes along who likes that I'm not afraid of competition.
Is this really non-C-compatible? Seems like it just needs a libc layer written in rust. Most C programs don't make syscalls directly anyways (on some OSes they aren't even able to). The rest of the question is definitely fair though.
According to The American Heritage Dictionary an operating system is:
"Software designed to control the hardware of a specific data-processing system in order to allow users and application programs to make use of it."
The question is how much Operating System is a software that delegates all "control the hardware" parts to the layer below.
I know there are wider definitions of OS, but my point is this is not going to replace Linux.
Without question projects like MotorOS are sill useful. Besides practical applications it's a nice idea. Just recently I wrote in another thread that I would love to have a glimpse into an alternative universe, where Pascal had won over C and everything was Pascal-based. The idea to have everything Rust-based is even more exciting.
From the description, this aims to recreate that but forgo the entirety of the linux+docker stack needed to actually be able to do FROM scratch in the first place
For work, I run a rust service on GCP. It could run on this instead of docker on Linux.
I'm going to assume without looking at this project that observability and operability would be worse than it already is though, so I'm not in a hurry to move. Anyway, GCP means cost of compute is a rounding error compared to cost of bandwidth, so I have no reason to find the edge of efficiency.
I don't think so. There is no stable interface to the drivers which makes their reuse in other projects a huge effort. With Linux you either get the whole kernel
or nothing and the whole thing is more than a HAL or BIOS.
I don't think drivers are what warrants Linux its popularity / indispensability. I worked in a few places where people, well, at least moderately, liked to write drivers. Also, consider that it would've been definitely possible to write an OS with a mechanism for module loading compatible with Linux drivers.
My take on this question would be the ease of use for application developers. This ease of use consists of plenty of somewhat ready-to-use libraries that cover plenty of use-cases, huge community that both produces documentation and will answer questions should you have any, multiple hardware vendor support, and of course, licensing.
It's possible to compete with Linux in very special, very narrow use-cases, but trying to win against Linux on every front would require an insane amount of effort by a very large group of people.
"Also, consider that it would've been definitely possible to write an OS with a mechanism for module loading compatible with Linux drivers."
Sure, you can do that and it will work exactly until the next change in the driver interface, which is not very long.
So you end up in maintenance hell and will still constantly be behind the curve, because Linux gets all the updates first. You will be less secure too, because you'll get the security patches later.
That being said, someone in a similar thread from a few days ago said one of the BSDs does that. Maybe it's not that bad.
// The scheduler.
//
// As the kernel supports wait/wake/swap, and no blocking in the kernel
// other than wait, any kind of more sophisticated scheduling policy
// can be implemented in the userspace (in theory; maybe tweaks are needed
// to make things perform well in practive).
//
// So the default scheduler here is rather simple, but should work
// for a lot of use cases.
//
// Priorities: see enum Priority
// Cpu affinity: a single CPU hint
// Sched groups: none: everything within a priority is round robin, so
// a process with many threads will negatively affect a process with few threads.
//
// This should be OK for most situations: if there is a need for isolation,
// just use another VM.
Out of curiousity, why would a small kernel take a whole 200ms to start on a modern computer? Wouldn't it need to initialize some metadata for the memory pages, mount the filesystem, and try to launch an init process? I suppose there might be an ethernet driver and possibly something to pipe logs to ("stdout" for the VM) to initialize. Shouldn't that all take a few microseconds?
Or is all the slowness in the host preparing the resources? (as in QEMU and KVM?)
In general, lots of hardware requires long waits to initialize. As in, program a value then you must wait 50 ms before querying the registers or the hardware will give you non-deterministic garbage with no error indication.
So, it depends on the hardware, though you are unlikely to need more than 1 second on anything other than truly degenerate hardware assuming you are initializing devices in parallel.
I pass through a GPU and USB hub to a VM running on a machine in the garage. An optical video cable and network compatible USB extender brings the interface to a different room making it my primary “desktop” computer (and an outdated laptop as a backup device). Doesn’t get more silent and cool than this. Another VM on the garage machine gets a bunch of hard drives passed through to it.
That said, hardware passthrough/VFIO is likely out of the current realistic scope for this project. VM boot times can be optimized if you never look for hardware to initialize in the first place. Though they are still likely initializing a network interface of some sort.
Many years ago I contributed to IncludeOS and that thing could boot a vm in a millisecond under the right circumstances, allowing for things like per request vm isolation. So I suspects it's a mix of hardware circumstances and the way the microkernel is implemented.
It could be cool if there was a WASM container built in Rust that could run in this OS. I don't really have a sense of how complicated a WASM container is, though it seems non-trivial to even decide what such a thing is, so it would be nice not to treat the WASM container itself as the OS (i.e., more room to safely experiment). WASM also seems like it dodges the ABI issue by being more explicitly about composition instead of shared binary data structures.
Yeah, just that sort of thing. I mean, heck, can you put #!wasmtime at the top of a wasm file? Would such a file literally run right now (assuming #! is supported, and wasmtime doesn't hit any MotorOS Rust limitations)?
I've seen this stated in a couple places, but IIRC the kernel is C++ and the UI is Dart. I assume lots of things in between could be in rust, but do you know which? daemons, services, drivers, window server?
...Docker, Nix OS, "serverless", etc. all exist because of Linux's complexity
Yeah, this seems like it's more directly competing with those than Linux. I'd want to see those addressed in the "Why?" -- that is, why MotorOs instead of Docker, etc.?
Motūrus OS is a microkernel-based operating system
By and large, microkernel and containers solve the same problem. Except, I wouldn't call containers a "solution", more like a workaround. Not in the sense that containers by themselves are a workaround, but the way they are used is a workaround for the same problem.
The way containers are used today, especially in the context of Kubernetes, is to finely slice the available physical resources. The orchestration allows to manage which part of the application gets what slice of the resource, thus attempting to cut down on waste that's typically generated in the world where resources are managed through VMs. Where the typical problem is that a single VM will require too many resources because of the bloated OS it needs to run and because it's hard to create VMs with very limited resources, since OSes usually come as a package deal.
So, containers "solve" the problem by giving up VM optimization -- instead it's usually beneficial to create very beefy VMs, on top of which then a new virtualization layer is created with containers. This minimized the VM waste, but doesn't get rid of it entirely, and, of course, creates a lot of complications with all the indirection resulting from two-tiered virtualization.
Microkernel is the opposite of this "solution" -- it's the attempt to make OSes more modular, and as such less demanding of resources. Ideally, in the world if microkernels you don't need containers (at least not for the thing they are usually used today): your VMs can slice the resources in the way at least as efficient as containers do (or, hopefully, even better).
So... to predict the next possible question: why containers are so popular and micorkernels aren't: it's because the later is harder on the applications (even when applications don't actually need something, they often use it because it's available in a full-blown OS, applications aren't usually written with resource scarcity in mind). Secondly, containers, essentially, exist on top of somewhat uniform, somewhat stable interface of Linux kernel. Micorkernel VMs would expose users to the zoo of ideas hardware vendors put into their products, making portability difficult. Finally, for all its flaws, Kubernetes is a big system with many (even if not so great) solutions for many problems. So, programmers who fear technology feel like it gives them a safety net and will allow them to program from a more comfortable position of modifying YAML files, copying the most upvoted answers from StackOverflow. There won't be such an easy cake in microkernel world.
Docker, Nix OS, "serverless", etc. all exist because of Linux's complexity
Docker and NixOS exist because of userspace problems with package management and serverless exists because businesses want to pay for compute on demand.
I'm the project author/dev. Thanks a lot for posting this, and for comments/discussions!
I see two main concerns raised here:
(a) long-term viability and support
(b) compilers, binary compatibility, etc.
While the first concern is definitely valid, and without a community this project will not succeed, I do believe that potential benefits of Motor OS (or a similarly focused/structured project) will eventually result in a widely used new operating system. There are major problems with Linux inside VMs (and sometimes outside), and the Linux devs are not focused on this enough to clean things up anytime soon. I work on Linux Kernel at my day job, I know.
Re: compiler instability, binary compatibility, etc.: I'm sorry, I don't understand what is the issue here. The latest Linux kernel can be compiled with different GCC or LLVM toolchains on x86_64, and the result will happily run old binaries compiled years ago with who knows what. repr(C) structs in rust are stable... So why so many concerns here?
Again, thank you all for your comments and questions - I'm happy to answer more (at least until my day job kicks in).
The lainding page at https://github.com/moturus/motor-os explicitly says that both networking and file I/O are slow and have to be improved. The only claim is about fast bootup, and the number is there, and can easily be verified.
Where do you see any unsupported claims re: performance?
"Maestro: A Linux-compatible kernel in Rust" (2023) https://news.ycombinator.com/item?id=38852360#38857185 ; redox-os, cosmic-de , Motūrus OS; MotorOS
It sounds like a cool project and I hope it continues development, but there is such a huge graveyard of such projects that have never gone anywhere that I struggle to get excited about them anymore. Replacing Linux is really hard, even for specific uses like cloud.
Replacing Linux is hard even for the BSDs, and those are very well established and considered basically next-in-line for similar tasks.
You think BSDs are next-in-line? Why is that? I have the distinct impression BSDs will remain niche. Something radical (like MotorOS?) seems more likely.
What do you think is running on your favorite game consoles (that aren't Xbox)? I'll give you a hint, it isn't Linux.
Nintendo is using a custom OS but with a huge chunk of user space borrowed from FreeBSD. Sony on the other hand just went and forked FreeBSD outright.
You might also want to look into what OS are being used for server environments. A lot more BSD there than you might have initially guessed.
unless it's a steamDeck, ofc. No love for consoles, though.
If I'm in the mood for being a little glib... where do you think that Linux distribution that SteamOS is based on got its user-space (or more seriously, its drivers?)
Also: the SteamDeck is by any reasonable standard a console. It just happens to run a windowed environment out of the box. Don't be that guy. If you want to pump Valve, focus instead on their contributions to the Wine project.
No idea why the personal involvement with valve pumping has come from. Atari "VCS All-In Bundle" is debian based, too. They are not playstation popular, of course. Like mentioned "no love for consoles", dont own one, not interested, either.
Are you saying that Arch gets its userspace and drivers from FreeBSD?
I guess there is some BSD software that is common in Linux, like OpenSSH tools and dhcpcd. But that's not unique to Arch.
And thanks to the BSD license, the project is getting zero back from Sony, while those Playstation profits get a big higher thanks to less R&D costs spent on OS code.
Same applies to clang/LLVM port to the Playstation, regarding everything that would expose console implementation details without an NDA.
It is probably for another product but there is at least one Sony email address, and a number email addresses from different corporate entities, on the FreeBSD contributors list. All 3 majors BSD OSes also list donators (can be financially or hardware).
So saying corps that use BSD code never give anything back because of license is not true. And an awful lot don't do any more or even hide their use of gpl licensed code anyway.
When, say, IBM contributed a lot of stuff into the Linux kernel in early 2000s, that stuff became immediately available to anyone. Whatever cool stuff Nintendo or Sony may introduce in their versions of BSD kernels, we don't even know, let alone seeing them contribute it back.
GPL works similarly to a patent pool: every participant sees that openly contributing to the pool is more profitable than being a renegade, as long as everyone else plays by the rules, too. MIT/BSD, while as open as possible, can easily promote a trade-secret type of environment, where any enhancements are never heard of, except under an NDA, and perish if their creators go under or lose interest.
Any sources on this, and on what parts were borrowed specifically?
I was under the impression that Nintendo did away with most of the Unix layers we know and love and went all-in on custom code and APIs, is that not the case?
The network stack is taken from BSD, but then again so was the network stack on Windows 2000/XP.
macOS and iOS are BSDs. Pretty good niche!
This is pretty much a myth. Both run a different kernel. macOS used be known to have a network stack (and maybe some stuff like a virtual file system) from freebsd but I am pretty sure most of the code has been replaced by now.
Having some BSD userland binaries doesn't make your OS a BSD. Otherwise Windows is just a fork of curl.
Last I checked, Darwin sources for tcp still look a lot like FreeBSD circa 2000 plus some Apple patches (MPTCP). No syncookies in 2024, because FreeBSD added those months after Apple forked the stack.
I argue that having a BSD license (in Darwin), BSD heritage (NeXTSTEP, FreeBSD, briefly NetBSD), and a mostly BSD userland 20+ years into the project makes this OS a BSD.
Darwin is not Mac OS X and vice-versa.
Maybe the BSD are too close to Linux to displace it.
It's not like the myriads of these hobby OSes (written in Rust or otherwise) are any different. They're all Unix clones to various degrees.
The author wrote this on reddit:
https://old.reddit.com/r/rust/comments/190znk5/mot%C5%ABrus_...
Shouldn't be much effort to make it compatible with Crablang, no?
/s
Unstable, undocumented, compiler-(version-)dependent ABIs are the reason Haiku for x86-32 still has to use GCC 2.95.
I’m never not going to welcome a hobby OS, especially in an interesting language (still have fond memories of House). By all means, go for it. Just... it’s important to be aware of the prior art here.
Haiku was stuck on old GCC only because they wanted to support old BeOS binaries? But binary backwards/forwards compatibility is not requirement for many use-cases.
Haiku without the BeOS applications wouldn't have that much to show.
This is the way to go on platforms that favour shipping binaries instead of source code.
Outside Linux/BSD, stable ABI across OS versions is a concern, hence why Apple, Google and Microsoft platforms all offer ways to keep this going.
I thought the Rust ABI wasn't even stable?
I was skeptical at first- the healthy approach to any new tech. But thinking again, the efficiency and security gains from stripping away layers of cruft after rather compelling
One thing I keep hoping to see in all of these kernels in Rust is an async first kernel. Is there something that makes this particularly difficult or do folks not see the value in it? I know from following along with Phil Oppermann’s OS in Rust series that is definitely possible, but these last few OS’ in Rust seem to not be attempting this, https://os.phil-opp.com/async-await/
There are so many posts about how async Rust isn't the right abstraction it would surprise me if someone doubled-down on it right now.
It would help if async Rust was actually a fully done feature, instead of half way there.
As Niko Matsakis puts it, async/await is Rust in hard mode, you don't need that when having to also worry about everything writing an OS from scratch entails.
Yes, there might be some significant gaps that make kernel development much more difficult. Off the top of my head, the current state of async traits is still being fleshed out, but it’s coming. The async_trait macros rely on boxing, so I could see that as a big downside in the kernel space. And I know that we don’t yet have an async streaming interface stabilized in the std lib.
I’m not sure if these are blocking issues or not, thus my question.
async trait macros are no longer needed for the majority of use cases as of 1.75. additionally, I doubt a hobby OS will hit bottlenecks where boxing like this is a problem. Async streaming not being standardized as an interface would only matter if the kernel wanted to use libraries, and I would assume most kernels will be light on external dependencies so it shouldn't matter.
If you are targeting a posix syscall interface, a lot of that doesn't really require async to pull off. You need to switch threads whenever an operation is blocked rather than run unrelated work on that thread. I suppose you can still model that with async, but it certainly doesn't feel as helpful.
Out of curiosity, what would be the benefit?
In theory, it would simplify concurrency primitives in the kernel space, especially on the main thread, which is something I’ve notice is not an area that a lot of kernels explore.
Maybe I'm misunderstanding your proposal, but it seems to undermine process isolation. If you trust processes to yield under a co-operative multitasking system, why not trust them to leave each other's memory alone?
I was thinking of the internal Kernel tasks that might block, where it might be possible to have things happen in parallel, like bringing various hardware components online. See the sibling discussion on 200ms startup times.
I tried with async in the kernel first, but the cruft that was needed two years ago was not worth it in the kernel itself. The net I/O is actually async-first, see here: https://github.com/moturus/moto-runtime/blob/main/src/net.rs
File I/O will move to this model later (the current file I/O code is quite old and mostly a placeholder).
I think I missed this on my quick pass through the code, I see a lot async usage here as well, https://github.com/moturus/moto-runtime/blob/main/src/net_as...
Very cool. Thanks for pointing this out.
That said, I would be thrilled to build this and other alt OSes and their userlands with Nix / Nixpkgs :).
Linux is not complex but there are some design issues with C ABI and glibc stability and symlinks.
Indeed I would say Linux is as simple as possible given various goals ... however, 'simple as possible' can still be complex!
Linux is definitely complex. Linux's ABI stability (good!) also means we cannot really blame C / glibc so much, it has internalized that problem.
I agree the goal posts make much of this complexity inevitable, but the solution is to stop trying to intersect everyone's objectives into a much narrower and daunting goal.
The big galaxy-brained goal should be:
1. Make more of Linux Rust.
2. Leverage Rust's superior expressive power to switch from a "configuration" mindset (Linux's current gazillion options on single giant code base) to a "composition mindset": we want to use types to combine various subsystems in various flavors.
3. Something like this and Linux proper can share lots of library code / subsystems. We can blur the line between "separate kernels" and "separate 'distrobutions' mixing and matching stand-alone Linux components".
I think this is the only sustainable way to reign in the accidental complexity.
Sounds interesting, but it also reminds me of what Linus once said when asked about fearing competition. From my memory his answer was something like: I really like writing device drivers. Few people like that and until someone young and hungry comes along who likes that I'm not afraid of competition.
You don't need very many device drivers to run in a VM.
The real question is, how useful is a non-C-compatible, non-Linux-compatible, VM-only OS. Maybe a little bit, for microservices?
Is this really non-C-compatible? Seems like it just needs a libc layer written in rust. Most C programs don't make syscalls directly anyways (on some OSes they aren't even able to). The rest of the question is definitely fair though.
According to The American Heritage Dictionary an operating system is:
"Software designed to control the hardware of a specific data-processing system in order to allow users and application programs to make use of it."
The question is how much Operating System is a software that delegates all "control the hardware" parts to the layer below.
I know there are wider definitions of OS, but my point is this is not going to replace Linux.
Without question projects like MotorOS are sill useful. Besides practical applications it's a nice idea. Just recently I wrote in another thread that I would love to have a glimpse into an alternative universe, where Pascal had won over C and everything was Pascal-based. The idea to have everything Rust-based is even more exciting.
It's relatively trivial to have a `FROM scratch` dockerised microservice in Rust. I'm not convinced having an entire OS brings that much to the table.
From the description, this aims to recreate that but forgo the entirety of the linux+docker stack needed to actually be able to do FROM scratch in the first place
For work, I run a rust service on GCP. It could run on this instead of docker on Linux.
I'm going to assume without looking at this project that observability and operability would be worse than it already is though, so I'm not in a hurry to move. Anyway, GCP means cost of compute is a rounding error compared to cost of bandwidth, so I have no reason to find the edge of efficiency.
So Linux will essentially just become a HAL / "bios"
I don't think so. There is no stable interface to the drivers which makes their reuse in other projects a huge effort. With Linux you either get the whole kernel or nothing and the whole thing is more than a HAL or BIOS.
No, Linux remains where it is.
AFAICT this could run on top of Xen, VMWare, or the hypervisor which runs Win 10/11.
I don't think drivers are what warrants Linux its popularity / indispensability. I worked in a few places where people, well, at least moderately, liked to write drivers. Also, consider that it would've been definitely possible to write an OS with a mechanism for module loading compatible with Linux drivers.
My take on this question would be the ease of use for application developers. This ease of use consists of plenty of somewhat ready-to-use libraries that cover plenty of use-cases, huge community that both produces documentation and will answer questions should you have any, multiple hardware vendor support, and of course, licensing.
It's possible to compete with Linux in very special, very narrow use-cases, but trying to win against Linux on every front would require an insane amount of effort by a very large group of people.
"Also, consider that it would've been definitely possible to write an OS with a mechanism for module loading compatible with Linux drivers."
Sure, you can do that and it will work exactly until the next change in the driver interface, which is not very long.
So you end up in maintenance hell and will still constantly be behind the curve, because Linux gets all the updates first. You will be less secure too, because you'll get the security patches later.
That being said, someone in a similar thread from a few days ago said one of the BSDs does that. Maybe it's not that bad.
I don't believe you and I don't even need to look at the code to know this is false.
Care to elaborate?
From the docs at the top of scheduler.rs
That's elaboration enough I think?
Out of curiousity, why would a small kernel take a whole 200ms to start on a modern computer? Wouldn't it need to initialize some metadata for the memory pages, mount the filesystem, and try to launch an init process? I suppose there might be an ethernet driver and possibly something to pipe logs to ("stdout" for the VM) to initialize. Shouldn't that all take a few microseconds?
Or is all the slowness in the host preparing the resources? (as in QEMU and KVM?)
In general, lots of hardware requires long waits to initialize. As in, program a value then you must wait 50 ms before querying the registers or the hardware will give you non-deterministic garbage with no error indication.
So, it depends on the hardware, though you are unlikely to need more than 1 second on anything other than truly degenerate hardware assuming you are initializing devices in parallel.
But it’s only for VMs. So what hardware are you initializing?
I pass through a GPU and USB hub to a VM running on a machine in the garage. An optical video cable and network compatible USB extender brings the interface to a different room making it my primary “desktop” computer (and an outdated laptop as a backup device). Doesn’t get more silent and cool than this. Another VM on the garage machine gets a bunch of hard drives passed through to it.
That said, hardware passthrough/VFIO is likely out of the current realistic scope for this project. VM boot times can be optimized if you never look for hardware to initialize in the first place. Though they are still likely initializing a network interface of some sort.
“MicroVM” seems to be a term used when as much as possible is stripped from a VM, such as with https://github.com/firecracker-microvm/firecracker
4 months ago we had FreeBSD booting in 25ms. That gives some insight where the bottlenecks might be: https://news.ycombinator.com/item?id=37319180
Because of the debug mode
Yes! And because of sandbagging :)
Many years ago I contributed to IncludeOS and that thing could boot a vm in a millisecond under the right circumstances, allowing for things like per request vm isolation. So I suspects it's a mix of hardware circumstances and the way the microkernel is implemented.
Timer calibration is a big time suck.
It could be cool if there was a WASM container built in Rust that could run in this OS. I don't really have a sense of how complicated a WASM container is, though it seems non-trivial to even decide what such a thing is, so it would be nice not to treat the WASM container itself as the OS (i.e., more room to safely experiment). WASM also seems like it dodges the ABI issue by being more explicitly about composition instead of shared binary data structures.
When you say wasm container, you mean something like wasmtime that provides a non-browser wasm runtime?
https://github.com/bytecodealliance/wasmtime
Yeah, just that sort of thing. I mean, heck, can you put #!wasmtime at the top of a wasm file? Would such a file literally run right now (assuming #! is supported, and wasmtime doesn't hit any MotorOS Rust limitations)?
Serendipitously I see this article about WASM and how it has all these nice properties as a runtime abstraction: https://wingolog.org/archives/2024/01/08/missing-the-point-o...
I don’t think swapping out the first order language does anything other than boxes out potential support.
It isn't hard to start writing a new operating system. It is very hard to support that operating system for the next 5 decades.
Linus Torvalds resolve is unbreakable
Yes, it's most impressive. I couldn't do it. That kind of long term focus is an incredible asset.
I think the most close to production ready "mostly" Rust OS is Fuchsia.
I've seen this stated in a couple places, but IIRC the kernel is C++ and the UI is Dart. I assume lots of things in between could be in rust, but do you know which? daemons, services, drivers, window server?
Yeah, this seems like it's more directly competing with those than Linux. I'd want to see those addressed in the "Why?" -- that is, why MotorOs instead of Docker, etc.?
My first thought was "can this run my containers?"
It basically says it already:
By and large, microkernel and containers solve the same problem. Except, I wouldn't call containers a "solution", more like a workaround. Not in the sense that containers by themselves are a workaround, but the way they are used is a workaround for the same problem.
The way containers are used today, especially in the context of Kubernetes, is to finely slice the available physical resources. The orchestration allows to manage which part of the application gets what slice of the resource, thus attempting to cut down on waste that's typically generated in the world where resources are managed through VMs. Where the typical problem is that a single VM will require too many resources because of the bloated OS it needs to run and because it's hard to create VMs with very limited resources, since OSes usually come as a package deal.
So, containers "solve" the problem by giving up VM optimization -- instead it's usually beneficial to create very beefy VMs, on top of which then a new virtualization layer is created with containers. This minimized the VM waste, but doesn't get rid of it entirely, and, of course, creates a lot of complications with all the indirection resulting from two-tiered virtualization.
Microkernel is the opposite of this "solution" -- it's the attempt to make OSes more modular, and as such less demanding of resources. Ideally, in the world if microkernels you don't need containers (at least not for the thing they are usually used today): your VMs can slice the resources in the way at least as efficient as containers do (or, hopefully, even better).
So... to predict the next possible question: why containers are so popular and micorkernels aren't: it's because the later is harder on the applications (even when applications don't actually need something, they often use it because it's available in a full-blown OS, applications aren't usually written with resource scarcity in mind). Secondly, containers, essentially, exist on top of somewhat uniform, somewhat stable interface of Linux kernel. Micorkernel VMs would expose users to the zoo of ideas hardware vendors put into their products, making portability difficult. Finally, for all its flaws, Kubernetes is a big system with many (even if not so great) solutions for many problems. So, programmers who fear technology feel like it gives them a safety net and will allow them to program from a more comfortable position of modifying YAML files, copying the most upvoted answers from StackOverflow. There won't be such an easy cake in microkernel world.
Docker and NixOS exist because of userspace problems with package management and serverless exists because businesses want to pay for compute on demand.
isn't this similar to other microkernel and something like wasi/wasm kinda addressing this the correct way ?
I'm the project author/dev. Thanks a lot for posting this, and for comments/discussions!
I see two main concerns raised here:
(a) long-term viability and support (b) compilers, binary compatibility, etc.
While the first concern is definitely valid, and without a community this project will not succeed, I do believe that potential benefits of Motor OS (or a similarly focused/structured project) will eventually result in a widely used new operating system. There are major problems with Linux inside VMs (and sometimes outside), and the Linux devs are not focused on this enough to clean things up anytime soon. I work on Linux Kernel at my day job, I know.
Re: compiler instability, binary compatibility, etc.: I'm sorry, I don't understand what is the issue here. The latest Linux kernel can be compiled with different GCC or LLVM toolchains on x86_64, and the result will happily run old binaries compiled years ago with who knows what. repr(C) structs in rust are stable... So why so many concerns here?
Again, thank you all for your comments and questions - I'm happy to answer more (at least until my day job kicks in).
Kind of cool as a hobby OS.
But would be nice to backup your claims regarding performance with actual numbers.
The lainding page at https://github.com/moturus/motor-os explicitly says that both networking and file I/O are slow and have to be improved. The only claim is about fast bootup, and the number is there, and can easily be verified.
Where do you see any unsupported claims re: performance?