- 229comments
- 13comments
- 125comments
- 30comments
- 115comments
- 93comments
- 474comments
- 83comments
- —discuss
- 6comments
- 292comments
- 344comments
- 19comments
- 20comments
- 2comments
- 84comments
- 41comments
- 38comments
- 98comments
- 6comments
- 4comments
- 108comments
- 273comments
- 5comments
- 89comments
- 73comments
- 2comments
- 87comments
- 14comments
- 105comments
What are the differences compare to Firecracker?
From the guest perspective, the differences are minimal. Even boot time of the guest (thinking about a custom-built minimalist Linux kernel here) is roughly the same.
On the host side, things are more interesting. Firecraker has a smaller TCB (Trusted Computing Base), is written in Rust, and is statically linked. On the other hand, QEMU provides more features (especially in the block layer, with more formats, network-based block devices, asynchronous I/O...), can be configured at build time to adapt it to a particular use case, and has a pretty good security record.
In the end, KVM userspace VMMs (Virtual Machine Monitors) are learning from each other, giving users more options to choose from. Everybody wins.
That's an interesting and I would argue, contrarian take?
https://www.theregister.co.uk/2017/01/30/google_cloud_kicked...
"QEMU has a long track record of security bugs, such as VENOM, and it's unclear what vulnerabilities may still be lurking in the code."
I think the slide 14 from the talk "Reports of my Bloat Have Been Greatly Exaggerated" [1] presented by Paolo Bonzini at KVM Forum 2019 gives some good perspective about QEMU's security track:
"Of the top 100 vulnerabilities reported for QEMU:
- 65 were not guest exploitable
- 3 were not in QEMU :)
- 5 did not affect x86 KVM guests
- 3 were not related to the C language
- Only 6 affected devices normally used for IaaS
The most recent of these 6 was reported in 2016"
The rest of this talk was also very interesting. I encourage everyone with 10 minutes to spare and an interest in VMMs to take a look at the slides.
[1] https://static.sched.com/hosted_files/kvmforum2019/c6/kvmfor...
wow
Which leaves about 30 that presumably were guest exploitable.
Don't get me wrong -- QEMU is useful. As a "kitchen sink" solution that runs anything, anywhere, with any useful combination of emulated {devices,processors,systems}.
However, this is also its biggest weakness. Which is why Google and Amazon all run their own custom VMMs for their IaaS services.
The microvm machine type as described here is a great step to improve this situation. The next step in my book would be to reconfigure QEMU's build system to allow building a binary that only supports the devices provided by microvm, and nothing else.
Indeed. Nice to see that the cross-pollination is happening.
For folks interested in what can be accomplished with userspace VMMs, a very minimalist example is the Solo5 project (https://github.com/Solo5/solo5), specifically the 'hvt' tender.
Oh, nice: props for including precise, fully worked example commands to actually use this. This is so helpful in terms of onboarding and being able to actually get people to use a project:)
If I was an eccentric billionaire, I'd reward good documentation with an open source doc prize.
It's so difficult to impress upon some people just important documentation is. And when they do it, they half-ass it by copying stuff from source code and not adding any context.
+100 to this. I’ve become a cheerleader for some projects just because the documentation was so good that it was easy to get started. It’s how I became a ZFS zealot among other things.
OT: where do you find the good ZFS documentation? Googling always seems to turn up Oracle's documentation, which may not match what's in FreeBSD or ZFS-on-Linux
The zfs man pages (at least on FreeBSD) are actually pretty decent for finding command usage. The FreeBSD handbook[1] covers some typical usages.
[1]: https://www.freebsd.org/doc/handbook/zfs.html
To be fair -- the docs for ZFS via the manpage are great but I found a lot of help also from forums of likeminded folks -- ServeTheHome (https://servethehome.com) is where I got hooked.
And then there are countless GitHub projects where the README.md file, which is usually first thing I read, that is super well documented and written for the noob (imo). The best example of that so far, especially for someone like me just getting started with the framework, is the gin Golang http framework: (https://github.com/gin-gonic/gin) -- that readme is full of useful examples.
Google does something similar: https://developers.google.com/season-of-docs/docs
I sure wish qemu were better documented. There's a lot of qemu documentation out there, but much is incomplete or out of date. A lot of Red Hat's documentation is virsh rather than qemu/kvm. It's fairly easy to get something simple up and going, but there are shortcomings when you try to do something more complicated.
Firecracker requires KVM, but QEMU can be used with instruction emulation, correct? Does that make this potentially a little easier to develop with in certain environments?
That's correct. The initial versions of the microvm patch series did require KVM, but the one that got upstreamed does work with TCG [1], thanks to the QEMU's maintainers feedback.
That said, I'm not sure for which kind of use cases it would be useful to run it this way, as the performance won't be amazing. I find TCG acceleration mainly useful for debugging and foreign systems emulation.
[1] https://wiki.qemu.org/Documentation/TCG
I wonder if that's a security benefit, too... I would expect that KVM equals more attack surface, and direct to the host kernel at that. Now, weighing likelihood of exploiting KVM through qemu vs qemu through TCG (or whatever)... I'm not qualified to asses, and probably depends on your threat model. But it sure feels good if you can run qemu with no special privileges.
What kind of workloads make sense for this runtime? DIYFaaS? Or small, isolated long-running processes?
Function as a service is certainly one place. Others include secure container runtimes (eg. Kata-containers), sandboxing applications in general, and libguestfs.