- 265comments
- 251comments
- 44comments
- 5comments
- 272comments
- 6comments
- 68comments
- 54comments
- 78comments
- 58comments
- 12comments
- 62comments
- 247comments
- 183comments
- 15comments
- 1comments
- 182comments
- 28comments
- 20comments
- 55comments
- 203comments
- 93comments
- 4comments
- 41comments
- 5comments
- 16comments
- 186comments
- 3comments
- 152comments
- 19comments
Interesting background discussion here for those wondering why systemd has its own DNS resolver: https://news.ycombinator.com/item?id=8203080
Or, in other words, since they want to integrate the entire OS into systemd, they need a DNS resolver there too.
As usual, it is all about cloud...
The submitter doesn't state affected versions, but judging from systemd's NEWS file, the resolved component was first introduced in 213 and later dubbed as "a pretty complete caching DNS and LLMNR stub resolver" in 216: http://lwn.net/Articles/609740/
Considering most users run on a backported 208-stable IIRC (I know RHEL 7 settled on that), I'm not sure how widespread this is. Still concerning that they felt to roll their own and not follow RFC guidelines, though.
Considering how plagued with vulnerabilities BIND was, I'd assume DNS is a hard thing to do.
That quote is gold. Most of DNS isn't implementing the protocol, but implementing all the protections against cache poisoning, many of which are not intuitive.
But as the oss-sec email says: "systemd-resolved does not implement any of the hardening recommendations of rfc5452."
Am I the only one who is getting fed up with everything under the sun having their own DNS resolver and DNS cache? It's border-lining madness right now.
Before you had a situation like this. ISP: Server and cache. Router: Client, server and cache. OS: Client and cache.
So if you ever encounter DNS failing, you only had 3 possible places to look for errors, or maybe not even errors, just stale data that may or may not correct itself over time. Flush the caches you can and hope for the best. Rinse and repeat.
But then came the browsers and decided that 3 levels of caching was good enough. Let's add more which can fail in the name of performance!
Combine this with mobile, mobility and data-roaming and now you not only have a DNS-cache (based on already tripple-cached data) which often is guaranteed to be wrong, your browser may actually have cached the wrong DNS-servers, so when you roam from Wifi to cell-data your precious LAN DNS servers are no longer available and your browser just looks stupid.
And now systemd adds to this. What the flying fuck?
All this nonsense because someone somewhere decided that a nice, shared, global OS level client and cache wasn't good enough. Handling this complexity once would just be too easy.
Can someone tell me why? Why wasn't it good enough? What was it that prompted someone to go reimplement all this wrongly, yet again?
Yes this is a rant, but if there are underlying technical reasons for this nonsense, I'm very, very interested to hear about it.
The commit's comment (http://cgit.freedesktop.org/systemd/systemd/commit/?id=32234...) says it all:
resolved: add DNS cache
Yes, that's the full commit message for what I think is the commit that adds the feature. I was trying to dig up an authorative source for the "why" you're asking, but I guess that failed. Perhaps it was discussed/motivated/described/something somewhere else, like a mailing list, I didn't search further.
Yes, that's the full commit message for what I think is the commit that adds the feature.
If true (that it is a 4 word commit message), that is a disgrace. Seriously, something as important as this should contain a lot more information, if not just a pointer to a place that provides it.
Lennart proclaimed therefore we all must follow. That appears to be how it works these days. No community, no discussion. Someone with commit access says bend over or fork it.
Actually fuck that. Which is why I'm sitting here on a FreeBSD machine.
I don't understand this attitude. Lennart has every right to write a DNS resolver if he feels like it. And you have every right not to use it. Nobody is being forced to do anything. (I'm not aware of any distribution currently using systemd-resolved by default, and you can disable it at build time and runtime.) So what's the problem exactly?
A couple of problems: (1) This resolver has security bugs which have been fixed in the existing daemons. That doesn't matter except that: (2) systemd utilities get an easier route into distros than competing utilities, even where the competition is much better or more secure. This really matters since Linux improves because of the Darwinian nature of the competition between the different components. You can't have one group of packages claiming they are the "one true standard", or this doesn't work.
Problems:
1. Solved problem. Contribute to something else.
2. Its another tentacle hanging off systemd. Some of us pay for our software (RHEL) so we'd in turn like to have some say in what we have to manage rather than prescriptive software development. We don't really get to choose to not use it because of the incestuous nature of the Linux ecosystem at the moment (I.e. Freedesktop/gnome/systemd).
3. Systemd is a buggy piece of crap. As are several of the "stable" bits of systemd I have encountered in RHEL7. I have quite a bit of experience with systemd and i really don't want it consuming any more things badly. Its been a complete pain in the arse to be honest. An init system has never been a problem until now. Its totally unreliable and incredibly difficult to debug.
4. Some of us get shit done, not use the ecosystem as a playground for egos and quote frankly incompetent twattery. This stuff can go on in the sidelines and mature independently, not knotted into the next "stable" drop because everything has suddenly been tangled with it.
Its a complete fuck up at all levels and I haven't even started on the technical marvels of introducing something with the complexity of COM into a process model that requires reliability.
This is not how to do software.
"Nobody is forced to do anything" is a very lame argument, because it is quite obvious that systemd is effectively being forced on the Linux world, because it has become a dependency of Gnome. Other distributions besides Red Hat will make their life very difficult if they don't implement systemd as well, so yes, practically, systemd is being forced upon Linux users. You could of course continue the argument by saying that nobody is forced to use Linux, or even a computer, but it gets silly.
Welcome to the world of Poettering.
The guy going "mea culpa" or anything close to that is like spotting a polar bear in Sahara.
You forgot about host files. Those typically over-rule the resolver so you have to check those as well.
Technically, those only over-rule the resolver from the local system libraries. They do not over-rule DNS queries.
Sure, but your average application does not do a DNS query directly, it simply uses gethostbyname and depends on those local system libraries to figure out what to return.
While I agree with you that these layers of caching can lead to headaches, Linux doesn't have a built-in cache (and distros often don't provide one by default). Systemd would be one way to provide this, instead of other daemons that have been used in the past, like nscd, dnsmasq or bind. So it's not adding yet another layer, just providing the layer at the local host.
As for technical reasons, I'm not sure what you want to hear, each layer implements a cache for performance and/or efficiency. No one throws in a DNS cache just because they feel like it.
Linux doesn't need a cache, you can simply run a local DNS server. The kernel should not do more than it needs to do. (It's bloated enough as it is)
I don't understand your comment. a DNS server is typically also a cache, and this is what systemd was attempting to provide, without the public server part. It's the same thing that distros were using e.g. dnsmasq for.
What does the kernel have to do with this? No one is proposing that the kernel should have a DNS cache.
GP says 'Linux doesn't have a built in cache'. I presume (hopefully correct) that he means 'the kernel', since Linux doesn't have anything 'built in' that you don't run as a user process. The kernel of course has plenty of stuff built in but (fortunately) it tries hard to stay away from things that can and should be done in userland.
Systemd is a user process and now tries to replicate functionality that we already had a perfectly good solution for.
Since a DNS server is typically also a cache we already had this functionality (and more), simply make it listen to a local port and nothing else. That way you don't need to resort to rebuilding things we already have with incompatible interfaces.
Linux hasn't been just a kernel for a while; it's a kernel with some associated userland utilities that are developed together and normally expected to be present. That's been the case in the filesystem part for a long time; the userland fs-related binaries are developed in sync with the kernel side. Systemd is doing something similar with e.g. cgroups.
Linux develops fewer things this way than the BSDs typically do, outsourcing a lot of userland to other projects like GNU, but it doesn't literally develop only kernelspace code. Imo it's generally an improvement; the situation with trying to get things like cgroups to work together with process management previously was terrible.
This is absolutely true, which is why linux is as large as it is and why you need certain stuff in order to bring up a functioning linux system. As for the filesystems they are modules plugged into the kernel (linked directly so you can boot our kernel straightaway or through a ram disk kludge as modules in linux, not the most elegant solution because technically that duplicates a buch of stuff but nobody seems to care too much) and a bunch of supporting programs, this has been the unix way since time immemorial, I think even the first unix I ever used had fsck.
The reason those have to be kept in sync is simply because you don't want to run fsck on a filesystem that was built with a newer kernel that possibly has datastructures that your fsck (or other filesystem aware utility) accesses directly without going through the kernel. Essentially such programs re-implement a bunch of kernel functionality because they access file systems that are not currently mounted.
Systemd on the other hand (in the case at hand) replicates the functionality of another user process that we already have, know how to use and have hardened over many millions of hours of use. That seems folly to me, it duplicates functionality, makes troubleshooting harder and increases the size of the codebase without actually improving anything.
What's next? The framebuffer? Keyboard interface? TCP stack?
There are no really hard reasons why those could not be user programs. It's fine with me if we make linux a micro kernel but let's do that in a way that we do not need to revisit the last decade or so of security issues and ill thought out interfaces. Once we start to need things like DBUS and X in order to bring up a server we've really lost the plot.
I'm afraid so: http://lists.freedesktop.org/archives/systemd-devel/2013-Nov...
Why does systemd need to provide it? I already provides enough things.
Are there any bugs in :
http://www.thekelleys.org.uk/dnsmasq/doc.html
I had great success using that. Both installing it on my own or run it part of Ubuntu's base install.
(And the answer of "but it already provides everything + kitchen sink, might as well provide DNS caching too" would probably not be what most people would want to hear).
Because it is the Emacs of init systems.
Depending in your standing as far as emacs goes, that may actually be something good.
I doubt you will find anyone interpreting this as such.
Emacs is the Emacs of init systems
http://www.informatimago.com/linux/emacs-on-user-mode-linux....
I'm not saying it needs to, or trying to defend systemd. All I'm saying that systemd is providing and alternative to running another resolver.
That's pretty much systwhatemd is doing, except it is just subset of dns server that does caching.
Exactly, so imnsho this is simply wrong and should be un-done. Systemd should do those things that can only be done at init time and/or those things that need to be there and no more and leave the rest of the lifting to those tools that we already know how to use and that have stood the test of time. Otherwise we end up with 2 sets of incompatible tools with one of them not only re-inventing the wheel but replaying a whole bunch of security issues from the past on top.
This is the worst case of feature creep in system land in a long time.
That is a good architectural design, but is in direct opposition to the projects mission statement of "embrace, extend, ..." you guess the third word.
If a group is actively trying to ruin something, noticing that they're trying to ruin things isn't going to stop them. That's a useful strategy with accidents and friendly groups, but not with actively aggressive oppositional destructive groups.
Mostly I'm just angry about having to waste the labor hours moving everything to Freebsd from Debian. Its interesting, but not exactly productive.
Funny, that was exactly my response to all this, if Linux insists on copying the worst of windows in the future then I'll be more than happy to go with FreeBSD. I'm a bit rusty but I'll survive.
As a FreeBSD evangelist I actually like systemd :-P
Isn't systemd actually doing it the BSD way (everything within one repository and tightly coupled)?
I think we've reached the point where we can start being pedantic about what "Linux" means.
I don't think the kernel developers care that much about what systemd is doing (except for the notorious clash about systemd interpreting a debug option meant for the kernel and crashing the boot...), so you can't say the kernel is copying or doing anything.
The distros, however, are possibly the ones you want to blame for pushing and supporting systemd. Which is actually funny because the way it's going, systemd is laying foundations for a new kind of linux distributions to which distributions will certainly have to adapt if they want to keep it.
It's scary for me, because everything I've done has been in a Linux, but it's also educational for me, because everything I've done has been in a Linux. If the systemd invasion results in a lot of attention going to the BSDs, it's not entirely a bad thing.
The biggest change will be that you might find it sometimes harder to get exotic hardware to work and that BSD tends to put stuff in different places. Most of your other unix knowledge (userland) will transfer just fine, the sysadmin stuff is a bit harder but learning another dialect of unix is along the lines of learning Spanish when you already know Italian.
Read the hier man page of your chosen BSD, first. It is a great eye opener. From there OpenBSD and FreeBSD have some great documentation (not sure / no slight intended to NetBSD or Dragon Fly BSD as I have not used them much).
But that's not in line with what systemd intends to be. It's meant to be the standard suite that you sort of plug in as a middleware between GNU and Linux.
Your proposal would require a radical paradigm shift as to the fundamental project goals of systemd, in which case it'd probably become something like uselessd.
There are already excellent solutions out there that have been battle-tested, such as unbound or even dnsmasq. Best as I can tell, resolved has no good reason to exist.
Every client operating system under the sun implements a DNS cache, its part of the intended design of DNS. This is just systemd's bikeshed of a function already available on Linux, via nscd.
Without caching, all DNS queries degrade to worst case behaviour, which is a recursive query involving 4-6 or more round trips, starting at the DNS roots (which, good luck finding them, since you don't have their addresses cached, which is exactly what the root hints file is).
As for invalidating the cache when network configuration changes arise, that sounds like a bug report for wherever you experienced it. Certainly I've never seen this kind of problem on OS X, nor any time in the past 10 years on Windows.
errr, xxxxBSD don't. libc sends every DNS query to wherever /etc/resolv.conf says to send it.
The short answer is: the glibc DNS interface sucks balls and systemd wanted extra features.
The long answer is:
Often, systems do not come feature-complete. There are many cases of a platform, or OS, or service, or framework, or some other piece of software being released without every possible imaginable feature being implemented. When this happens, some other software may need to re-implement some core piece of technology in order to add functionality to it. Usually care is taken to not interfere with other systems or cause additional burden to the user. In the case of DNS, our use of DNS and different networks simultaneously requires additional functionality in order to process requests independently and efficiently. As the DNS protocol itself has an inherent flaw allowing for cache poisoning, it's non-trivial for people to implement without running into this security hole. And as someone else mentioned, the kernel and glibc don't have their own DNS cache.
I didn't downvote you for the rant because you weren't being mean or insulting and you genuinely seem to want to know what's going on. But next time you feel yourself posting out of emotional reaction, I recommend typing it out in a draft and save it for a half hour, and see if you might want to reword it later to be less angry/ranty.
Actually Unix out of the box (unlike for example windows) does not have system level dns cache, people often setup things like bind, unbound, dnsmasq, nscd etc.
Systemd here tries to be the official standard system level caching.
I agree though, about your extra caching sentiment to a degree. Actually as long the caches are RFC complaint (e.g. they obey TIL - looking at you nscd & java) or should be fine.
"Am I the only one..."
No.
But the solution is not to complain, but to use /etc/hosts, run your own root and your own cache.
Any software that overrides /etc/hosts is not worth using, in my opinion. There are no "underlying technical reasons" for not letting the user control their own name resolution that benefit the user. Of course, there could be reasons that benefit someone else.
Nit: Routers generally don't do DNS, though they do have (rarely needed) clients.
Pretty weird for Systemd to have its own cache and resolver. If you need a local resolver and a cache simply run a local DNS server rather than to re-invent the wheel (in a broken way at that). I'm sure they have their reasons but this is not the unix way.
It wouldn't be systemd if it didn't reinvent the wheel.
Systemd is just not the Unix Way. It's some shit program that requires to be run as PID 1, and takes over the system in the forms of an octopus. And, you need some gods-awful binary program to even read their logfiles. Yes, it's like the Windows registry.
Even Debian has succumbed to Systemd.
Yes, it's like the Windows registry.
Nope, that would be GSettings/dconf. It's limited to GNOME, thankfully.
Actually systemd is more like DCOM which is much worse.
The registry is fucking marvellous in comparison.
Nah, the equivalent of DCOM is dbus.
Which is what systemd is built on. At least on windows COM stays in user space but no, fuck it, lets use kdbus on Linux...
Sure, that's what built on, and that's why the equivalence was incorrect. You'd might as well say that systemd = Linux.
Systemd is a fascist.
I've been thinking a lot about the Windows Registry (and /etc, etc) and I've come to the conclusion that the systems which get it the most "right" is the app model (e.g. Android, iOS, Windows 8).
You see systems like Docker containers, which try to isolate a complete application and its configuration files together, and really that is just a hacky solution to the lack of inherent silo-ing within the underlying design.
While silo-ed apps have their own limitations (e.g. two "apps" using the same configurations) most of them can be overcome with tinkering for the edge cases.
I'd never design a system in 2014 with either Windows Registry (too messy, impossible to manage), or /etc (too inconsistent with itself, also often results in orphan files).
The correct analogy would be Event Viewer, not the Registry.
I am afraid sir, that comparing the system take over to the 'forms of an octopus' is most unfair to our dear Cephalopod friends.
I encourage you all to please use the more appropriate comparison : Cthulhu / Cthulhud
http://www.urbandictionary.com/define.php?term=Cthulhu%27d
This is a perfect example of why the systemd approach of putting a bunch of disparate components under a single tightly-coupled umbrella is bad engineering. Generally, projects focused on doing one thing are able to do it well, while projects that attempt to do many things have a hard time doing any of them well. Unfortunately, as the steady stream of cache poisoning vulnerabilities in other DNS servers have demonstrated, implementing DNS securely is really hard. A less-than-perfect attempt at implementing a caching DNS resolver will be insecure. For this reason, it's a task best left to a mature project run by people with DNS-specific experience who can focus on doing that one thing well.
Why? This wasn't caused by systemd's complexity or having too much components interconnect poorly. It was a bug in one of their implementations that could have equally happened had it been a stand-alone DNS cache resolver.
I'd totally buy that if BIND didn't have such a checkered history. I mean with BIND 9 they scrapped the entire project and started again because it was so insecure...
Yet they're only attempting to do "one thing" so according to you they should have done it well.
This line oddly contradicts the line that directly proceeds it.
According to the previous line they aren't doing that one thing well...
The reason why a project with a bunch of tightly-coupled disparate components will have a hard time doing anything well is that an expert who could do a good job isn't going to spend their time working on an implementation that can only be used along with a bunch of unrelated baggage. They're going to work on an independent implementation that can be used by as many people as possible.
Take DJB. He's an expert and djbdns has a stellar security record. He's not going to spend his time trying to make systemd's resolver secure. Instead systemd is going to end up repeating all the same mistakes BIND has made. At least BIND has a substantial head start over systemd-resolved.
So they would work on systemd then? Since it is set to have a far larger customer base in the near and long term than most independent cache resolvers (just see the list of distro's who have taken it up).
Potentially. But I don't think you've successfully shown that that is inherently due to systemd's design, complexity, or componentization. Instead it is likely due to systemd's relative immaturity of a codebase and that as discussed previously most cache resolvers have had a checkered history of security issues (and assuming systemd is "as bad" as everyone else, it will have security issues).
Is the systemd project passionate about DNS? Is the curl team passionate about protocols and files? Is the opensmtpd team passionate about e-mail?
I think two of the three are yes answers. The great strength of UNIX has always been people's ability to flock to places where their passion is shared and improve and learn (institutional knowledge) about their passions.
I don't think systemd will have passionate people for these extra services and it seems they don't have the institutional knowledge that others have gained since this is not a new kind of bug.
As has already been shown through example, having a single unified purpose does not make something immune (or even resistant) from security issues.
Tons of projects with a single focus have had countless security problems, some so bad the entire project was completely scrapped.
So the entire argument is based on a flawed premise.
I think history favors projects, companies, and individuals that concentrate one something as opposed to big entities that try to be all things to everyone. This is how small companies are able to dislodge larger ones. Focus is important in anything. The argument is far from flawed, as limited counter-examples are hard to find.
Yes. And when it happened before, it was The Bug of the Century. Repeating that mistake is either stupidly incompetent or proof the systemd developers were literally born yesterday.
New development is expected to have some bugs, but it shouldn't sound like a tribute band playing the greatest hits of yesteryear. And they can't exactly fallback on the old standby of "oh, it's a legacy code base."
Totally agree. It's mind blowing to realize that because of the init system we have on our system we are now vulnerable to DNS poisoning.
Do one thing and do it well.
The init system is not vulnerable. The local dns caching daemon that you can optionally install and that shares the name and maybe some code with the init system is vulnerable.
It's like saying "package dnsmasq.deb is vulnerable, so sysv-init.deb is now vulnerable, they share an extension"
Caches are bugs waiting to happen. Rob Pike @rob_pike 21 Mar 2014
By the same author
"There's no such thing as a simple cache bug."
"Caches aren't architecture, they're just optimization."
This isn't really even a cache bug. It's a bug in a cache, but it's a stupid bug to have in any DNS implementation, cache or not.
Of all the components of systemd that doesn't need to exist, resolved is definitely one of them. Why they're reinventing the wheel rather than allowing the user to set up something that works and is battle-tested, such as unbound, is beyond me.
I don't necessarily disagree with the heart of your statement, but it's worth mentioning that systemd does not require one to use resolved. The user is still "allowed" to run unbound, dnsmasq, etc., and the preferred solution will be a question for distribution maintainers and users to answer.
I understand concerns about systemd's scope, but most of the "questionable" functionality is optional.
Personally, I'd prefer that they concentrate on stuff that's properly within the scope of the project. This is way, way outside what the reasonable scope of the project ought to be.
Small comfort. When a project has the incredibly poor judgment (and, frankly, hubris) to write a caching DNS resolver without even the most basic cache hardening protections, and then declare it "a pretty complete caching DNS and LLMNR stub resolver"[1] instead of just leaving it to people who know what they're doing, it calls into question the rest of the project. This is a major reason why so many of us are wary of systemd.
[1] http://lwn.net/Articles/609740/
I find the design of systemd-resolved to be very strange. It uses dbus to talk to glibc, and it seems to be a new, from-scratch implementation of a DNS resolver. To be clear, I don't really think it matters whether systemd-resolved is under the systemd umbrella, but I do think that the design has a lot of unnecessary NIH syndrome.
It turns out that there's a very well-specified protocol by which clients can ask a local cache on their system to answer DNS queries. That protocol is called DNS :) I don't see why routing something DNS-like over dbus makes any sense in contrast to doing it using DNS itself on port 53.
Fedora is experimenting with running unbound as a local caching resolver [1]. This gives caching, DNSSEC validation, and all the benefits from the fact that unbound is probably much better hardened than the average libc or application-side DNS client implementation.
[1] http://fedoraproject.org/wiki/Features/DNSSEC_on_workstation...
Poettering and crew seems to have such a hardon for dbus that they want to get their own variant into the Linux kernel (kdbus).
Hell, i have recently seen a fedora bug regarding the use of su -. Where Poettering argued that people should not use su because it broke dbus.
Instead he seemed to advocate that people ssh into 127.0.0.1 to do their thing with a different account.
Systemd... a very large mass of c code being written in a short amount of time with huge functionality creep. Yeah, this is going to end/begin really well. I swear all of the people commenting about it being a good thing are not seasoned developers.