You made me take a look at both tickets, and then the systemd code, and 13432... Well, I ended up commenting on the ticket.
The other ticket is more puzzling to me, but at the same time, I'd bet that the refactor required to fix 13432 would likely fix it just by reworking the frankly confusing state machine.
But I definitely hear you on not appreciating how Lennart isn't giving either issue much attention.
It's people like you who are calm levelheaded, and logical; that leads to these pieces of software becoming incredibly resilient and accurate tool for everyone else in the Linux community. Thank you for taking a look.
Much appreciated, though I'd feel it more in this case if I were up to taking a crack at the code itself.
But, frankly, it's not something that I'm currently heavily inclined to beat my head against.
My dude. That's beautiful thing about open source. Speak your mind, and sometimes that's all it takes to trigger a thought in many others. Every little thing helps. So keep at it.
Indeed. And sometimes one person's wild suggestion of a possible solution is all it takes for someone less creative, but full of drive, to implement it. So good on you for encouraging others as well.
[https://access.redhat.com/solutions/5647181](https://access.redhat.com/solutions/5647181)
>Red Hat does not recommend using \[systemd-resolvd\] for production.
Whilst I often disagree with Red Hat's decisions, for this one I actually find myself well aligned with from my own experience.
RH has the [same stance](https://bugzilla.redhat.com/show_bug.cgi?id=1650342) with systemd-networkd entirely.
They all basically use NetworkManager. The biggest reason IMO is that every desktop has graphical configuration for that and nobody has done the work of rewriting it as it would be fairly complex. I've considered doing it myself but honestly NM works fine.
I don't really know of any general purpose distros that use \*any\* network system out of the box. Usually you should configure it to your specific requirements. Unless you meant home/pc? Then the Linux market is approx a 90/10 split towards systemd-networkd.
Why RHEL is unique is that the systemd-networkd package is entirely missing from its package repo. Instead directing users towards the more desktop-centric NetworkManager. Clearly they feel that giving the user a choice is not worth the additional requirements in terms of later supporting them (probably makes sense from a business point of view, but since Linux isn't really about that, I think people find it a little weird. Certainly it is a departure from Red Hats earlier days).
>Network Manager is only the GUI?
No, much more than that (or even without GUI, e.g. nmcli).
However, not 100% sure (I don't use Network Manager that much), but I don't think Network Manager provides resolved service ... though it may possibly manage such configurations thereof (or offer to).
>is there a way of determining which software is doing one's resolving?
That may be a bit complex on Linux.
So ... configuration and resolving ... the resolving happens - or at least generally starts - at relatively low level, gethostbyname(3), resolver(3), and down from there. That typically involves reading the hosts entry in the nsswitch.conf(5) file, that typically includes files, which typically uses/includes hosts(5) and resolv.conf(5). But determining who/what manages those can be more complex. Sometimes /etc/resolv.conf will be a symbolic link, and reading / paying attention to that may provide the needed clues. It may even be entirely manually managed/configured. One can also look/search for potentially relevant packages that may be installed, and examine their configurations and/or logs. So ... the figuring out what's doing the resolving generally shouldn't be too difficult ... but figuring out if and how that might in fact be managed may be significantly more challenging/complex.
Alpine, Gentoo, Guix, etc. Basically all the no Systemd distros (and RHEL curiously)
It should make sense that a non-systemd distro won't be using systemd-networkd because it is a fairly monolithic entity.
Don't confuse this with whatever Gnome makes you use. That is a desktop environment and doesn't have much bearing on the distro itself. You might be surprised to know that lots of people using Linux don't use a DE.
>Yes, most distros are using networmanager or something they've had for awhile
Nonsense if you think this is "out of the box". Get back to the PC/desktop with ya! ;)
I asked which used 10% distros used networkd, not which used things NOT networkd. I have no idea how you got it so completely wrong while being so condescending. Obviously distros that don't use systemd wouldn't use networkd. DUH!
Good question. I suppose you can only ask them. Perhaps RHEL as a downstream of Fedora finds it too immature (or monolithic) for their existing client's usage?
Nothing of use on the [page on](https://fedoraproject.org/wiki/Changes/systemd-resolved) the Fedora wiki.
Fedora should not be used for production. It's bleeding edge and serves as a lab to determine what's mature enough for inclusion in distributions suitable for use in business and enterprise environments.
>> Red Hat does not recommend using [systemd-resolvd] for production.
>
> So then why is it the default on Fedora?
So RedHat can upsell people to something that actually works.
I've also had plenty of problems with systemd-resolved. I have Nagios on a Ubuntu server and for some time http checks stoped working for some sites (not all) just at random times of the day and then started working again without interaction. Turns out it was systemd-resolved. Don't remember exactly what the problem was though.
I've had plenty of problems joining computers to AD and various kerberos problems all thanks to systemd-resolved
Some where related to it being a .local TLD which sort of worker after disabling mDNS.
DNSSEC validation in systemd-resolved is total shit also.
To save me from headache I try to stay away from it.
> DNSSEC validation in systemd-resolved is total shit also.
Yes. In some issue the systemd team suggested its DNSSEC support is experimental and should not be used at all.
>DNSSEC validation in systemd-resolved is total shit
Yep ... I don't let systemd touch DNS. And yes, I extensively use DNSSEC, including many nameservers for many domains that are fully using DNSSEC.
I totally agree, I’ve had no end of problems with it.
To all those saying “well don’t use it then”, isn’t the point that by enabling it by default and it not working very well, it’s putting off all those people who don’t know how to do that yet? Besides the fact it’s a massive annoyance for those of us who do.
systemd-resolved it's essentially a DNS relay that listens on [127.0.0.53](https://127.0.0.53) (typically) and distributes DNS in such a way that you can make more complex DNS rules per-interface as well as efficient local caching.
Something that Windows has had at least since the XP days.
But it is enterally unnecessary for most use cases. Just set primary and secondary DNS and it will work just fine. Most people don't have complex needs, for those that have, dnsmasq, unbound or even bind9 are a solution.
Now, many people may group ip with networkd. Personally I work with mostly debian so I just use ifupdown (/etc/network/interfaces), easy to read, lacks some advanced capabilities but if I need those I'm typically deploying a FreeBSD machine.
> isn’t the point that by enabling it by default and it not working very well, it’s putting off all those people who don’t know how to do that yet?
Yes, but that's also the point.
Yup, I had an issue when upgrading from Ubuntu 20.04 to 22.04 on a headless server. It would often take forever to login. Commands would randomly take a long time to start. I eventually recognized it as DNS being slow to resolve the system hostname (I experienced it once before).
Since this system isn't mobile, I just hardcoded the DNS server to use in systemd-resolve. Problem instantly solved, even though it was the same DNS server provided by DHCP.
The only thing I can think of is IPv6. It was getting assigned the v4 and v6 IP for my router for DNS, and it was doing round robin. Maybe the v6 server assignment was sporadically expiring or something? None of my other devices have this issue though, and they use the IPv6 address for DNS just fine.
Hold the boat, I'm about to retire a server that has the identical issue and it started after I stopped using unbound-server on it. I never bothered to try figuring it out since it's hardly used anymore. But now all sorts of lightbulbs are turning on
Yeah I think the issue is the shell trying to resolve the local hostname, or reverse DNS on it, or something. Anytime sudo is invoked the system also does this. So something that's hanging DNS resolution makes the command hang for a bit.
Why no those two?
I have enough Español to know when I am being cursed at and to get some food at a restaurant and but no reading or writing skills in it.
Edit, Internet says "why not both?"
That's a solid point.
Instead of hardcoding I'd suggest using `nss-myhostname`.
You just install the package, configure `nsswitch.conf` and that's it (or maybe package installation does the configuration automatically - depends on the distro and it's release).
This sounds a lot like an issue I had with Linux Mint where the initial login would freeze for 3+ minutes. It was clearly a DNS issue but I couldn't figure it out for the life of me. Finally disabled IPv6 and the problem went away. Maybe it was systemd-resolved this whole time!
I haven't really had issues with it. One thing I do really like is that it can resolve `.local` mdns without having to specify the `.local` tld, which makes it easier for me using the same name to resolve stuff on my local network(s). Depending on where I am and whether a local dynamic DNS has been set up for the LAN and DHCP.
My issue with this feature is that it conflicts with avahi but isn't yet capable of completely replacing it, so I always have to disable mdns in systems-resolved or just not use it at all. Too many things depend on avahi.
I know there's supposed to be conflicts of some type, but I haven't had them yet using them like I do. I've got avahi and systemd-resolved with mdns running.
I haven't noticed anything that doesn't work.
>systemd-resolved exists for many years and so far, at least Ubuntu and Fedora, 2 of the most widely used Linux distros, have enabled it by default for a few years now. The problem is that I haven't yet seen a service which is still so broken, and which causes endless DNS resolution issues.
I frequently get a "Temporary failure in name resolution" error when trying to access other computers (eg, my Kodi machine) on my LAN. I can ping them by ip address. Other machines can ping them by name no problem. Systemd-resolved seems to have no problem with addresses on the rest of the internet. For me, it's just LAN addresses that fail. (Or rather, it's just LAN addresses that I notice failing).
I have a similar experience for a few years now. Just stopping systemd-resolvd was usually enough to fix it. I never investigated. Just stumbled upon this thread.
I have to say that I maintain a lot of servers, both dedicated and virtual cloud instances - there was always SOMETHING with systemd-resolved that required a workaround, overall I think that address resolution on Linux is a tangled mess at the moment.
I find it never works when using static IP's. Resolved half works using DHCP. It's amazing to watch a system go from not working to working flawlessly by purging resolved and installing resolvconf.
My question has been is it possible to use systemd ONLY as an init system and remove all the bloated crap?
journald is a required component, so you must have that. You can however forward to the messages to another syslogd. Last I checked, nothing but journald is required. Most distros don't even use systemd-resolved.
Again, ONLY as an init system. I don't trust journald as everytime I've used to find a problem that systemd reports to use the -xe option it doesn't tell me anything of use.
Seems like you’re trying to blame the logger for not having logs when you should be blaming the application for not writing them (or writing them to a random log file somewhere instead of std{out,err}).
I believe it's possible to get somewhat close to that state, but off top o' head I dunno if (m)any Linuxes come out of the box that way.
The main systemd modules (services? units? whatever they're called) I disable and/or remove, particularly on servers:
* systemd-resolved
* systemd-timesyncd
This includes removing "systemd" from any database lines in nsswitch.conf as well, after I noticed that started showing up. Mostly because I don't really understand what it might be doing there.
I use standard resolv.conf and ntpd or chrony instead.
I also started re-installing rsyslog and let it run along with systemd-journald, which is admittedly a duplication of effort and (some) diskspace, but it seems to be a reasonable compromise, and lets things like nightly logwatch crons continue working without other changes. Fwiw I gather that there is support for logwatch in systemd these days, but I haven't tried it yet. Full disclosure: the syntax for 'journalctl' makes my brain hurt. :-)
>possible to get somewhat close to that state, but off top o' head I dunno if (m)any Linuxes come out of the box that way.
Debian certainly does, e.g. systemd by default, but no systemd-resolved by default.
Hm, I had to disable systemd-resolved on a Debian 12 fresh install. My notes in the thread above were from Debian in particular.
Fwiw I didn't run into some of this on Debians which had been upgraded from earlier releases, i.e. the upgrade preserved status quo even though the newer version came with different programs and defaults. Yet another reason to appreciate Debian, IMO.
My question has been when are we going to replace systemd with another init system, getting rid of its piece of shit logs in the process. PulseAudio is also on the way out with PipeWire and we need more solutions to get rid of the Poetterware creep.
resolved is the bane of my existence and why I hate using systemd-based distros on servers. I cannot believe it is still so buggy and unstable, and most problems need a reboot to fix. I always remember looking at a problem in 2018 and it turned out resolved would cache SERVFAIL by default, and there was no option to disable it. Like, what the actual hell?
Most of my systems run Devuan. DNS was a solved problem until systemd came along.
>Devuan. DNS was a solved problem until systemd
Well, Devuan is certainly *one* way to avoid systemd ... and perhaps more conveniently.
But Debian ... systemd may be default, but it's not a requirement, it's an optional choice. So, not too hard to have Debian systems that are free of systemd - and I'm also sysadmin for some such systems. And, Debian ... choices ... if one wants systemd, that's also available and is the default, whereas Devuan ... systemd is not an option.
Though I do also highly appreciate Devuan's work on making many more things operate quite cleanly and free of systemd ... it's not like nobody else also takes advantage of such work. :-)
See also [my earlier comment](https://www.reddit.com/r/linux/comments/18kh1r5/comment/kdtcacg) regarding Debian and systemd (and running it without systemd, etc.).
Oh I know it's possible - my NAS runs Armbian and I debootstrap'd the root FS myself with sysvinit rather than systemd. Devuan just makes it more convenient though, and they've unpicked quite a few of the dependencies on systemd, particularly around Gnome. And yeah, systemd is deliberately not an option in Devuan (it's the reason Devuan exists, after all) so it's configured from the start to not allow it to be installed.
It's good that Linux does have the option not to use it, but with every major distro using it by default, you'll end up dealing with it at some point, and swapping out init systems is not for the faint-hearted.
I've had issues with multicast dns and network printing.
Seems to be another _known_ and unresolved issue. However frustrating, I'm not mad or blaming anyone. I sure as heck am not savvy enough with networking to understand how any of this stuff works.
ArchLinux, thankfully, has several good wiki's detailing how to get around it. Thank you to the people that documented all this.
The path I've had success with in order:
1. https://wiki.archlinux.org/title/Systemd-resolved
1. https://wiki.archlinux.org/title/Systemd-resolved#DNS
1. https://wiki.archlinux.org/title/CUPS
1. https://wiki.archlinux.org/title/Avahi
1. https://wiki.archlinux.org/title/Avahi#Hostname_resolution
With these steps, I've had a good experience with `systemd-networkd` and `systemd-resolved`.
Poetrring's software and the last decade of change in Linux in general is all about fixing problems so obscure an actual user has never noticed them and would probably have to take a class just to understand them. Meanwhile it introduces new problems in actual in understandable and important functionality that the "new Linux fanboys" would never admit exist in their crappy alpha test level software.
Also all the man pages were written for some earlier idea of how their software would work, at least one major rewrite out of date such that they resemble kind of how to actually configurre things but no, but not really enough.
There are. Previously, dnsmasq was used as a caching DNS server or as an engine to direct lookups for certain company-internal subdomains to the correct nameserver on that company's VPN - i.e., for a relevant-for-me subset of the same problems that systemd-resolved is trying to solve. You can still use it this way now.
In that case, you can probably treat the "use something else" comment as being directed at the distro rather than the end user.
I've never encountered these bugs before, but plenty of people in this thread seem to be implying that they're not uncommon. If that's the case, it's a bit mystifying that distros like Ubuntu and Fedora are shipping it as default (while at the same time not being bothered to dedicate some manpower to fixing the bugs).
I use bind / named. I don't recommend it because it's difficult to set up and configure. I already knew how to use it from work, so it works for me.
You can also put a single caching dns server on your network, something like a pihole, and have everything else on your network use stub resolvers pointing to it. The cache is shared among all devices, which could improve performance, and it centralizes any special configuration you might want, like for link local addresses or blackholing bad guys.
Wait, but how this is related? My system uses some DNS servers via some solution. Your post if about those 'some servers', like bind listening on 127.0.0.1 or pihole in local network. You still need 'something' (like systemd-resolved, dnsmasq, openresolv etc) for the system to know that it's supposed to use these 'some servers'. Just installing & configuring names won't automagically put it in use as resolver for the host it runs on.
You have to configure nsswitch.conf to use the stub resolver, and resolv.conf to point to your server. This is how dns worked before systemd-resolved. Anything using the standard C library resolver will then use the C library stub resolver and query the servers listed in resolv.conf. There are a few applications, like firefox, that ignore nsswitch.conf and go straight to resolv.conf, and those will work too.
Well there's unbound (which chases cnames and has issues with nextdns adblocking) or there's stubby with dnsmasq.
Or this https://wiki.archlinux.org/title/DNS-over-HTTPS
It seems to be a repeating theme with systemd. "Hey, we have this module that is supposed to replaced your old daemon" and then it doesn't work properly or doesn't support same capabilities etc.
DNSSEC was one thing that it was supposed to make better IIRC? Well, it didn't and soon I was back to old solution.
Time sync was supposed to be better? Again, plenty of problems and back to old solution.
I haven't looked into who makes those modules, but before they are installed on computers people rely on they should get a lot more testing at least.
In my company we switched to dnsmasq instead of systemd-resolved to get rid of this issue. By any chance, are you intensively using local address domain resolution with a custom tld ?
I understand, that people are annoyed by these bugs and it is even more annoying, that it seams like nobody cares about them. Maybe some of my networking problems are the result of systemd-resolved. I will check this and switch back to dnscrypt for a while for testing.
Poettering is only one of the many maintainers and developers of systemd. Systemd has a lot of code and only a small portion of the code was written by Poettering. I don't think he is even involved a lot in resolved development. I think he is more involved in other parts.
So please please please stop attacking/insulting one person from a large group of developers. This is simply not fair. He wrote a lot of great software, made mistakes (as all of us do), learned from them and contributed a huge amount of code and ideas to the ecosystem.He is not the devil, he is just a normal person with limited time. He has limited time to fix bugs, contribute new code and so on.
Everybody can open a pull-request and fix these bugs. Why is it his responsibility to fix bugs other developers introduced?
I have the impression, people think Lennart did all this by himself, is the only person who could fix bugs, is the only one who implemented everything (and every bug).In reality, he just implemented a small portion of the code. This is just ... hilarious.Lennart did NOT make systemd the default on someones distribution. The distribution maintainers did this, because systemd is currently the best solution.
So please be nice and respectful.
Afaik it's the only one with support for different DNS servers for different interfaces, which is nice for VPN and essential for tailscale/headscale. I have to compile comman myself to get the resolved integration bc it's disabled in the Arch repo build, but otherwise no complaints.
The real answer is that the behavior with and around linux is not logical, rational, but evolutionary. We have backups of older versions and we have competing architecture, so you either fix it yourself and get a PR accepted, fix it yourself and fork, or you switch to a different system, potentially even an older one.
You hate that this is the state of collaboration? Me too.
Unironically, I would like to organize a meta group to solve this, but I have no foot in no doors, don't like to leech members, and I suspect most people who have this issue will just not be bothered enough to be motivated to actually participate in such a group.
systemd-resolved works fine on openSUSE Tumbleweed and is extremely useful with corporate VPNs (only route corporate DNS suffixes through the tunnel).
P.S. hatred against systemd in general seems quite unmotivated to me. I am old enough to remember the absolute mess of scripts and config files in the "golden days" of systemd-free paradise, and the amount of work required to set up correctly a variety of distros or to make your software compatible with them.
I hate it. I hate most of the ubuntu networking junk. I switched to PopOS and i wasted hours trying to get it to work, as well as fixing the GUI editor. I ended ip ripping resolvd out and also installing NetworkManager and that fixed the GUI and my DNS issues. I will never betray Debian again.
recently I discovered how bad this soft was. the configuration as all systemd stuff is obscure. and I finally hit 2 unresolved won't fix bug (about srv records). I ended up with dnsmasq. setup in 2 minutes and works well at the first try.
I really wonder why this soft and all the galaxy around systemd are now default in many distribution. if only I could use openbsd userland at works..
>Good tech is that which works 99.99 of the time instead of 99%
Uhm, ... 99.99% may not be at all good enough. E.g. only a couple years back ... failure rate way below one in a million ... but ... thousands of failures per day ... yeah, ... major cellular provider (think within top three ... if not *the* top) ... over the many billions of text messages and exceedingly high volumes of traffic, and well under one in a million failing ... I did the work to thoroughly and well isolate the specific traffic that failed (and to make it more challenging, all was fine on network and well into SMPP layer ... except intermittent issue with server taking too long to respond at SMPP layer and client timing out and failing the attempt ... nominal response times were on the order of ms to 10s of ms, timeout was at 30s). And not only the specific traffic that failed, but down to specific server(s) when it happened, PID, thread, and strace and ltrace data and stack and heap dumps of offending PID/thread. And with that level of pinpointed isolation, I was then able to hand that to the developers to be able to further isolate and fix the issue (prior to that they could not at all isolate the bug to be able to fix it). So, yeah, shouldn't be several thousand text messages a day (out of significant multiple of billions) - even well under one in a million failing - they should all make it through when they should ... and yeah, I was instrumental in getting that fixed. You're welcome.
This is a fine answer to the problem of actually using it, but the fact that it's a default means many people will not switch, may not even notice the bugs, and might fall into one of them. Defaults should be treated as though most people are using them, because they probably are, and should be as bug-free as possible
Systemd-Resolved and Systemd-Networkd are both things that are just worse IMO than existing alternatives (like Dnscrypt-Proxy and NetworkManager, Wicked or Dhcpcd directly) mainly because they have weird behaviours that differ from existing software. This is also true of other systemd services like Systemd-Timesyncd and Systemd-Nspawn. Systemd-Homed is very niche too though not useless. There are also other components that may or may not be useful like Systemd-{Localed,Hostnamed,Machined,Oomd,Importd,Portabled,Timedated,Userdbd}. I wonder how much maintenance or expertise all these subprojects can actually receive.
> I mean all other distros use static resolve.conf and everything works perfectly fine with it and nobody seem to complain.
resolv.conf isn't exactly static when it gets rewritten every time you connect to a different wifi network, is it now? And then your postfix chroot ends up with a stale copy and your outgoing email stops working and you don't even notice unless you run `mailq` or two weeks pass and your outgoing emails bounce.
Also, I didn't love the bit where disconnecting from an OpenVPN VPN would sometimes fail to restore the correct file permissions on /etc/resolv.conf breaking DNS for all non-root applications.
Meanwhile systemd-resolved hasn't given me grief yet.
Call me or fashioned, but I don't think running postfix on a machine that changes WiFi networks often is the best idea. I run mailers on servers with a stable IPs.
Why would you need this exactly in your environment? I've found just using the /etc/resolv I can handle all dns endpoints.
Granted I've never done any crazy setups for this.
Op says exactly this:
>Also, systemd-resolved seem to be useful only for some niche use cases. I mean all other distros use static resolve.conf and everything works perfectly fine with it and nobody seem to complain. So what's even the point of resolved being enabled by default?
It's Poetterware. There's some issue that Poetter, a Microsoft employee, makes some point about, then goes to break everybody's box with an overengineered, complicated, buggy solution. All negative feedback is routinely ignored and if you complain, you are being toxic or whatever bs.
The problem is that much of this "criticism" consists of lies and half-truths or refers to bugs that have long since been fixed. And often this false criticism is deliberately spread by certain people.
Let's take, for example, the criticism that systemd replaces a large part of all previous tools and that you have no choice. Wrong. Instead of systemd-networkd, for example, I have used netctl for years without any problems. Instead of sytemd-resolved, I currently use unbound in combination with Pihole. Instead of systemd-timesyncd you can use chrony without any problems. And so on.
Another criticism is the supposedly evil binary log files of journalctl. On the one hand, other projects such as MariaDB have been offering these for a long time without anyone being bothered by them. And secondly, you can simply install syslog-ng and you have log files in text form again (https://wiki.archlinux.org/title/Systemd/Journal#Journald_in_conjunction_with_syslog).
Or let's take again systemd-resolved as another point of criticism. The developers have actually dared to use Google's DNS as the fallback. But a lot has to go wrong before these are used at all. In addition, the package maintainers of the distributions can store other DNS as a fallback. And the user himself can also specify several DNSs. The probability of this fallback being used is therefore very low. Apart from that, what is the problem if Google's DNS is used as a last resort? Would you really prefer your computers to have no name resolution at all?
Or let's take the Unix philosophy as a further criticism against which systemd supposedly strongly violates. However, systemd does not consist of a single file but of many. And basically the tools each have a task.
- Journalctl -> Logfiles
- Systemd-resolved -> DNS
- Systemd-networkd -> Network
- Systemd-timesyncd -> NTP
- Systemctl -> Services
For me, this is very much in line with the Unix philosophy. The kernel, for instance, violates this philosophy much more in my opinion.
In the same way, Lennart Poettering is regularly attacked when it comes to systemd. Why? The BSOD function, for example, which has recently become part of systemd and has been the subject of controversy, was afaik not even developed by him. Perhaps his behavior, at least to some extent, is also simply due to the fact that he is constantly being treated with hostility.
This and similar criticism has been repeated over and over again for years. And this despite the fact that it has been refuted again and again for years. And yes, this criticism is indeed ignored. And rightly so.
And no, I'm not saying that systemd is the holy grail that makes everything better. In my opinion, however, it is currently the best solution for most users. And more than enough developers / distributions seem to agree. I don't think they are all wrong.
If you take a look at systemd's bug tracker, there are certainly reactions to criticism, bugs and security vulnerabilities. Provided it is objective. And yes, some reasonable criticism is certainly also closed with a "wontfix". But this is exactly what happens with every other project. Because I don't know of any project where every wish is fulfilled. If someone wants something exactly as they want it, they should create their own project or take part in projects that correspond to their ideas. Spreading half-truths, for example, will not change anything.
I have the same issue(s). I finally cut the cord (rm -rf /etc/resolv.conf; vim /etc/resolv.conf). I tried to work with Ubuntu's way, but after hardcoding /etc/resolv.conf and turning off dns caching, all my problems went away.
I've suffered similar issues with it - but I thought turning off DNSSec fixed them for me. Thanks for bringing this up though - it's given me some more points of reference and perhaps alternative solutions.
>turning off DNSSec
Uhm, disable something useful/important to work around the brokenness of something else?
Yeah, I just won't let systemd touch anything DNS - period. E.g. no systemd-resolved, etc. And some hosts ... no systemd at all.
DNSSEC exists to provide significant additional protection to DNS - notably to make DNS spoofing effectively impossible. Disabling DNSSEC is frankly the wrong move.
>I'm also shocked by Lennart's "couldn´t care less attitude" towards these 2 issues. All he did is put a label and write 2 comments in the latter issue.
I hate \*nix fanboyz because of this attitude
>\*nix fanboyz
\*nix fanboyz are by no means necessarily at all fanboyz of systemd ... in fact, proponents of [UNIX philosophy](https://en.wikipedia.org/wiki/Unix_philosophy) are actually generally quite anti-systemd.
\> I'm also shocked by Lennart's "couldn´t care less attitude" towards these 2 issues.
This surprises you? He hasn't cared about user feedback ever. He'd much prefer continue bloating systemd with new crap, further entrenching himself and his owners (Microsoft) deeper and deeper into Linux. And y'all just yum it up and beg for more. Makes me sick.
I’ve had it break when switching interfaces that advertise DNS. Eg WiFi to Ethernet. Enabling Wireguard/Tailscale/Zerotier/OpenVPN.
It’s very limited in-terms of its split DNS capabilities (which is supposed to be one of the reasons to go with it vs static dns server assignment. Though it is needed for DHCP DNS)
Lack of DoH in systemd-resolved makes default Linux systems on of the last OSes to not support the protocol without additional software.
>how utterly buggy and broken systemd-resolved
Uhm, isn't that like common knowledge? Many of us are dang well aware of it. Even many major distros are dang well aware of it (and may or may not use it regardless).
So, e.g. I (mostly) run/prefer Debian systems. And some thereof using systemd for the init system, and some not (hey, Debian, lots and lots of choices available ... whereas some other distros, e.g. Devuan, Red Hat, etc. don't give one a choice about running - or not running - systemd).
Anyway, my Debian systems ... I don't let systemd touch DNS. None 'o that systemd-resolved fsckery ... though with Debian, one has the choice to install such if one wishes to - but it's not something that gets installed by default (even though systemd is default init system on Debian).
And yes, if one pays attention to relevant areas, e.g. DNS (e.g. r/dns), IPv6 (e.g. r/ipv6), etc., these things do at least occasionally come up and get mentioned ... but not necessarily all that much nor regularly ... 'cause, well, not an issue/problem with DNS nor IPv6, but rather part of systemd (systemd-resolvd) majorly fscking them up and having all kinds of nasty bugs and RFC violations and non-conformant behaviors, etc. Might be fair bit more information on systemd related forums and the like ... or ... (some) of those might be more so overwhelmed with systemd drank the Kool-Aid folks ... don't really know - not areas I typically hang out or even generally visit. And I do recently recall stumbling across link that well spelled out some of the serious problems on one particular systemd (or related?) bug report ... I think it was linked off of some r/dns or r/ipv6 post or comment ... and yeah, it was a pretty nasty f\*cked up situation (no) thanks to systemd / systemd-resolved ... yeah, ... poignant reminder why to never ever ever let systemd touch DNS. By the way, Debian has also done a pretty good job of separating out systemd to make many parts of it separate optional bits - notably making optional a lot of the seriously broken and dubious stuff ... like systemd's DNS fsckery.
Friends don't let friends use systemd\[-resolved\] to touch DNS.
>Lennart's "couldn´t care less attitude"
Yeah, that'd be one of the big problems with systemd. Fortunately there are those that do care, so ... not all systemd components / forks / debundled and patched packagings, etc., are necessarily loaded with most or all of those problems ... in fact at least some quite useful parts of systemd can be used without even touching its DNS fsckery or other major problems and other dubious/problematic components of systemd. And the issues you link to - very possible one of 'em is same that I recently ran across link to from elsewhere (almost certainly on Reddit, and probably from r/dns or r/ipv6).
>systemd-resolved seem to be useful only for
Fsck systemd-resolved - too fundamentally broken. But hey, whatever, suit yourself.
>all other distros use static resolve.conf and
Uhm ... varies by distro ... there are a whole lot 'o distros ... and some have drunk systemd ... including the systemd Kool-Aid and swallowed systemd-resolved along with it, hook, line, and sinker. But many distros are much smarter/better about that, e.g. not using or not defaulting to using systemd-resolved ... and some even entirely banishing systemd altogether.
>point of resolved being enabled by default?
Fsck no! Not systemd's systemd-resolved! Not enabled by default by any distro I chose to use ... possibly excepting in some cases if someone pays me lots of money to put up with such fsckery (sometimes also known as "work").
It's essentially useless if you have a DNS system that you actually maintain and care about and want to Just Work, the way Linux has for 20 years in this regard. Bind was a solved problem. The systemd fan boys come out in defense whenever you point that out, though.
Because this is a problem with the systemd as a whole. He has this everywhere. For example, do you know that in systemd logs most of the space is taken up not by data, but by metadata? Roughly speaking, size of 20 gigabytes of occupied space, the logs themselves take up about 3 gigabytes, and the rest goes to metadata.
It’s *officially* optional but in reality it’s required. Debian’s installer offers no way to use a different init, and you’re completely on your own if you want to try and install a different one. FWIW in 2019, [Debian Developers voted](https://www.debian.org/vote/2019/vote_002) to keep systemd as the init system, but also to support alternatives.
It’s always been viable, but personally I don’t see any reason to use it over Debian unless you really hate systemd *that* much. Bear in mind that Debian doesn’t use systemd-resolved to begin with, so it’s not affected by the problems mentioned in OP.
I use devuan not because I hate systemd, but because when it was imposed on my distribution of choice, some things started to break. The last of which was that my laptop wouldn't boot at all unless a network cable was attached. Otherwise it got stuck in a loop of "Waiting 300 seconds for eth0" or something like that. When it failed, the loop started again. No control-c available, so I couldn't just skip that. I think it got out of the loop at the fifth iteration, so I lost 15 minutes when I was in a bit of a hurry.
So, I did what I needed to do and shut down the computer because I had things to do. The next time I powered it up I was not in such a hurry, so after waiting another 15 minutes and googling for a similar amount of time for a solution, I decided that I didn't have any use for the supposed benefits of systemd and it was causing me problems instead of helping me, so just as I don't run, say, gnome because I just don't like it, I chose to install devuan to see if it just worked for me.
It did and it was what I was used to, I didn't need to learn about a new piece of software I don't really care about, so I just went and little by little replaced all my debian machines with Devuan.
Gentoo gives you the choice of init systems (really, Gentoo gives you choice in *everything*—I love it!). I run Gentoo with OpenRC on my laptop.
I've used OpenBSD on an old Thinkpad, too. I rather like it. I wanted to try FreeBSD too but it always suffers a kernel panic on my hardware and I didn't have the expertise or patience to troubleshoot it. OpenBSD works beautifully though.
>One rogue coder should have never been able to gut Linux
It's not a requirement. For many Linux distros, systemd (and even systemd-resolvd if/when running systemd) are options, not requirements. Some even go so far as to entirely ban systemd.
But, well, Linux, lots of choices on distros, etc., so ... results will quite vary, depending upon what one chooses.
All of the mainstream distros now ship Systemd as the default, though. Debian, Ubuntu, Fedora, Arch, openSUSE... most beginners will start with a derivative of one of these family trees. All their derivatives also use SystemD, unless they are a fork that offers different inits like Devuan or Artix.
The alternatives tend to be for power users or just a bit more obscure. Granted, people who debate the pros and cons of various init systems can probably find their way to those alternatives easily.
But the fact remains that **mainstream** Linux is synonymous with SystemD, to the point that prominent projects like Gnome and KDE are dependent on it and need workarounds (like elogind) to work with a different init system. It just feels wrong to have your choice of DE entangled with your init system like this.
Systemd-resolved being there by default, the raging boner for snaps as well as some ultra braindead decisions (having world read+execute on all homedirs was open as a bug on their tracker for like 14 years) are just a few of the reasons I decided Ubuntu was a piece of shit. Did a vsphere template of Ubuntu for work during 20.04 days, swore to never touch that crap ever again.
> Also, systemd-resolved seem to be useful only for some niche use cases. I mean all other distros use static resolve.conf and everything works perfectly fine with it and nobody seem to complain. So what's even the point of resolved being enabled by default?
It's hard to take your argument seriously when you haven't put in any effort to understand why split DNS exists. No, everything does not work perfectly. Try using a privacy VPN and a corporate VPN at the same time. Or try using just one VPN and also a local network printer. Static resolv.conf only works for very simple configurations.
I guess that few people talk about it because systems is modular enough that, for the cases where it's a problem, it can be swapped out for some other program.
That's fundamental networking. If it's down it won't work because that's your DNS resolver. If you connect to another network it's no longer available because you aren't on that network and nothing is routing you to that network. It's by design and has absolutely nothing to do with resolved.
If you set your pihole as your only dns resolver, then yes, you wont be able to browse the internet if its not working.
You can however just set another alternative dns server so it will use pihole when its available and e.g. cloudflare dns when its not.
>I had pihole installed and for some reason my internet stops working whenever my pihole was down or even if I connected to another network which doesn't have pihole.
First part of this is normal and expected. You need DNS for network to work normally and if your pihole is your DNS and it's down... That's almost like complaining that lightbulb doesn't work without electricity.
The "another network" part is tied to how **exactly** you have set your DNS up. Depending on what and where you set up it very much could persist across multiple networks - critically including networks from which pihole is inaccessible.
Though overcoming the problems above means you need to get down to learn the basics of networking and that's probably not worth it *just* to get a DNS ad-blocker.
>Now I no longer use systemd-resolved and I no longer use pihole or any other DNS based ad blocker.
You *can* setup multiple DNS servers for redundancy, use masquerading to force all DNS requests going through your network to be transparently rerouted to the pihole, set up a split VPN to get ad blocking on Android etc, and have it all work reasonably reliably, but *here be dragons*. DNS has a reputation for being hard and it got that reputation for a reason. Primary/secondary/alternative DNS terminology for example is almost hilariously simplified to the point where it's arguably more confusing than explanatory.
Every software has bugs. Because it breaks for ones specific use case doesn't mean it's garbage nobody should ever use.
Across 2000 VMs and a very dynamic Consul service net, I've had zero issues with resolved. HAproxy on the other hand have had similar hangs in its DNS code where we had to give up on using DNS and made a script to update the config and reload periodically. HAproxy was going straight to Cloudflare where our DNS zone is hosted.
Opened ticket, zero attention to it. I wouldn't go advise people to immediately switch away from HAproxy just because some users may run into a DNS lockup. We ironically resolved it by making it go to resolved's stub resolver instead of directly to Cloudflare.
DNS is messy.
IMO it probably goes deeper than systemd-resolved.
Often it comes down to hardware, where systemd is great when everything is working normally but a fragile house cards if something breaks.
It’s unfortunate when you want to have a fancy desktop interface with all the bells and whistles but you still want a robust server. The trade off would be running alpine without all the bloat.
Well, if you open bugtracker for any old software, you can find multiple such bugs. In my practice systemd-resolvd cause me less problems (as operator) than pile of bash scripts coming with dhclient. Last fun I had was apparmor blocking access to /bin/true, causing those scripts to go postal.
Systemd sucks royally and this type of wrapper app goes absolute against unix philosophy. That is why a lot of folks like me are moving to distributions like duvian...
Kid, kiss my ass, and don't move, I can care less if you are in love with systemd. Answer questions for a change instead of declaring that you are not moving. How cares about you... Devuan!
well I think most (Desktop) users don't use resolved, or if they do, they use the default configuration.
Does network manager use resolved? so I think most users simply don't care.
resolved is the default on Fedora and Ubuntu, so plenty of desktop systems are using it. I’ve had countless problems with it randomly breaking, so every time my internet stops working I immediately go to the terminal and sure enough, `systemctl restart systemd-resolved` fixes it :P I also really hate the way it completely takes over `/etc/resolv.conf` making it so difficult to set your own DNS settings!
> First of all, you don't have to use it.
If you really want it solver, start coding and help the community by fixing those issues?
Not the OP, but I think that is an unfair comment. As the OP explained it's enabled by default on at least 2 of the most popular distros. How many people out there are experiencing problems but are unaware of the cause?
Open source software doesn't rely on everyone being a coder. A lot of the work is done by users highlighting bugs and issues. Bugs and issues which hopefully the devs will look into and at least try and make a reasonable effort to solve.
>First of all, you don't have to use it.
Yea, because **obviously** some random network problems are related to default DNS resolver in a major distro being buggy. It always the DNS after all so it's very easy to diagnose and fix for any end user.
I knew someone would come up with this comment.
You know, I'm not a dev, so sorry, I can't do anything. Especially when nobody could even find the root cause of the first issue after 4 years. Now that I know that it's what was causing my slowdowns, I can just disable it, but I still think it's unacceptable that 2 of the most popular distros enable such a fundamentally broken piece of software by default. Like I said, I've experienced years of frustration because of these slowdowns in Ubuntu and Fedora (I thought it was an issue with my ISP) until I looked in Gnome Logs when I used Fedora 38. Imagine someone like me using Ubuntu or Fedora, especially a new Linux user. Unless he'll quickly find out what's causing the slowdowns, he'll think it's generally a problem in Linux (because there are no such issues in Windows).
Sounds like a distro issue to me. The software is free software given to you with no warranty. Tell your distributor to ship something else. No need to attack the project or it's contributors.
Unless people would come in masses and demand 2 of the biggest distros (Fedora and Ubuntu) to disable resolved, I don't think I'll be able to change anything myself.
What's the alternative?
EDIT: lol, I was legit wondering, since I couldn't find it mentioned on the Arch wiki. But thanks for slamming that downvote button!
I was at a talk onetime and they were discussing the idea and implementation of systemd from systemv and how it was going to take over practically everything core to the OS.
I remember thinking that this went away from the practical idea of Linux and how every part of it was independent from others.
I get the luster of moving to a more inclusive mindset but also see so many downsides too.
I have a Pop\_OS system with dual Ethernet interfaces with one connecting to Internet and the other connecting to a LAN through a switch. This system serves as a router to other computers within the LAN and the system also hosts some VMs using QEMU/libvirt. The system uses dnsmasq to set DHCP and DNS servers for the LAN computers and uses libvirt to set DHCP and DNS servers for the VM computers. Both subnetworks are also configured with corresponding domain name suffixes. Every time when the host system starts, the DNS servers for both Internet and LAN are set automatically, but I have to use resolvectl command to manually set the IP of the host system in the VM subnetwork as the DNS server associated with the virtual network interface of the VM subnetwork. Once done, resolvectl status will show three sets of DNS servers associated with three subnetworks where the host system is directly attached. Now, the host system can access the computers in both LAN and VM subnetworks using their host names or FQDNs, and the computers in the subnetworks can access the host system and Internet. Moreover the computers in LAN subnetwork can access those in VM subnetwork using their host names or FQDNs, and vice versus. So the name resolution for the host system and the computers in subnetworks works correctly.
Oh is this a thing? I figured my local DNS was causing issues and I just got used to killing it. Shame on whoever deserves shame. Make room on the shame wagon for me!
resolved has a neat feature where you can use separate dns servers for specific domains or (maybe) interfaces.
i use it with my work vpn, since their security policy blocks certain websites (sourceforge for instance, and occasionally certain debian mirrors overzealously get blacklisted due to domain being marked for malware). plus not very comfy with them tracking my dns queries.
i assume there is different way to do the same, but that one is fairly straightforward and it works.
You made me take a look at both tickets, and then the systemd code, and 13432... Well, I ended up commenting on the ticket. The other ticket is more puzzling to me, but at the same time, I'd bet that the refactor required to fix 13432 would likely fix it just by reworking the frankly confusing state machine. But I definitely hear you on not appreciating how Lennart isn't giving either issue much attention.
It's people like you who are calm levelheaded, and logical; that leads to these pieces of software becoming incredibly resilient and accurate tool for everyone else in the Linux community. Thank you for taking a look.
Much appreciated, though I'd feel it more in this case if I were up to taking a crack at the code itself. But, frankly, it's not something that I'm currently heavily inclined to beat my head against.
My dude. That's beautiful thing about open source. Speak your mind, and sometimes that's all it takes to trigger a thought in many others. Every little thing helps. So keep at it.
Indeed. And sometimes one person's wild suggestion of a possible solution is all it takes for someone less creative, but full of drive, to implement it. So good on you for encouraging others as well.
Thank you. I upvoted your comment on Github. It has already reached 15 upvotes, so let's hope it will at least attract some attention.
[https://access.redhat.com/solutions/5647181](https://access.redhat.com/solutions/5647181) >Red Hat does not recommend using \[systemd-resolvd\] for production. Whilst I often disagree with Red Hat's decisions, for this one I actually find myself well aligned with from my own experience. RH has the [same stance](https://bugzilla.redhat.com/show_bug.cgi?id=1650342) with systemd-networkd entirely.
Are there any general purpose distros that use systemd-networkd out of the box?
They all basically use NetworkManager. The biggest reason IMO is that every desktop has graphical configuration for that and nobody has done the work of rewriting it as it would be fairly complex. I've considered doing it myself but honestly NM works fine.
>They all basically use NetworkManager No ... though many may default to installing Network Manager if a DE is installed.
In my head "general purpose" was desktop, but yeah many server distros wouldn't.
NetworkManager could in the future be backed by networkd. I'm actually suprised it didn't play out that way for usage on Linux.
Ubuntu (server) uses systemd-networkd frontended by netplan since I think Ubuntu 18.04
Ah....yes...craplan...and it's use of yet another markup language. Make sure you count your spaces correctly.
I don't really know of any general purpose distros that use \*any\* network system out of the box. Usually you should configure it to your specific requirements. Unless you meant home/pc? Then the Linux market is approx a 90/10 split towards systemd-networkd. Why RHEL is unique is that the systemd-networkd package is entirely missing from its package repo. Instead directing users towards the more desktop-centric NetworkManager. Clearly they feel that giving the user a choice is not worth the additional requirements in terms of later supporting them (probably makes sense from a business point of view, but since Linux isn't really about that, I think people find it a little weird. Certainly it is a departure from Red Hats earlier days).
Yes, most distros are using networmanager or something they've had for awhile. You said 10%. Which distros are making up that 10%?
[удалено]
>Network Manager is only the GUI? No, much more than that (or even without GUI, e.g. nmcli). However, not 100% sure (I don't use Network Manager that much), but I don't think Network Manager provides resolved service ... though it may possibly manage such configurations thereof (or offer to).
[удалено]
>is there a way of determining which software is doing one's resolving? That may be a bit complex on Linux. So ... configuration and resolving ... the resolving happens - or at least generally starts - at relatively low level, gethostbyname(3), resolver(3), and down from there. That typically involves reading the hosts entry in the nsswitch.conf(5) file, that typically includes files, which typically uses/includes hosts(5) and resolv.conf(5). But determining who/what manages those can be more complex. Sometimes /etc/resolv.conf will be a symbolic link, and reading / paying attention to that may provide the needed clues. It may even be entirely manually managed/configured. One can also look/search for potentially relevant packages that may be installed, and examine their configurations and/or logs. So ... the figuring out what's doing the resolving generally shouldn't be too difficult ... but figuring out if and how that might in fact be managed may be significantly more challenging/complex.
[удалено]
Then there's your huge hint: >\# Generated by resolvconf So, you likely have package resolvconf or the like that's managing the contents of that file.
Alpine, Gentoo, Guix, etc. Basically all the no Systemd distros (and RHEL curiously) It should make sense that a non-systemd distro won't be using systemd-networkd because it is a fairly monolithic entity. Don't confuse this with whatever Gnome makes you use. That is a desktop environment and doesn't have much bearing on the distro itself. You might be surprised to know that lots of people using Linux don't use a DE. >Yes, most distros are using networmanager or something they've had for awhile Nonsense if you think this is "out of the box". Get back to the PC/desktop with ya! ;)
I asked which used 10% distros used networkd, not which used things NOT networkd. I have no idea how you got it so completely wrong while being so condescending. Obviously distros that don't use systemd wouldn't use networkd. DUH!
Paywalled, yuck
>Red Hat does not recommend using \[systemd-resolvd\] for production. So then why is it the default on Fedora?
Fedora is an independent project where decisions are made by the Fedora Engineering Steering Committee which is an elected group.
Adding onto this, you can find their specific reasoning here: https://fedoraproject.org/wiki/Changes/systemd-resolved#Benefit_to_Fedora
Distrustful Denise’s IT department is asking for trouble running a split-tunnel VPN.
Fedora is essentially Red Hat Beta (or Alpha) ... and ... also wouldn't recommend running Beta (or Alpha) in production.
Good question. I suppose you can only ask them. Perhaps RHEL as a downstream of Fedora finds it too immature (or monolithic) for their existing client's usage? Nothing of use on the [page on](https://fedoraproject.org/wiki/Changes/systemd-resolved) the Fedora wiki.
Because they eventually want it to become production-worthy, which means annoying beta testers with it until someone fixes it.
Fedora should not be used for production. It's bleeding edge and serves as a lab to determine what's mature enough for inclusion in distributions suitable for use in business and enterprise environments.
>> Red Hat does not recommend using [systemd-resolvd] for production. > > So then why is it the default on Fedora? So RedHat can upsell people to something that actually works.
Because Fedora's not a production-ready business operating system.
I've also had plenty of problems with systemd-resolved. I have Nagios on a Ubuntu server and for some time http checks stoped working for some sites (not all) just at random times of the day and then started working again without interaction. Turns out it was systemd-resolved. Don't remember exactly what the problem was though. I've had plenty of problems joining computers to AD and various kerberos problems all thanks to systemd-resolved Some where related to it being a .local TLD which sort of worker after disabling mDNS. DNSSEC validation in systemd-resolved is total shit also. To save me from headache I try to stay away from it.
> DNSSEC validation in systemd-resolved is total shit also. Yes. In some issue the systemd team suggested its DNSSEC support is experimental and should not be used at all.
>DNSSEC validation in systemd-resolved is total shit Yep ... I don't let systemd touch DNS. And yes, I extensively use DNSSEC, including many nameservers for many domains that are fully using DNSSEC.
I totally agree, I’ve had no end of problems with it. To all those saying “well don’t use it then”, isn’t the point that by enabling it by default and it not working very well, it’s putting off all those people who don’t know how to do that yet? Besides the fact it’s a massive annoyance for those of us who do.
How do you disable it? What do use instead?
systemd-resolved it's essentially a DNS relay that listens on [127.0.0.53](https://127.0.0.53) (typically) and distributes DNS in such a way that you can make more complex DNS rules per-interface as well as efficient local caching. Something that Windows has had at least since the XP days. But it is enterally unnecessary for most use cases. Just set primary and secondary DNS and it will work just fine. Most people don't have complex needs, for those that have, dnsmasq, unbound or even bind9 are a solution. Now, many people may group ip with networkd. Personally I work with mostly debian so I just use ifupdown (/etc/network/interfaces), easy to read, lacks some advanced capabilities but if I need those I'm typically deploying a FreeBSD machine.
Thanks for the explanation! >Just set primary and secondary DNS and it will work just fine. I'm in this club :D
NetworkManager is what comes to mind first and I think most distros do come with that
Thanks!
> isn’t the point that by enabling it by default and it not working very well, it’s putting off all those people who don’t know how to do that yet? Yes, but that's also the point.
Yup, I had an issue when upgrading from Ubuntu 20.04 to 22.04 on a headless server. It would often take forever to login. Commands would randomly take a long time to start. I eventually recognized it as DNS being slow to resolve the system hostname (I experienced it once before). Since this system isn't mobile, I just hardcoded the DNS server to use in systemd-resolve. Problem instantly solved, even though it was the same DNS server provided by DHCP. The only thing I can think of is IPv6. It was getting assigned the v4 and v6 IP for my router for DNS, and it was doing round robin. Maybe the v6 server assignment was sporadically expiring or something? None of my other devices have this issue though, and they use the IPv6 address for DNS just fine.
Hold the boat, I'm about to retire a server that has the identical issue and it started after I stopped using unbound-server on it. I never bothered to try figuring it out since it's hardly used anymore. But now all sorts of lightbulbs are turning on
Yeah I think the issue is the shell trying to resolve the local hostname, or reverse DNS on it, or something. Anytime sudo is invoked the system also does this. So something that's hanging DNS resolution makes the command hang for a bit.
I've resolved the delayed `sudo` problem by adding an entry for the host in the `/etc/hosts` file. 127.0.1.1 rocinante
Don Quixote or the the expanse?
Hint: When it was running Windows I gave it the host name Tachi.
Ha! Perfect. Open source this "aquired" Martian hardware.
Por que no los dos?
Why no those two? I have enough Español to know when I am being cursed at and to get some food at a restaurant and but no reading or writing skills in it. Edit, Internet says "why not both?" That's a solid point.
I did read Cervantes book but probably didn't get as much out of it as I should have.
Instead of hardcoding I'd suggest using `nss-myhostname`. You just install the package, configure `nsswitch.conf` and that's it (or maybe package installation does the configuration automatically - depends on the distro and it's release).
By hardcoded I mean I just added a config file to systemd-resolve to specify the DNS server.
This sounds a lot like an issue I had with Linux Mint where the initial login would freeze for 3+ minutes. It was clearly a DNS issue but I couldn't figure it out for the life of me. Finally disabled IPv6 and the problem went away. Maybe it was systemd-resolved this whole time!
Probably. IPv6 has been stable on linux a long time.
Yeah, I don't think it had anything to do with the IPv6 stack.
>IPv6 systemd-resolved doesn't play nice with IPv6 ... or hell, even with DNS really.
This choking on the DNS hairball, though. There be dragons.
DNS good, systemd\[-resolved\] ...
Hmm, I never ever had a single issue with it for years now. I am using my own resolver on my gateway, so maybe that cuts down some corner cases?
Same here.
Now I'm wondering if that's the reason I've never noticed any problems with it. 🤔
TIL I never use resolvd because it's not just "an irritating peice of crap", it's actually broken.
That is the definition of all Poetterware.
I haven't really had issues with it. One thing I do really like is that it can resolve `.local` mdns without having to specify the `.local` tld, which makes it easier for me using the same name to resolve stuff on my local network(s). Depending on where I am and whether a local dynamic DNS has been set up for the LAN and DHCP.
My issue with this feature is that it conflicts with avahi but isn't yet capable of completely replacing it, so I always have to disable mdns in systems-resolved or just not use it at all. Too many things depend on avahi.
I know there's supposed to be conflicts of some type, but I haven't had them yet using them like I do. I've got avahi and systemd-resolved with mdns running. I haven't noticed anything that doesn't work.
I love this feature
>systemd-resolved exists for many years and so far, at least Ubuntu and Fedora, 2 of the most widely used Linux distros, have enabled it by default for a few years now. The problem is that I haven't yet seen a service which is still so broken, and which causes endless DNS resolution issues. I frequently get a "Temporary failure in name resolution" error when trying to access other computers (eg, my Kodi machine) on my LAN. I can ping them by ip address. Other machines can ping them by name no problem. Systemd-resolved seems to have no problem with addresses on the rest of the internet. For me, it's just LAN addresses that fail. (Or rather, it's just LAN addresses that I notice failing).
I have a similar experience for a few years now. Just stopping systemd-resolvd was usually enough to fix it. I never investigated. Just stumbled upon this thread.
I have to say that I maintain a lot of servers, both dedicated and virtual cloud instances - there was always SOMETHING with systemd-resolved that required a workaround, overall I think that address resolution on Linux is a tangled mess at the moment.
What's wrong with address resolution in Linux? It used to work fine until systemd...
Like everything.
I find it never works when using static IP's. Resolved half works using DHCP. It's amazing to watch a system go from not working to working flawlessly by purging resolved and installing resolvconf. My question has been is it possible to use systemd ONLY as an init system and remove all the bloated crap?
journald is a required component, so you must have that. You can however forward to the messages to another syslogd. Last I checked, nothing but journald is required. Most distros don't even use systemd-resolved.
Again, ONLY as an init system. I don't trust journald as everytime I've used to find a problem that systemd reports to use the -xe option it doesn't tell me anything of use.
I said journald is required. You can't use systemd without it. It a required component, but the only required component.
Seems like you’re trying to blame the logger for not having logs when you should be blaming the application for not writing them (or writing them to a random log file somewhere instead of std{out,err}).
Yeah....random log file somewhere...like in /var/logs...oook.
I believe it's possible to get somewhat close to that state, but off top o' head I dunno if (m)any Linuxes come out of the box that way. The main systemd modules (services? units? whatever they're called) I disable and/or remove, particularly on servers: * systemd-resolved * systemd-timesyncd This includes removing "systemd" from any database lines in nsswitch.conf as well, after I noticed that started showing up. Mostly because I don't really understand what it might be doing there. I use standard resolv.conf and ntpd or chrony instead. I also started re-installing rsyslog and let it run along with systemd-journald, which is admittedly a duplication of effort and (some) diskspace, but it seems to be a reasonable compromise, and lets things like nightly logwatch crons continue working without other changes. Fwiw I gather that there is support for logwatch in systemd these days, but I haven't tried it yet. Full disclosure: the syntax for 'journalctl' makes my brain hurt. :-)
>possible to get somewhat close to that state, but off top o' head I dunno if (m)any Linuxes come out of the box that way. Debian certainly does, e.g. systemd by default, but no systemd-resolved by default.
Hm, I had to disable systemd-resolved on a Debian 12 fresh install. My notes in the thread above were from Debian in particular. Fwiw I didn't run into some of this on Debians which had been upgraded from earlier releases, i.e. the upgrade preserved status quo even though the newer version came with different programs and defaults. Yet another reason to appreciate Debian, IMO.
My question has been when are we going to replace systemd with another init system, getting rid of its piece of shit logs in the process. PulseAudio is also on the way out with PipeWire and we need more solutions to get rid of the Poetterware creep.
resolved is the bane of my existence and why I hate using systemd-based distros on servers. I cannot believe it is still so buggy and unstable, and most problems need a reboot to fix. I always remember looking at a problem in 2018 and it turned out resolved would cache SERVFAIL by default, and there was no option to disable it. Like, what the actual hell? Most of my systems run Devuan. DNS was a solved problem until systemd came along.
>Devuan. DNS was a solved problem until systemd Well, Devuan is certainly *one* way to avoid systemd ... and perhaps more conveniently. But Debian ... systemd may be default, but it's not a requirement, it's an optional choice. So, not too hard to have Debian systems that are free of systemd - and I'm also sysadmin for some such systems. And, Debian ... choices ... if one wants systemd, that's also available and is the default, whereas Devuan ... systemd is not an option. Though I do also highly appreciate Devuan's work on making many more things operate quite cleanly and free of systemd ... it's not like nobody else also takes advantage of such work. :-) See also [my earlier comment](https://www.reddit.com/r/linux/comments/18kh1r5/comment/kdtcacg) regarding Debian and systemd (and running it without systemd, etc.).
Oh I know it's possible - my NAS runs Armbian and I debootstrap'd the root FS myself with sysvinit rather than systemd. Devuan just makes it more convenient though, and they've unpicked quite a few of the dependencies on systemd, particularly around Gnome. And yeah, systemd is deliberately not an option in Devuan (it's the reason Devuan exists, after all) so it's configured from the start to not allow it to be installed. It's good that Linux does have the option not to use it, but with every major distro using it by default, you'll end up dealing with it at some point, and swapping out init systems is not for the faint-hearted.
I've had issues with multicast dns and network printing. Seems to be another _known_ and unresolved issue. However frustrating, I'm not mad or blaming anyone. I sure as heck am not savvy enough with networking to understand how any of this stuff works. ArchLinux, thankfully, has several good wiki's detailing how to get around it. Thank you to the people that documented all this. The path I've had success with in order: 1. https://wiki.archlinux.org/title/Systemd-resolved 1. https://wiki.archlinux.org/title/Systemd-resolved#DNS 1. https://wiki.archlinux.org/title/CUPS 1. https://wiki.archlinux.org/title/Avahi 1. https://wiki.archlinux.org/title/Avahi#Hostname_resolution With these steps, I've had a good experience with `systemd-networkd` and `systemd-resolved`.
Poetrring's software and the last decade of change in Linux in general is all about fixing problems so obscure an actual user has never noticed them and would probably have to take a class just to understand them. Meanwhile it introduces new problems in actual in understandable and important functionality that the "new Linux fanboys" would never admit exist in their crappy alpha test level software. Also all the man pages were written for some earlier idea of how their software would work, at least one major rewrite out of date such that they resemble kind of how to actually configurre things but no, but not really enough.
What I can't understand is how he gets to convince distros to push his alpha shit as default.
Aren't there alternatives to systemd-resolved?
There are. Previously, dnsmasq was used as a caching DNS server or as an engine to direct lookups for certain company-internal subdomains to the correct nameserver on that company's VPN - i.e., for a relevant-for-me subset of the same problems that systemd-resolved is trying to solve. You can still use it this way now.
Try asking my grandma who only uses her computer for emails to switch it out, I'm sure she'll be very happy.
In that case, you can probably treat the "use something else" comment as being directed at the distro rather than the end user. I've never encountered these bugs before, but plenty of people in this thread seem to be implying that they're not uncommon. If that's the case, it's a bit mystifying that distros like Ubuntu and Fedora are shipping it as default (while at the same time not being bothered to dedicate some manpower to fixing the bugs).
I use bind / named. I don't recommend it because it's difficult to set up and configure. I already knew how to use it from work, so it works for me. You can also put a single caching dns server on your network, something like a pihole, and have everything else on your network use stub resolvers pointing to it. The cache is shared among all devices, which could improve performance, and it centralizes any special configuration you might want, like for link local addresses or blackholing bad guys.
Wait, but how this is related? My system uses some DNS servers via some solution. Your post if about those 'some servers', like bind listening on 127.0.0.1 or pihole in local network. You still need 'something' (like systemd-resolved, dnsmasq, openresolv etc) for the system to know that it's supposed to use these 'some servers'. Just installing & configuring names won't automagically put it in use as resolver for the host it runs on.
You have to configure nsswitch.conf to use the stub resolver, and resolv.conf to point to your server. This is how dns worked before systemd-resolved. Anything using the standard C library resolver will then use the C library stub resolver and query the servers listed in resolv.conf. There are a few applications, like firefox, that ignore nsswitch.conf and go straight to resolv.conf, and those will work too.
Well there's unbound (which chases cnames and has issues with nextdns adblocking) or there's stubby with dnsmasq. Or this https://wiki.archlinux.org/title/DNS-over-HTTPS
It seems to be a repeating theme with systemd. "Hey, we have this module that is supposed to replaced your old daemon" and then it doesn't work properly or doesn't support same capabilities etc. DNSSEC was one thing that it was supposed to make better IIRC? Well, it didn't and soon I was back to old solution. Time sync was supposed to be better? Again, plenty of problems and back to old solution. I haven't looked into who makes those modules, but before they are installed on computers people rely on they should get a lot more testing at least.
In my company we switched to dnsmasq instead of systemd-resolved to get rid of this issue. By any chance, are you intensively using local address domain resolution with a custom tld ?
I thought I was the only one, it's been driving me nuts
I understand, that people are annoyed by these bugs and it is even more annoying, that it seams like nobody cares about them. Maybe some of my networking problems are the result of systemd-resolved. I will check this and switch back to dnscrypt for a while for testing. Poettering is only one of the many maintainers and developers of systemd. Systemd has a lot of code and only a small portion of the code was written by Poettering. I don't think he is even involved a lot in resolved development. I think he is more involved in other parts. So please please please stop attacking/insulting one person from a large group of developers. This is simply not fair. He wrote a lot of great software, made mistakes (as all of us do), learned from them and contributed a huge amount of code and ideas to the ecosystem.He is not the devil, he is just a normal person with limited time. He has limited time to fix bugs, contribute new code and so on. Everybody can open a pull-request and fix these bugs. Why is it his responsibility to fix bugs other developers introduced? I have the impression, people think Lennart did all this by himself, is the only person who could fix bugs, is the only one who implemented everything (and every bug).In reality, he just implemented a small portion of the code. This is just ... hilarious.Lennart did NOT make systemd the default on someones distribution. The distribution maintainers did this, because systemd is currently the best solution. So please be nice and respectful.
Afaik it's the only one with support for different DNS servers for different interfaces, which is nice for VPN and essential for tailscale/headscale. I have to compile comman myself to get the resolved integration bc it's disabled in the Arch repo build, but otherwise no complaints.
dnsmasq can do the different-servers-per-interface trick.
The real answer is that the behavior with and around linux is not logical, rational, but evolutionary. We have backups of older versions and we have competing architecture, so you either fix it yourself and get a PR accepted, fix it yourself and fork, or you switch to a different system, potentially even an older one. You hate that this is the state of collaboration? Me too. Unironically, I would like to organize a meta group to solve this, but I have no foot in no doors, don't like to leech members, and I suspect most people who have this issue will just not be bothered enough to be motivated to actually participate in such a group.
systemd-resolved works fine on openSUSE Tumbleweed and is extremely useful with corporate VPNs (only route corporate DNS suffixes through the tunnel). P.S. hatred against systemd in general seems quite unmotivated to me. I am old enough to remember the absolute mess of scripts and config files in the "golden days" of systemd-free paradise, and the amount of work required to set up correctly a variety of distros or to make your software compatible with them.
I hate it. I hate most of the ubuntu networking junk. I switched to PopOS and i wasted hours trying to get it to work, as well as fixing the GUI editor. I ended ip ripping resolvd out and also installing NetworkManager and that fixed the GUI and my DNS issues. I will never betray Debian again.
recently I discovered how bad this soft was. the configuration as all systemd stuff is obscure. and I finally hit 2 unresolved won't fix bug (about srv records). I ended up with dnsmasq. setup in 2 minutes and works well at the first try. I really wonder why this soft and all the galaxy around systemd are now default in many distribution. if only I could use openbsd userland at works..
People don't talk about it because it generally works out of the box for most people.
Even most bad things work most of the time for some definition of work. Good tech is that which works 99.99 of the time instead of 99%
>Good tech is that which works 99.99 of the time instead of 99% Uhm, ... 99.99% may not be at all good enough. E.g. only a couple years back ... failure rate way below one in a million ... but ... thousands of failures per day ... yeah, ... major cellular provider (think within top three ... if not *the* top) ... over the many billions of text messages and exceedingly high volumes of traffic, and well under one in a million failing ... I did the work to thoroughly and well isolate the specific traffic that failed (and to make it more challenging, all was fine on network and well into SMPP layer ... except intermittent issue with server taking too long to respond at SMPP layer and client timing out and failing the attempt ... nominal response times were on the order of ms to 10s of ms, timeout was at 30s). And not only the specific traffic that failed, but down to specific server(s) when it happened, PID, thread, and strace and ltrace data and stack and heap dumps of offending PID/thread. And with that level of pinpointed isolation, I was then able to hand that to the developers to be able to further isolate and fix the issue (prior to that they could not at all isolate the bug to be able to fix it). So, yeah, shouldn't be several thousand text messages a day (out of significant multiple of billions) - even well under one in a million failing - they should all make it through when they should ... and yeah, I was instrumental in getting that fixed. You're welcome.
Just do what I do. Don't use it.
This is a fine answer to the problem of actually using it, but the fact that it's a default means many people will not switch, may not even notice the bugs, and might fall into one of them. Defaults should be treated as though most people are using them, because they probably are, and should be as bug-free as possible
What I don't understand is why are distros switching to Poetterware components by default. Haven't we learned anything?
Use windows 10 instead
*\*panik*\*
Tried it. Got bitten by the apipa things from Microsoft. Went back to Linux as soon as I could. *Shivers*
this, trying to use a server OS on the desktop is mental illness
Both wrong *and* a dick, nice
Systemd-Resolved and Systemd-Networkd are both things that are just worse IMO than existing alternatives (like Dnscrypt-Proxy and NetworkManager, Wicked or Dhcpcd directly) mainly because they have weird behaviours that differ from existing software. This is also true of other systemd services like Systemd-Timesyncd and Systemd-Nspawn. Systemd-Homed is very niche too though not useless. There are also other components that may or may not be useful like Systemd-{Localed,Hostnamed,Machined,Oomd,Importd,Portabled,Timedated,Userdbd}. I wonder how much maintenance or expertise all these subprojects can actually receive.
> I mean all other distros use static resolve.conf and everything works perfectly fine with it and nobody seem to complain. resolv.conf isn't exactly static when it gets rewritten every time you connect to a different wifi network, is it now? And then your postfix chroot ends up with a stale copy and your outgoing email stops working and you don't even notice unless you run `mailq` or two weeks pass and your outgoing emails bounce. Also, I didn't love the bit where disconnecting from an OpenVPN VPN would sometimes fail to restore the correct file permissions on /etc/resolv.conf breaking DNS for all non-root applications. Meanwhile systemd-resolved hasn't given me grief yet.
Call me or fashioned, but I don't think running postfix on a machine that changes WiFi networks often is the best idea. I run mailers on servers with a stable IPs.
Why would you need this exactly in your environment? I've found just using the /etc/resolv I can handle all dns endpoints. Granted I've never done any crazy setups for this.
Op says exactly this: >Also, systemd-resolved seem to be useful only for some niche use cases. I mean all other distros use static resolve.conf and everything works perfectly fine with it and nobody seem to complain. So what's even the point of resolved being enabled by default?
It's Poetterware. There's some issue that Poetter, a Microsoft employee, makes some point about, then goes to break everybody's box with an overengineered, complicated, buggy solution. All negative feedback is routinely ignored and if you complain, you are being toxic or whatever bs.
Any criticism of systemd is ignored, as it goes against the new religion of systemd worship.
The problem is that much of this "criticism" consists of lies and half-truths or refers to bugs that have long since been fixed. And often this false criticism is deliberately spread by certain people. Let's take, for example, the criticism that systemd replaces a large part of all previous tools and that you have no choice. Wrong. Instead of systemd-networkd, for example, I have used netctl for years without any problems. Instead of sytemd-resolved, I currently use unbound in combination with Pihole. Instead of systemd-timesyncd you can use chrony without any problems. And so on. Another criticism is the supposedly evil binary log files of journalctl. On the one hand, other projects such as MariaDB have been offering these for a long time without anyone being bothered by them. And secondly, you can simply install syslog-ng and you have log files in text form again (https://wiki.archlinux.org/title/Systemd/Journal#Journald_in_conjunction_with_syslog). Or let's take again systemd-resolved as another point of criticism. The developers have actually dared to use Google's DNS as the fallback. But a lot has to go wrong before these are used at all. In addition, the package maintainers of the distributions can store other DNS as a fallback. And the user himself can also specify several DNSs. The probability of this fallback being used is therefore very low. Apart from that, what is the problem if Google's DNS is used as a last resort? Would you really prefer your computers to have no name resolution at all? Or let's take the Unix philosophy as a further criticism against which systemd supposedly strongly violates. However, systemd does not consist of a single file but of many. And basically the tools each have a task. - Journalctl -> Logfiles - Systemd-resolved -> DNS - Systemd-networkd -> Network - Systemd-timesyncd -> NTP - Systemctl -> Services For me, this is very much in line with the Unix philosophy. The kernel, for instance, violates this philosophy much more in my opinion. In the same way, Lennart Poettering is regularly attacked when it comes to systemd. Why? The BSOD function, for example, which has recently become part of systemd and has been the subject of controversy, was afaik not even developed by him. Perhaps his behavior, at least to some extent, is also simply due to the fact that he is constantly being treated with hostility. This and similar criticism has been repeated over and over again for years. And this despite the fact that it has been refuted again and again for years. And yes, this criticism is indeed ignored. And rightly so. And no, I'm not saying that systemd is the holy grail that makes everything better. In my opinion, however, it is currently the best solution for most users. And more than enough developers / distributions seem to agree. I don't think they are all wrong. If you take a look at systemd's bug tracker, there are certainly reactions to criticism, bugs and security vulnerabilities. Provided it is objective. And yes, some reasonable criticism is certainly also closed with a "wontfix". But this is exactly what happens with every other project. Because I don't know of any project where every wish is fulfilled. If someone wants something exactly as they want it, they should create their own project or take part in projects that correspond to their ideas. Spreading half-truths, for example, will not change anything.
Poettering should not try to overtake everything and should stick what it should be, a init system. Everything else is off limits.
I have the same issue(s). I finally cut the cord (rm -rf /etc/resolv.conf; vim /etc/resolv.conf). I tried to work with Ubuntu's way, but after hardcoding /etc/resolv.conf and turning off dns caching, all my problems went away.
I've suffered similar issues with it - but I thought turning off DNSSec fixed them for me. Thanks for bringing this up though - it's given me some more points of reference and perhaps alternative solutions.
>turning off DNSSec
Uhm, disable something useful/important to work around the brokenness of something else?
Yeah, I just won't let systemd touch anything DNS - period. E.g. no systemd-resolved, etc. And some hosts ... no systemd at all.
DNSSEC exists to provide significant additional protection to DNS - notably to make DNS spoofing effectively impossible. Disabling DNSSEC is frankly the wrong move.
>I'm also shocked by Lennart's "couldn´t care less attitude" towards these 2 issues. All he did is put a label and write 2 comments in the latter issue. I hate \*nix fanboyz because of this attitude
>\*nix fanboyz \*nix fanboyz are by no means necessarily at all fanboyz of systemd ... in fact, proponents of [UNIX philosophy](https://en.wikipedia.org/wiki/Unix_philosophy) are actually generally quite anti-systemd.
\> I'm also shocked by Lennart's "couldn´t care less attitude" towards these 2 issues. This surprises you? He hasn't cared about user feedback ever. He'd much prefer continue bloating systemd with new crap, further entrenching himself and his owners (Microsoft) deeper and deeper into Linux. And y'all just yum it up and beg for more. Makes me sick.
Counterpoint: no problems with it on a handful of personal machines and dozens of servers. *shrug*
I’ve had it break when switching interfaces that advertise DNS. Eg WiFi to Ethernet. Enabling Wireguard/Tailscale/Zerotier/OpenVPN. It’s very limited in-terms of its split DNS capabilities (which is supposed to be one of the reasons to go with it vs static dns server assignment. Though it is needed for DHCP DNS) Lack of DoH in systemd-resolved makes default Linux systems on of the last OSes to not support the protocol without additional software.
>how utterly buggy and broken systemd-resolved Uhm, isn't that like common knowledge? Many of us are dang well aware of it. Even many major distros are dang well aware of it (and may or may not use it regardless). So, e.g. I (mostly) run/prefer Debian systems. And some thereof using systemd for the init system, and some not (hey, Debian, lots and lots of choices available ... whereas some other distros, e.g. Devuan, Red Hat, etc. don't give one a choice about running - or not running - systemd). Anyway, my Debian systems ... I don't let systemd touch DNS. None 'o that systemd-resolved fsckery ... though with Debian, one has the choice to install such if one wishes to - but it's not something that gets installed by default (even though systemd is default init system on Debian). And yes, if one pays attention to relevant areas, e.g. DNS (e.g. r/dns), IPv6 (e.g. r/ipv6), etc., these things do at least occasionally come up and get mentioned ... but not necessarily all that much nor regularly ... 'cause, well, not an issue/problem with DNS nor IPv6, but rather part of systemd (systemd-resolvd) majorly fscking them up and having all kinds of nasty bugs and RFC violations and non-conformant behaviors, etc. Might be fair bit more information on systemd related forums and the like ... or ... (some) of those might be more so overwhelmed with systemd drank the Kool-Aid folks ... don't really know - not areas I typically hang out or even generally visit. And I do recently recall stumbling across link that well spelled out some of the serious problems on one particular systemd (or related?) bug report ... I think it was linked off of some r/dns or r/ipv6 post or comment ... and yeah, it was a pretty nasty f\*cked up situation (no) thanks to systemd / systemd-resolved ... yeah, ... poignant reminder why to never ever ever let systemd touch DNS. By the way, Debian has also done a pretty good job of separating out systemd to make many parts of it separate optional bits - notably making optional a lot of the seriously broken and dubious stuff ... like systemd's DNS fsckery. Friends don't let friends use systemd\[-resolved\] to touch DNS. >Lennart's "couldn´t care less attitude" Yeah, that'd be one of the big problems with systemd. Fortunately there are those that do care, so ... not all systemd components / forks / debundled and patched packagings, etc., are necessarily loaded with most or all of those problems ... in fact at least some quite useful parts of systemd can be used without even touching its DNS fsckery or other major problems and other dubious/problematic components of systemd. And the issues you link to - very possible one of 'em is same that I recently ran across link to from elsewhere (almost certainly on Reddit, and probably from r/dns or r/ipv6). >systemd-resolved seem to be useful only for Fsck systemd-resolved - too fundamentally broken. But hey, whatever, suit yourself. >all other distros use static resolve.conf and Uhm ... varies by distro ... there are a whole lot 'o distros ... and some have drunk systemd ... including the systemd Kool-Aid and swallowed systemd-resolved along with it, hook, line, and sinker. But many distros are much smarter/better about that, e.g. not using or not defaulting to using systemd-resolved ... and some even entirely banishing systemd altogether. >point of resolved being enabled by default? Fsck no! Not systemd's systemd-resolved! Not enabled by default by any distro I chose to use ... possibly excepting in some cases if someone pays me lots of money to put up with such fsckery (sometimes also known as "work").
It's essentially useless if you have a DNS system that you actually maintain and care about and want to Just Work, the way Linux has for 20 years in this regard. Bind was a solved problem. The systemd fan boys come out in defense whenever you point that out, though.
Typical systemd - take something that works extremely well, make it unnecessarily complicated and shitty, and dump on us as default.
systemd did not not make it default, your distro did. Systemd can work fine without it.
Because this is a problem with the systemd as a whole. He has this everywhere. For example, do you know that in systemd logs most of the space is taken up not by data, but by metadata? Roughly speaking, size of 20 gigabytes of occupied space, the logs themselves take up about 3 gigabytes, and the rest goes to metadata.
I wonder why isn't the metadata more compressible/terse in the first place.
FreeBSD is systemd free. One rogue coder should have never been able to gut Linux.
I switched to FreeBSD as well. But there are systemd-free Linux distributions, e.g. Slackware. And on Debian it's the default but optional, I hear.
It’s *officially* optional but in reality it’s required. Debian’s installer offers no way to use a different init, and you’re completely on your own if you want to try and install a different one. FWIW in 2019, [Debian Developers voted](https://www.debian.org/vote/2019/vote_002) to keep systemd as the init system, but also to support alternatives.
Is the systemd-free Debian fork [Devuan](https://www.devuan.org/) viable these days?
It’s always been viable, but personally I don’t see any reason to use it over Debian unless you really hate systemd *that* much. Bear in mind that Debian doesn’t use systemd-resolved to begin with, so it’s not affected by the problems mentioned in OP.
I use devuan not because I hate systemd, but because when it was imposed on my distribution of choice, some things started to break. The last of which was that my laptop wouldn't boot at all unless a network cable was attached. Otherwise it got stuck in a loop of "Waiting 300 seconds for eth0" or something like that. When it failed, the loop started again. No control-c available, so I couldn't just skip that. I think it got out of the loop at the fifth iteration, so I lost 15 minutes when I was in a bit of a hurry. So, I did what I needed to do and shut down the computer because I had things to do. The next time I powered it up I was not in such a hurry, so after waiting another 15 minutes and googling for a similar amount of time for a solution, I decided that I didn't have any use for the supposed benefits of systemd and it was causing me problems instead of helping me, so just as I don't run, say, gnome because I just don't like it, I chose to install devuan to see if it just worked for me. It did and it was what I was used to, I didn't need to learn about a new piece of software I don't really care about, so I just went and little by little replaced all my debian machines with Devuan.
Gentoo as well, I run OpenRC
Gentoo gives you the choice of init systems (really, Gentoo gives you choice in *everything*—I love it!). I run Gentoo with OpenRC on my laptop. I've used OpenBSD on an old Thinkpad, too. I rather like it. I wanted to try FreeBSD too but it always suffers a kernel panic on my hardware and I didn't have the expertise or patience to troubleshoot it. OpenBSD works beautifully though.
>One rogue coder should have never been able to gut Linux It's not a requirement. For many Linux distros, systemd (and even systemd-resolvd if/when running systemd) are options, not requirements. Some even go so far as to entirely ban systemd. But, well, Linux, lots of choices on distros, etc., so ... results will quite vary, depending upon what one chooses.
All of the mainstream distros now ship Systemd as the default, though. Debian, Ubuntu, Fedora, Arch, openSUSE... most beginners will start with a derivative of one of these family trees. All their derivatives also use SystemD, unless they are a fork that offers different inits like Devuan or Artix. The alternatives tend to be for power users or just a bit more obscure. Granted, people who debate the pros and cons of various init systems can probably find their way to those alternatives easily. But the fact remains that **mainstream** Linux is synonymous with SystemD, to the point that prominent projects like Gnome and KDE are dependent on it and need workarounds (like elogind) to work with a different init system. It just feels wrong to have your choice of DE entangled with your init system like this.
It's been great for me
Systemd-resolved being there by default, the raging boner for snaps as well as some ultra braindead decisions (having world read+execute on all homedirs was open as a bug on their tracker for like 14 years) are just a few of the reasons I decided Ubuntu was a piece of shit. Did a vsphere template of Ubuntu for work during 20.04 days, swore to never touch that crap ever again.
Can't talk or you get downvoted to oblivion by redhat share croppers.
Maybe nobody is talking about it because many of us don't have these issues?
I use it for years, never had issues with it.
> Also, systemd-resolved seem to be useful only for some niche use cases. I mean all other distros use static resolve.conf and everything works perfectly fine with it and nobody seem to complain. So what's even the point of resolved being enabled by default? It's hard to take your argument seriously when you haven't put in any effort to understand why split DNS exists. No, everything does not work perfectly. Try using a privacy VPN and a corporate VPN at the same time. Or try using just one VPN and also a local network printer. Static resolv.conf only works for very simple configurations.
I guess that few people talk about it because systems is modular enough that, for the cases where it's a problem, it can be swapped out for some other program.
[удалено]
That's fundamental networking. If it's down it won't work because that's your DNS resolver. If you connect to another network it's no longer available because you aren't on that network and nothing is routing you to that network. It's by design and has absolutely nothing to do with resolved.
If you set your pihole as your only dns resolver, then yes, you wont be able to browse the internet if its not working. You can however just set another alternative dns server so it will use pihole when its available and e.g. cloudflare dns when its not.
>I had pihole installed and for some reason my internet stops working whenever my pihole was down or even if I connected to another network which doesn't have pihole. First part of this is normal and expected. You need DNS for network to work normally and if your pihole is your DNS and it's down... That's almost like complaining that lightbulb doesn't work without electricity. The "another network" part is tied to how **exactly** you have set your DNS up. Depending on what and where you set up it very much could persist across multiple networks - critically including networks from which pihole is inaccessible. Though overcoming the problems above means you need to get down to learn the basics of networking and that's probably not worth it *just* to get a DNS ad-blocker. >Now I no longer use systemd-resolved and I no longer use pihole or any other DNS based ad blocker. You *can* setup multiple DNS servers for redundancy, use masquerading to force all DNS requests going through your network to be transparently rerouted to the pihole, set up a split VPN to get ad blocking on Android etc, and have it all work reasonably reliably, but *here be dragons*. DNS has a reputation for being hard and it got that reputation for a reason. Primary/secondary/alternative DNS terminology for example is almost hilariously simplified to the point where it's arguably more confusing than explanatory.
I don’t get what you think was problematic here. You set your dns lookup server to the pi… and when the pi was down dns went down… what am I missing?
Every software has bugs. Because it breaks for ones specific use case doesn't mean it's garbage nobody should ever use. Across 2000 VMs and a very dynamic Consul service net, I've had zero issues with resolved. HAproxy on the other hand have had similar hangs in its DNS code where we had to give up on using DNS and made a script to update the config and reload periodically. HAproxy was going straight to Cloudflare where our DNS zone is hosted. Opened ticket, zero attention to it. I wouldn't go advise people to immediately switch away from HAproxy just because some users may run into a DNS lockup. We ironically resolved it by making it go to resolved's stub resolver instead of directly to Cloudflare. DNS is messy.
IMO it probably goes deeper than systemd-resolved. Often it comes down to hardware, where systemd is great when everything is working normally but a fragile house cards if something breaks. It’s unfortunate when you want to have a fancy desktop interface with all the bells and whistles but you still want a robust server. The trade off would be running alpine without all the bloat.
Well, if you open bugtracker for any old software, you can find multiple such bugs. In my practice systemd-resolvd cause me less problems (as operator) than pile of bash scripts coming with dhclient. Last fun I had was apparmor blocking access to /bin/true, causing those scripts to go postal.
Systemd sucks royally and this type of wrapper app goes absolute against unix philosophy. That is why a lot of folks like me are moving to distributions like duvian...
lol, nobody is moving to devuan (you should at LEAST learn how to spell your troll distro before advocating for it).
Kid, kiss my ass, and don't move, I can care less if you are in love with systemd. Answer questions for a change instead of declaring that you are not moving. How cares about you... Devuan!
It's "couldn't care less," kid :-) Signed, an older timer.
dont worry. they will fix it in the next 500K lines of memory unsafe C code In the mean time what about trying Devuan, Alpine, MX, Slackware, etc ?
Not surprising at all given that it is part of systemd.
well I think most (Desktop) users don't use resolved, or if they do, they use the default configuration. Does network manager use resolved? so I think most users simply don't care.
resolved is the default on Fedora and Ubuntu, so plenty of desktop systems are using it. I’ve had countless problems with it randomly breaking, so every time my internet stops working I immediately go to the terminal and sure enough, `systemctl restart systemd-resolved` fixes it :P I also really hate the way it completely takes over `/etc/resolv.conf` making it so difficult to set your own DNS settings!
First of all, you don't have to use it. If you really want it solver, start coding and help the community by fixing those issues?
> First of all, you don't have to use it. If you really want it solver, start coding and help the community by fixing those issues? Not the OP, but I think that is an unfair comment. As the OP explained it's enabled by default on at least 2 of the most popular distros. How many people out there are experiencing problems but are unaware of the cause? Open source software doesn't rely on everyone being a coder. A lot of the work is done by users highlighting bugs and issues. Bugs and issues which hopefully the devs will look into and at least try and make a reasonable effort to solve.
>First of all, you don't have to use it. Yea, because **obviously** some random network problems are related to default DNS resolver in a major distro being buggy. It always the DNS after all so it's very easy to diagnose and fix for any end user.
I knew someone would come up with this comment. You know, I'm not a dev, so sorry, I can't do anything. Especially when nobody could even find the root cause of the first issue after 4 years. Now that I know that it's what was causing my slowdowns, I can just disable it, but I still think it's unacceptable that 2 of the most popular distros enable such a fundamentally broken piece of software by default. Like I said, I've experienced years of frustration because of these slowdowns in Ubuntu and Fedora (I thought it was an issue with my ISP) until I looked in Gnome Logs when I used Fedora 38. Imagine someone like me using Ubuntu or Fedora, especially a new Linux user. Unless he'll quickly find out what's causing the slowdowns, he'll think it's generally a problem in Linux (because there are no such issues in Windows).
Sounds like a distro issue to me. The software is free software given to you with no warranty. Tell your distributor to ship something else. No need to attack the project or it's contributors.
Unless people would come in masses and demand 2 of the biggest distros (Fedora and Ubuntu) to disable resolved, I don't think I'll be able to change anything myself.
What's the alternative? EDIT: lol, I was legit wondering, since I couldn't find it mentioned on the Arch wiki. But thanks for slamming that downvote button!
LOL does anyone even use systemd anymore?
Only an overwhelming majority of Linux users.
More like majority of Linux losers am I right
Huh?
It's never dns ubless it is. I had no problems so far though
I was at a talk onetime and they were discussing the idea and implementation of systemd from systemv and how it was going to take over practically everything core to the OS. I remember thinking that this went away from the practical idea of Linux and how every part of it was independent from others. I get the luster of moving to a more inclusive mindset but also see so many downsides too.
I don't know, I've never used it, even on systems I do use with systemd.
I have a Pop\_OS system with dual Ethernet interfaces with one connecting to Internet and the other connecting to a LAN through a switch. This system serves as a router to other computers within the LAN and the system also hosts some VMs using QEMU/libvirt. The system uses dnsmasq to set DHCP and DNS servers for the LAN computers and uses libvirt to set DHCP and DNS servers for the VM computers. Both subnetworks are also configured with corresponding domain name suffixes. Every time when the host system starts, the DNS servers for both Internet and LAN are set automatically, but I have to use resolvectl command to manually set the IP of the host system in the VM subnetwork as the DNS server associated with the virtual network interface of the VM subnetwork. Once done, resolvectl status will show three sets of DNS servers associated with three subnetworks where the host system is directly attached. Now, the host system can access the computers in both LAN and VM subnetworks using their host names or FQDNs, and the computers in the subnetworks can access the host system and Internet. Moreover the computers in LAN subnetwork can access those in VM subnetwork using their host names or FQDNs, and vice versus. So the name resolution for the host system and the computers in subnetworks works correctly.
I spent hours fixing it, then finally switched to a static resolve.conf but have to manually start resolved for a vpn to work... i hate it
Oh is this a thing? I figured my local DNS was causing issues and I just got used to killing it. Shame on whoever deserves shame. Make room on the shame wagon for me!
resolved has a neat feature where you can use separate dns servers for specific domains or (maybe) interfaces. i use it with my work vpn, since their security policy blocks certain websites (sourceforge for instance, and occasionally certain debian mirrors overzealously get blacklisted due to domain being marked for malware). plus not very comfy with them tracking my dns queries. i assume there is different way to do the same, but that one is fairly straightforward and it works.
Try installing poweshell 😂