レナート   Wunschkonzert, Ponyhof und Abenteuerspielplatz   ﻟﻴﻨﺎﺭﺕ

Thu, 28 Apr 2011

Why systemd?

systemd is still a young project, but it is not a baby anymore. The initial announcement I posted precisely a year ago. Since then most of the big distributions have decided to adopt it in one way or another, many smaller distributions have already switched. The first big distribution with systemd by default will be Fedora 15, due end of May. It is expected that the others will follow the lead a bit later (with one exception). Many embedded developers have already adopted it too, and there's even a company specializing on engineering and consulting services for systemd. In short: within one year systemd became a really successful project.

However, there are still folks who we haven't won over yet. If you fall into one of the following categories, then please have a look on the comparison of init systems below:

And even if you don't fall into any of these categories, you might still find the comparison interesting.

We'll be comparing the three most relevant init systems for Linux: sysvinit, Upstart and systemd. Of course there are other init systems in existance, but they play virtually no role in the big picture. Unless you run Android (which is a completely different beast anyway), you'll almost definitely run one of these three init systems on your Linux kernel. (OK, or busybox, but then you are basically not running any init system at all.) Unless you have a soft spot for exotic init systems there's little need to look further. Also, I am kinda lazy, and don't want to spend the time on analyzing those other systems in enough detail to be completely fair to them.

Speaking of fairness: I am of course one of the creators of systemd. I will try my best to be fair to the other two contenders, but in the end, take it with a grain of salt. I am sure though that should I be grossly unfair or otherwise incorrect somebody will point it out in the comments of this story, so consider having a look on those, before you put too much trust in what I say.

We'll look at the currently implemented features in a released version. Grand plans don't count.

General Features

sysvinit Upstart systemd
Interfacing via D-Bus no yes yes
Shell-free bootup no no yes
Modular C coded early boot services included no no yes
Read-Ahead no no[1] yes
Socket-based Activation no no[2] yes
Socket-based Activation: inetd compatibility no no[2] yes
Bus-based Activation no no[3] yes
Device-based Activation no no[4] yes
Configuration of device dependencies with udev rules no no yes
Path-based Activation (inotify) no no yes
Timer-based Activation no no yes
Mount handling no no[5] yes
fsck handling no no[5] yes
Quota handling no no yes
Automount handling no no yes
Swap handling no no yes
Snapshotting of system state no no yes
XDG_RUNTIME_DIR Support no no yes
Optionally kills remaining processes of users logging out no no yes
Linux Control Groups Integration no no yes
Audit record generation for started services no no yes
SELinux integration no no yes
PAM integration no no yes
Encrypted hard disk handling (LUKS) no no yes
SSL Certificate/LUKS Password handling, including Plymouth, Console, wall(1), TTY and GNOME agents no no yes
Network Loopback device handling no no yes
binfmt_misc handling no no yes
System-wide locale handling no no yes
Console and keyboard setup no no yes
Infrastructure for creating, removing, cleaning up of temporary and volatile files no no yes
Handling for /proc/sys sysctl no no yes
Plymouth integration no yes yes
Save/restore random seed no no yes
Static loading of kernel modules no no yes
Automatic serial console handling no no yes
Unique Machine ID handling no no yes
Dynamic host name and machine meta data handling no no yes
Reliable termination of services no no yes
Early boot /dev/log logging no no yes
Minimal kmsg-based syslog daemon for embedded use no no yes
Respawning on service crash without losing connectivity no no yes
Gapless service upgrades no no yes
Graphical UI no no yes
Built-In Profiling and Tools no no yes
Instantiated services no yes yes
PolicyKit integration no no yes
Remote access/Cluster support built into client tools no no yes
Can list all processes of a service no no yes
Can identify service of a process no no yes
Automatic per-service CPU cgroups to even out CPU usage between them no no yes
Automatic per-user cgroups no no yes
SysV compatibility yes yes yes
SysV services controllable like native services yes no yes
SysV-compatible /dev/initctl yes no yes
Reexecution with full serialization of state yes no yes
Interactive boot-up no[6] no[6] yes
Container support (as advanced chroot() replacement) no no yes
Dependency-based bootup no[7] no yes
Disabling of services without editing files yes no yes
Masking of services without editing files no no yes
Robust system shutdown within PID 1 no no yes
Built-in kexec support no no yes
Dynamic service generation no no yes
Upstream support in various other OS components yes no yes
Service files compatible between distributions no no yes
Signal delivery to services no no yes
Reliable termination of user sessions before shutdown no no yes
utmp/wtmp support yes yes yes
Easily writable, extensible and parseable service files, suitable for manipulation with enterprise management tools no no yes

[1] Read-Ahead implementation for Upstart available in separate package ureadahead, requires non-standard kernel patch.

[2] Socket activation implementation for Upstart available as preview, lacks parallelization support hence entirely misses the point of socket activation.

[3] Bus activation implementation for Upstart posted as patch, not merged.

[4] udev device event bridge implementation for Upstart available as preview, forwards entire udev database into Upstart, not practical.

[5] Mount handling utility mountall for Upstart available in separate package, covers only boot-time mounts, very limited dependency system.

[6] Some distributions offer this implemented in shell.

[7] LSB init scripts support this, if they are used.

Available Native Service Settings

sysvinit Upstart systemd
OOM Adjustment no yes[1] yes
Working Directory no yes yes
Root Directory (chroot()) no yes yes
Environment Variables no yes yes
Environment Variables from external file no no yes
Resource Limits no some[2] yes
umask no yes yes
User/Group/Supplementary Groups no no yes
IO Scheduling Class/Priority no no yes
CPU Scheduling Nice Value no yes yes
CPU Scheduling Policy/Priority no no yes
CPU Scheduling Reset on fork() control no no yes
CPU affinity no no yes
Timer Slack no no yes
Capabilities Control no no yes
Secure Bits Control no no yes
Control Group Control no no yes
High-level file system namespace control: making directories inacessible no no yes
High-level file system namespace control: making directories read-only no no yes
High-level file system namespace control: private /tmp no no yes
High-level file system namespace control: mount inheritance no no yes
Input on Console yes yes yes
Output on Syslog no no yes
Output on kmsg/dmesg no no yes
Output on arbitrary TTY no no yes
Kill signal control no no yes
Conditional execution: by identified CPU virtualization/container no no yes
Conditional execution: by file existance no no yes
Conditional execution: by security framework no no yes
Conditional execution: by kernel command line no no yes

[1] Upstart supports only the deprecated oom_score_adj mechanism, not the current oom_adj logic.

[2] Upstart lacks support for RLIMIT_RTTIME and RLIMIT_RTPRIO.

Note that some of these options are relatively easily added to SysV init scripts, by editing the shell sources. The table above focusses on easily accessible options that do not require source code editing.

Miscellaneous

sysvinit Upstart systemd
Maturity > 15 years 6 years 1 year
Specialized professional consulting and engineering services available no no yes
SCM Subversion Bazaar git
Copyright-assignment-free contributing yes no yes

Summary

As the tables above hopefully show in all clarity systemd has left behind both sysvinit and Upstart in almost every aspect. With the exception of the project's age/maturity systemd wins in every category. At this point in time it will be very hard for sysvinit and Upstart to catch up with the features systemd provides today. In one year we managed to push systemd forward much further than Upstart has been pushed in six.

It is our intention to drive forward the development of the Linux platform with systemd. In the next release cycle we will focus more strongly on providing the same features and speed improvement we already offer for the system to the user login session. This will bring much closer integration with the other parts of the OS and applications, making the most of the features the service manager provides, and making it available to login sessions. Certain components such as ConsoleKit will be made redundant by these upgrades, and services relying on them will be updated. The burden for maintaining these then obsolete components will be passed on the vendors who plan to continue to rely on them.

If you are wondering whether or not to adopt systemd, then systemd obviously wins when it comes to mere features. Of course that should not be the only aspect to keep in mind. In the long run, sticking with the existing infrastructure (such as ConsoleKit) comes at a price: porting work needs to take place, and additional maintainance work for bitrotting code needs to be done. Going it on your own means increased workload.

That said, adopting systemd is also not free. Especially if you made investments in the other two solutions adopting systemd means work. The basic work to adopt systemd is relatively minimal for porting over SysV systems (since compatibility is provided), but can mean substantial work when coming from Upstart. If you plan to go for a 100% systemd system without any SysV compatibility (recommended for embedded, long run goal for the big distributions) you need to be willing to invest some work to rewrite init scripts as simple systemd unit files.

systemd is in the process of becoming a comprehensive, integrated and modular platform providing everything needed to bootstrap and maintain an operating system's userspace. It includes C rewrites of all basic early boot init scripts that are shipped with the various distributions. Especially for the embedded case adopting systemd provides you in one step with almost everything you need, and you can pick the modules you want. The other two init systems are singular individual components, which to be useful need a great number of additional components with differing interfaces. The emphasis of systemd to provide a platform instead of just a component allows for closer integration, and cleaner APIs. Sooner or later this will trickle up to the applications. Already, there are accepted XDG specifications (e.g. XDG basedir spec, more specifically XDG_RUNTIME_DIR) that are not supported on the other init systems.

systemd is also a big opportunity for Linux standardization. Since it standardizes many interfaces of the system that previously have been differing on every distribution, on every implementation, adopting it helps to work against the balkanization of the Linux interfaces. Choosing systemd means redefining more closely what the Linux platform is about. This improves the lifes of programmers, users and administrators alike.

I believe that momentum is clearly with systemd. We invite you to join our community and be part of that momentum.

posted at: 23:16 | path: /projects | permanent link to this entry | 128 comments

Posted by foo at Fri Apr 29 03:27:11 2011
As much as I agree with you about Bazaar/Git, I think it it is very unfair to label it in the same way as CVS.

Posted by Dax Kelson at Fri Apr 29 04:04:57 2011
As a multi-distribution Linux training company that authors multiple courseware titles I'm well aware of the various NEEDLESS incompatibilities between Linux distributions.

Setting aside all the cool features, as you mention, the opportunity for Linux standardization is HUGE with systemd and I fully support that idea. The less frivolous incompatibilities the better!

Posted by Jonathan at Fri Apr 29 04:15:06 2011
I think sysvinit's development takes place in a Subversion repository. See http://savannah.nongnu.org/projects/sysvinit (and try not to be distracted by the CVS repo tracking a single index.html file).

Posted by Lennart at Fri Apr 29 04:29:18 2011
Jonathan: you are right. And I even have it checked out. Somehow I forgot about that... Fixed it now.

Posted by alk at Fri Apr 29 06:28:15 2011
I'm totally in favor of systemd idea except...

I'm not sure putting everything boot-related into single package and reimplementing it in C is very good idea. Shell scripts may be slow, but they're very hackable with only plain editor. With systemd fixing/hacking/working-around something will likely require C compiler. There's also possibility that C code will have more bugs that shell code.

Posted by Roland Taylor at Fri Apr 29 06:33:50 2011
How did I know you were going to drop something about Ubuntu in here. I won't say I'm a fan boy or anything (and not a fan of upstart for sure), but...

Can't we just have a clean post without the "with one exception" crap?

Seriously...

Posted by Kelly Clowers at Fri Apr 29 06:36:59 2011
>There's also possibility that C code will have more bugs that shell code.

Sounds backwards to me. Shell is just not that great for large scale programming, and lacks many of the tools that are available for checking C. Sure, it could have more bugs, but it doesn't seem there is any reason to expect that.

Posted by Roland Taylor at Fri Apr 29 06:38:48 2011
Also, sorry if I sounded a bit angry there - just tired of the pointless GNOME/Ubuntu war. Or should I risk saying, Red Hat/Canonical, RPM/Deb - war...

IT'S ALL LINUX! MAKE LOVE (well... friendliness) NOT WAR :)!

Posted by Kelly Clowers at Fri Apr 29 06:39:49 2011
@Roland: How is anyone supposed to have a discussion of various init systems without talking about Ubuntu?

Posted by Jeremy Bicha at Fri Apr 29 06:59:37 2011
This article is misleading. You imply that every distro in existence except Ubuntu has switched to systemd which is wrong. Debian and openSUSE haven't; openSUSE uses upstart in fact. And as far as market share goes, Debian, Ubuntu & derivatives are huge.

In fact, you can't blame Canonical for not having switched to systemd which has been out for a few months and suddenly dropping upstart which they've poured a lot of time, money, and inertia into. Even if it were the obvious technically superior solution, give it time without needing to attack your colleagues working for other projects.

Posted by pankaj at Fri Apr 29 06:59:56 2011
systemd rocks

Just the other day i thought of writing a native systemd unit file for 3proxy (my first - finished it in 5 mins) and then found someone had already written it (not yet pushed it to repo), and it was exactly same as i came up with - except for the description field.

This is just awesome. i can assure that writing sysvinit shell scripts can never be as easy

Posted by Simon at Fri Apr 29 07:01:30 2011
@Roland - while Lennart is prone to snarky remarks about Ubuntu, I don't think this is one of them. In this context, it would be negligent not to mention that one of the biggest distros isn't expected to move to systemd, and I don't think there's anything wrong with how that's been worded.

Posted by Simon at Fri Apr 29 07:06:25 2011
@Jeremy - the article never says (or implies) that every distro has switched to systemd. It says that most of the major distros have decided to switch, not that they've already done so.

Posted by Sankar at Fri Apr 29 07:45:51 2011
@Jeremy: openSUSE 11.4 (currently stable release) is not using systemd by default, but anyone can switch to systemd with a simple boot parameter. For the next release we will switch to systemd by default.

One [stupid] question though, why would, "CPU Affinity" matter in boot ?

Posted by Mathias Hasselmann at Fri Apr 29 08:17:53 2011
It's indeed quite sad that Ubuntu isn't in the boat yet: Mark's pragmatism pushed it light years ahead in terms of user experience. Tried several other distros last weekend, feed up with all this Ubuntu vs. the others wars, but have to say I was shocked, seriously shocked how far behind the other big distributions. I'd have expected the huge success of Ubuntu let them join the competition for best UX, but in fact they stagnated somewhere at the state of the level of 2005 or something. It was shocking. Seriously shocking experience.

Posted by albert at Fri Apr 29 08:57:54 2011
@Jeremy in addition to this. I do think that Ubuntu/Canonical remember a piece of software they introduce to early in their distribution. I am speaking about PulseAudio. Nobody can blame them to be very very careful with systemd after this. PA is a nigthmare to integrate and work only in Fedora because of Lennard doing the intgration himself. The same pattern is appearing here and nobody knows how it is difficult to integrate properly in any other distribution and if there are a lot of change to do inside because of a too much "Red-Hat" orientation.

Posted by Johannes at Fri Apr 29 09:16:55 2011
@albert: damn, how I waited for the PA argument...

But seriously the sidekicks against Ubuntu and Bazaar seems a bit unnecessary in this technical blog post...

Posted by Mark__T at Fri Apr 29 09:31:07 2011
@albert PA works fine in foresight too, so arguing only Lennard can integrate doesn't help here

Posted by iMil at Fri Apr 29 10:23:50 2011
How convenient are those spreadsheets hu ? How about adding the following lines:

. Portability
. Proper coding standards
. Need to modify FHS
. Consistent configuration files
. Huge dependency list

Oh and to be fair, on the "pro's" side:

. Software name is "systemd"

Cheers

Posted by simple_linux_user at Fri Apr 29 10:26:22 2011
I can program in c/c++ or simple bash, what I know is:
pidgin and player can sound together with pulseaudio ,this is just what I want.

systemd unit file is very simple,anyone can understand

I know very little of linux boot process,
but I can still understand systemd unit file in 10 seconds

anyone who knows everything of linux boot process,
can understand a sysv init script in 10 seconds?

you are right,bash is hackable easy
but just  syntax understood easy
not logical understood easy

1000+ line bash  vs 10 line- unit file

bash is still earier ?

Posted by Tomasz at Fri Apr 29 10:51:49 2011
Lennart, regarding “disabling/masking services without editing files”. Does http://upstart.at/2011/03/11/override-files-in-ubuntu-natty/ count for upstart?

Posted by Dave Neary at Fri Apr 29 11:26:15 2011
Wow! Is there anything systemd doesn't do? I guess there's no "comes with a kitchen sink" category?

Dave.

Posted by drago01 at Fri Apr 29 11:46:39 2011
@alk:

IMO nothing that is part of the system should be written as shell script; and no init scripts aren't configuration files so they don't have to be easily editable (administrators should edit config files for configuration; if that doesn't work fix the app in question to be sanely configurable).

I cannot edit kernel modules without a C compilier either ;)

Posted by Ole Laursen at Fri Apr 29 11:58:33 2011
I think you forgot babysitting support in your long list?

As one tends to learn the hard way when running a webserver, Apache/nginx and similar friends aren't entirely bug free (or perhaps it's cosmic rays). Sometimes you get a crash, not often, but maybe a couple per year. That's why you currently need something like monit on a server to restart the vital processes if they go down - but it's really annoying having to configure monit for this basic protection.

I think upstart is supposed to have some babysitting support, but last time I looked, the process watch support didn't really work too well, rendering it unusable with basic web server services (at least if the comments here are right):

https://bugs.launchpad.net/upstart/+bug/406397

Babysitting support is the number 1 feature I'm hoping to get from systemd when Debian makes the jump.

Posted by Jean-Daniel Browne at Fri Apr 29 12:13:02 2011
Hi,

What is the best way to express high level, business specific process monitoring with systemd?

Monit is a cool system which allows the following in its config file

  if totalmem > 60% then alert
  if children > 50 for 5 cycles then restart
  if cpu usage > 98% for 3 cycles then restart
  if failed url http://localhost/healthcheck
  timeout 10 seconds
  for 6 cycles then restart

Note the high level URL check test, very specific to the application.

But monit and others need their own set of config files and commands, which leads to a bit of confusion as to who does what.

Posted by Andrew Clayton at Fri Apr 29 12:14:13 2011
@Dave Neary:
systemd is the EMACS of init systems!

I for one welcome our new system overlord.

Liked the SCM comparison ;)

Posted by oiaohm at Fri Apr 29 12:45:59 2011
anyone who wants to keep up the script disagreement.  Please remember http://bellard.org/tcc/  C can be used as a script just as much as bash can.  Performance is a major difference.  tcc as a shell still runs rings around bash.  Defects those are getting harder to get away with as compilers improve.

Really I see this as a start.  freeipa.org and other techs can still be merged in.  systemd is a chance for Linux to leave the Unix history behind.  And move forwards.  Really freeipa techs merged in from the start line would be nice.

Posted by frok at Fri Apr 29 12:46:58 2011
> GNOME agents

What about KDE/XFCE/LXDE/Unity/etc.?

Sounds for me like systemd is what we call Eierlegende Wollmilchsau with a dependency-hell.

Posted by arekm at Fri Apr 29 13:09:33 2011
Will systemd work fine inside of linux-vserver guest?

sysvinit and upstart do work there.

Posted by Tobias at Fri Apr 29 13:47:12 2011
Systemd wins! Only one red box!!!

Posted by ncopa at Fri Apr 29 13:52:38 2011
With that long list it seems like systemd is too big an bloaty for me. Systemd has some nice concepts but I really don't want that bloat.

Posted by Anonymous at Fri Apr 29 13:53:24 2011
You mention "gapless service upgrades"; I have a lot of interest in that, and I'd love to hear more about it.

Posted by Anonymous at Fri Apr 29 13:59:21 2011
While I like systemd very much, I think you've marked many things "no" in sysvinit which it does actually support, either directly or via init scripts.  Yes, sysvinit uses shell scripts, but you still have to count the features that it implements using those shell scripts. :)

Read-Ahead at least gets a footnote, since readahead normally provides an init scripts to invoke it.

Init scripts also normally handle mount, fsck, swap, temporary files, random seeds, and so on.

Posted by Anonymous at Fri Apr 29 13:59:54 2011
You mention timer-based activation; do you provide crontab compatibility?

Posted by Anonymous at Fri Apr 29 14:08:05 2011
Does the LUKS support and passphrase handling work for the root filesystem?

Posted by Lennart at Fri Apr 29 14:34:25 2011
alk: shell code invites work-arounds and using canons toshoot on sparrows. The C code is usually much simpler and robust than shell code actually can be.

Roland: uh? Ubuntu is the one exception, I think people want to know which one it is, then it would be a lot more mysterious for no reason.

Jeremy: Debian includes it in the distribution, which is as much as one can ask for them for now. OpenSUSE has made public that they'll switch in the next release, and that's what I said. Meego has now switched too in the development version.

Sankar: CPU Affinity in service files is useful to assign CPU 0,1 to Apache, and CPU 2,3 to MySQL and stuff like that.

iMil: Portability? all three have been ported to the archs Linux runs on. And we are focussing here on Linux, nothing else. Coding standards? No sure what you are referring to, systemd follows a modified kernel coding style, and Upstart a modified GNU coding style. While I prefer the former, it's not really something that is clearly better or worse. (sysvinit uses no real coding standard). Yes, systemd makes improvements on top of the FHS. And so does Ubuntu now even though they aren't using systemd. And all other distros, like Debian have adopted /run in their devel distros, too. And consistent config files? systemd definitely wins here, the others don't even bother. Regarding deps: systemd requires only libdbus and glibc/libcap as deps. sysvinit only glibc. Upstart requires glibc, libdbus and libnih. So I think systemd looks prtty good on those points too.

Posted by Lennart at Fri Apr 29 14:39:38 2011
Tomasz: I'd say that's editing of config files for disabling something. Neither sysvinit nor systemd require that.

Jean-Daniel: I think the monitoring does not belong in systemd itself, but the respawning does. i.e. figuring out whether a service works fine usually requires service-specific code and hence doesn't really fit that well in systemd. However, then restarting things definitely is something systemd should do on behalf of that monitor.

frok: the GNOME agent works fine under other desktops. And if you want to code your own, read http://www.freedesktop.org/wiki/Software/systemd/PasswordAgents

Posted by Lennart at Fri Apr 29 14:49:44 2011
arekm: it boots fine in an nspawn container. This probably means that the vserver-guest stuff works too.

ncopa: really? bloat? Replacing a mess of shell scripts with a handful of cleanly-written, easy to read, robust C utilities you call bloat? If you drop all the stuff systemd replaces then the overall size of your distro will be much smaller. And even psychologically it will be a much tighter, minimal system. Also, systemd is modular. You can enable the features you want and the ones you don't. So really, the word "bloat" is completely inappropriate here.

Anonymous: regarding gapless service upgrades: a properly written service using bus and socket activation can be upgraded in systemd without losing any kind of traffic. We actually do this on every boot on F15 by default: the tiny early-boot kmsg syslogd is replaced on-the-fly by the full fledged rsyslog daemon -- without losing a single message, and without having the /dev/log socket inaccessible at any time. Something similar works for version upgrades where you can stop the old rsyslog and start the new one without having to close any sockets or losing client traffic. Something similar is possible for all services using sockets or dbus for interfacing (i.e.: practically all of them).

Anonymous: there's a paragraph already about hacking process parameters into sysv init scripts. And sysvinit does not bring any standard implementation of read-ahead or random seeed or the other stuff with it. That's usually done with additional, distro-specific scripting.

Anonymous: no contrab compatibility, no. And no plans either. If you want crontab, use cron.

Anonymous: to decrypt the root disk you'd have to run systemd in the initrd. Which you should be able to, but nobody has done so far.

Posted by Richard at Fri Apr 29 15:26:34 2011
Re Graphical UIs, I think you're slightly unfair - there are plenty of 3rd party ones (eg Gnome, KDE, and most distro control panels). However, I love the idea of systemd.

Writing compatible init-scripts is a real pain. For example,  I've just written a daemon which runs a private instance of opentracker - I take great care not to tread on the main instance (if present), and to use different ports, and so that "service my-opentracker start/stop/status" looks at the user who is running it as well as the executable and the pidfile). But the distro's init script stops opentracker using "killall opentracker", so the main service kills my one, even though I respect the main one!

Posted by Anon at Fri Apr 29 16:04:31 2011
I guess another advantage to systemd is there is no need for portreserve ( https://bugzilla.redhat.com/show_bug.cgi?id=103401 ).

I believe Chromium OS will use upstart.

Posted by Daniel at Fri Apr 29 16:19:28 2011
So, systemd violated the unix design principle and try to do everything in a single process?

pid 1 process should be failsafe. putting everything in it is prone to error.

Posted by stenny at Fri Apr 29 16:20:51 2011
I hereby offer "specialized professional consulting and engineering services" for sysvinit.

Posted by Anonymous at Fri Apr 29 16:23:26 2011
The future sounds very promising and I'm willing to believe that during F16/F17 things look just great.

However, I must ask what was the plan with dracut? Has it reached its peak or will it be modified somehow to make it more aligned with systemd based universe?

Thanks.

Posted by Matthias Urlichs at Fri Apr 29 17:28:35 2011
So your table basically says that you can't find a single feature of any other init system that systemd lacks.

I'm sure that's somewhat biased. The real world isn't that one-sided.

Also, flagging Bazaar with a red tag is just plain wrong. It's a DVCS with features that are reasonably comparable to git. I'm a git junkie myself and don't like baz at all, but that's uncalled-for.

Also, *exist_e_nce.

Posted by sgtrock at Fri Apr 29 17:32:18 2011
"So, systemd violated the unix design principle and try to do everything in a single process?

pid 1 process should be failsafe. putting everything in it is prone to error."

+10,000,000,000,000!

Yes, I will conceded that sysvinit is prone to all the problems that you cite.  However, it's also inherently stable from a systems point of view as well as being infinitely flexible and extensible. 

Show me how you keep to the basic UNIX design philosophy, please.

Posted by Tomasz at Fri Apr 29 17:37:03 2011
Lennart: about masking/disabling still. In upstart you don't edit service file itself, you create .override file. It is almost the same as creating link to /dev/null in systemd.

sgtrock: It is not everything-in-one-process. % ls /lib/systemd/systemd-* | wc -l
24

Posted by zob at Fri Apr 29 17:47:04 2011
I like systemd and so on but one comparison makes me tick.

SCM Subversion Bazaar git

Subversion works well. Bazaar works well. GIT works well.

Why the hell is GIT in green then? Oh just because it's the newest and sounds cool?

Well that sort of irrelevant item should be "all in white" or just not there. It makes systemd's author look like a dumbass advertiser instead of an engineer. And really, systemd should convince engineers, not the general crowd who couldn't care less.

Please, just fix it :)

Posted by eclipse at Fri Apr 29 18:25:33 2011
I haven't been following Linux development lately, but I have been impressed by F15 startup time! Kudos to you Lennart!

Posted by Elijah Newren at Fri Apr 29 19:24:30 2011
zob: I can't speak for Lennart, but I believe the last two items of the "Miscellaneous" category are best viewed as being targeted towards development community building, as opposed to much of the rest of the article which is aimed more at end-user adoption.  I agree that in terms of end-user adoption, those two items aren't directly relevant, but for development community building they are.  (Many developers will avoid or outright refuse to contribute to projects requiring copyright assignment.  Similarly, since the Linux kernel and freedesktop.org have adopted git, one could easily argue that the people most able/willing to contribute to systemd are very likely to prefer git as their version control system).  In that sense, there's really nothing that needs to be fixed, other than perhaps making it clearer why those two items are included.

Posted by Anonymous at Fri Apr 29 20:13:40 2011
I'd suggest that Bazaar at least deserves credit for not being Subversion; marking them both red hardly seems fair to Bazaar.  Yellow? :)

Posted by Anonymous at Fri Apr 29 20:16:34 2011
So, if I do socket-based activation of a web server on port 80, I can restart that web server without losing a single request?

Posted by Anonymous at Fri Apr 29 20:18:21 2011
Yes, I realize that sysvinit doesn't ship with scripts handling swap, fsck, and so on; however, you already cover that under "standardization".  Distributions always have scripts handling those, and sysvinit invokes them.  Credit where credit is due.

Posted by Chris at Fri Apr 29 20:21:42 2011
In order to use systemd you have to read a book about how it works. Hence the numerous blog posts about it by Lennart prior to this one.
It very clear that systemd is very advanced. I couldnt help noticing though the mention of consolekit and XDG, dbus and the fact that the tools to tweak it (systemd-analyze, systemadm) are written in python and gtk.
How are those even relevant for init?

Posted by Anonymous at Fri Apr 29 20:26:42 2011
Why does sysvinit not have a "yes" for umask, nice, environment variables, and similar?  You can set those settings in upstart or systemd as part of the file for that service, and the same goes for sysvinit.  Again, sysvinit uses shell as its "configuration"; you've already argued very convincingly against that choice (I certainly agree with you there), but nevertheless sysvinit uses shell.

Posted by Anonymous at Fri Apr 29 20:30:30 2011
Upstart provides integration with Plymouth.

Posted by Michal at Fri Apr 29 20:34:06 2011
Hi,

From my POV systemd has great documentation, but I am afraid that many users don't want to read all your blog posts and manuals to figure out how systemd works. It seems to me that you should write something like "systemd for dummies" :)

Maturity - last little setback with mysql tells me that there should be something like a large testfarm where most popular services will be tested before new systemd release.

Best regards,
Michal

Posted by linuxuser at Fri Apr 29 20:43:58 2011
I seriously think that it would be insane to replace a running init-stack with systemd at this moment. Systemd provides a lot of critical functionality and reimplements many existing functionality. The current boot-stack took years to mature and my guess is that it will take quite some time for systemd to reach the same maturity.
Of course it would help systemd if it could be tested on all distributions. But using it as default? Not yet!

Posted by Michal at Fri Apr 29 20:55:04 2011
BTW. If someone is looking for free systemd services, here are mine
https://github.com/eventhorizonpl/systemd-services

Here can also be found free services written by a few brave souls
https://fedoraproject.org/wiki/User:Johannbg/QA/Systemd/compatability

Best regards,
Michal

Posted by Comodet at Fri Apr 29 21:44:16 2011
Even thou I like systemd and I already played with it and wrote some reviews I have one complain that remains: systemd consumes more RAM than sysvinit. I do not care about system size on "disks" - flash is cheap, but RAM is a problem on embedded devices.

Posted by Alberto Mardegan at Fri Apr 29 22:04:44 2011
I didn't get a chance to run systemd yet (running on "the exception"), and I'm sure it's a pity. I hope (and believe) that Ubuntu will deploy it soon.
At least MeeGo is almost there :-)

The system you guys designed is a real piece of art! When I first read the initial announcement I was ecstatic! :-)

Posted by stenny at Fri Apr 29 22:27:09 2011
lennart:

are you implying it is not possible to produce a handful of cleanly-written, easy to read, robust shell utilities?  why would I move away from 'one tool for one job'?  what the hell is the point of using dbus when you're just going to cram all available functionality into init?  also, shell scripts are modular.  you chmod +x the ones you want and chmod -x the ones you don't.  so really, systemd is a complete waste of energy here.

Posted by Lennart at Fri Apr 29 22:31:27 2011
Daniel: no, in systemd things are modular, most system bootupt stuff is done in parallelizable seperate processes. Readahead, fsck, quota, the binfmt stuff, the tmpfiles stuff, all of that are small binaries which we can be spawned in parallel. Parallelization is key. And that means you want process. Not a ridiculous amount of processes, and not serialized processes (as shell scripts do), but a reasonable amount of parallelizable processes. So yeah, PID 1 manages processes, and very few very basic things (like mounting API VFS, and loading the SELinux policy), but not much beyond that.

Anonymous: dracut is tightly integrated with systemd. For example profiling data is passed along, as well as results of the root fsck and a few other things. As systemd developer I can only recommend distros to adopt dracut if they are looking for an initrd.

Matthias: I am listening. I mean, besides the really basic design decisions Upstart has nothing that systemd hasn't. The reason for that among other things is that I actually looked at Upstart, and added to systemd everything I liked about it. But hey, I'd be really happy to see Scott post a similar blog story, so that we can see the other side of the medal.

Posted by Michael at Fri Apr 29 22:33:59 2011
Are there plans to standardize things like the EnvironmentFile-directory? On some systems it is /etc/sysconfig, on others /etc/conf.d etc. I really don't care what it will be, but it would the unit-files more portable if it was standardized...

Posted by Barney at Fri Apr 29 22:34:10 2011
Your article does compare bare-metal sysvinit and upstart to systemd without any of their companion scripts, using specific systemd-oriented criterias as if they were universal goals.

It is no way to conduct an objective comparison. It would be like comparing VIM and MS Office based on the later point of view : they are different products with different goals, different design philosophy and they target very different markets.

Here is what it would look like, to demonstrate the futility of such an approach :
http://i.imgur.com/usftZ.png

Posted by Lennart at Fri Apr 29 22:40:57 2011
sgtrock: dude, most of the components we ship are run in separate processes, from separate binaries. Just because we ship them along with systemd doesn't mean that they'd all be in the same binary and all run as PID 1.

zob: bazaar is simply a PITA for hackers who work on all sides of the OS, because it is not just much less usable than git, but also yet another tool to deal with. Quite frankly, I don't want to care about VCSes, they should just work, and not get in my way. And git is pretty good at that. bzr not, and it requires me to learn even more stuff. But anyway, this story is not about bzr, but it's fun to see how easily people are to tick of with a little colour. Before I posted this we actually bet that this would be the first thing people would comment on -- and they did! Consider it candy for the trolls...

Anonymous: yes, if the web server is properly written (which isn't too hard) then systemd will make it possible to restart it without losing a single HTTP request. How awesome is that? This is actually pretty powerful stuff. For example newer udev versions we reworked so that they too can be restarted without losing a single hw event. It's superduper neat.

Posted by Lennart at Fri Apr 29 22:47:37 2011
Chris: systemd is a platform of the basic stuff to setup and maintain the Linux userspace. It includes an init system, and also additional tools, like profiling tools, UIs, or all the early-boot C utilities. Actually the point of the whole article was to point people to this fact.

Anonymous: There's a paragraph under that respective table explaining why I think umask is not really that nicely configurable on sysvinit: it requires you hacking shell code. And that althought the primary place for user/admin config is /etc/sysconfig/ resp. /etc/default, and not the actual init scripts anymore.

Anonymous: I couldn't find any mention of Ply in the sources, can you point me to them? I'd be happy to correct the table should I be wrong.

Michal: systemd is low-level stuff. Only technical savvy people interface directly with it. For them we write the documentation.  Somebody like my mother who uses her computer to type texts and browse the web, should never have to deal with systemd itself, or even have heard about it. That's why there are no docs for her.

Posted by Lennart at Fri Apr 29 22:54:25 2011
stenny: uh? We don't cram everything into PID1? systemd uses separate processes for separate stuff, already to make things nicely parallelizable. And we use D-Bus as IPC to control the init system. That's all.

Michael: I will soonishly write another blog posts explaining why I think /etc/sysconfig, /etc/default and so on are a bad idea, and should go away in the long run.

Barney: dude, in the introduction of this article I already made clear that I am one of the creators of systemd, and thus cannot be unbiased. Note sure what you are asking for. In a truly pluralistic way you should just ask for the sysvinit or Upstart folks to also blog and refute this story of mine, but not complain to me that I have opinions and priorities, and publicize them.

Posted by Stefan at Fri Apr 29 23:06:59 2011
Bazaar in red? Seriously? At least it helps to get a feeling about the quality of this post...

Posted by Sean McNamara at Fri Apr 29 23:07:12 2011
Hi,

As a result of discussions on Phoronix, I'm interested in asking people a question about their use of systemd with server workloads (mostly for my own curiosity).

If you're an application developer, a software integrator for a distribution, or a sysadmin who deploys custom software, please chime in regarding this question:

Which features of systemd have you used for your server workload, that would be difficult, hackish, or impossible with upstart or sysvinit? Just take a glance at Lennart's table, pull out some features and talk to them. If you haven't actually deployed systemd yet, but are planning to, that's also fine -- as long as your project is somewhat serious.

I'm just looking for some "field data" about how this is being used in practice, so only those who are seriously looking into this from an implementor / administrator's perspective need reply. Thank you so much in advance! :)

P.S. -- If you're a maintainer of systemd (e.g. Lennart), feel free to reply, but I'm really looking for the "third party" angle from folks who are not directly invested in the systemd project. :)

Posted by Lennart at Fri Apr 29 23:22:36 2011
Sean, F15 isn't even released yet. And even if it is I suspect the fewest admins will install that on servers right-away. I think you are asking for this too early. But here are three things that I suspect will be very useful for admins: have the output of any service you run go directly to syslog, or the cluster/remote support in systemctl, or the automount support.

Posted by Anonymous at Fri Apr 29 23:55:38 2011
Lennart: The Plymouth integration lives in Plymouth; upstart provides a dbus interface for progress reporting.

Posted by Anonymous at Fri Apr 29 23:57:10 2011
Lennart: I agree that sysvinit requires hacking shell code to configure many things, including nice, umask, affinity, and anything else you might want.  sysvinit uses shell.  You already have an entry in the table for the use of shell, and I agree with you on that one.  But if you then ignore any functionality which depends on shell, sysvinit doesn't get to claim any features, because it implements them in shell.

Hypothetical example: suppose that I love interpreted languages, and want to promote their use as a feature of an init system so that you can edit scripts without recompiling or adding configuration.  ("Ewwww" is the appropriate reaction, but bear with me for the example.)  So, I could have a feature entry for "written in an interpreted language", and naturally upstart and systemd would get a "no".  However, in that example, I still need to give systemd credit for supporting configuration of nice, umask, and so on, even though configuring them requires editing configuration files rather than an interpreted language. :)

At the very least it deserves a footnote, but I think quite a few of the "no" entries in sysvinit seem like "yes, by editing shell scripts like everything else in sysvinit".

Posted by Artem S. Tashkinov at Sat Apr 30 00:14:43 2011
Re: Anonymous at Fri Apr 29 23:57:10 2011

systemd still allows you to use shell scripts for your favorite services, thus I don't quite understand what more you need ;)

Posted by Lennart at Sat Apr 30 00:49:17 2011
Ananonymous: and that's merged in Plymouth upstream? Can you show me the code? Note that I was looking only on released code, not on patches.

Regarding editing init scripts: the point I was trying to make is that in sysvinit init scripts are nowadays not really considered configuration anymore but code. This is made explicit by the introduction of /etc/sysconfig/xxx resp., /etc/default/xxx which include var assignments used by the init scripts. Hence what I am saying is that you cannot set the umask/affinity/nice and so on in /etc/sc/xxx & /etc/d/xxx, but only in the actual sources of the init script.

Posted by awer at Sat Apr 30 01:25:08 2011
I think upstart is a important piece that is missing from a OS like Linux. I trust you know what has to be done to have a robust init system.

P.D. What's up with PulseAudio? Do you have some plans for it?

Posted by Don Kongo at Sat Apr 30 01:49:44 2011
I would think this was the ultimate no-brainer, if it wasn't for the fact it's coming from the guy behind Pulseaudio, which also easily boasted a similar chart against any competitor (and IS a great idea and achievment) but still is a royal pain for many users and for any problem, someone is just "using it wrong" and development all but stopped when something shinier came up.

I'm afraid this is more of the same, even though I'm personally conviced that systemd is the sane approach compared to Upstart which seems ill conceived - in fact, I think systemd probably owes a lot of the great design to the mistakes of Upstart.

But really: Pulse. Think about it.

Posted by Sean McNamara at Sat Apr 30 01:54:51 2011
Thanks for your insights, Lennart! :)

awer, I think you mean "I think systemd..." in your first sentence? Or else you are referring to the systemd competition from Canonical, upstart, which has been bouncing around for six years or so. Why would you think that upstart is an important missing piece when systemd does so much more than it ? Just curious...

Or maybe you made a typo ;)

Posted by xyroth at Sat Apr 30 04:37:15 2011
re plymouth, having support for this is obviously favourable to distros, but I know a lot of users who's prefered mode of working is 80x25 real text mode, and plymouth 200+x25 fake text mode just doesn't cut the mustard.

in fact, a number of them are currently having to dual boot with different boot options for when they need the gui and when they don't, and are looking to ditch plymouth based distro's unless an easy to find option to ditch super small text is found.

re common init scripts, the potential is there, but unless more of the scripts become common you will just end up with distro specific systemd or upstart scripts instead of the existing mess.

re the graph, it does look like you have said "for the features I'm interested in, systemd passes and the others don't". As the systems were developed by different people for different purposes, it would have been more useful to have had a real comparison of the strengths and weaknesses of the different systems rather than just your perception of the strengths of systemd.

Posted by Jef Spaleta at Sat Apr 30 05:48:44 2011
Question,
Has Red Hat made a statement committing  to rolling out systemd as an option to RHEL 6 at some point in the future?

Having it in Fedora is good, but I think the coming of age moment will be when Red Hat engineering makes a commitment to provide support for systemd for its customers.  If customers have to wait for RHEL 7 to see this go into production systems, that's not exactly a near time goal.

It will be interesting to see if Canonical gets systemd into an Ubuntu LTS release before Red Hat
makes a commitment to support systemd on deployed RHEL systems

-jef

Posted by raketenforscher at Sat Apr 30 06:27:58 2011
Cool. In the next step you'll add XML config files to be as cool as Apple's launchd ...

Posted by raketenforscher at Sat Apr 30 06:29:38 2011
oh, also great idea to compile your init scripts. so if something breaks you will have to fuck around with gdb instead of $EDITOR. is this project a troll?

Posted by Dax Kelson at Sat Apr 30 07:51:49 2011
@Jef Spaleta

Funny. I had the similar thought yesterday... "If RH waits until RHEL7 to integrate this, then SLES12 will ship with systemd first."

My guess is that systemd will make an appearance in RHEL6.3 to RHEL6.5 timeframe.

Posted by Sebastian Bertho at Sat Apr 30 09:29:06 2011
Does systemd support the debian configuration scheme of /etc/default/foo (e.g. for command line arguments?) or do I have to edit the process definition file for that?

Posted by Linus Walleij at Sat Apr 30 09:52:52 2011
Thanks Lennart for continuing to pushing change in Linux userspace, IMO systemd is as a big positive revolution in Linux userspace as the introduction of CFS was in the kernel, but arguably a lot harder to push through due to plain social factors such as conservatism and NIH-phenomena.

Posted by Michal at Sat Apr 30 12:05:30 2011
@Dax Kelson

"My guess is that systemd will make an appearance in RHEL6.3 to RHEL6.5 timeframe"

I would not be surprised if systemd will appear as technology preview. But it will not be easy to support both - different inits, different init scripts.

Posted by Colin Watson at Sat Apr 30 14:42:21 2011
Yes, Plymouth integration for Upstart was merged into Plymouth upstream seven weeks ago.  Please correct your table.

  http://cgit.freedesktop.org/plymouth/commit/?id=61e58596723efd504078ef3866913a0fa24da70c

Posted by Lennart at Sat Apr 30 15:50:53 2011
raketenforscher: nope, XML is not useful as configuration language. And I am not sure where you got the idea of compiled init scripts from, but uh, systemd is not doing anything like that.

Sebastian: you can import /etc/sysconfig/ and /etc/default/ in unit files. But I will shortly publish another blog story explaining why I think those files are actually a bad idea and we should get rid of them in the long run.

Colin: fixed. Sorry for the mistake!

Posted by intgr at Sat Apr 30 16:04:33 2011
@Sebastian
> Does systemd support the debian configuration scheme of /etc/default/foo
> (e.g. for command line arguments?)

Yes. You can define EnvironmentFile=/etc/default/foo and then use those arguments like ExecStart=/bin/foobar $FOOBAR_ARGS

Posted by awer at Sat Apr 30 18:39:04 2011
Sean McNamara, oops, you are right, I mean systemd. I got confused with all that odd names :)

Posted by ... at Sat Apr 30 19:03:46 2011
I can't help it, but some of what you call features seems like disadvantages. E.g.: "shell-free" - this actually gives the impression, that you restricted it somehow and moved towards non-standard solutions.

Posted by bst at Sat Apr 30 19:41:52 2011
As IE9? :)

http://www.infoq.com/news/2010/06/HTML5-Compliance-Dispute

Posted by Bogdan at Sat Apr 30 20:16:25 2011
Your features comparison is total bullshit.
I really like systemd and would like to see it adopted by many distros, but this comparison is super exaggerated and completely irrelevant in many cases.

Posted by Gustavo Sverzut Barbieri at Sun May 1 02:22:40 2011
People, you need to stop and read what was written, or ask for clarifications, instead of roughly guess what you want to say/troll :-)

  1. Shell is good, I can write everything in shell! Sure, you can do everything in assembly as well. So what's the point of saying that? If people would like to troll-back, they would say "shell is written in C and you can get that back", or "you can always call system()". Stupid, right?

  2. Saying you can add "umask" or "ulimit" in /etc/init.d/SERVICE is similar to say "I can change /usr/sbin/SERVICE and add umask() in there". Although technically possibly it is not recommended and that's why people moved to /etc/sysconfig || /etc/defaults. That's what Lennart tried to say with his argument, but maybe he was not clear enough. Everything is possible given effort, see my point #1 above.

  3. By sysvinit here he mean the provided package, not how can you use 3rd part services to do it. Everything is possible given effort, you can even write something like systemd that is called from sysvinit. Sysvinit is very simple indeed, that was always the point, but it caused problems on their own. Do you really see a point why all the distros handle FileSystem mounting differently? And although these are shell, have you ever tried to hack them? I did, Lennart did and others did and the consensus is "these are all legacy crap put together with tapes", if you change one thing it will break the other with no clear reason. People called it "maturity" but it's actually "unmaintainable shit -- don't touch". All we put in Systemd is what variated without any good reason in different distros (mount, crypto, fsck...)

  4. we're not denying or prohibiting anyone to use shell in their system services. You can still do it. We're just not obligating/mandating you to do it anymore. As expected you can use ExecStart=/usr/bin/my-script, or even use ExecStart=/bin/sh -c "a; b; c". But you're not forced to use it anymore. You're not forced to copy a skeleton with the same old start/stop/reload functions, etc. But if you wish, go for it!

  5. the unix principle is still applicable. We have one tool to do one job, see /lib/systemd/systemd-* binaries, they do one single task and are executed in different process, and in parallel. Actually we're following the best tool for the best job! Have you ever tried to parse things like /etc/fstab in shell, do some processing like dependency of mounts, all in shell? It is more code and yet less efficient as it will execute few processes (grep, sed, awk) without actual need... so more complexity with slowness is good? god, no!!! :-)

  6. people mentioned about bloat, particularly RAM usage and embedded systems. I'd like to see facts about this. In my experience working with embedded systems and porting them to systemd, it proved to be lighter on every single aspect! We trash less the VM, use less the CPU, etc... Would you care to show how more memory is used? Do you guess this because of DBus or kernel dependencies like CGroups?  Likely if you're embedded system is so small that those make it no-go, then you're likely not even using sysvinit but your own single-app as the init process and you have a completely custom system with XIP binaries, single app/service and all, right?

And it was not mentioned, but Gentoo is also interested to have it. It's not official yet, but there are great interest in the bugzilla and systemd overlay works amazingly well ;-) (shameless plug, I did the initial port for Gentoo!)

Posted by Gustavo Sverzut Barbieri at Sun May 1 02:28:08 2011
Ah, and I forgot to say a BIG THANK YOU to the project to help Linux overall by the leading principle "let's fix not workaround".

We should have a list of other projects that were improved given systemd existence and development. From bugs in btrfs that were spotted by parallel mount to core dependencies such as udev/dbus to simple standards like /run.

Not isolating itself in an isle and working around problems is an awesome thing that few projects do.

Posted by Seblu at Sun May 1 23:35:53 2011
lennart, why choose to make dbus a dependance of systemd?
I see the growing interest in a desktop environment on dbus but not on servers.

Posted by Lennart at Sun May 1 23:40:29 2011
Seblu: D-Bus is an IPC system. IPC is something you need in modern computing, regardless of server or desktop. I am not sure which confused folks have told you D-Bus wasn't for the server, but as I see it it's just a fine IPC, and there's no other contender for the job, and hence the decision for D-Bus is a no-brainer. If you claim that D-Bus wasn't for the server, then I can only respond back: why do you say that?

Posted by oiaohm at Mon May 2 02:50:57 2011
Dbus not for servers I also find strange.  Look at almost every single windows server.  You find graphical running.  This is what windows administrators are use to.  When they come to Linux they take shock horror.

The split between desktop and server users has been a major problem.  Graphical local and remote management of services and like would be good.  Particular if remote interface could be from a Windows, OS X, Android and Standard Linux clients.  So allowing those reluctance users simpler time.

Please be aware anyone pushing for system v to remain.  One of the biggest problem is system differences between distributions in the init system so making building generic management software for the init process very complex.

Lennart the idea of D-bus not for servers normally comes from its full name "Desktop Bus".  Of course I don't see any valid reason for servers not to use D-bus were its a suitable solution.

I still see systemd as a starting block.  There is a lot more work after systemd to make a system that is some what user-friendly to administrator.

Posted by Seblu at Mon May 2 03:23:58 2011
Lennart, as far i know, dbus means "Desktop Bus". It's the grand-son of DCOP and we can read in IntroductionToDbus on FreeDesktop.org : " is designed for use as a unified middleware layer underneath the main free desktop environments."

In the general spirit is a tool for integrating applications in a desktop environment, not an essential component of a running linux system.

In a pragmatic approch, there is few project listed in
http://www.freedesktop.org/wiki/Software/DbusProjects which run on a server. In addition it appears that udev remove its dbus support. Finally, all servers i used doesn't have dbus installed.

I suggest that it's rarely used on servers (except if you run kde), i'm not claiming that it should not.

I discovered systemd recently and currently it sounds promising, so my goal is not to troll/criticize but understand. My question may have been improperly installed.

Why you want to "force" dbus to be mandatory and not let it as an optional depend?

If i have installed dbus libs, systemd can expose it's services on dbus, but if not, or if it's disabled, it don't care.

Is dbus a core part of systemd or just a module allow tierce program to connect it?

Posted by oiaohm at Mon May 2 10:41:50 2011
Seblu problems come into existence.  You need a IPC of some form to control the init process.

"unified middleware layer underneath".  Really it makes sense.  Why does systemd have to have its own IPC independent to dbus.  That is more what has to be explained.

Mind you systemd is not the only item using dbus in a hard way.  NetworkManager  used by lot of distributions for network management also uses dbus a lot.

Seblu I really would like to see services using dbus more.  So we can have a unified system for real time control of services.

dbus combined with policykit also allows a layer of secuirty in command and control other ways have trouble doing.  Like allowing the web developer to restart web server and database servers.  Yet not allowed todo any other operation.  This kind of control would be complex todo any other way.  policykit can also limit commands other ways.

Issue here creating another IPC interface for systemd would possibly provide a back door around policykit restrictions.

That you are using console does not mean you don't need more options to control command and control of the system.

There has also been consideration give to creating an item called kbus.  A kernel version of dbus.  Since this would be way higher performing than current dbus.  So in time having or not having dbus might not be an option.

Posted by Meelis Roos at Mon May 2 11:38:18 2011
D-Bus in a server seems an out of place idea to me.
There is no clear permission model. To get some permissions in a flexible way, it needs policykit, consolekit etc around it and in addition to leading to bloat, it also becomes unmanageable with this bunch of XML files and not having a clear overview of things. So, security-wise, d-bus seems to be a disaster waiting to happen, and I do not want it into my servers.

Performance-wise, it has been bad. Moving the implementation into kernel exposes lack of good interfaces for lean and generic interface to be set into stone - rather it seems like ad-hoc xml messaging. And user-space daemon for IPC is also a big no-no if you want to develop a reliable IPC. So in the current form, d-bus is not a good IPC design.

Posted by Meelis Roos at Mon May 2 11:41:49 2011
One major problem with Ubuntu's upstart is debuggability. Builtin debug trace has helped me once but has failed to help all other times. Cannot even use strace to debug it. And it's not just me - upstart+mountall has some still unfixed problem if you look into their bug-tracker.

So, how is systemd's debuggability? Can I strace it, suspend it for many minutes without affect system functionality?

Posted by Harald Hoyer at Mon May 2 14:49:40 2011
@Meelis Roos: It's possible to strace pid 1 again with recent kernels.

Posted by Lennart at Mon May 2 15:00:48 2011
Seblu: we need a high-level IPC between systemd and systemctl, or between systemd and any other client. It needs to have a high-level notion of invoking methods, supporting signals and introspection. It needs to enforce policy, and authentication. And I see no other contender for that job. Do you? D-Bus is well supported everywhere, has bindings for almost every language, and is perfectly fine software. It is the only candidate and a really good candidate. The question is not whether to use D-Bus or not. The question is which IPC system to use. Because we need one. And that question only has one possible answer really.

Quite frankly, if you criticise D-Bus, then you should have better arguments than "I heard it's not right for the server", or "it's evil". Technical arguments matter. Not FUD.

Also, note that Upstart already pulled D-Bus in, so there's nothing new here really.

And one more thing: I don't "force" anything on anyone. Neither in quotation marks, nor without quotation marks. systemd is an offer. You get it for free. If you don't like the offer, don't take it.

I am sorry, but systemd is bound to D-Bus, and not optional. We use it as internal IPC as well as official interface. And that's not going to change.

meelis: d-bus allows you to enforce fine-grained policy, per-method, per-client. If you just use raw sockets and roll your own crappy IPC on top of that you will not be able to use anything like that. So D-Bus offers you substantially more than you'd get without it. I don't share your negative assessments of D-Bus. And really, the entire discussion is futile, since there's nothing else we could use instead really.

systemd is gdb'able and strace'able just fine. I do it all the time. Even if systemd crashes thinks will mostly work, except that you obviously cannot spawn new services with it.

Posted by oiaohm at Mon May 2 15:56:36 2011
Meelis Roos Please go read http://dbus.freedesktop.org/doc/dbus-specification.html  You will find you got lots wrong.  Like for starters wire protocal of dbus does not use xml at all.  And the wire protocal of dbus is particularly designed to allow it to reject requests.  Its a very clean protocol.  dbus wire protocol has a lot in common with email.  Plain text viewable clearly what is going on. No Xml.  So compact.

There is no point trying it have a technical debate with a person who does not have a clue what they are talking about.  Sorry to say almost everyone of your claims you made about dbus are false Meelis Roos.

Posted by anon2 at Mon May 2 16:02:18 2011
@Anonymous

Hacking shell scripts is a no-no if you run an autoupdated server and updates overwrite those scripts. If updates don't overwrite them (but put them in *.new for example) you need to update them manually every time there's an update, which is also a no-no if you want an autoupdated production server.

@Sean McNamara

I use daemontools and I wrote a custom run script which standardizes everything about services on my system, including startup configuration, inetd and standalone service types, transparent ssl support for inetd-type services, parallelization, monitoring, restarting, logging, log rotation, ip access control, resource limits etc. Installing a completely new and not-supported-yet service involves copying an existing /etc/service/ subdirectory (for the service directory tree with correct permissions), deleting logs therein (if there are any, multiple logging subsystems supported including syslog in which case there are no separate service logfiles), editing a couple of variables in ./env/ including a one-line PROGRAM variable that starts the service (that's usually the only change required if it's a standalone service and not an inetd-type one), and ln -s /etc/service/newservice to /service/ and I get all the benefits mentioned above and more.

That is 1 minute max if I do it manually and don't use a support Makefile that I wrote that creates a default directory tree for me (and optionally converts an inetd entry, or whole inetd.conf file, to my setup) in which case I only need to edit the one-liner-file (or not even that if it's a convert-from-inetd) that contains the command line that starts the service binary and link the new service to /service.

The entire setup consists of a system-wide run-file (used by all services for both the service and logging), a system-wide support Makefile (again, used for both the service and logging) and a per-service cfg file. Cfg file is used because it's easier to handle (by a human) a single file than a directory of one-liner files and it provides helpful comments and defaults, but it's easier to programatically edit a one-liner file than a full-fledged cfg file with associated comments etc, so an UI is trivial to write that either edits the cfg file itself or the directory of one-line cfg files. Then just either 'make cfg' to convert an env/ dir to a cfg file (for example for backup, because the cfg file itself is otherwise not used by daemontools whereas env/ is) or 'make env' to convert a cfg file to an env/ dir (for example to revert changes or init a new service from the cfg file).

I only use (sysv)init for the very basic boot/shutdown system configuration and handling which my setup doesn't do (in particular shutdown is a bit of a soft spot).

Because systemd seems to support all the features that my private setup does, and more, I'm considering switching to systemd if it works fine and can be built and configured on slackware without (too) much fuss, or even better, if someone convinces Patrick to put it in /extra.

Posted by Gerald at Tue May 3 14:59:15 2011
Very much looking forward to seeing this on Gentoo!

I read all your "systemd for Administrators" posts and really liked what I saw :-)

Posted by Y. Nemo at Tue May 3 18:13:43 2011
How does systemd benefit in any substantial way users who have already tweaked and tuned the SysV init system to their needs?

It seems systemd is more appropriate for large distributions intending to target a significant user and configuration base.

My most pressing concern is the added complexity of systemd as well as the inherent difficulty of debugging and configuring such a system. (side-note: though I haven't read the 'Systemd for Administrators' series, it seems to get quite complex). The SysV system stands out because bash is so generic. To modify the CPU/IO schedular, cgroups, uid/gid,automatic restarting etc... edit the appropriate bash script.


p.s. Which versions of systemd and other packages need no patching to work - mainly referring to sysloging.

Posted by Horst H. von Brand at Tue May 3 21:28:32 2011
@Y.Nemo: Benefit is that systemd is much more transparent, and easier to tweak (yes, "we all read shell scripts", but just look at the difference in having to read and understand 986 lines of shell to start up crond (854 in /etc/rc.d/init.d/functions, 132 in /etc/rc.d/init.d/crond proper) vs 11 lines in /lib/systemd/system/sshd.service (plus 19 in /lib/systemd/ssytem/syslog.target, which is referenced by the above, for the terminally paranoid). And tweaking anything in some of the init scripts is something I've learned to never do, and that the hard way. Most of the intricacies in the functions is just abstracted away and handled by systemd itself, really handles dependencies (not just "(try to) start/stop stuff in this magically divined order on runlevel change" and leaves you helpless when starting/stopping stuff by hand)  plus it groups the processes into groups for managing automatically among other goodies. It's vastly less complex to manage and can do things SysVinit can't even dream of, but also very different from what you (or I, for that matter) have ingrained.

Posted by Chris Cox at Wed May 4 00:46:25 2011
There are clearly pros and cons between sysvinit and newbies like upstart and systemd.

To me this isn't different from tar-balls vs. rpm vs. dpkg.

Is systemd a full on replacement of sysvinit... clearly no... but arguably yes if it allows fallback to sysvinit scripts.  It's different.  I view it like KDE4 vs. KDE3.  Not a direct replacement of form and function, but rather something different.

The pains... the pain of change, especially something that branches out a bit from the scope of what sysvinit covered is that ISVs will have to update their installation documentation.  Some things will become easier and some things will be more difficult.

systemd tries to solve a lot of age old problems... but you know, since they are "age old" people are used to working around those problems or have dealt with them long ago even with sysvinit. 

There are advantages to arbitrary shell based scripting systems.... and it does afford the experienced end user the opportunity to "hack" a server back to health.  While there are nice hooks for systemd, I do believe there will be unexpected quirks due to the added feature complexity.  There's a REASON why openSUSE 11.4 does NOT use systemd by default... you can read their page about the items that FALL OUT (no easy answer) when switching to systemd.

So... YMMV.  I think for simple things, systemd will work just fine.  For software unknown to systemd, there could be problems.... but maybe not... just requires more effort to figure out how to implement the pieces missing/ignored/altered inside of systemd.

For those needing the additional features of systemd (with regards to service protection, reliability, etc.)... of course, systemd brings a LOT of good stuff to the table.  The question is: How big of a problem was/is that?  Or have we just been "dealing" with sysvinit and accepted its limitations?  Are we truly happy with sysvinit?  Or does it really frustrate us?  IMHO, there's NOT a lot of sysvinit frustration out there.  So... finding the TRUE benefits of systemd and SELLING everyone is the key.  Showing people how the whole world will become much, much, much better with systemd.  Right now... I'm not seeing it.

Make the sale.  Decreased startup times... yes... we know that item... need more talking points, you know.. REAL life scenarios.  We can say, "it'll restart your services"... but frankly, my servers never go down.... and I don't think I'm alone.  In fact, if a service does crash out, do we REALLY want it to restart?  Need to show a tangible REAL and wide spread benefit.... perhaps something that everyone will WANT/NEED that sysvinit simply CANNOT do... and then maybe you can make the sale.

Otherwise, we're replacing something well understood with something new just because we wanted to do that.

Posted by oiaohm at Wed May 4 08:54:37 2011
Fall out in OpenSuse 11.4 is really not having transition interface processing working.  Not failures in systemd itself.  Big one is chkconfig status for active service not matching systemd status for active service.  Transition issue. Not unsolvable.  http://en.opensuse.org/openSUSE:Systemd_status  Yes I have been through the page.

Really most of the time people I find calling for system v to init remain.  Don't understand how many failures it is causing and how much its pissing off ISV's.

The dbus item that is complained about a lot I look forward to as a ISV.

Currently each distribution does there own tweaks on the system v init system to try to fix up limitations and issues it has.  So as a ISV if I need to change a global setting simple as turning a service on.  Ok what directory do I need to put that in.  /etc/rc2.d or is that /etc/rc3.d or is that /etc/rc4.d or is that /etc/rc5.d  Ok bugger do the lot.  Now what services should be before the service I need to run.  Bugger again.  because I got to know a magic number.  That is not the same between distributions.  That magic number says where in the boot my stuff starts.  So now I ship with everything including httpd server so I know I have everything starting the right way for me.  But now we have issues.

Basically for a ISV system v init is complete trash.  The little bit of overhead to support systemd I really don't care about.  Systemd I simple run if my thing trys to access something that is not started yet it sorts it out.

Really why does each distribution have to have its own management system for init with its own unque naming and bents.  Systemd will bring more commonality than us ISV's have ever had.  So maybe at long last we can use the host distribution provided httpd servers and other items instead of our own copies.

A project after this should be to make ISV live simple between distributions.  I have a php with mysql/postgresql website I want to run on a Linux distribution.  If everything is nice this should be a walk in park.  To get the php in the webserver get the database loaded and setup.  Currently its a complete trip through hell with variations for different distributions so it works.  Some of those variations is in the system v init system.  dbus does make it simpler for a installer to find out if a service is installed and if it is running.  Because dbus if its not their my installer does not die.

Configuration of services option is another major area that needs a proper common solution to make ISV developers live simpler.

Shell script hacking is something ISV's don't want happening either.  Since its makes more files that have to be checked for errors and can get screwed up by updates.  Its been a really good thing systemd does splitting executable and control data.  Doing this in shell script is kinda impossible.

Allowing system v init scripts is about allowing transitional.

One of the big things is that systemd here as a ISV will give me some secuirty options that are generic independent to what LSM distribution is using.  Cgroup interfaces as a ISV I could not depend on being able to alter as a ISV either.

Basically system v init needs a 40 foot hole dug and all records if its existence burnt and put in the hole filled in and no clue to its existence left.  Something that works for ISV without distribution knolledge putting in its place.  systemd at this stage ticks what use ISV's want.

And if distributions can when doing systemd please get your heads together can come up with common service naming.  So appache is not appache on one distribution appache2 on another httpd on another.  What are you trying todo here.  Be confusing so ISV have to have complex processing scripts to generically install from a LSB rpm.

Posted by T_W at Fri May 6 03:26:47 2011
Could you come up with a similar comparison table for PulseAudio vs the alternative sound systems?  I'd love to see if systemd is as likely to hose up my system as your last creation.

BTW, thanks for PulseAudio.  Who needed a working sound system anyways?

Posted by Kamilion at Fri May 6 11:47:05 2011
That last post by oiaohm hit the nail on the head.

Right now, you need a howto to do just about anything.

For example; here's some of mine:
http://blog.sllabs.com/2010/06/fun-with-nginx-upstart-and-lucid.html
http://blog.sllabs.com/2010/08/weaving-redmine-rvm-ruby-rails-rack.html
http://blog.sllabs.com/2010/08/catching-up-with-configuration-curve.html

If I want to do any of these things myself, I have to refer to my own howtos.

It really shouldn't be this hard.

This should work:
sudo apt-get install nginx php5-cgi php5-pgsql php5-sqlite php5-suhosin php5-imap php5-mcrypt php5-gd postgresql phppgadmin

If I hit 127.0.0.1 after that post-systemd, I'd expect to be looking at phppgadmin through a php-fpm unix socket. At the very most, after editing the nginx config file and reloading it.
One file edit, MAX.

No messing with the init system, no shell scripting, nothing but gksudo gedit /etc/nginx/nginx.conf

One of my favorites is adding this to nginx.conf:
include /var/www/sites/*/config/*;

If I ran F15, the nginx unit can even test the config and signal nginx to reload it on a valid config. In this case, nginx would validate all the includes too. No more gksudo.

All I'd have to do was click 'save' in gedit as a normal user. Imagine that.

And guess what? My howtos are ubuntu specific. Why? Because I can't be bothered writing them for every distro.

It shouldn't be this way. Let's make progress.

Posted by oiaohm at Sat May 7 02:24:42 2011
T_W I know pulseaudio seams bad.  But compared to esound, artd, MAS, NAS and NMM.  Its down right brilliant.

jackaudio does still have many key advantages.  Yet even feeding alsa applications into jackaudio is trouble.

Of course I have the same problem as every sound server though history.  Lock terminal let another user login and now they don't have sound because my account is still holding onto the sound output.

Maybe systemd providing cgroups around users maybe a solution to that problem can be formed. 

Users access to sound even if kernel does not mix.  The kernel must be able to control access.  Ideal world would be running jackaudio in 1 user pulseaudio in another and another using alsa direct and all them using the same sound card  without having cat fights over who is controlling the audio.  Active user on screen has the audio at min.

Basically audio requires a kernel level fix.  And systemd maybe a step to get the struts in place to take advantage of that fix.  Until that is accepted all the sound server work in the world is going to solve none of the traditional problems.

Posted by aniou at Mon May 9 14:51:44 2011
@Kamilion

Yours enthusiasm really warms my heart, but are you really sure about F15 and nginx? Now, I'm logged in F15-beta and I see only "old" initscripts. No conditional reloading in systemd for nginx nor haproxy at all.

Posted by oiaohm at Mon May 9 15:10:38 2011
aniou mind looking at postgresql.  Remember the flow lines.  nginx receives message wakes up php then php needs postgresql running.  So first one to need special handing would be postgresql.

This is the thing most of the advantages of systemd are a few levels in.  So systemd updates can be targeted at items like databases that kind of need applications to pause while they are starting instead of crashing.  Then over time expanded for connection dependability with nginx and the like.

Also remember systemd if its launched stuff threw the old init scripts can still perform all the operations they provide.  So interface does not have to be different.  Systemd control commands work against init scripts.

Posted by Anonymous at Mon May 9 15:14:17 2011
Commentary about this posting from the Ubuntu camp:

http://undacuvabrutha.wordpress.com/2011/04/29/why-ubuntu-should-continue-with-upstart-for-11-10/

Would be very interesting to hear Lennart's comment on their comments :-)

Posted by Stuart Gathman at Wed May 11 00:20:00 2011
Ubuntu (and any other contrarian distros) are doing us a favor by providing a basis of comparison.  You can verify whether one really is better than the other in operation.

Posted by oiaohm at Wed May 11 05:32:28 2011
Big thing to remember is upstart Ubuntu is using lacks "Copyright-assignment-free contributing".  So other distributions cannot trust there future to upstart.

Since due to Copyright-Assignment legally Canonical could pull the rug out from under them.  So upstart is really not competition for everyone other than Ubuntu.

There are very few resisting distributions.  System V itself basically is dead man walking too many problems also most systems don't use it.

bussybox init is very lean and mean.  Being able to cut-down to out run that it will be impressive.  For embed usages bussybox init is going to be one of the harder ones to beat.  Low memory usage combined not requiring runlevels or anything else fancy.  Most likely would have to be a sub build of systemd.

The problem ubuntu has is that systemd is only a year old and already systemd is well ahead in features.  Being a living init project compared to systemd that was basically a dead init project as more distributions come onboard development should speed up.

I see systemd as a start of a long trip.  New start.

Sandboxing user applications is something that has to come as well.  Users should have the right to cgroup off an application so it cannot see their documents or whatever.

Lets make the user the commander of the ship not the applications.

Posted by legolas558 at Sat Jul 16 09:52:06 2011
Oh, please, let's fix the Linux mess! It's nice to read about somebody actually pursuing this battle! :)

I am testing this on Arch Linux, great work Paul, thanks!

Posted by NunnaBizness at Tue Jul 19 09:46:00 2011
Quoting Lennart's own words from this blog as of 19-July-2011 (just in case he edits it to cover his tracks):

"Technical arguments matter. Not FUD."

Lennart, your tables look like FUD produced by any major commercial software company. Oh, I forgot. You work for one don't you?

I poke at you because you have not provided the technical arguments to support your claims yet you demand it of others entering comments here.

How many claims do you make in these tables about systemd? 50? 100? More? Whatever it is I would like, no DEMAND, that you follow your own words and provide technical arguments for every claim you make about systemd where it is given a "yes".

To argue back without providing proof only shows that you are a hypocrite with an ego larger than the entire universe.

Posted by pomidorek at Wed Nov 2 16:53:12 2011
there is no way I will leave sysvinit and go with systemd - years with aix, hp-ux, sco  and Gentoo.. systemd along with fedora15 that I am forced to use for one or other reason making me sick, never been a fan of anything Redhat related, and now systemd... maybe one day...

Posted by rudi at Sat Nov 12 22:12:55 2011
Hehe, very nice that feature list!
But to make its meaning more clear to the readear you should use red color for "yes" and green for "no".

Posted by Pawel at Mon Dec 12 09:20:12 2011
@pomidorek

What a valuable comment. Nobody forces you to use Fedora. You can stick with old and ugly sysvinit if you want, but I'm really happy Linux got superior system and service manager. It's great it's Linux only. :)

Posted by Nat at Tue Jan 3 20:04:23 2012
The SCM comparison doesn't say anything about systemd/sysvinit/upstart themselves, and likely none of the SCMs listed would attract/detract potential contributors.  Though to your credit you caveat the entire post with your [potential] bias - you are undercutting your factual credibility with this particular "comparison."

Posted by Alex.T. at Tue Jan 10 12:36:16 2012
Lennart: "ncopa: really? bloat? Replacing a mess of shell scripts with a handful of cleanly-written, easy to read, robust C utilities..."

Lennart, please do not substitute one term with another one--they are not "C utilities" but "binary utilities". Personally, I can not read ELF files. Glad, if you can.

Posted by chrys at Wed Jan 25 08:18:42 2012
decades of init scripts down the tubes - I still need the functionality they provide. Everything in parallel just doesnt cut it -- even in earlier Fedora releases various scripts had to have sleep [cf man 1 sleep] inserted just to counter somebody's overactive imagination on parallelization!!!!!
forcing systemd use is not the answer - optioning systemd use IS.

Posted by FeRD at Thu Feb 9 00:34:03 2012
Well, that blew. I tried to submit a somewhat detailed comment just now, and was greeted with only a lovely "error occurred" unexplained-rejection. :-(

Lazarus has the text, of course, so there's nothing lost there. I'm just wondering why I'm so unloved! sad

Posted by FeRD at Thu Feb 9 00:36:06 2012
Huh. Of course, that one goes through. Let's try the original again, I guess...

@Nat:

I completely disagree. Somewhat to my own surprise, even, I've noticed that I'm far more likely to take a look at a new project, poke around in the source tree, and consider checking out and building a copy for myself if, say, the project is hosted over at github than if it's in some svn repo somewhere. (Even at Sourceforge, I tend to groan a tiny bit when I stumble across one of the older projects that's still trapped in Subversion or, even more horrifying, CVS land!)

And I'm infinitely more likely to roll up my own changes or development and submit it back to the project if my working copy is a git repo, because if I did any significant work within the source tree I probably git init'd it even if the origin was a different SCM!

But, to match your tone: You undermine your comment by seizing on a single line-item element from a chart in a fairly lengthy article — one data point out of more than 100! — and choosing to go all inexplicably butthurt about that one, relatively insignificant throwaway point.

Leave a Comment:

Your Name:


Your E-mail (optional):


Comment:


As a protection against comment spam, please type the following number into the field on the right:
Secret Number Image

Please note that this is neither a support forum nor a bug tracker! Support questions or bug reports posted here will be ignored and not responded to!

It should be obvious but in case it isn't: the opinions reflected here are my own. They are not the views of my employer, or Ronald McDonald, or anyone else.

Please note that I take the liberty to delete any comments posted here that I deem inappropriate, off-topic, or insulting. And I excercise this liberty quite agressively. So yes, if you comment here, I might censor you. If you don't want to be censored your are welcome to comment on your own blog instead.

Lennart's Blog | Lennart's Homepage | Lennart's Photos | Impressum/Imprint
Lennart Poettering <mzoybt (at) 0pointer (dot) net>
Syndicated on Planet GNOME, Planet Fedora, planet.freedesktop.org, Planet Debian Upstream. feed RSS 0.91, RSS 2.0
Archives: 2005, 2006, 2007, 2008, 2009, 2010, 2011

Valid XHTML 1.0 Strict!   Valid CSS!