Discussion:
Bug#727708: init system other points, and conclusion
(too old to reply)
Ian Jackson
2013-12-28 22:50:30 UTC
Permalink
I have reported on my impressions and experiences of both systemd and
upstart in my previous messagges.

I'd like to run through the remaining points I want to make. I'll
then summarise and set out my primary conclusion.


Firstly, unlike the systemd maintainers, I think portability to
non-Linux systems is important. It may be that our existing non-Linux
ports are not very widely used, undermained, and/or not of production
quality. However, I think it is important for us to keep those
options open. Of course that provides a space for people to work on
them and use them, directly, but more importantly it keeps Debian's
options open for the future. And the competition helps keep Linux
honest, which is important because Linux is effectively unforkable,
has a poor history of responsiveness to concerns of some of its
downstream userbases, and has a nearly-unuseable governance setup.

This by itself means that systemd would have to have very strong other
advantages for me to want to choose it. And I recognise that this
point of view is not necessarily widely shared. However, happily, I
find that no conflict arises for me between my desire for portability
and the other relevant criteria.


It is important for a component like an init system to be flexible: it
needs to act as glue between disparate software projects, and to
accomodate their quirks.

My experiences with systemd's Debian maintainers (and, indirectly,
systemd's upstream) have been far from satisfactory in this regard.
Instead of taking a flexible approach, and being willing to provide a
range of glue facilities and approaches for different daemon
upstreams, the systemd community seems doctrinaire. Daemon authors
are expected to do as they are told by systemd upstream, rather than
systemd upstream making things comfortable for daemon developers.

This is IMO the opposite of the proper attitude.

The same appears to be the case for systemd's interactions with Debian
as a downstream. Pressure has had to be applied on issues such as
separate /usr (and much documentation still contains claims that this
is "broken"); I asked for systemd to be able to execute programs from
PATH rather than requiring unit files to specify the absolute path and
was rebuffed (#...)

This is exacerbated by the fact that systemd's Debian maintainers are
(IMO) much too deferential to upstream.

Finally, the systemd community seems to havve a programme of replacing
many other facilities throughout the system (including ones which
other software already does better - see eg binfmt-support, #716812,
and Colin's comments, which I agree with). This is to be discouraged
IMO.

In short, the systemd community feels, to me, arrogant and
controlling. I am far from the first to say something like this, but
I've now experienced things for myself and I concur with the
criticism.


upstart's minimalism is very appealing to me.

It does, however, have a number of missing features. Those I have in
mind are:
- ability to log daemon output to syslog
- multiple socket activation (systemd socket activation protocol)
- socket activation for IPv6 (and datagram sockets)

Of these Russ rightly points out that lack of IPv6 support is rather
poor; it is arguably not suitable for release in jessie without this.

However, crucially, these are all simple matters of programming,
without difficult design decisions. They IMO don't reveal structural
problems with upstart's approach to things.

Furthermore, in my view the responses of Debian's upstart maintainers
to my bug reports about upstart have been exemplary. There has been,
for example, no resistance (from them or AFAICT from upstream) to
supporting the systemd socket activation protocol.

I am confident, therefore, that if we choose upstart in jessie, these
lacunae (and any others we will discover) will be fixed. These
problems are certainly not blockers for selecting upstart as the
default in jessie.


So, to recap this and my previous mails and summarise:
* upstart is simpler than systemd (which leads to fewer bugs, etc.)
* upstart integration fits better into a daemon source code
* upstart is easier to package for than systemd
* upstart's community is much better to work with
* systemd's non-portability is (for me) a near-blocker
* upstart's remaining disadvantages are readily fixable SMOP
* upstart is therefore ready for adoption in jessie
* sysvinit has many longstanding bugs and deficiences
* openrc is not ready (I couldn't evaluate it due to lack of a manual)

I therefore conclude that the default init system for jessie should be
upstart.


Thanks,
Ian.
Russ Allbery
2013-12-29 02:52:24 UTC
Permalink
Post by Ian Jackson
Firstly, unlike the systemd maintainers, I think portability to
non-Linux systems is important. It may be that our existing non-Linux
ports are not very widely used, undermained, and/or not of production
quality. However, I think it is important for us to keep those options
open. Of course that provides a space for people to work on them and
use them, directly, but more importantly it keeps Debian's options open
for the future. And the competition helps keep Linux honest, which is
important because Linux is effectively unforkable, has a poor history of
responsiveness to concerns of some of its downstream userbases, and has
a nearly-unuseable governance setup.
This by itself means that systemd would have to have very strong other
advantages for me to want to choose it. And I recognise that this point
of view is not necessarily widely shared. However, happily, I find that
no conflict arises for me between my desire for portability and the
other relevant criteria.
I find this statement curious, given your recommendation of upstart, since
my understanding is that neither upstart nor systemd have been ported to
non-Linux systems.

Is the porting work started by Dmitrijs Ledkovs farther along than I had
thought? The latest update I heard from November was that he had a fork
of libnih that passed its test suite once several key features were
disabled (inotify, abstract domain sockets), and very little of that work
had been merged upstream. (It's possibly worth noting here that the
libnih upstream and Debian package maintainer is still Scott James
Remnant, who previously expressed skepticism of a straight port to
kFreeBSD.)

I've been giving a lot of thought to the portability issue as well, but
where I'm currently at is rather different than your sentiments above. My
feeling is that there is a slight advantage to upstart here in that the
port has been started, but it's slight. In both cases, there are
Linux-specific APIs embedded deeply in the project, and in both cases
porting would require substantial effort.

The upstart porting effort appears to be working in exactly the right way:
providing functionality in glibc and the kernel on FreeBSD that supports
the APIs that libnih needs. That would also be the first step in porting
systemd to kFreeBSD/glibc. It's great work and benefits both projects as
well as many others, but I'm not sure it's a meaningful discriminator.

I feel like you may be overly optimistic about future development in
upstart and overly pessimistic about future development in systemd here.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Michael Stapelberg
2013-12-29 02:54:36 UTC
Permalink
Hi Ian,

My apologies in advance for answering only to one email, but quoting
several. It could be that you had some threading in mind which my reply
might break.
* systemd's readiness protocol is an unattractive implementation
target for an upstream daemon author. I think this is an important
weakness, if it remains unaddressed.
We talked about this in #732157 already, but I think it is worth
summarizing it here: for modern daemons, libsystemd-daemon is a widely
available library which can be trivially added. In case dependencies
absolutely need to be kept at a minimum, the less preferably alternative
of dropping that library’s code into the daemon’s code base exists. If
both of those are unsatisfactory, one can implement it by oneself, which
you deem unattractive. I tend to agree, but I think you are overstating
this as an “important weakness”. Most daemons do not even need any
readiness notification whatsoever. Using socket activation or forking
after initialization is enough.
I'm treating the config fragments as part of the packaging, rather
than as something upstreamish, because in practice they are inherently
un-debuggable without access to a system running the corresponding
daemon. They are also small enough that any distro which cares could
easily ship their own (and might need to).
I don’t understand that reasoning. By “daemon”, do you mean init system
or actual daemon for which the service file is intended? Also, why would
a distro ship its own? Even in the rare case that an upstream-provided
service file is (and stays!) unsuitable for Debian, a patch is all that
is necessary.
There was less support from the Debian policy manual. Perhaps there
is some other systemd Debian packaging guidance somewhere which I
didn't find.
There is https://wiki.debian.org/Systemd/Packaging which is the first
Google hit for “debian systemd packaging”. Until a while ago, the page
was changing a lot while we were still hashing out details about
packaging. By now, I’d say we could add this to the policy manual.
The unit files were somewhat harder to write. It wasn't just that the
systemd unit file offered a great many more options, although that was
a factor. The two-unit split of systemd socket activation was more
fiddly. And systemd wants each directive to be in a particular
"[section]".
See also http://0pointer.de/blog/projects/systemd-for-admins-3.html
(“How Do I Convert A SysV Init Script Into A systemd Service File?”).
I do admit that this is hard to find if you don’t know it exists. It
_is_ linked to from the main systemd website, though, but I cannot
expect that everybody reads the entire body of documentation.
Also, the approach to the systemd integration introduces a new runtime
package dependency on "init-system-helpers", which despite its
generic-sounding name actually contains only helpers for systemd and
is maintained in Debian by the systemd maintainers.
It has already been pointed out elsewhere in the thread, but given that
I introduced this package, I’d like to stress again that I will gladly
accept other init system’s helper programs to this package. The package
name and description is not misleading; I really mean it.
The same appears to be the case for systemd's interactions with Debian
as a downstream. Pressure has had to be applied on issues such as
separate /usr (and much documentation still contains claims that this
is "broken"); I asked for systemd to be able to execute programs from
PATH rather than requiring unit files to specify the absolute path and
was rebuffed (#...)
#732981 is the bug reference that you didn’t include by accident, I
think :).
This is exacerbated by the fact that systemd's Debian maintainers are
(IMO) much too deferential to upstream.
[…]
In short, the systemd community feels, to me, arrogant and
controlling. I am far from the first to say something like this, but
I've now experienced things for myself and I concur with the
criticism.
Given that I am the one who responded to both your referenced bug
reports, let me provide my perspective in this bug. Upstream does not
want to provide the features you asked for, and I’ll not comment on
that, apart from stating that it’s not obvious that they are a good idea
(one can see that from the discussion on the bugs).

You then asked for these features to be carried as a patch in the Debian
systemd package, and both requests were rejected. I think this is what
you refer to when saying “the systemd Debian maintainers are much too
deferential to upstream”. The reason why we don’t want to carry these
patches is that they significantly alter the public API between systemd
and the daemon authors — but in Debian only!

As stated in the bug, my rule of thumb is whether people need to
differentiate between Debian and “all other distributions”. E.g., with
your simpler readiness notification proposal, daemon authors could use
that on Debian, but would need to conditionally compile code for Debian
while they also still need to ship code for the current API. This is not
a simplification at all. Similarly, with the $PATH thing, we’d introduce
the need to patch _every_ single upstream unit in our Debian
packages. Even worse, unit file authors would be surprised when their
unit file does not work on other systems when it was written and tested
on Debian (where one could use $PATH).
Furthermore, in my view the responses of Debian's upstart maintainers
to my bug reports about upstart have been exemplary. There has been,
for example, no resistance (from them or AFAICT from upstream) to
supporting the systemd socket activation protocol.
I’d like to point out that with features that are universally accepted
(and especially by upstream), the response of systemd contributors was
exemplary, too. Zbigniew Jędrzejewski-Szmek in particular has reacted to
documentation clarification requests and even feature requests like the
globbing of units in record time. Thanks for that!
--
Best regards,
Michael
Russ Allbery
2013-12-29 03:11:15 UTC
Permalink
Post by Michael Stapelberg
You then asked for these features to be carried as a patch in the Debian
systemd package, and both requests were rejected. I think this is what
you refer to when saying “the systemd Debian maintainers are much too
deferential to upstream”. The reason why we don’t want to carry these
patches is that they significantly alter the public API between systemd
and the daemon authors — but in Debian only!
As stated in the bug, my rule of thumb is whether people need to
differentiate between Debian and “all other distributions”. E.g., with
your simpler readiness notification proposal, daemon authors could use
that on Debian, but would need to conditionally compile code for Debian
while they also still need to ship code for the current API. This is not
a simplification at all. Similarly, with the $PATH thing, we’d introduce
the need to patch _every_ single upstream unit in our Debian
packages. Even worse, unit file authors would be surprised when their
unit file does not work on other systems when it was written and tested
on Debian (where one could use $PATH).
I agree with Michael on this point. I think he would be doing a
disservice to both Debian and upstreams to carry these sorts of changes as
distribution-specific patches.

Support for SIGSTOP is, I think, a debatable point, and I understand Ian's
position. I also understand Lennart's position. If I were Lennart, I
probably would have added it, but I can understand why he wants to
discourage it. I don't think it's a good long-term solution. I do think
a strong argument could be made for adding it, though... but not as a
distribution-specific patch. That seems like a recipe for user confusion.

On $PATH searching, I just completely disagree with Ian. Adding $PATH
dependencies to unit files is something I consider a straight-out bad
idea. Patching unit files to adjust paths, even if it requires
distribution-specific patches, is significantly better. I would have told
him no to this request were I systemd upstream as well. It makes the
whole init system more fragile in ways that aren't helpful.
Post by Michael Stapelberg
I’d like to point out that with features that are universally accepted
(and especially by upstream), the response of systemd contributors was
exemplary, too. Zbigniew Jędrzejewski-Szmek in particular has reacted to
documentation clarification requests and even feature requests like the
globbing of units in record time. Thanks for that!
I concur with this. I have had exceptionally good interactions with both
systemd upstream and the systemd package maintainers.

I think Ian had the misfortune of having the two points he cared the most
about both be objections to fundamental design decisions and places where
upstream felt that implementing his approach would make the system more
fragile. In one case, I agree with them. Rejecting such requests does
not make for a bad upstream. I would argue that's upstream's job.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Raphael Hertzog
2013-12-29 09:36:28 UTC
Permalink
Hi,
Post by Ian Jackson
It does, however, have a number of missing features. Those I have in
- ability to log daemon output to syslog
- multiple socket activation (systemd socket activation protocol)
- socket activation for IPv6 (and datagram sockets)
Of these Russ rightly points out that lack of IPv6 support is rather
poor; it is arguably not suitable for release in jessie without this.
However, crucially, these are all simple matters of programming,
without difficult design decisions. They IMO don't reveal structural
problems with upstart's approach to things.
This is pretty much in opposition with the way that the tech-ctte has
approached problems in the past. In used to only decide based on existing
code that was ready to be deployed.

This is troublesome because, in my eyes, you now look very much like the
systemd fanboys that you criticize, except that you are in the upstart
camp.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Tollef Fog Heen
2013-12-29 15:15:17 UTC
Permalink
]] Ian Jackson
Post by Ian Jackson
This is exacerbated by the fact that systemd's Debian maintainers are
(IMO) much too deferential to upstream.
That's because the bits of systemd you've asked to change isn't just a
piece of software. It's a standardised API, and you're effectively
asking us to change that API. I think it's pretty clear why that is a
bad idea.
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
Nikolaus Rath
2013-12-29 22:55:24 UTC
Permalink
Post by Ian Jackson
It does, however, have a number of missing features. Those I have in
- ability to log daemon output to syslog
- multiple socket activation (systemd socket activation protocol)
- socket activation for IPv6 (and datagram sockets)
Of these Russ rightly points out that lack of IPv6 support is rather
poor; it is arguably not suitable for release in jessie without this.
However, crucially, these are all simple matters of programming,
without difficult design decisions. They IMO don't reveal structural
problems with upstart's approach to things.
I don't understand this argument. Even turning systemd into a 1:1
upstart copy is "simple programming" (and vice versa), so by that argument
there'd be no technical reason to pick one init system over the other at
all. In some sense, it seems to me that you are proposing that the
Debian *Technical* Committee make its decision for the init system based
on the *developers* of each project, because the technical differences
are mere metters of programming.

(I'm not sure why the "structural" and "design decisions" would make a
difference either, changing from one existing design to another existing
design is also just programming).

Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

»Time flies like an arrow, fruit flies like a Banana.«
Russ Allbery
2013-12-30 00:10:10 UTC
Permalink
We seem to be at the point of the process where at least those of us who
did early investigation are stating conclusions. I think I have enough
information to state mine, so will attempt to do so here.

This is probably going to be rather long, as there were quite a few
factors that concerned me and that I wanted to investigate.

The brief summary is that I believe Debian should adopt systemd as its
default init system on Linux. There are two separate conceptual areas in
which I think systemd offers substantial advantages over upstart, each of
which I would consider sufficient to choose systemd on its own. Together,
they make a compelling case for systemd. This position would have
substantial implications for upgrade paths and for non-Linux ports; I'll
discuss a bit of that below, but most of it in the separate branch of this
bug report that Ian opened on that topic.

Below, I first discuss the other choices before us besides systemd and
upstart. Then I look at a straight technical comparison between the two
init systems, and finally look at issues of maintenance, community,
ecosystem, and portability. The three main criteria on which I was
evaluating both systems were technical capabilities, surrounding
ecosystem, and portability. The latter two turned out to be deeply
entangled, so I discuss them together.


1. Other Choices

First, other choices besides systemd and upstart.

There were three replacement init systems proposed to the Technical
Committee to replace sysvinit, plus the existing status quo. The third
option, OpenRC, is a more conservative and less revolutionary change than
either systemd or upstart. It continues to use the existing sysvinit init
process but replaces the startup script management with a more robust
shell library and additional features.

I think the OpenRC developers are great people and I wish them all the
success in the world with their project, but I just don't think it's
ambitious enough for Debian's needs. If we're going to the effort of
replacing init systems and changing our startup scripts, a bare minimum
requirement for me is that we at least address the known weaknesses of the
sysvinit mechanism, namely:

* Lack of integration with kernel-level events to properly order startup.
* No mechanism for process monitoring and restarting beyond inittab.
* Heavy reliance on shell scripting rather than declarative syntax.
* A fork and exit with PID file model for daemon startup.

My impression of OpenRC is that it is not attempting to solve these issues
in the same way that systemd and upstart are. To the extent that these
issues are on the OpenRC roadmap, it's not as far along as either systemd
or upstart is. It's difficult to evaluate since the OpenRC documentation
is rather sparse and lacks the comprehensive manual available to both
systemd and upstart, which is itself a sign of a lack of project maturity.

I don't think that switching to OpenRC offers enough clear benefit over
the status quo.

That raises the other obvious option: sticking with sysvinit. I've made
my position on this fairly clear in other threads, so I won't reiterate it
here at length. The short version is that I turned to other tools to
manage daemons years ago because sysvinit was simply inadequate, and my
feeling on that hasn't changed. The model of fork and exit without clear
synchronization points is inherently racy, the boot model encoded into
sysvinit doesn't reflect a modern system boot, and maintaining large and
complex init scripts as conffiles has been painful for years. Nearly
every init script, including the ones in my own packages, have various
edge-case bugs or problems because it's very hard to write robust service
startup in shell, even with the excellent helper programs and shell
libraries that Debian has available. A quick perusal of
/etc/init.d/skeleton and the complex case statements and careful attention
to status codes required for a proper init script makes this case clear.

I think the choice of a default init system for Linux is a choice between
systemd and upstart. We would be doing ourselves and our users a
disservice to stick with the status quo, or even a moderate update of the
status quo to add a simpler service definition. The limitations have been
well-known for years, and I think it's telling that most other operating
systems, even fairly conservative ones, have moved away from the System V
init script model.

The last option that was before us was supporting multiple init systems.
I consider this a variation on a transition plan, with a possibly infinite
time horizon, and will discuss this separately when I talk about
transition plans.


2. Core Service Management Functionality

As reported to this bug, I did a fairly extensive evaluation of both
upstart and systemd by converting one of my packages, which provides a
network UDP service, to a native configuration with both systems. While
doing so, I tried to approach each init system on its own terms and
investigate what full, native support of that init system would look like,
both from a Debian packaging perspective and from an upstream perspective.
I also tried to work through the upgrade path from an existing init script
with an external /etc/default configuration file and how that would be
handled with both systemd and upstart.

I started this process with the expectation that systemd and upstart would
be roughly evenly matched in capabilities. My expectation was that I
would uncover some minor differences and variations, and some different
philosophical approaches, but no deeply compelling differentiation.

To my surprise, that's not what happened. Rather, I concluded that
systemd has a substantial technical advantage over upstart, primarily in
terms of available useful features, but also in terms of fundamental
design.

2.1. General Impressions

systemd feels like a software package that has been used and pounded on in
a wide variety of real-world situations, and has grown the flexibility and
adaptibility that is required to make a wide variety of use cases work.
upstart, on the other hand, has a minimal design and a ready escape to
shell scripting, which may have discouraged directly tackling a broader
array of use cases. Regardless, there are a bunch of cases that systemd
handles cleanly with simple configuration that would require shell script
fragments or other workarounds in Ubuntu, which in turn makes the startup
configurations less reliable and harder to debug.

I was quite impressed throughout the process of developing systemd unit
files. Every time I realized I needed some piece of functionality to
configure the daemon properly, systemd already had it.

2.2. Major Functionality Gaps

Here are the major pieces of functionality that I think would have to be
added to upstart for rough feature parity:

* Socket activation, by which I don't mean lazy start of daemons, although
it enables that, but init management of socket setup so that daemons can
start in parallel.

This has been discussed elsewhere on the thread, but I want to note here
that systemd's approach is bold and innovative. We've had multiple
discussions in Debian lists in the past where people have felt somewhat
depressed or discouraged about Debian's lack of innovation or
unwillingness to tackle sweeping improvements. After having studied and
implemented socket activation, I think this is one of those
opportunities, and we should not pass it by.

There are a variety of advantages to socket activation that have been
discussed elsewhere, and I'm not going to repeat them all here. But one
I want to call out is the advantage for an enterprise systems
administration environment. Right now, in order to configure bind
addresses or IPv6 behavior for my services, I have to dig into the
individual configuration syntax or command-line flags of each separate
daemon, and often there's no easy way to set these parameters without
making intrusive changes to daemon startup. Socket activation lets me
manage all of this through a simple configuration override that I drop
into /etc via (for example) Puppet, and the syntax is the same for every
service that uses it. It would obviously take quite some time to get
there, but that's a really nice vision of the future, and one that would
make a real difference for Debian use cases I care about.

upstart has a socket activation protocol, but it would need an
almost-complete redesign in order to be used the way that systemd's can
be used. It doesn't support passing multiple sockets (required for
complex daemons, some IPv6 scenarios, and binding to multiple but not
all interfaces), it doesn't support IPv6 at all, it doesn't support UDP
sockets, and its configuration syntax is inadequate to represent the
parameters that would be useful in a real-world case. It also doesn't
separate the socket configuration from the daemon configuration, which
makes it harder for a local systems administrator to control binding
behavior without changing other properties of daemon initialization.

* Integrated daemon status. This one caught me by surprise, since the
systemd journal was functionality that I expected to dislike. But I was
surprised at how well-implemented it is, and systemctl status blew me
away. I think any systems administrator who has tried to debug a
running service will be immediately struck by the differences between
upstart:

lbcd start/running, process 32294

and systemd:

lbcd.service - responder for load balancing
Loaded: loaded (/lib/systemd/system/lbcd.service; enabled)
Active: active (running) since Sun 2013-12-29 13:01:24 PST; 1h 11min ago
Docs: man:lbcd(8)
http://www.eyrie.org/~eagle/software/lbcd/
Main PID: 25290 (lbcd)
CGroup: name=systemd:/system/lbcd.service
└─25290 /usr/sbin/lbcd -f -l

Dec 29 13:01:24 wanderer systemd[1]: Starting responder for load balancing...
Dec 29 13:01:24 wanderer systemd[1]: Started responder for load balancing.
Dec 29 13:01:24 wanderer lbcd[25290]: ready to accept requests
Dec 29 13:01:43 wanderer lbcd[25290]: request from ::1 (version 3)

Both are clearly superior to sysvinit, which bails on the problem
entirely and forces reimplementation in every init script, but the
systemd approach takes this to another level. And this is not an easy
change for upstart. While some more data could be added, like the
command line taken from ps, the most useful addition in systemd is the
log summary. And that relies on the journal, which is a fundamental
design decision of systemd.

And yes, all of those log messages are also in the syslog files where
one would expect to find them. And systemd can also capture standard
output and standard error from daemons and drop that in the journal and
from there into syslog, which makes it much easier to uncover daemon
startup problems that resulted in complaints to standard error instead
of syslog. This cannot even be easily replaced with something that
might parse the syslog files, even given output forwarding to syslog
(something upstart currently doesn't have), since the journal will
continue to work properly even if all syslog messages are forwarded off
the host, stored in some other format, or stored in some other file.
systemd is agnostic to the underlying syslog implementation.

* Security defense in depth. Both upstart and systemd support the basics
(setting the user and group, process limits, and so forth). However,
systemd adds a multitude of additional defense in depth features,
ranging from capability limits to private namespaces or the ability to
deny a job access to the network. This is just a simple matter of
programming on the upstart side, but it still contributes to the general
feature deficit; the capabilities in systemd exist today. I'm sure I'm
not the only systems administrator who is expecting security features
and this sort of defense in depth to become increasingly important over
the next few years.

Here again, I think we have an opportunity for Debian to be more
innovative and forward-looking in what we attempt to accomplish in the
archive by adopting frameworks that let us incorporate the principles of
least privilege and defense in depth into our standard daemon
configurations.

There are also a plethora of minor features and tuning available in
systemd but not in upstart. None of this is as significant as the points
mentioned above, and none of it is as difficult to implement, but it's not
currently implemented, and I think it speaks to systemd having been tested
against a broader array of use cases.

2.3. Event vs. Dependency Model

There is one UI design difference between systemd and upstart that's less
clear-cut, but which I think will surprise people. systemd is built
around familiar dependencies between services, and starts services in
dependency order. There are some twists, such as allowing a service to
create a reverse dependency (make another service depend on it), but it's
the basic design that's familiar to any packager, or to users of languages
like Puppet. upstart, on the other hand, uses a message bus model:
services are started when particular events are received, and dependencies
are expressed by listing the events required to trigger startup (or some
other action).

Conceptually, both of these designs are equivalent. They both construct a
DAG that's used to order service startup. However, upstart complicates
matters by having two types of messages on its message bus: signals and
methods (technically, there are also hooks, but the distinction doesn't
matter for this point). Signals behave like the typical asynchronous
message bus event, or like a dependency: they trigger services to start,
but the service issuing the signal does not care whether anyone listens or
not. Methods do not; methods are effectively synchronous calls and the
service issuing a method event waits until the method event has been acted
on before continuing.

The UI problem with this approach is that it creates a pitfall with rather
noticable consequences. If someone ever confuses a signal event and a
method event and starts a service on a method event instead, it is then
very easy to block startup of some fundamental system service because its
method event never completes due to deadlock. This is made somewhat more
likely by the fact that method events are the default in initctl emit
commands, whereas signal events require a flag.

Again, this is not a fundamental issue with either system; either
representation is mathematically convertable into the other. But it's
difficult to mess up dependencies in quite the same way. One can create
cycles, but unless one is modifying the dependencies of core services,
it's hard to create a cycle that involves a core service. upstart
provides a way to shoot oneself in the foot by blocking startup of a core
service by listening to the wrong type of event. This model doesn't, so
far as I could find, offer any clear advantages over a dependency
structure in compensation.

2.4. Configuration File Model

There is one place where I came into this evaluation preferring the
upstart design over the systemd design, and came away with a continued
preference, but a more mild one: the configuration file model. systemd
uses an /etc overrides /lib model, where all unit configurations are
installed in /lib and only local overrides and some configuration goes
into /etc. upstart uses the (more familiar to Debian) model where the
daemon configuration is a conffile in /etc.

Both approaches have real advantages, but I think the upstart approach has
slightly more. The systemd model means that one no longer has to add
various guards to daemon configurations to allow for the possibility that
the package has been uninstalled but not purged. Those continue to be
necessary with upstart (and continue to be written in shell; systemd
actually has a nicer language for doing this, even though it's not
needed). However, the upstart approach makes it easier to preserve and
merge local changes with upstream changes. In the systemd model, the
local administrator has line-by-line granularity on overrides of systemd
unit configurations, which while solving much of the problem does not help
with the specific case of wanting to change the flags passed to the
daemon. If the package later changes the flags in some orthogonal way,
it's easy for the system to miss that change. This is something that,
under systemd, will probably require development of new tools to warn the
adminsitrator of what's happened. upstart avoids this problem by having
the whole configuration be managed as a conffile.

I think the upstart approach is better, but I think the drawbacks of the
systemd approach could be mostly overcome with some fairly simple Debian
tools.

2.5. Summary

I think the technical comparison between upstart and systemd as both
projects exist today substantially favors systemd, at both the feature and
design level. When picking between both products as they currently exist
on the basis of their current capabilities and future adaptibility, I have
no qualms about picking systemd.


3. Ecosystem and Portability

One of the primary concerns from the start of this conversation has been
around portability of any new init system. One advantage of the extreme
simplicity of sysvinit is that it's extremely portable; this advantage
continues to be shared by OpenRC. Both of the more-functional init
systems are Linux-specific. However, upstream attitudes towards
portability differ. This ties directly into the development models of
both systemd and upstart, the community momentum, and the larger
surrounding ecosystem.

3.1. Ecosystem Reality Check

One of the points that I think may have been obscured in the discussion,
but which is important to highlight, is that basically all parties have
agreed that Debian will adopt large portions of systemd. systemd is an
umbrella project that includes multiple components, some more significant
than others. Most of those components are clearly superior to anything we
have available now on Linux platforms and will be used in the distribution
going forward.

In other words, this debate is not actually about systemd vs. upstart in
the most obvious sense. Rather, the question, assuming one has narrowed
the choices to those two contenders, is between adopting all the major
components of systemd including the init system, or adopting most of the
major components of systemd but replacing the init system with upstart.
Either way, we'll be running udev, logind, some systemd D-Bus services,
and most likely timedated and possibly hostnamed for desktop environments.

I think this changes the nature of the discussion in some key ways. We're
not really talking about choosing between two competing ecosystems.
Rather, we're talking about whether or not to swap out a core component of
an existing integrated ecosystem with a component that we like better.

Now, I am generally on the side that says loose coupling of ecosystems is
an inherent good. However, I don't agree that it's such an inherent good
that we should disassemble things just for the sake of having disassembled
things. At feature parity, and absent any compelling reason to swap
components, I think we should take the path of least resistance and use
the integrations that other people have already developed. Debian has
more than enough hard integration problems to solve without creating new
ones for ourselves unnecessarily. But that's the key word: unnecessarily.
If we do have a reason for doing this, we should seriously consider it.

Therefore, I believe the burden of proof is on upstart to show that it is
a clearly superior init system along some axis, whether that be
functionality or portability or flexibility or maintainability, to warrant
going to the effort of disassembling a part of the systemd ecosystem and
swapping in our own component.

3.2. Portability

This is a difficult topic to clearly discuss, since it is, in essence, all
future speculation at this point.

I should state up front that, in making these sorts of decisions around
free software projects, I have a relatively high future discount rate. In
other words, I give substantially less credit to something that does not
exist now but could exist in the future. I don't discount it to zero, but
I do discount it relatively strongly. Others may not.

I do this because free software projects and volunteer projects are
inherently unpredictable. The free software world is stuffed to the gills
with roadmaps that never actually happened, through no fault of any of the
people involved. It's easy to agree that something would be a good idea,
and another matter to actually drive it through to completion.

Right now, neither systemd nor upstart work on non-Linux platforms.
Therefore, right now, adopting either of them means that we either
jettison our non-Linux ports or adopt a transition plan that retains
support for sysvinit scripts. Right now, there is minimal difference
between the two projects in terms of portability; they both make extensive
use of Linux-specific APIs and have hooks for Linux-specific actions.

However, there is a porting effort for upstart to kFreeBSD underway, and
the current upstart maintainers have indicated more interest in
portability than the systemd maintainers. That's been a point of
significant friction over systemd (and was, in the past, also a point of
friction with the previous upstart upstream, although that's subsequently
changed). So there is a real advantage for upstart here, but it's one
that has to be discounted because it's potential future work that could
happen, but which is certainly not guaranteed to happen.

Another point worth considering here is that the best way, from Debian's
perspective, of porting either project to kFreeBSD or the Hurd is to
implement the currently Linux-specific interfaces on those platforms in
some fashion. (An inotify and epoll API that uses kqueue under the hood,
for example.) To the extent that this is possible, it benefits both
upstart and systemd equally, as well as many other programs in the
archive that are written to currently Linux-specific APIs. This is an
approach that's been common for years in different porting scenarios; I
use it myself to maintain compatibility with both MIT Kerberos and Heimdal
in the Kerberos-related packages I maintain.

Finally, note the ecosystem point above. To maintain feature parity
across Debian's ports, there already appears to be widespread agreement
that components of systemd will have to be ported, particularly logind and
possibly some of the other services. Now, that's not quite the same thing
as porting the init system: it's possible those components use fewer
Linux-specific interfaces (I've not checked), it's possible that
alternative implementations of the same functionality can be provided
(which IIRC is what happened with udev in some fashion), and not being
able to run major desktop environments is not the same thing as not being
able to boot. But I do think it blunts some of the porting argument. The
non-Linux ports are going to have to port, fork, or replace systemd
components anyway, regardless of the choice of init system, or drop out of
feature parity with the Linux ports.

So, in short, I consider portability to be a possible benefit of upstart,
but I'm inclined to discount that advantage for several reasons. One,
it's not yet actually materialized and still may not, and two, systemd
porting looks like it's going to be on the table regardless. I therefore
think that we should deal with this issue through how we structure a
transition plan, rather than taking it as a reason to choose upstart over
systemd. More on that in another message.

3.3. Project Momentum

One of the reasons why I'm leery of the future portability argument for
upstart, and one of the reasons why I'm leery of upstart in general, is
that I'm quite worried upstart will prove to be a blind alley.

I think there are several reasons to be concerned here. None of them is
persuasive in isolation, but taken together I think they raise significant
cause for concern:

* Red Hat adopted upstart but never did a wholescale conversion, and then
abandoned upstart in favor of systemd. Obviously, one should not put
too much weight on this; Red Hat is a commercial company that has a
wealth of reasons for its actions that do not apply to Debian. But I
think it's still worth noting that the only non-Ubuntu major adopter of
upstart backed away from it.

* upstart is older than systemd but has significantly fewer features.
Now, the danger of this sort of metric is that features can be added as
"padding" without any real significance or advantage. But having spent
serious time with both systems, I don't believe that's the case here.
systemd is not adding extraneous features; rather, it's adding
significant, useful functionality and real-world adaptability, and
upstart is trailing despite being an older project.

* systemd has a broader community. SuSE and Red Hat are both converting,
there is significant interest across the general Linux community, major
upstreams of Debian such as GNOME and KDE are adopting systemd support
(and in some cases even requiring it), and systemd is tackling
significant problems, such as logind, that everyone agrees need to be
solved. By comparison, upstart is effectively used only by Ubuntu, and
there isn't the same sort of enthusiasm or attempts to tackle broad
problems happening at present in the upstart community so far as I can
see. This is reasonable if upstart is mature and mostly complete
software, but that was not my personal experience.

* There appears to be some direct tension between GNOME upstream and
upstart, not mostly due to upstart itself but because of corporate
direction at Canonical. Again, this can easily be overstated. But I do
think that Debian will want to continue to support GNOME going forward,
and doing that with upstart will clearly require more work within the
project than doing that with systemd. This is another case where we
shouldn't shy away from the work if it's necessary, but we also
shouldn't adopt unnecessary work.

Over the past few months, I've also put out some feelers to other
colleagues, and the uniform reaction I got in response is that systemd is
a better technical solution than upstart. I think this speaks to the
general momentum around systemd, and will directly affect our ease of
integration in the future. I know that after my personal experience with
both projects, I'm excited to add systemd support to my projects as
upstream, and not particularly enthused about upstart from an upstream
perspective since it doesn't offer me any clear benefits.

3.4. Summary

I'm concerned that, if we adopt upstart, in two or three years we'll end
up wanting to do the same thing that Red Hat did, back out, and switch to
systemd. That would be a huge amount of wasted effort. Even worse would
be to end up in that situation and decide that the conversion is too much
work, and then just settle for an init system that is harder to integrate
and provides less functionality.

I remain unconvinced of the long-term growth curve of the upstart project.
I don't think it's going to be abandoned completely, at least unless
Ubuntu decides to switch (which seems unlikely at the moment) or Canonical
dissolves (which also seems unlikely). I do think there's a significant
danger that it will stagnate and fall behind in terms of desired features,
particularly since this appears to already be happening. I don't have
faith in the path that takes upstart from where it is now to something
with feature parity with systemd as it is now, let alone something that's
clearly better than systemd. And I think Debian as a project should be
aiming for better, not merely sufficient.

The portability issues are significant. However, I don't think they
provide a clear advantage to upstart. It's possible that they will in the
future, at which point the ecosystem argument becomes much more difficult
and much narrower. But the fact remains that we'll be using large
components of systemd across the distribution anyway, which means that
swapping out the init system doesn't add as much portability as one might
hope, and increases our integration burden.

I think we should make wise decisions about which areas we want to invest
project effort. I dislike investing significant project effort in
catch-up efforts that, when complete, merely get us back to where we would
have been if we'd chosen a different solution. I don't think that's wise
stewardship of project resources. I want to see Debian focus its efforts
on places where we can make a real difference, where we can be leaders.
That means adopting the best-of-breed existing solutions and building on
top of them, not reinventing wheels and thereby starting from a trailing
position.


4. Conclusion

If I'm correct in my analysis of the community and ecosystem dynamics, I
think upstart needs to show that it is a significantly better technical
choice than systemd in order to warrant the additional project work that
will be required to build on top of upstart. Given feature parity, I
believe we should adopt systemd so that we can focus our efforts on
interesting new problems rather than on redoing integrations that other
people have already done.

My personal analysis did not show that upstart was significantly better
than systemd, or even at feature parity. Rather, I believe it is
currently trailing systemd substantially in multiple areas, some of which
will require significant design changes.

Given that, I believe systemd is the clear choice, despite the portability
issues that we will incur by choosing it. However, I think that means we
need to be very careful about how we handle a transition. I intend to
comment on that in a separate message (which will probably be tomorrow
given how long writing this message took).
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Tollef Fog Heen
2013-12-30 08:40:29 UTC
Permalink
]] Russ Allbery

First, thanks to both you and Ian for the quite comprehensive
write-ups.
If the package later changes the flags in some orthogonal way, it's
easy for the system to miss that change. This is something that,
under systemd, will probably require development of new tools to warn
the adminsitrator of what's happened. upstart avoids this problem by
having the whole configuration be managed as a conffile.
You might want to have a look at systemd-delta. On my quite boring home
machine:

: ***@xoog ~ > systemd-delta
[MASKED] /etc/udev/rules.d/75-persistent-net-generator.rules → /lib/udev/rules.d/75-persistent-net-generator.rules

1 overridden configuration files found.
: ***@xoog ~ >

It can also output diffs.
I think the upstart approach is better, but I think the drawbacks of the
systemd approach could be mostly overcome with some fairly simple Debian
tools.
We were initially considering using ucf and checking if the file in /etc
had changed (before we had per-line overrides and such), but with the
more complex overrides now available, I think systemd-delta is a better
solution. I guess we could integrate that into the packaging somehow
and present a useful UI if you've overridden a line that changes.

[...]
But I do think it blunts some of the porting argument. The non-Linux
ports are going to have to port, fork, or replace systemd components
anyway, regardless of the choice of init system, or drop out of
feature parity with the Linux ports.
It's not like we have feature parity today. Hurd has the whole
translators setup. kFreeBSD has jails and ZFS. Nobody is arguing that
we must ship those with Linux too, so why should the feature parity have
to go in the other direction? Personally, I think the more interesting
use case for kFreeBSD is on the server side, and not as a GNOME
workstation. I also realise a file system is not on the same magnitude
for a distribution as an entire desktop environment, but we're looking
at degrees here anyway, not a black and white picture.
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Russ Allbery
2013-12-30 16:51:13 UTC
Permalink
Post by Tollef Fog Heen
]] Russ Allbery
Post by Russ Allbery
I think the upstart approach is better, but I think the drawbacks of
the systemd approach could be mostly overcome with some fairly simple
Debian tools.
We were initially considering using ucf and checking if the file in /etc
had changed (before we had per-line overrides and such), but with the
more complex overrides now available, I think systemd-delta is a better
solution. I guess we could integrate that into the packaging somehow
and present a useful UI if you've overridden a line that changes.
Right, several people have pointed me at systemd-delta, but your last
sentence was the tool that I was driving at. systemd-delta tells you
whether you're currently overriding something, which is a useful building
block, but which doesn't restore conffile upgrade handling. What's needed
to keep existing Debian behavior is a tool that tells you whether the
maintainer and the local system administrator have changed the
configuration in conflicting ways so that something is now being
overridden that was not being overridden before.

In other words, systemd-delta does a two-way diff, but what's needed here
is a three-way diff plus integration into the packaging system so that, as
with conffile changes now, the upgrade (by default) stops and prompts the
local systems administrator with the orthogonal differences and lets them
choose how to handle the situation.

I don't think this would be particularly difficult to implement on top of
systemd-delta, although the details may be tricky to get right and may
require a few iterations.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Russ Allbery
2013-12-30 17:05:57 UTC
Permalink
Post by Russ Allbery
* Red Hat adopted upstart but never did a wholescale conversion, and then
abandoned upstart in favor of systemd. Obviously, one should not put
too much weight on this; Red Hat is a commercial company that has a
wealth of reasons for its actions that do not apply to Debian. But I
think it's still worth noting that the only non-Ubuntu major adopter of
upstart backed away from it.
[...]
Post by Russ Allbery
By comparison, upstart is effectively used only by Ubuntu, [...]
Both of these statements are incorrect.

I'm sure that somewhere in the many vast threads that we've had over the
init system, someone pointed out to me that Google's Chromium OS and
Chrome OS use upstart. Now that I've been reminded of this, I think Steve
also mentioned it in the discussion of how upstart's upstream and
packaging development were handled. However, it had completely dropped
out of my mind and I was unaware of it while writing the above statements.

I apologize for the misrepresentation.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
gregor herrmann
2013-12-30 18:09:21 UTC
Permalink
Post by Russ Allbery
Post by Russ Allbery
By comparison, upstart is effectively used only by Ubuntu, [...]
Both of these statements are incorrect.
I'm sure that somewhere in the many vast threads that we've had over the
init system, someone pointed out to me that Google's Chromium OS and
Chrome OS use upstart.
Maemo5 also uses upstart (a quite old version but after all, almost
everything there is quite old).

Cheers,
gregor
--
.''`. Homepage: http://info.comodo.priv.at/ - OpenPGP key 0xBB3A68018649AA06
: :' : Debian GNU/Linux user, admin, and developer - http://www.debian.org/
`. `' Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
`- NP: Supertramp: Ain't Nobody But Me
Ian Jackson
2013-12-30 18:58:37 UTC
Permalink
Post by Russ Allbery
We seem to be at the point of the process where at least those of us who
did early investigation are stating conclusions. I think I have enough
information to state mine, so will attempt to do so here.
Thanks.
Post by Russ Allbery
First, other choices besides systemd and upstart.
I agree with your comments here; it appears you've investigated OpenRC
in more detail than I have but I'm happy to take your word for it.
Post by Russ Allbery
2. Core Service Management Functionality
...
Post by Russ Allbery
To my surprise, that's not what happened. Rather, I concluded that
systemd has a substantial technical advantage over upstart, primarily in
terms of available useful features, but also in terms of fundamental
design.
I think we have fundamentally different ideas about what the
replacement init ought to be like. I don't think either of us will
convince the other.
Post by Russ Allbery
3. Ecosystem and Portability
...
Post by Russ Allbery
3.1. Ecosystem Reality Check
...
Post by Russ Allbery
Rather, we're talking about whether or not to swap out a core component of
an existing integrated ecosystem with a component that we like better.
Unless you are proposing to make systemd mandatory for all Debian
installations, this is work that needs to be done anyway.

Also, I get the impression me that the "integration" of much of this
functionality into the systemd source package has been done for
political rather than technical reasons. Indeed to the extent that
there is a problematically tight technical coupling between these
components, I find it difficult to avoid doubting the good faith of
the people making those decisions. At the very least, I worry that
the systemd project as a whole is far too willing to impose decisions
of all kinds on its downstreams.

Under those kind of circumstances I am willing for the Debian project
as a whole to go to quite some effort (and, indeed, impose some effort
even on the maintainers of systemd in Debian) to retain the
flexibility that I think is important. That flexibility certainly
includes the ability for a user to drop use of parts of systemd that
they don't find desirable.
Post by Russ Allbery
Now, I am generally on the side that says loose coupling of ecosystems is
an inherent good. However, I don't agree that it's such an inherent good
that we should disassemble things just for the sake of having disassembled
things. At feature parity, and absent any compelling reason to swap
components, I think we should take the path of least resistance and use
the integrations that other people have already developed. Debian has
more than enough hard integration problems to solve without creating new
ones for ourselves unnecessarily. But that's the key word: unnecessarily.
If we do have a reason for doing this, we should seriously consider it.
If these problems have been created with reckless disregard for the
flexibility needs of downstreams and users, then I take the opposite
view.
Post by Russ Allbery
3.2. Portability
...
Post by Russ Allbery
So, in short, I consider portability to be a possible benefit of upstart,
but I'm inclined to discount that advantage for several reasons. One,
it's not yet actually materialized and still may not,
My understanding was that Dimitri had got upstart running on BSD.
I can't imagine that this problem won't be solved (in Debian) for
jessie. We're not talking about some nonexistent hypothetical
patchset.
Post by Russ Allbery
3.3. Project Momentum
I don't see the outlook here the same way as you do.

Furthermore, I am much less worried about Debian going it alone
(although, of course, it's not alone) than you seem to be. We have
historically been entirely unafraid to do our own better things, even
if it is more work and takes us longer.

I felt that way when Debian was very much a minority interest. That's
far from the case nowadays; I've heard it asserted that Debian-based
distros now account for a majority of traditional distro installs. I
guess numbers on that are all speculative but it is certainly true
that we have a thriving ecosystem of our own.

We have got where we are by doing things the way we think is best, not
by simply following the herd.

Of course you say that you prefer systemd, which still leads you to
the opposite conclusion. But I wanted to explicitly deal with this
argument.

Ian.
Russ Allbery
2013-12-30 19:56:33 UTC
Permalink
Post by Ian Jackson
Post by Russ Allbery
First, other choices besides systemd and upstart.
I agree with your comments here; it appears you've investigated OpenRC
in more detail than I have but I'm happy to take your word for it.
I should be clear that I've not actually run the system. I've tried to
track down documentation and tried to understand what the project goals
were. It's quite possible that I've gotten some details wrong, and I
would welcome any corrections from the OpenRC folks.
Post by Ian Jackson
Post by Russ Allbery
Rather, we're talking about whether or not to swap out a core component
of an existing integrated ecosystem with a component that we like
better.
Unless you are proposing to make systemd mandatory for all Debian
installations, this is work that needs to be done anyway.
What I'm hearing from both GNOME and KDE maintainers is that assuming
systemd would save them a great deal of work. I think having systemd be
the only supported and tested option for at least some large package sets
that heavily use the systemd infrastructure upstream is a not-unlikely
long-term outcome.

In other words, I think the long-term portability future of GNOME (and to
a much lesser extent KDE, where the issue will be bitrot of unused
configurations more than an intentional maintenance choice, and where I
expect the system to at worst degrade functionality rather than stop
working) to non-systemd platforms is already in some jeopardy, because
it's not clear that resources will be available upstream to maintain those
projects outside of the APIs that systemd supports. That was, after all,
the forcing moment that brought this whole debate to a head.

One can, of course, have a wide variety of reactions to that. As someone
who considers portability to be an inherent good, I find it sad, although
I don't have a full appreciation for what problems GNOME is trying to
solve by increasing its reliance on systemd. But I'm highly dubious that
Debian will just single-handedly fix that, or that we would drop GNOME
because we don't like upstream's integration decisions.
Post by Ian Jackson
Also, I get the impression me that the "integration" of much of this
functionality into the systemd source package has been done for
political rather than technical reasons.
I hear that you have this impression, but I completely disagree. I find
the amount of bad will and assumption of malice aimed at the systemd
maintainers to be intensely depressing and unwarranted by any of the
interactions that I've seen, and I've been doing a fair bit of reading.

Lennart passionately argues his side. So do you. So do a lot of us. We
all tend to have strong technical opinions and a great deal of confidence
in our own judgement.

Personally, I'm putting contentions that systemd is doing a power grab or
is merging things for political reasons into the same bucket as
contentions that Technical Committee opinions are driven by current or
past employer relationships. I would prefer to assume good faith among
the people involved and understand their projects on their own terms.
Post by Ian Jackson
At the very least, I worry that the systemd project as a whole is far
too willing to impose decisions of all kinds on its downstreams.
All upstreams do this. I have yet to meet an upstream for any piece of
software that doesn't care about that software working uniformly on its
supported platforms. systemd is entirely in line with normal upstream
practice of trying to pick the best solution to a given problem and
supporting it thoroughly, rather than incurring the maintenance burden of
supporting multiple ways to do the same thing.

This always involves imposing some decisions on downstreams unless
downstreams are willing to fork around that decision. It's already meant
that some of Debian's decisions are being imposed on Red Hat because they
were judged to be better solutions.

In places where Debian needs to take a different approach for reasons of
backward compatibility or existing integrations, obviously we should, and
we should be sure that we have the flexibility to do so. For example, I
think we should disable the current systemd binfmt support in favor of our
existing working solution. I have not seen any indication that would be a
problem.

You made multiple proposals that do not reflect what Debian is doing
*today*, and which are not *necessary* for Debian. I don't think those
are good test cases for upstream's willingness to accomodate Debian's
needs.
Post by Ian Jackson
My understanding was that Dimitri had got upstart running on BSD.
The latest that I have seen on this porting effort is here:

http://blog.surgut.co.uk/2013/11/libnih-upstart-dependency-ported-to.html

I asked previously on this bug if someone had later news. Do you have
more information than that?

The above is definitely not "upstart running on BSD." That's upstart's
underlying support library mostly working on BSD after disabling two
features (that are required for upstart). I haven't not heard whether the
porting of upstart proper has started. That will certainly be much easier
once libnih is ported, but there are, for example, direct uses of epoll in
upstart proper. (I don't know if kFreeBSD already has a portability layer
for epoll in terms of kqueue.)
Post by Ian Jackson
Furthermore, I am much less worried about Debian going it alone
(although, of course, it's not alone) than you seem to be. We have
historically been entirely unafraid to do our own better things, even if
it is more work and takes us longer.
I think "worried" is an incorrect characterization of what I said. What I
said, rather, was:

I think we should make wise decisions about which areas we want to
invest project effort. I dislike investing significant project effort
in catch-up efforts that, when complete, merely get us back to where
we would have been if we'd chosen a different solution. I don't think
that's wise stewardship of project resources. I want to see Debian
focus its efforts on places where we can make a real difference, where
we can be leaders. That means adopting the best-of-breed existing
solutions and building on top of them, not reinventing wheels and
thereby starting from a trailing position.

upstart is a great project, of which its maintainers are deservedly proud.
However, I don't agree that it's better than systemd. My own research
convinced me of the opposite. But putting that aside, my point in that
section is that, given feature parity (and I mean "feature" expansively,
including adaptibility to Debian's needs), we should choose systemd
because it has more project momentum and better existing integration,
which means that we will have to spend less energy on it and will have
more energy to spend on other things.

It's absolutely worth doing our own, better things. What I don't want to
do is our own *worse* things. Or, for that matter, our own equivalent
things, when we could instead use work that already exists. It's a waste
of energy.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
cameron
2013-12-30 21:12:50 UTC
Permalink
Post by Russ Allbery
http://blog.surgut.co.uk/2013/11/libnih-upstart-dependency-ported-to.html
I asked previously on this bug if someone had later news. Do you have
more information than that?
The above is definitely not "upstart running on BSD." That's
upstart's
underlying support library mostly working on BSD after disabling two
features (that are required for upstart).
I know inotify is working on kFreeBSD with the libinotify-kqueue
software (recently packaged in Debian).
https://github.com/dmatveev/libinotify-kqueue

That leaves only abstract domain name sockets left, which should not be
too complicated.

Upstart still does not run on kFreeBSD, though.
Steve Langasek
2013-12-30 23:30:51 UTC
Permalink
Post by Russ Allbery
Post by Ian Jackson
Post by Russ Allbery
Rather, we're talking about whether or not to swap out a core component
of an existing integrated ecosystem with a component that we like
better.
Unless you are proposing to make systemd mandatory for all Debian
installations, this is work that needs to be done anyway.
What I'm hearing from both GNOME and KDE maintainers is that assuming
systemd would save them a great deal of work. I think having systemd be
the only supported and tested option for at least some large package sets
that heavily use the systemd infrastructure upstream is a not-unlikely
long-term outcome.
It's clear that being able to assume availability of certain dbus services
is labor-saving for GNOME and KDE upstreams, and I see no reason for them
not to standardize on such a requirement assuming the dbus services have
sane APIs (which they do).

What I object to is referring to this as "assuming systemd". They are
systemd APIs, sure, but with one exception the systemd implementations are
not presently tied to the use of systemd as init, and there is an alternate
implementation of that one exception (systemd-shim).

From comments made by various GNOME upstream developers on this, I think
they are being suitably cautious about avoiding scope creep where the
systemd dependencies are concerned. So in what sense are the GNOME and KDE
requirements not already being met on top of upstart? The only outstanding
issue I see is with non-Linux ports, because logind is not portable; that's
definitely a problem for running GNOME on kfreebsd, but it's actually a
rather narrowly-scoped porting problem and one that the ports need to deal
with one way or another if they want GNOME to continue working.

I'm sure there are GNOME upstream contributors who would be happy to see
GNOME become systemd-only. But I think their motivations in letting the
lines be blurred in such a way are purely political, not technical; and I
give GNOME upstream as a whole the benefit of the doubt that they would not
so weaken their project to suit someone's political agenda.
Post by Russ Allbery
One can, of course, have a wide variety of reactions to that. As someone
who considers portability to be an inherent good, I find it sad, although
I don't have a full appreciation for what problems GNOME is trying to
solve by increasing its reliance on systemd. But I'm highly dubious that
Debian will just single-handedly fix that, or that we would drop GNOME
because we don't like upstream's integration decisions.
I find the notion that Debian doing this "single-handedly" is an obstacle to
be a bizarre one. Debian has more than one pair of hands, it's a veritable
multi-tentacled beast. Have we so atrophied as a community that we'll wail
and gnash our teeth about a non-portable dependency, but have no porters
willing to actually provide a native implementation of a (quite small) dbus
API?

You've said that you think the portability question doesn't weight strongly
in favor of either upstart or systemd, because neither port is done yet.
But the amount of work that would be involved in porting systemd to kfreebsd
is an order of magnitude greater than the work involved in porting upstart.
logind is just one small component of systemd, which someone could provide
an API-compatible implementation for without having to contend with the
question of cgroups on non-Linux kernels. If even that is out of reach for
the Debian porters, how could we ever expect a full BSD port of systemd to
materialize?

I think the reality is that adopting systemd as default init for Debian is
nothing short of a death sentence for the non-Linux ports. The move to a
different init will have a snowball effect in the distribution, as packages
stop being tested with sysvinit and popular support grows for dropping
compatibility with sysvinit altogether. This is not a problem if the ports
are in a position to come along on this transition with a moderate
investment in porting init. But the porting required for systemd as a whole
is far from moderate, and I believe that faced with such a requirement the
non-Linux ports would wither and die. I know there are Debian developers
(including many in the GNOME/systemd "camp") who think this is a desirable
outcome. I have no personal attachment to the non-Linux ports, but I do
think that as an existing part of our Debian community, the TC should at
least give these ports a fighting chance, because this kind of competition
is healthy.
Post by Russ Allbery
Post by Ian Jackson
My understanding was that Dimitri had got upstart running on BSD.
http://blog.surgut.co.uk/2013/11/libnih-upstart-dependency-ported-to.html
I asked previously on this bug if someone had later news. Do you have
more information than that?
The above is definitely not "upstart running on BSD." That's upstart's
underlying support library mostly working on BSD after disabling two
features (that are required for upstart). I haven't not heard whether the
porting of upstart proper has started. That will certainly be much easier
once libnih is ported, but there are, for example, direct uses of epoll in
upstart proper. (I don't know if kFreeBSD already has a portability layer
for epoll in terms of kqueue.)
AIUI, upstart itself built out of the box once libnih was available, because
all the portability issues are encapsulated in libnih. (The single use of
epoll in the upstart source is in the upstart-socket-bridge, which is an
optional component from upstart's perspective and certainly not needed for
booting a Debian base system - so presumably Dimitri just built with this
bridge disabled.)

With libinotify-kqueue (that Dimitri maintains), kfreebsd 10 (in
experimental), eglibc 2.18 (also in experimental), and Dimitri's branch of
libnih, it seems that all the portability requirements for upstart are met,
except for abstract socket support. There are evidently still some porting
problems that aren't detected at build time, because upstart doesn't yet
boot on kfreebsd. But all things considered, the port is rather far along.
Post by Russ Allbery
upstart is a great project, of which its maintainers are deservedly proud.
However, I don't agree that it's better than systemd. My own research
convinced me of the opposite. But putting that aside, my point in that
section is that, given feature parity (and I mean "feature" expansively,
including adaptibility to Debian's needs), we should choose systemd
because it has more project momentum and better existing integration,
which means that we will have to spend less energy on it and will have
more energy to spend on other things.
It's absolutely worth doing our own, better things. What I don't want to
do is our own *worse* things. Or, for that matter, our own equivalent
things, when we could instead use work that already exists. It's a waste
of energy.
I think this downplays the significance of the integration work that has
already been done in Ubuntu on top of upstart, that Debian will be able to
adopt as-is (or with whatever bugfixes the maintainers find along the way).
My look at Fedora 20 confirmed my belief that the integration necessary to
have a robust systemd boot across all the configurations that Debian
supports has not actually been done yet. systemd upstream advocates
shipping systemd units upstream where they can be consumed unmodified by
distributions, but in practice Debian is going to be on the hook for
debugging the very long tail of integration problems. It's hard to gauge
whether the energy saved by not bringing upstart up to feature parity
outweighs the energy spent on bringing systemd integration up to snuff, but
I definitely don't think there's a clear point here in systemd's favor.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org
Russ Allbery
2013-12-31 00:04:05 UTC
Permalink
Post by Steve Langasek
From comments made by various GNOME upstream developers on this, I think
they are being suitably cautious about avoiding scope creep where the
systemd dependencies are concerned. So in what sense are the GNOME and
KDE requirements not already being met on top of upstart? The only
outstanding issue I see is with non-Linux ports, because logind is not
portable; that's definitely a problem for running GNOME on kfreebsd, but
it's actually a rather narrowly-scoped porting problem and one that the
ports need to deal with one way or another if they want GNOME to
continue working.
I don't see what you're saying here as substantively different than what I
said in my writeup about the ecosystem. I feel like we're both presenting
the same facts through different lenses.
Post by Steve Langasek
Post by Russ Allbery
One can, of course, have a wide variety of reactions to that. As
someone who considers portability to be an inherent good, I find it
sad, although I don't have a full appreciation for what problems GNOME
is trying to solve by increasing its reliance on systemd. But I'm
highly dubious that Debian will just single-handedly fix that, or that
we would drop GNOME because we don't like upstream's integration
decisions.
I find the notion that Debian doing this "single-handedly" is an
obstacle to be a bizarre one. Debian has more than one pair of hands,
it's a veritable multi-tentacled beast. Have we so atrophied as a
community that we'll wail and gnash our teeth about a non-portable
dependency, but have no porters willing to actually provide a native
implementation of a (quite small) dbus API?
Please recall the context here: this whole aside started with an objection
to my contention that adopting upstart requires disassembly and redoing of
an integration that we would otherwise not have to disassemble. Nowhere
in my message did I say that we would or could not do that disassembly if
we adopted upstart; I said that it was work that we otherwise wouldn't
have to do.

That's the intended context of my point above: I don't think we're going
to port GNOME to a non-systemd infrastructure, in the sense of carrying
significant patches to GNOME to adopt it to, say, not using logind. I
think GNOME will continue to use systemd APIs heavily, which makes GNOME
less portable. That means that systems that are not capable of running
those systemd components will need to either port them or develop
alternatives.

I don't consider this wailing or gnashing of teeth, but rather a realistic
look at what efforts the project is talking about committing to, as
opposed to supporting people working on if they so choose.
Post by Steve Langasek
You've said that you think the portability question doesn't weight
strongly in favor of either upstart or systemd, because neither port is
done yet. But the amount of work that would be involved in porting
systemd to kfreebsd is an order of magnitude greater than the work
involved in porting upstart.
I haven't investigated this myself, but that matches the impressions I've
gotten from discussion both here and in Lennart's blog. Nothing I'm
writing here is intended as disagreement with this point. Porting systemd
looks harder.
Post by Steve Langasek
logind is just one small component of systemd, which someone could
provide an API-compatible implementation for without having to contend
with the question of cgroups on non-Linux kernels.
Yes, as I said explicitly in my writeup, it may well be the case that
porting some of the components of systemd will be substantially easier
than porting the init system.
Post by Steve Langasek
If even that is out of reach for the Debian porters, how could we ever
expect a full BSD port of systemd to materialize?
I never said that a port of logind was out of reach for Debian porters.
I'm saying that there's a lot of work here, and we don't yet know whether
or how any of it is going to happen. We should not close off possible
future directions unnecesarily, but we need to make a decision based on
the current state without assuming that roadmaps will necessarily come to
fruition.

*If* Debian adopts systemd as an init system, I do think that it's
possible (how likely, I don't know) we'll end up with GNOME unavailable on
the kFreeBSD port because people choose not to do the effort of separating
out and porting the components required. It's up to the porters, and
depends on what role they see for the port. If, for example, they're
primarily targeting servers, this may not be a significant loss. The
point is that it would be their call.

If Debian adopts upstart, obviously we're going to be required to do the
work of separating the init system from the rest of systemd, which will
make some of that effort possibly easier for the kFreeBSD porters
(although that work will not magically port logind to kFreeBSD, or adjust
for any future requirements for D-Bus that might materialize, etc.).

Either way, real effort will be required to get GNOME working on
kFreeBSD. I'm not saying that effort won't happen. I'm saying that we
have choices about what effort we're going to *commit* to, as opposed to
leaving to the discretion of the porters, and that I am dubious of this
argument that going with upstart as the init system makes that work
substantially easier.
Post by Steve Langasek
I think the reality is that adopting systemd as default init for Debian
is nothing short of a death sentence for the non-Linux ports. The move
to a different init will have a snowball effect in the distribution, as
packages stop being tested with sysvinit and popular support grows for
dropping compatibility with sysvinit altogether. This is not a problem
if the ports are in a position to come along on this transition with a
moderate investment in porting init. But the porting required for
systemd as a whole is far from moderate, and I believe that faced with
such a requirement the non-Linux ports would wither and die.
This is definitely something that might happen. It's a risk of the
approach that I outline. I'm less pessmistic than you are about it, but
this is definitely a risk.

My belief, and again I welcome concrete reasons why I'm not correct, is
that adopting upstart poses a similar risk for the Hurd port as adopting
systemd, and I care just as much about the Hurd port as kFreeBSD. And
while kFreeBSD is in better shape due to the already-begun upstart port,
there are still significant porting challenges in the way of maintaining
feature parity in the kFreeBSD port at the level that it has today. Many
of those challenges are going to remain regardless of which init system we
pick.
Post by Steve Langasek
I have no personal attachment to the non-Linux ports, but I do think
that as an existing part of our Debian community, the TC should at least
give these ports a fighting chance, because this kind of competition is
healthy.
Well, I won't get into how much I loathe the word competition in this
context, but as you can see from my other message this morning, I'm in
wholehearted agreement with giving the ports a fighting chance.
Post by Steve Langasek
AIUI, upstart itself built out of the box once libnih was available,
because all the portability issues are encapsulated in libnih. (The
single use of epoll in the upstart source is in the
upstart-socket-bridge, which is an optional component from upstart's
perspective and certainly not needed for booting a Debian base system -
so presumably Dimitri just built with this bridge disabled.)
With libinotify-kqueue (that Dimitri maintains), kfreebsd 10 (in
experimental), eglibc 2.18 (also in experimental), and Dimitri's branch
of libnih, it seems that all the portability requirements for upstart
are met, except for abstract socket support. There are evidently still
some porting problems that aren't detected at build time, because
upstart doesn't yet boot on kfreebsd. But all things considered, the
port is rather far along.
Thank you for the update! It sounds like it's farther along than the
previous information I had, although not as far along as Ian had thought.

I want to mention again something that you'd dismissed as possibly a joke
earlier, but which I was actually serious about: I think a world in which
we use systemd on Linux and upstart on non-Linux ports, should upstart
prove much more portable, actually makes sense. As I said in my other
writeup, I believe the systemd feature advantage is sufficient to choose
systemd in isolation, without the other issues discussed. I also believe
that maintaining a systemd unit plus an upstart configuration is, modulo
testing (which I realize is a huge issue), much easier than maintaining a
single sysvinit script.

I do want to reiterate here, though, that my position on transitions,
multiple init system support, and so forth are exactly the same if upstart
works on kFreeBSD but not on Hurd as if it works on both of them. I
consider them to have equal place here. (However, demonstrated
portability to kFreeBSD may -- to what extent I don't know -- indicate
that the port to Hurd will be relatively easy.)
Post by Steve Langasek
I think this downplays the significance of the integration work that has
already been done in Ubuntu on top of upstart, that Debian will be able
to adopt as-is (or with whatever bugfixes the maintainers find along the
way). My look at Fedora 20 confirmed my belief that the integration
necessary to have a robust systemd boot across all the configurations
that Debian supports has not actually been done yet. systemd upstream
advocates shipping systemd units upstream where they can be consumed
unmodified by distributions, but in practice Debian is going to be on
the hook for debugging the very long tail of integration problems. It's
hard to gauge whether the energy saved by not bringing upstart up to
feature parity outweighs the energy spent on bringing systemd
integration up to snuff, but I definitely don't think there's a clear
point here in systemd's favor.
I believe the level of features that systemd brings to the table is
substantially beyond what's available from upstart at the current time,
and am confident that, even including the level of effort required to
integrate systemd without the work that Ubuntu has done for upstart, we
will end up ahead on the Linux ports. My defense of that position is
basically the sum total of my previous two messages, so I won't repeat it,
just note that you have correctly identified a core disagreement between
the two of us.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Colin Watson
2013-12-31 03:17:39 UTC
Permalink
Post by Russ Allbery
My belief, and again I welcome concrete reasons why I'm not correct, is
that adopting upstart poses a similar risk for the Hurd port as adopting
systemd, and I care just as much about the Hurd port as kFreeBSD. And
while kFreeBSD is in better shape due to the already-begun upstart port,
there are still significant porting challenges in the way of maintaining
feature parity in the kFreeBSD port at the level that it has today. Many
of those challenges are going to remain regardless of which init system we
pick.
I haven't looked at this in much detail, and I suspect Dimitri hasn't
yet although IIRC he did express some interest in doing so. But I
haven't seen anyone else try to outline the scope of a port, so let me
try to do so for the sake of general understanding. As far as I know,
the hardest parts would be inotify, ptrace, and prctl
(PR_SET_CHILD_SUBREAPER).

inotify is used to notice changes to configuration files. This is
certainly helpful for users, but it isn't critical as "initctl
reload-configuration" works without it. We could probably do without
this with the aid of a dpkg trigger.

ptrace is used for "expect fork" and "expect daemon"; as I indicated in
another post, I think it would be preferable to avoid these in Debian
and quite possibly to compile them out. (This would mean we wouldn't be
able to translate Ubuntu jobs quite as directly, and a number of
important jobs would definitely need to be changed, but the conversion
isn't usually particularly difficult.)

prctl (PR_SET_CHILD_SUBREAPER) is used to make SIGCHLD notification work
properly when Upstart is supervising a user session. This isn't a
required feature and could easily be compiled out until suitable kernel
support is available (this actually seems like the sort of thing that
could be done in the Hurd without too much difficulty, but I haven't
looked into it). If absent, it might well impede the ability to do an
advanced desktop port, but it wouldn't get in the way of porting the
bulk of services.

There might also be odds and ends around the details of wait status
handling.

So, I'll certainly concede that I haven't actually tried this, but from
what I know of Upstart/libnih's system dependencies, I think that the
Hurd could be accommodated with at worst possibly some loss of
non-critical features.
Post by Russ Allbery
I want to mention again something that you'd dismissed as possibly a joke
earlier, but which I was actually serious about: I think a world in which
we use systemd on Linux and upstart on non-Linux ports, should upstart
prove much more portable, actually makes sense. As I said in my other
writeup, I believe the systemd feature advantage is sufficient to choose
systemd in isolation, without the other issues discussed. I also believe
that maintaining a systemd unit plus an upstart configuration is, modulo
testing (which I realize is a huge issue), much easier than maintaining a
single sysvinit script.
I wouldn't discard this option out of hand, but I think it would need
significant tool support to be practical (e.g. heuristic generation of
Upstart job files from systemd units), unless we expect all service
maintainers to have kFreeBSD/Hurd VMs lying around. Keeping the
sysvinit scripts and using Upstart as the init daemon is another
possibility, and at least in that case the sysvinit scripts are probably
still lying around. We don't even necessarily have to choose between
those up-front.
--
Colin Watson [***@debian.org]
cameron
2013-12-31 04:35:16 UTC
Permalink
Post by Colin Watson
Post by Russ Allbery
My belief, and again I welcome concrete reasons why I'm not
correct, is
that adopting upstart poses a similar risk for the Hurd port as adopting
systemd, and I care just as much about the Hurd port as kFreeBSD.
And
while kFreeBSD is in better shape due to the already-begun upstart port,
there are still significant porting challenges in the way of
maintaining
feature parity in the kFreeBSD port at the level that it has today.
Many
of those challenges are going to remain regardless of which init system we
pick.
I haven't looked at this in much detail, and I suspect Dimitri hasn't
yet although IIRC he did express some interest in doing so. But I
haven't seen anyone else try to outline the scope of a port, so let me
try to do so for the sake of general understanding. As far as I know,
the hardest parts would be inotify, ptrace, and prctl
(PR_SET_CHILD_SUBREAPER).
inotify is used to notice changes to configuration files. This is
certainly helpful for users, but it isn't critical as "initctl
reload-configuration" works without it. We could probably do without
this with the aid of a dpkg trigger.
inotify use can easily be ported to kqueue within Upstart, or
libinotify-kqueue can be used.
Post by Colin Watson
ptrace is used for "expect fork" and "expect daemon"; as I indicated in
another post, I think it would be preferable to avoid these in Debian
and quite possibly to compile them out. (This would mean we wouldn't be
able to translate Ubuntu jobs quite as directly, and a number of
important jobs would definitely need to be changed, but the conversion
isn't usually particularly difficult.)
If we are able to settle on a readiness protocol and fully (or almost
fully) implement it, then ptrace will become irrelevant. I am sure that
if upstream is in agreement on the proto, then Ubuntu and Debian can
collaborate to update the jobs (and daemons) for their next releases.
Post by Colin Watson
prctl (PR_SET_CHILD_SUBREAPER) is used to make SIGCHLD notification work
properly when Upstart is supervising a user session. This isn't a
required feature and could easily be compiled out until suitable kernel
support is available (this actually seems like the sort of thing that
could be done in the Hurd without too much difficulty, but I haven't
looked into it). If absent, it might well impede the ability to do an
advanced desktop port, but it wouldn't get in the way of porting the
bulk of services.
Unity, likely the only desktop environment using Upstart as a session
init, is not in Debian. The sacrifice of this functionality on
non-linux systems is perfectly acceptable.
Post by Colin Watson
There might also be odds and ends around the details of wait status
handling.
Two things come to mind: use of epoll in upstart-socket-bridge, and no
abstract namespace sockets.
Post by Colin Watson
So, I'll certainly concede that I haven't actually tried this, but from
what I know of Upstart/libnih's system dependencies, I think that the
Hurd could be accommodated with at worst possibly some loss of
non-critical features.
Post by Russ Allbery
I want to mention again something that you'd dismissed as possibly a joke
earlier, but which I was actually serious about: I think a world in which
we use systemd on Linux and upstart on non-Linux ports, should upstart
prove much more portable, actually makes sense. As I said in my other
writeup, I believe the systemd feature advantage is sufficient to choose
systemd in isolation, without the other issues discussed. I also believe
that maintaining a systemd unit plus an upstart configuration is, modulo
testing (which I realize is a huge issue), much easier than
maintaining a
single sysvinit script.
I wouldn't discard this option out of hand, but I think it would need
significant tool support to be practical (e.g. heuristic generation of
Upstart job files from systemd units), unless we expect all service
maintainers to have kFreeBSD/Hurd VMs lying around. Keeping the
sysvinit scripts and using Upstart as the init daemon is another
possibility, and at least in that case the sysvinit scripts are probably
still lying around. We don't even necessarily have to choose between
those up-front.
Cheers,
Cameron Norman
Colin Watson
2014-01-01 12:56:39 UTC
Permalink
Post by cameron
Post by Colin Watson
inotify is used to notice changes to configuration files. This is
certainly helpful for users, but it isn't critical as "initctl
reload-configuration" works without it. We could probably do without
this with the aid of a dpkg trigger.
inotify use can easily be ported to kqueue within Upstart, or
libinotify-kqueue can be used.
Note that I'm talking about the Hurd here. kqueue is a kFreeBSD thing,
as far as I know. (Compare e.g.
https://lists.debian.org/debian-hurd/2013/10/msg00021.html)
Post by cameron
Post by Colin Watson
prctl (PR_SET_CHILD_SUBREAPER) is used to make SIGCHLD notification
work properly when Upstart is supervising a user session. This isn't
a required feature and could easily be compiled out until suitable
kernel support is available (this actually seems like the sort of
thing that could be done in the Hurd without too much difficulty, but
I haven't looked into it). If absent, it might well impede the
ability to do an advanced desktop port, but it wouldn't get in the
way of porting the bulk of services.
Unity, likely the only desktop environment using Upstart as a
session init, is not in Debian. The sacrifice of this functionality
on non-linux systems is perfectly acceptable.
While this is true today, we need to look to the future. Using Upstart
as a session init is not conceptually tied to Unity in any way, and I
expect that other desktop environments will want to use more advanced
session supervision soon enough.
--
Colin Watson [***@debian.org]
Cameron Norman
2014-01-01 17:38:05 UTC
Permalink
Post by Colin Watson
Post by cameron
Post by Colin Watson
inotify is used to notice changes to configuration files. This is
certainly helpful for users, but it isn't critical as "initctl
reload-configuration" works without it. We could probably do without
this with the aid of a dpkg trigger.
inotify use can easily be ported to kqueue within Upstart, or
libinotify-kqueue can be used.
Note that I'm talking about the Hurd here. kqueue is a kFreeBSD thing,
as far as I know. (Compare e.g.
https://lists.debian.org/debian-hurd/2013/10/msg00021.html)
Post by cameron
Post by Colin Watson
prctl (PR_SET_CHILD_SUBREAPER) is used to make SIGCHLD notification
work properly when Upstart is supervising a user session. This isn't
a required feature and could easily be compiled out until suitable
kernel support is available (this actually seems like the sort of
thing that could be done in the Hurd without too much difficulty, but
I haven't looked into it). If absent, it might well impede the
ability to do an advanced desktop port, but it wouldn't get in the
way of porting the bulk of services.
Unity, likely the only desktop environment using Upstart as a
session init, is not in Debian. The sacrifice of this functionality
on non-linux systems is perfectly acceptable.
While this is true today, we need to look to the future. Using Upstart
as a session init is not conceptually tied to Unity in any way, and I
expect that other desktop environments will want to use more advanced
session supervision soon enough.
I place less importance on the session init capabilities of Upstart, as
alternative home grown solutions ww.ritten by the environments are in place
and are portable. Furthermore, portability of these environments hinges on
a session and seat manager, ConsoleKit support in GNOME is quickly
disappearing, and ConsoleKit itself is completely abandoned. What this
means, to me, is there is a lot more effort required to accomplish this
than to port Upstart's session init capabilities or use the current
portable session inits.

Sincerely,
Cameron Norman

--
Steve Langasek
2013-12-31 04:32:46 UTC
Permalink
Post by Russ Allbery
Post by Steve Langasek
From comments made by various GNOME upstream developers on this, I think
they are being suitably cautious about avoiding scope creep where the
systemd dependencies are concerned. So in what sense are the GNOME and
KDE requirements not already being met on top of upstart? The only
outstanding issue I see is with non-Linux ports, because logind is not
portable; that's definitely a problem for running GNOME on kfreebsd, but
it's actually a rather narrowly-scoped porting problem and one that the
ports need to deal with one way or another if they want GNOME to
continue working.
I don't see what you're saying here as substantively different than what I
said in my writeup about the ecosystem. I feel like we're both presenting
the same facts through different lenses.
I do see a substantive difference between "GNOME depends on systemd" and
"GNOME depends on interfaces a, b, and c, which are available as part of
systemd".
Post by Russ Allbery
Post by Steve Langasek
Post by Russ Allbery
One can, of course, have a wide variety of reactions to that. As
someone who considers portability to be an inherent good, I find it
sad, although I don't have a full appreciation for what problems GNOME
is trying to solve by increasing its reliance on systemd. But I'm
highly dubious that Debian will just single-handedly fix that, or that
we would drop GNOME because we don't like upstream's integration
decisions.
I find the notion that Debian doing this "single-handedly" is an
obstacle to be a bizarre one. Debian has more than one pair of hands,
it's a veritable multi-tentacled beast. Have we so atrophied as a
community that we'll wail and gnash our teeth about a non-portable
dependency, but have no porters willing to actually provide a native
implementation of a (quite small) dbus API?
Please recall the context here: this whole aside started with an objection
to my contention that adopting upstart requires disassembly and redoing of
an integration that we would otherwise not have to disassemble. Nowhere
in my message did I say that we would or could not do that disassembly if
we adopted upstart; I said that it was work that we otherwise wouldn't
have to do.
That's the intended context of my point above: I don't think we're going
to port GNOME to a non-systemd infrastructure, in the sense of carrying
significant patches to GNOME to adopt it to, say, not using logind. I
think GNOME will continue to use systemd APIs heavily, which makes GNOME
less portable. That means that systems that are not capable of running
those systemd components will need to either port them or develop
alternatives.
I don't consider this wailing or gnashing of teeth, but rather a realistic
look at what efforts the project is talking about committing to, as
opposed to supporting people working on if they so choose.
Ok. I think our core point of disagreement here, then, is in our assessment
of how much work we think this actually is (for the Linux case, not for the
non-Linux case). I think the actual package maintenance to make this happen
is not even a weekend's worth of free time, and therefore represents a
negligible committment of resources on Debian's part, given that this
dissassembly/integration has already been done in Ubuntu.
Post by Russ Allbery
*If* Debian adopts systemd as an init system, I do think that it's
possible (how likely, I don't know) we'll end up with GNOME unavailable on
the kFreeBSD port because people choose not to do the effort of separating
out and porting the components required. It's up to the porters, and
depends on what role they see for the port. If, for example, they're
primarily targeting servers, this may not be a significant loss. The
point is that it would be their call.
Right. I think that if we adopt systemd, GNOME will be the least of the
kfreebsd porters' problems, because of the overarching init integration
questions.
Post by Russ Allbery
My belief, and again I welcome concrete reasons why I'm not correct, is
that adopting upstart poses a similar risk for the Hurd port as adopting
systemd, and I care just as much about the Hurd port as kFreeBSD.
Well, certainly there is risk that no one will be interested in doing the
work to port libnih+upstart to the Hurd. Picking either upstart or systemd
will not guarantee that we have porters willing to do the work to keep up.
But I think the success of the in-progress kfreebsd port shows that upstart
poses a much lower portability barrier than systemd; if we want non-Linux
ports to exist on the same terms as the Linux ones (i.e., without needing
backwards-compatibility solution for sysvinit), I think upstart does give
them a much better fighting chance.
Post by Russ Allbery
I want to mention again something that you'd dismissed as possibly a joke
earlier, but which I was actually serious about: I think a world in which
we use systemd on Linux and upstart on non-Linux ports, should upstart
prove much more portable, actually makes sense. As I said in my other
writeup, I believe the systemd feature advantage is sufficient to choose
systemd in isolation, without the other issues discussed. I also believe
that maintaining a systemd unit plus an upstart configuration is, modulo
testing (which I realize is a huge issue), much easier than maintaining a
single sysvinit script.
I agree that maintaining a systemd unit plus an upstart job is better than
maintaining an init script. I just can't see any way through to a world
where these will both actually be maintained (the testing problem),
particularly if upstart use is relegated to the non-Linux ports. It's hard
for me to view "Ubuntu, Debian GNU, and Debian GNU/kFreeBSD use upstart; Red
Hat, Fedora, SuSE, and Debian GNU/Linux use systemd" as anything but a loss
for upstart. With only non-Linux ports of Debian on upstart's side and all
the other potential collaborators among the traditional distros opting for
systemd, I think that would leave Canonical confronting some hard questions
about whether to continue investing in upstart development.

Earlier in the discussion, you posited that the same argument for Canonical
Post by Russ Allbery
Post by Steve Langasek
bzr lost the DVCS war not because Canonical was unwilling to invest in
bzr, but because git had an early lead in performance and flexibility
that made it the runaway choice for developers, and by the time bzr was
catching up in functionality, network effects meant it was too late.
Once it became clear that bzr would not deliver on its promise of being
a universal DVCS because the world had already adopted git as a de facto
standard, it was reasonable for Canonical to stop investing in bzr
development.
True. However, I'll point out that a similar argument can be made for
systemd.
My answer to this is that, as things stand today, this argument does *not*
apply, because Debian hasn't made a choice for either systemd or upstart
yet. Whichever option Debian chooses, that is the option that is going to
carry the day, because Debian does integration better, and across a wider
range of software, than any other distro out there. If Debian chooses
systemd, then these integration efforts become focused on systemd, and
systemd wins. If Debian chooses upstart, then upstart wins. And if Debian
chooses upstart only for the non-Linux ports (which attract the attention of
only a small minority of Debian developers), then systemd wins - and any
spare cycles Ubuntu might win back by having Debian and Ubuntu in alignment
on init systems go out the window, leaving less time to invest in upstart
development.

So I don't think I would vote "systemd on linux, upstart on non-linux" above
"systemd, non-linux ports to figure out how to be compatible", because I
fear that would be leading the non-Linux ports on. I don't think it's fair
to them to recommend upstart as a path forward when upstart's own future
would be uncertain under such circumstances, on top of them already having
to shoulder the burden of doing all the testing of upstart jobs that are
used nowhere else in Debian.

(As a personal aside, whichever of upstart and systemd is chosen by the TC
as the default, I intend to wholeheartedly adopt it for my own use in
Debian. I love the upstart codebase, for all the same reasons that you
found when you looked at it, but I'm not on a quixotic quest here. If
Debian chooses systemd, then any time I spend on debugging init system bugs
in Debian is best spent debugging them on systemd, where it will bring the
most benefit to our users.)

Cheers,
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org
cameron
2013-12-31 00:35:28 UTC
Permalink
Post by Steve Langasek
Post by Ian Jackson
Post by Ian Jackson
Post by Russ Allbery
Rather, we're talking about whether or not to swap out a core
component
Post by Ian Jackson
Post by Russ Allbery
of an existing integrated ecosystem with a component that we like
better.
Unless you are proposing to make systemd mandatory for all Debian
installations, this is work that needs to be done anyway.
What I'm hearing from both GNOME and KDE maintainers is that
assuming
systemd would save them a great deal of work. I think having systemd be
the only supported and tested option for at least some large
package sets
that heavily use the systemd infrastructure upstream is a
not-unlikely
long-term outcome.
It's clear that being able to assume availability of certain dbus services
is labor-saving for GNOME and KDE upstreams, and I see no reason for them
not to standardize on such a requirement assuming the dbus services have
sane APIs (which they do).
What I object to is referring to this as "assuming systemd". They are
systemd APIs, sure, but with one exception the systemd
implementations are
not presently tied to the use of systemd as init, and there is an alternate
implementation of that one exception (systemd-shim).
From comments made by various GNOME upstream developers on this, I think
they are being suitably cautious about avoiding scope creep where the
systemd dependencies are concerned. So in what sense are the GNOME and KDE
requirements not already being met on top of upstart? The only outstanding
issue I see is with non-Linux ports, because logind is not portable; that's
definitely a problem for running GNOME on kfreebsd, but it's actually a
rather narrowly-scoped porting problem and one that the ports need to deal
with one way or another if they want GNOME to continue working.
I'm sure there are GNOME upstream contributors who would be happy to see
GNOME become systemd-only. But I think their motivations in letting the
lines be blurred in such a way are purely political, not technical; and I
give GNOME upstream as a whole the benefit of the doubt that they would not
so weaken their project to suit someone's political agenda.
Post by Ian Jackson
One can, of course, have a wide variety of reactions to that. As someone
who considers portability to be an inherent good, I find it sad, although
I don't have a full appreciation for what problems GNOME is trying to
solve by increasing its reliance on systemd. But I'm highly
dubious that
Debian will just single-handedly fix that, or that we would drop GNOME
because we don't like upstream's integration decisions.
I find the notion that Debian doing this "single-handedly" is an obstacle to
be a bizarre one. Debian has more than one pair of hands, it's a veritable
multi-tentacled beast. Have we so atrophied as a community that we'll wail
and gnash our teeth about a non-portable dependency, but have no porters
willing to actually provide a native implementation of a (quite small) dbus
API?
You've said that you think the portability question doesn't weight strongly
in favor of either upstart or systemd, because neither port is done yet.
But the amount of work that would be involved in porting systemd to kfreebsd
is an order of magnitude greater than the work involved in porting upstart.
logind is just one small component of systemd, which someone could provide
an API-compatible implementation for without having to contend with the
question of cgroups on non-Linux kernels. If even that is out of reach for
the Debian porters, how could we ever expect a full BSD port of systemd to
materialize?
systemd lists logind as non-reimplementable, and that was pretty much
proven when Ubuntu tried to reimplement it and ended up reimplementing
or pulling in a ton of systemd anyway. Seeing that, both KWin and
Mutter have denounced portability to non-Linux when running on Wayland.
They will soon be outright non-portable (when ConsoleKit support is
dropped, which, AFAIK, is soon in GNOME).
Post by Steve Langasek
I think the reality is that adopting systemd as default init for Debian is
nothing short of a death sentence for the non-Linux ports. The move to a
different init will have a snowball effect in the distribution, as packages
stop being tested with sysvinit and popular support grows for dropping
compatibility with sysvinit altogether. This is not a problem if the ports
are in a position to come along on this transition with a moderate
investment in porting init. But the porting required for systemd as a whole
is far from moderate, and I believe that faced with such a
requirement the
non-Linux ports would wither and die. I know there are Debian
developers
(including many in the GNOME/systemd "camp") who think this is a desirable
outcome. I have no personal attachment to the non-Linux ports, but I do
think that as an existing part of our Debian community, the TC should at
least give these ports a fighting chance, because this kind of
competition
is healthy.
Post by Ian Jackson
Post by Ian Jackson
My understanding was that Dimitri had got upstart running on BSD.
http://blog.surgut.co.uk/2013/11/libnih-upstart-dependency-ported-to.html
I asked previously on this bug if someone had later news. Do you have
more information than that?
The above is definitely not "upstart running on BSD." That's upstart's
underlying support library mostly working on BSD after disabling two
features (that are required for upstart). I haven't not heard whether the
porting of upstart proper has started. That will certainly be much easier
once libnih is ported, but there are, for example, direct uses of epoll in
upstart proper. (I don't know if kFreeBSD already has a
portability layer
for epoll in terms of kqueue.)
AIUI, upstart itself built out of the box once libnih was available, because
all the portability issues are encapsulated in libnih. (The single use of
epoll in the upstart source is in the upstart-socket-bridge, which is an
optional component from upstart's perspective and certainly not needed for
booting a Debian base system - so presumably Dimitri just built with this
bridge disabled.)
With libinotify-kqueue (that Dimitri maintains), kfreebsd 10 (in
experimental), eglibc 2.18 (also in experimental), and Dimitri's branch of
libnih, it seems that all the portability requirements for upstart are met,
except for abstract socket support. There are evidently still some porting
problems that aren't detected at build time, because upstart doesn't yet
boot on kfreebsd. But all things considered, the port is rather far along.
Post by Ian Jackson
upstart is a great project, of which its maintainers are deservedly proud.
However, I don't agree that it's better than systemd. My own research
convinced me of the opposite. But putting that aside, my point in that
section is that, given feature parity (and I mean "feature"
expansively,
including adaptibility to Debian's needs), we should choose systemd
because it has more project momentum and better existing
integration,
which means that we will have to spend less energy on it and will have
more energy to spend on other things.
It's absolutely worth doing our own, better things. What I don't want to
do is our own *worse* things. Or, for that matter, our own
equivalent
things, when we could instead use work that already exists. It's a waste
of energy.
I think this downplays the significance of the integration work that has
already been done in Ubuntu on top of upstart, that Debian will be able to
adopt as-is (or with whatever bugfixes the maintainers find along the way).
My look at Fedora 20 confirmed my belief that the integration
necessary to
have a robust systemd boot across all the configurations that Debian
supports has not actually been done yet. systemd upstream advocates
shipping systemd units upstream where they can be consumed unmodified by
distributions, but in practice Debian is going to be on the hook for
debugging the very long tail of integration problems. It's hard to gauge
whether the energy saved by not bringing upstart up to feature parity
outweighs the energy spent on bringing systemd integration up to snuff, but
I definitely don't think there's a clear point here in systemd's favor.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer
http://www.debian.org/
Steve Langasek
2013-12-31 01:05:54 UTC
Permalink
Post by cameron
systemd lists logind as non-reimplementable, and that was pretty
much proven when Ubuntu tried to reimplement it and ended up
reimplementing or pulling in a ton of systemd anyway.
All this proves is that Ubuntu developers have the good sense to not
reinvent the wheel unnecessarily. No one ever tried to reimplement logind
for Ubuntu, at all. Why should they, when logind is already a perfectly
usable implementation of logind?
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org
intrigeri
2013-12-30 23:00:25 UTC
Permalink
Hi,

(Sorry if I am duplicating a point that was already made.
These threads are huge, and don't fit entirely into my memory.)
Post by Ian Jackson
Post by Russ Allbery
Rather, we're talking about whether or not to swap out a core component of
an existing integrated ecosystem with a component that we like better.
Unless you are proposing to make systemd mandatory for all Debian
installations, this is work that needs to be done anyway.
"Needs to be done anyway", possibly, but I find it important to make
it clear that, depending on what decision is made, it affects the
project as a whole, and many of its members, in very different ways:

* In one case (upstart is chosen as the default init system), we, as
a project, are committed to do this work, at the very least because
Policy mandates it, and we want to release without too many
RC-buggy packages.

Some maintainers happily do it because they are glad that upstart
was picked, some do it reluctantly as they would have preferred
systemd and they feel this all is unnecessary work put on our
collective shoulders, some would have preferred systemd but console
themselves because it feels so good to move away from sysvinit
eventually, etc.

Regardless of what our personal preference is, we have to do this
work, because else it's a RC bug and we can't release.

* In the other case (systemd is chosen as the default init system),
any individual or self-organized team may tackle this work, if their
desires or needs make them feel committed to provide choice and
flexibility, for the init component and potentially the kernel too,
to Debian users.

I believe this effort is similar in many ways to porting, and as
such I trust it will be treated with "deference and reasonable
accommodation" (thanks, Russ) in our community.

The difference lies in who are the people who "need" to do this work
"anyway", and who else may instead dedicate their time to other tasks,
lead by their own desires and needs.

[Now moving away from my clarification point, and largely off-topic.
Feel free to ignore.]

This specific part of the broader init system discussion boils down to
"what are our collective goals and priorities?", or "what do we find
important enough to put much energy into?". We are currently not that
good at finding exciting collective answers to this, and even at
finding good ways to make such decisions.

I mean: if it were just me, my proposal would be "let's make Debian
better empower its users to protect their privacy in the post-Snowden
world", and to me it feels way more exciting and relevant a thing we
could do for our users and Free Software than "let's allow Debian
users to replace systemd's init component with something else".
How far off-topic my answer is shows, I believe, how hard it is to
focus on the mere technical decision that presently needs to be made
(and rightfully was sent to the TC), given how broad its non-technical
implications could be not only on the Debian project, Debian users,
Debian derivatives and Free Software, but on the world at large.

End of the digression explaining how hard it is not to digress.

Cheers,
--
intrigeri
| GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
| OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc
Ian Jackson
2013-12-31 16:58:17 UTC
Permalink
Post by intrigeri
(Sorry if I am duplicating a point that was already made.
These threads are huge, and don't fit entirely into my memory.)
That's fine, of course.
Post by intrigeri
Post by Ian Jackson
Unless you are proposing to make systemd mandatory for all Debian
installations, this is work that needs to be done anyway.
"Needs to be done anyway", possibly, but I find it important to make
it clear that, depending on what decision is made, it affects the
* In one case (upstart is chosen as the default init system), we, as
a project, are committed to do this work, at the very least because
Policy mandates it, and we want to release without too many
RC-buggy packages.
I think you have misunderstood. Or perhaps I hae misunderstood you.
The "work" that I'm saying needs to be done anyway is the work to
disentange the parts of systemd which are required by (say) GNOME from
the parts which are only relevant for systemd as init.

This is work that would have to be done by the systemd maintainers in
Debian.
Post by intrigeri
The difference lies in who are the people who "need" to do this work
"anyway", and who else may instead dedicate their time to other tasks,
lead by their own desires and needs.
I think that it is right that the costs of undoing systemd's bundling,
be borne by the systemd community (including systemd's advocates and
maintainers in Debian) rather than by Debian (or the rest of the Free
Software world) at large.

Ian.
Tollef Fog Heen
2013-12-31 20:09:52 UTC
Permalink
]] Ian Jackson
Post by Ian Jackson
I think you have misunderstood. Or perhaps I hae misunderstood you.
The "work" that I'm saying needs to be done anyway is the work to
disentange the parts of systemd which are required by (say) GNOME from
the parts which are only relevant for systemd as init.
This is work that would have to be done by the systemd maintainers in
Debian.
I find it quite clear that this should be done and maintained by the
people who want to run such an configuration. I am not running any
non-systemd desktop systems and would be in a very poor positition to be
able to do this work. I also have no interest in it, which I think
should be pretty clear given my previous mails on the subject.

We've also said quite clearly that we'd gladly accept a co-maintainer
who wants to maintain this configuration, but nobody has shown up yet.
(Martin Pitt has worked a bit with us on getting the patches from Ubuntu
integrated, but AIUI, he's not been able to offer a long-term commitment
to maintaining the patches, and I think it's a very bad idea to merge a
patchset that nobody in the team wants to maintain long-term.)
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
intrigeri
2014-01-01 14:44:07 UTC
Permalink
Hi,
Post by Ian Jackson
I think you have misunderstood. Or perhaps I hae misunderstood you.
The "work" that I'm saying needs to be done anyway is the work to
disentange the parts of systemd which are required by (say) GNOME from
the parts which are only relevant for systemd as init.
I was talking about the very same work, so I think we're understanding
each other just fine on this point :)

Your reply helped me focus and clarify my analysis (that is not "the
project as whole" / "porters and upstart lovers" anymore), though.
Thank you.

It also helped me understand what, I think, a few things we disagree
on: first, who would have the "moral" responsibility of doing this
work; second, whether it matters at all; third, who would have to do
this work in practice.

In what follows, I'll try to keep my personal preferences (that don't
matter at all as far as the TC decision is concerned) aside, but
I don't expect to be entirely successful. Sorry about that in advance.
My conscious intent here is to help make this discussion more fruitful
and focused on what matters in practice; but it's obvious that my
analysis is somehow affected by the investments I've personally made
in the last 14 months (since then, I have been very happily running
systemd on almost every Wheezy or newer system that I am responsible
for). For the sake of full-disclosure, I have to add that I am a core
developer of a Debian derivative that relies on GNOME and does not
intend to switch any time soon.
Post by Ian Jackson
Post by intrigeri
The difference lies in who are the people who "need" to do this work
"anyway", and who else may instead dedicate their time to other tasks,
lead by their own desires and needs.
I think that it is right that the costs of undoing systemd's bundling,
be borne by the systemd community (including systemd's advocates and
maintainers in Debian) rather than by Debian (or the rest of the Free
Software world) at large.
First, I beg to disagree about who, in full rightness, would have the
"moral" responsibility to do this work. But as shown below, we don't
care much about what you or I think.

Second, regardless of what my or Ian's or the TC's or $DEITY's opinion
on this moral matter is, that's very much unimportant: in my belief,
the time and energy people put into Debian is rarely lead by a sense
of "moral" responsibility, especially when the work that "needs to be
done" is something they do *not* feel to be part of what they have
committed to in the first place. I happen to very much doubt that the
systemd community feels that they are committed to support the
use-case we are discussing (systemd minus its init component).

Hence, my third point is that in practice:

* If upstart is chosen as the default init: it might be that the
systemd community shows little interest in doing this work (and, to
be fair, I would find this totally understandable, and not
surprising at all). Then, it is not unlikely that the people who end
up doing this work are those who maintain reverse dependencies of
the systemd stack, such as desktop environments (at least GNOME and
Unity, in Debian and/or Ubuntu). They might have to do this because
of the combination of 1. they want to keep their packages in working
state in Debian and/or Ubuntu; 2. the decision to use upstart
creates the need for someone to do this work; 3. for whatever
reason, nobody else is doing it.

If this were to happen, regardless of anyone's take on what full
rightness would have called for, then the cost of the initial and
ongoing work would be held by an important subset of our community,
that is anyone who wants Debian to deliver a given modern DE.
I realize that it's not the same as the Debian project as a whole,
contrary to what I initially wrote. But it's a lot more people than
just the systemd maintainers in Debian (I'm not comparing to the
broader systemd community here, for reasons I've given above).

* If systemd is chosen as the default init: I am very doubtful that
a decision of the TC regarding who has the "moral" responsibility of
it would be enough to motivate the systemd community to tackle this
work if they don't feel it's part of the mission they are committed
to. It might be, but I don't think it's a reasonable assumption to
make.

So, with the information I have in hand, I'm sticking to my original
point here: in practice, this would be a job for porters, for anyone
who wants to allow users of Debian to give this amount of
flexibility, and for any derivative who wants to use another init
system; while others treat this effort with "deference and
reasonable accommodation", and can get themselves busy, as they
wish, with other means to scratch their own itch and/or make the
world a better place.

My conclusion is that 1. in practice, the situation is far from being
as simple as: the systemd community will have to do it anyway; and 2.
the consequences of the TC decision is likely to affect very
different, though non-disjoint, 1. sets of people and, 2. (IMO more
importantly) use cases Debian has been supporting, and many
derivatives have been building upon for many years.

Cheers,
--
intrigeri
| GnuPG key @ https://gaffer.ptitcanardnoir.org/intrigeri/intrigeri.asc
| OTR fingerprint @ https://gaffer.ptitcanardnoir.org/intrigeri/otr.asc
Ansgar Burchardt
2013-12-31 17:21:15 UTC
Permalink
Post by Ian Jackson
Post by intrigeri
The difference lies in who are the people who "need" to do this work
"anyway", and who else may instead dedicate their time to other tasks,
lead by their own desires and needs.
I think that it is right that the costs of undoing systemd's bundling,
be borne by the systemd community (including systemd's advocates and
maintainers in Debian) rather than by Debian (or the rest of the Free
Software world) at large.
What about the cgroup management functionality that newer versions of
logind require? Should the systemd maintainers also reimplement it in
upstart?

And if upstart wants to use parts of systemd, why shouldn't the upstart
maintainer do the work for this? Or they could fork logind which they
suggested before... This would also allow having a newer systemd in
Debian.

Ansgar
Ian Jackson
2013-12-31 18:31:48 UTC
Permalink
Post by Ansgar Burchardt
What about the cgroup management functionality that newer versions of
logind require? Should the systemd maintainers also reimplement it in
upstart?
This is a somewhat separate issue, but: I think bundling the single
cgroup writer into systemd is a very poor design choice. I think the
bad consequences of that choice should be borne by the people who made
it.

Ian.
cameron
2013-12-31 20:13:20 UTC
Permalink
On Tue, Dec 31, 2013 at 10:31 AM, Ian Jackson
Post by Ian Jackson
Post by Ansgar Burchardt
What about the cgroup management functionality that newer versions of
logind require? Should the systemd maintainers also reimplement it in
upstart?
This is a somewhat separate issue, but: I think bundling the single
cgroup writer into systemd is a very poor design choice. I think the
bad consequences of that choice should be borne by the people who made
it.
Nitpick: cgroups had multiple arbitrators when systemd was written to
use it, and only recently did that change. I agree it was a poor design
choice, and the systemd devs did not outright oppose it (as they should
have IMO), but it was not technically systemd's fault.

Best regards,
Cameron Norman
Josselin Mouette
2013-12-31 20:13:52 UTC
Permalink
Post by Ian Jackson
Post by Ansgar Burchardt
What about the cgroup management functionality that newer versions of
logind require? Should the systemd maintainers also reimplement it in
upstart?
This is a somewhat separate issue, but: I think bundling the single
cgroup writer into systemd is a very poor design choice. I think the
bad consequences of that choice should be borne by the people who made
it.
By writing this, it strikes me that you must have seriously
misunderstood some fundamental concepts of systemd. The new logind
behavior is unrelated to the “single cgroup writer” matter, because
there is no single cgroup writer as of today. I spent quite some time to
summarize facts on cgroup management at Andreas’ request, and it seems
you haven’t even read them. I find this very rude from a member of the
technical committee to not try to understand the technical issues before
deciding what other people are supposed to do.

Which brings me to the other point: you are not going to decide what
people want to spend their time on. If systemd is selected as the
default, the systemd maintainers are not going to ask Steve to fix their
upgrades problem for them. And if upstart is selected, you will
certainly not ask members of the systemd community - from which Debian
would have just excluded itself - to fix Debian’s problems with not
having systemd.

For an example I know, if having a working GNOME on Linux means a
dependency on systemd, then it will have a dependency on systemd. If the
TC overrules that, like it did the last time one of its members felt
offended by a dependency in a package he doesn’t use, the alternative
will have to be developed and made available by someone. From my
discussions so far with other members of the GNOME team, that someone
will not be a member of that group.

Let’s say that GNOME migrates to systemd user sessions, like what is
planned for GNOME 2.12 (yes, the version we intend to ship in jessie,
ain’t that sweet). You can decide to cripple GNOME with Ubunbu patches
instead, but that won’t be GNOME anymore; just an unbranded Ubuntu
desktop. And you will not ask the people who spend their time providing
a serious, upstream-friendly alternative to that desktop to spend it on
dumping Ubuntu packages in Debian instead.

So unless the TC wants to remove a great number of packages from the
archive, you need to take into account the fact that some voluntary
manpower is required to implement your decision.

Cheers,
--
.''`. Josselin Mouette
: :' :
`. `'
`-
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
cameron
2013-12-31 20:30:22 UTC
Permalink
Post by Josselin Mouette
Post by Ansgar Burchardt
Post by Ansgar Burchardt
What about the cgroup management functionality that newer
versions of
Post by Ansgar Burchardt
logind require? Should the systemd maintainers also reimplement
it in
Post by Ansgar Burchardt
upstart?
This is a somewhat separate issue, but: I think bundling the single
cgroup writer into systemd is a very poor design choice. I think the
bad consequences of that choice should be borne by the people who made
it.
By writing this, it strikes me that you must have seriously
misunderstood some fundamental concepts of systemd. The new logind
behavior is unrelated to the “single cgroup writer” matter,
because
there is no single cgroup writer as of today. I spent quite some time to
summarize facts on cgroup management at Andreas’ request, and it
seems
you haven’t even read them. I find this very rude from a member of
the
technical committee to not try to understand the technical issues before
deciding what other people are supposed to do.
Which brings me to the other point: you are not going to decide what
people want to spend their time on. If systemd is selected as the
default, the systemd maintainers are not going to ask Steve to fix their
upgrades problem for them. And if upstart is selected, you will
certainly not ask members of the systemd community - from which Debian
would have just excluded itself - to fix Debian’s problems with not
having systemd.
Nobody is preventing anybody from spending time on re-integrating
systemd and its services.
Post by Josselin Mouette
For an example I know, if having a working GNOME on Linux means a
dependency on systemd, then it will have a dependency on systemd. If the
TC overrules that, like it did the last time one of its members felt
offended by a dependency in a package he doesn’t use, the
alternative
will have to be developed and made available by someone. From my
discussions so far with other members of the GNOME team, that someone
will not be a member of that group.
Do not assume that people who dislike GNOME's systemd dependency do not
use GNOME. I do not use GNOME, but my sanity depends on gsettingsd (my
touchpad is broken and only gsettingsd can fix it), as well as my
aesthetic happiness (I like Pantheon shell, which uses gsettingsd and
gnomecc).
Post by Josselin Mouette
Let’s say that GNOME migrates to systemd user sessions, like what is
planned for GNOME 2.12 (yes, the version we intend to ship in jessie,
ain’t that sweet). You can decide to cripple GNOME with Ubunbu
patches
instead, but that won’t be GNOME anymore; just an unbranded Ubuntu
desktop. And you will not ask the people who spend their time
providing
a serious, upstream-friendly alternative to that desktop to spend it on
dumping Ubuntu packages in Debian instead.
If GNOME depends on systemd, that is fine, but GNOME does not get to
dictate Debian's choice. The GNOME maintainers need to do whatever is
necessary for GNOME to run on Debian, including maintaining systemd and
systemd units, patching to work with Upstart, or removing functionality
that depends on systemd (which should not at all be basic
functionality, but only small stuff).

Even if that non-basic functionality is removed, **it will still be
GNOME**. At least, it should be based on GNOME's own technical
conclusion.

It seems like you are saying that because GNOME has decided to depend
on systemd, that Debian needs to cater to that decision. Debian does
not. In fact, Debian //should not//, because GNOME consciously decided
(or will decide) to break non-systemd compatibility, and knew that
doing so would break GNOME on Debian (in its current state).

Cheers,
Cameron Norman
Post by Josselin Mouette
So unless the TC wants to remove a great number of packages from the
archive, you need to take into account the fact that some voluntary
manpower is required to implement your decision.
Cheers,
--
.''`. Josselin Mouette
`. `'
`-
--
with a subject of "unsubscribe". Trouble? Contact
Russ Allbery
2013-12-31 20:57:50 UTC
Permalink
Post by Josselin Mouette
Which brings me to the other point: you are not going to decide what
people want to spend their time on. If systemd is selected as the
default, the systemd maintainers are not going to ask Steve to fix their
upgrades problem for them. And if upstart is selected, you will
certainly not ask members of the systemd community - from which Debian
would have just excluded itself - to fix Debian’s problems with not
having systemd.
I want to second this, because I agree with it completely. It is a core
principle of Debian, featuring prominantly in the constitution:

Nothing in this constitution imposes an obligation on anyone to do
work for the Project. A person who does not want to do a task which
has been delegated or assigned to them does not need to do
it. However, they must not actively work against these rules and
decisions properly made under them.

I'm pretty sure that there are project resources available to integrate
with systemd and that there are resources available to integrate with
upstart (and there may be resources to integrate with both, although I
really don't know). But those resources are probably going to be
different in at least some cases. People are not, generally speaking,
going to spend significant amount of time and effort working on source
bases they don't like or in pursuit of goals that they don't agree with.

This is one of the points I made in my writeup about the ecosystem. These
are exactly the sort of resources that we will have to muster and maintain
going forward in order to adopt usptart. Those resources may or may not
be available in Debian today. I think that depends on how much surgery
this will require to systemd and GNOME, and possibly later to KDE and to
other packages, and what the ongoing maintenance burden will look like.
Obviously, we can share work with Ubuntu in some of these areas, which
reduces the demand for Debian-specific resources. But we certainly should
not assume that either the current GNOME maintainers or the current
systemd maintainers will be those resources. They may be, or they may
decide to go work on something else; either way, it's entirely up to them,
as it should be.

We currently have multiple dedicated groups already in place in Debian who
have clearly said they will do the work to integrate their components with
systemd, and who have either not expressed an opinion or who have
indicated they're not interested in integrating with upstart. And, I will
also point out, we have other dedicated groups in Debian who are very
concerned about the impact the adoption of systemd would have on *their*
work and *their* projects.

The degree to which this all should affect our decision-making process is
limited. That's particularly true in this case, which is one of those
decisions where, whatever we choose, some group of people in Debian who
have put a ton of work into something they care about are going to be at
least somewhat unhappy. That may be the systemd maintainers, the GNOME
team, the Hurd porters, the kFreeBSD porters, the upstart maintainers,
those who care deeply about tight Debian and Ubuntu integration, or any
number of other groups. It will probably be several of those.

Occasionally, there are decisions with sweeping consequences, and we have
to have a method for making them. The TC is that method, which can then
be overridden by General Resolution if warranted.

However, it does affect how we should talk about these decisions, and it
does affect the reality check on which approach has the most cost and on
what resources are available.

The TC doesn't get to order people to do work. We are not managers, and
Debian is not a corporation. At most we can authorize NMUs or otherwise
work around people who are preventing *other* people from doing work.
And, as such, I really dislike seeing people make statements about what
other specific people should be required to do in Debian. Making
statements about what work one believes needs to be done is a different
matter, but one should not assume that this work is going to be done by
the people currently maintaining the relevant packages if they don't agree
with it. Or if, for that matter, they get busy, they get sick, they have
children, they retire, or they decide not to do that work for any other
reason.

For example, one possible outcome here is that whatever group cares about
doing the upstart integration introduces a new systemd source package
that's split and modified in such a way so as to work with upstart as the
init system, and which conflicts with systemd as it is currently packaged.

Furthermore, the TC is also not a code review body for Debian, nor is it
our role to impose our personal asthetic or design views on the project.
If any of us in the Debian project wanted to work on something with a
dictated, top-down design, there are numerous other projects we could be
working on, including several that offer wages for such effort. One of
the core qualities of Debian is that we only dictate standards to people
when they're necessary for integration, security, or to reduce impact on
other core teams such as ftp-master, release, or security. Otherwise, we
try to let people go about their work in the way that suits them best.
This has various significant drawbacks, but the huge advantage that it
makes working on Debian fun rather than a job.

We have to make a decision here because Debian can only have one *default*
init system, and all of our packages need to work with it. Maybe that
decision necessarily involves some amount of central control over how
integrations are done, including switching maintainers or introducing new
packages in order to implement those integrations. But that's an
unfortunate consequence of necessity, not an exciting opportunity to make
everyone do everything the Right Way in an area where there are profound
disagreements.

I'm quite sure I'm not the only one who is regularly dictated design
decisions in my regular job and has no interest in being subjected to the
same thing in my hobby.

Some amount of it is necessary in order to accomplish our shared goals of
producing an integrated distribution. And some of it falls under
reasonable accomodation; for example, I believe the solution that we
arrived at with Network Manager was reasonable accomodation to the desires
of other people in Debian, and continue to defend it. But on *exactly the
same grounds*, I will defend introduction of shared library dependencies
to daemons for integration with systemd.

And we need to be very careful about what the boundaries of reasonable
accomodation are. Major surgery to one's packages contrary to fundamental
design goals that one has pursued in creating those packages is not
reasonable accomodation, and it's something we should only pursue if the
greater goal truly both warrants it and requires it, and with the full
understanding that this will often mean that the current maintainers will
step down and someone else will need to do the actual work.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Tollef Fog Heen
2014-01-01 01:23:50 UTC
Permalink
First of all, thanks a lot for writing this mail. It expresses a lot of
my thoughts and feelings on the subject a lot more eloquently than I am
able to do myself. You're a wordsmith and a master of words. I am
not.

]] Russ Allbery
Post by Russ Allbery
Occasionally, there are decisions with sweeping consequences, and we have
to have a method for making them. The TC is that method, which can then
be overridden by General Resolution if warranted.
Given how the voting ratio so far looks, I've been giving the whole GR
process a bit of thought lately and at least I am unlikely to pursue it,
simply because I don't think it's a good way to spend my and the
project's energy. If you decide that you want to go with upstart,
that's obviously not what I want, but I'll rather take the consequences
of that than go one more round and appeal to the developer body at
large. As our common astronomy geek friend puts it, I don't have the
people beans to spend on that. Somebody else might.

Personally, I wish the TC was a bit more careful with the «people» angle
of their rulings. You spent a lot of goodwill in the GNOME camp when
pushing through the latest NM Depends->Recommends change, and I think
it'd be a shame if we ended up losing or demotivating a good bunch of
good developers again.
Post by Russ Allbery
The TC doesn't get to order people to do work. We are not managers, and
Debian is not a corporation. At most we can authorize NMUs or otherwise
work around people who are preventing *other* people from doing work.
I think what you're saying here is really important. What Ian is asking
about is for the systemd maintainers to do a lot of work we don't want
to do, and if we end up with a resolution that basically tells the
systemd maintainers to support a chopped-up package, it'd be massively
demotivating for me personally. From the message from Joss, it seems
like the GNOME people feel the same way.

[...]
Post by Russ Allbery
For example, one possible outcome here is that whatever group cares about
doing the upstart integration introduces a new systemd source package
that's split and modified in such a way so as to work with upstart as the
init system, and which conflicts with systemd as it is currently packaged.
The way things are going, I don't think that'd be a terrible situation,
tbh.
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Russ Allbery
2014-01-01 01:50:29 UTC
Permalink
Post by Tollef Fog Heen
Given how the voting ratio so far looks, I've been giving the whole GR
process a bit of thought lately and at least I am unlikely to pursue it,
simply because I don't think it's a good way to spend my and the
project's energy.
There's one point that I think is worth noting there, although I'm not
advocating for a GR and it would be much better if the TC arrived at a
conclusion that the project could get behind. That's that it's common,
for decision-making bodies and for appeal processes, to have a small group
of people go off and do discovery of the facts so that there's a basis of
researched information on which to judge an appeal.

Obviously, one of the reasons why I'm trying to synthesize my
understanding and write up summaries is because I think it's useful for
the TC decision itself. But it's also a useful artifact for the developer
body as a whole, should it want to review the TC decision. I think this
process has already brought a lot more light to the exact issues,
concerns, and tradeoffs involved, and subsequent decisions can be made
with reference to the artifacts of the TC deliberation, without expecting
each developer to need to go off and do research on their own.

We've already worked through a variety of misconceptions and false starts
that were not obvious from the debian-devel discussion. If it does come
to a project-wide vote, I think we will have more of a consensus on the
facts as we know them today, and people can then vote on how they want to
weigh those facts against each other.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Russ Allbery
2014-01-02 16:05:01 UTC
Permalink
Sometimes I also wonder if a GR might be a better way to deal with the
decision as this feels more and more like an "political" or "opinion"
decision rather then a technical decision to me as tech-ctte members
have found both upstart and systemd to be suitable so far and only minor
reasons were responsible for the final decision.
For what it's worth, I actually disagree with this. I believe upstart is
technically unsuitable to be Debian's init system.

Now, the upstart maintainers are committing to implement a whole bunch of
new functionality in upstart to *make* it suitable if we adopt it, so
that's a situation that could change. But, at the present time, I think
upstart is clearly technically inferior to systemd, and not in just minor
ways.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Russ Allbery
2014-01-02 16:09:44 UTC
Permalink
And, despite the fact that the decision has become very politicised (to
some extent along the lines of preexisting camps of strongly disagreeing
contributors), I think it is primarily a technical decision.
I think this is a remarkable statement given that you're the primary one
who has been politicizing it. I am really unsure how to respond usefully
to the outright attacks you have been posting against both systemd
upstream and other maintenance teams in Debian. The degree to which you
are assuming not only bad faith but malicious motives among other people
in the free software community is deeply disturbing to me.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Raphael Hertzog
2014-01-02 18:02:07 UTC
Permalink
Post by Russ Allbery
And, despite the fact that the decision has become very politicised (to
some extent along the lines of preexisting camps of strongly disagreeing
contributors), I think it is primarily a technical decision.
I think this is a remarkable statement given that you're the primary one
who has been politicizing it. I am really unsure how to respond usefully
to the outright attacks you have been posting against both systemd
upstream and other maintenance teams in Debian. The degree to which you
are assuming not only bad faith but malicious motives among other people
in the free software community is deeply disturbing to me.
+1

Thank you for saying it so clearly while still avoiding the ad-hominem
attack. I didn't find a good way to say it without picking on Ian's
behaviour but you express it very well.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Discover the Debian Administrator's Handbook:
→ http://debian-handbook.info/get/
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Russ Allbery
2014-01-02 16:15:25 UTC
Permalink
(a) Make a decision based on our assessment of the merits; that
includes considering the strength and health of the communites
behind each project. But for me it doesn't include consideration
of their raw popularity or unpopularity in the Debian ecosystem.
Expect a GR if our decision is too unpopular.
(b) Decline to make a decision and propose a GR.
Colin was casting about for a plausible intermediate approach perhaps
involving a straw poll. I'm explaining why I think that's not a good
idea; perhaps even a worse idea than (b).
I don't think any of the TC are going to propose (b). Perhaps we
should put (b) on the TC ballot for form's sake; I guess most of the
TC would vote it above FD.
I had actually been considering proposing (b) be on the ballot based on
Guillem and Anthony's reaction.

If the criteria that we're using to decide between systemd and upstart is
about discomfort with systemd's upstream development practices and their
attempt to create a unified and consistent experience, I think a GR may be
a better way to decide this question. That's not a technical question;
that's a question of overall project direction, a question about whether
we force loose coupling even when our upstreams are preferring tight
coupling. And if we are going to drop some core flexibility inside Debian
to get tighter integration and (at least IMO) technically superior
solutions. It's also to some extent a question about how much portability
should drive our core software adoption, which again is more of a
wide-ranging question about project direction than it is a purely
technical decision.

That's starting to sound much more like the sort of decision that should
be decided by GR rather than by a small group of people. And as
previously mentioned, we have the advantage of being able to go into the
GR with a large set of well-researched data that we didn't have before.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Ian Jackson
2014-01-02 16:25:13 UTC
Permalink
Post by Russ Allbery
I don't think any of the TC are going to propose (b). Perhaps we
should put (b) on the TC ballot for form's sake; I guess most of the
TC would vote it above FD.
I had actually been considering proposing (b) be on the ballot based on
Guillem and Anthony's reaction.
I have no objection to (b) being on the ballot. (Obviously.) If we
are to reject it it would be good to do so explicitly.

Ian.
Tollef Fog Heen
2014-01-02 16:51:11 UTC
Permalink
]] Colin Watson
and I think it'd be a shame if we ended up losing or demotivating a
good bunch of good developers again.
Pretty much every time the CTTE makes a ruling, someone is going to be
annoyed or demotivated. Easy, non-contentious decisions never make it to
us.
If there are concrete ways we can be better at making sure that it's
clear developers concerns are being heard and taken into account which
we are not already doing, I'd certainly like to hear them.
Is there any useful way we could take a reasonably quick non-binding
straw poll of developers? Sort of an "if we voted a particular way, is
it likely that we would be on the wrong side of developer opinion".
In addition to the popcon numbers referenced from Sjoerd, we have the
numbers from Michael's systemd survey in May 2013. The numbers there
were 35%/30%/33% for yes/dunno/no for systemd as default init when only
counting DD/DMs/package maintainers

Of course, those numbers might have changed since then.
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
Colin Watson
2014-01-02 21:34:48 UTC
Permalink
Post by Tollef Fog Heen
In addition to the popcon numbers referenced from Sjoerd, we have the
numbers from Michael's systemd survey in May 2013. The numbers there
were 35%/30%/33% for yes/dunno/no for systemd as default init when only
counting DD/DMs/package maintainers
Of course, those numbers might have changed since then.
Thanks. That's marginal enough that I guess it's not likely to be an
obviously dominating factor, so bearing in mind Ian's comments I'm happy
to drop this suggestion.
--
Colin Watson [***@debian.org]
Steve Langasek
2014-01-01 03:01:51 UTC
Permalink
Post by Josselin Mouette
Post by Ian Jackson
Post by Ansgar Burchardt
What about the cgroup management functionality that newer versions of
logind require? Should the systemd maintainers also reimplement it in
upstart?
This is a somewhat separate issue, but: I think bundling the single
cgroup writer into systemd is a very poor design choice. I think the
bad consequences of that choice should be borne by the people who made
it.
By writing this, it strikes me that you must have seriously
misunderstood some fundamental concepts of systemd. The new logind
behavior is unrelated to the “single cgroup writer” matter, because
there is no single cgroup writer as of today.
It's not true that it's unrelated. In v205, logind hands off the cgroup
heirarchy creation to PID 1, precisely because it's preparing for the
anticipated future kernel requirement of a single cgroup writer. If we're
expecting this "single cgroup writer" requirement to manifest, then it makes
sense to move cgroup writing out of logind. The problem is with moving it
to PID1, causing increasingly tight coupling between the components of
systemd.
Post by Josselin Mouette
Let’s say that GNOME migrates to systemd user sessions, like what is
planned for GNOME 2.12 (yes, the version we intend to ship in jessie,
ain’t that sweet).
"It's important to note that with these patches, we still support
non-systemd systems (as well as older systemd). How far into the future we
do so is an open question, but it should not be too difficult to leave
non-systemd systems with the previous model over the next few cycles."

https://wiki.gnome.org/ThreePointEleven/Features/SystemdUserSession

I suppose it's possible that GNOME upstream are actually insane enough to
decide that in future versions they will dictate an init system to the
distributions, but that is not actually an issue for 3.12. There are
advantages to using systemd for user session management, particularly when
coupled with Wayland... but these can just as well be delivered on top of
upstart rather than systemd. It does not follow that GNOME upstream should
dictate to Debian that they adopt systemd rather than upstart.
Post by Josselin Mouette
So unless the TC wants to remove a great number of packages from the
archive, you need to take into account the fact that some voluntary
manpower is required to implement your decision.
I think the current Debian GNOME team has a not-undeserved reputation for
being obstructionist with respect to bugfixes that require divergence from
upstream's stated direction. If the team demonstrated they were open to
contributions of the kind you described, volunteers to do the work would not
be hard to come by.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org
Josselin Mouette
2014-01-02 10:58:45 UTC
Permalink
Post by Steve Langasek
It's not true that it's unrelated. In v205, logind hands off the cgroup
heirarchy creation to PID 1, precisely because it's preparing for the
anticipated future kernel requirement of a single cgroup writer.
This change would have happened sooner or later. The fact that logind
used to work without systemd as init was purely coincidental, since
logind was designed as an integral part of systemd from the very
beginning (particularly because of the cgroups design). This specific
change might has been triggered by anticipation for a kernel change (a
change that still doesn’t exist), but if not for cgroups, it would have
been for another reason.
Post by Steve Langasek
If we're
expecting this "single cgroup writer" requirement to manifest, then it makes
sense to move cgroup writing out of logind. The problem is with moving it
to PID1, causing increasingly tight coupling between the components of
systemd.
Let’s imagine for a minute that the systemd developers would have
listened to this request and put the cgroup manager outside of PID 1.

Since systemd starts up everything it spawns using cgroups, the very
first thing it would need to do would be to start up the cgroups manager
at boot time, before anything else. For systems with systemd in the
initramfs, the cgroups manager needs to be in the initramfs too.

A new protocol would be introduced for PID 1 to talk to the cgroups
manager, and every time a process is spawned, every time a query comes
from an interface, PID 1 would talk to the cgroups manager through this
protocol. Other processes would talk to PID 1 through one protocol, and
to the cgroups manager through another one, having to gather the
information afterwards, unable to obtain it in an atomic way.

What, exactly, would be the benefit of such a situation for systemd
users?
Post by Steve Langasek
Post by Josselin Mouette
Let’s say that GNOME migrates to systemd user sessions, like what is
planned for GNOME 2.12 (yes, the version we intend to ship in jessie,
ain’t that sweet).
"It's important to note that with these patches, we still support
non-systemd systems (as well as older systemd). How far into the future we
do so is an open question, but it should not be too difficult to leave
non-systemd systems with the previous model over the next few cycles."
Oh but of course we can cripple our packages by disabling systemd
support and live without user sessions. We can always do that. That was
exactly my point.
Post by Steve Langasek
There are
advantages to using systemd for user session management, particularly when
coupled with Wayland...
And maybe after upstart, you intend to try forcing Mir upon us when
GNOME packages introduce a dependency on Wayland?
Post by Steve Langasek
but these can just as well be delivered on top of
upstart rather than systemd. It does not follow that GNOME upstream should
dictate to Debian that they adopt systemd rather than upstart.
You’re acting like a good salesperson, but I’m afraid your product is
inferior. These advantages cannot be provided by upstart. Just like for
system services, the available features for user sessions are not
comparable.
Post by Steve Langasek
I think the current Debian GNOME team has a not-undeserved reputation for
being obstructionist with respect to bugfixes that require divergence from
upstream's stated direction.
This criticism is totally undeserved. The GNOME team is involved with
GNOME upstream and understands where that stated direction comes from.
And usually agrees with it, but not every time: we have maintained large
sets of patches in Debian because of disagreements with the way
PulseAudio and GDM were managed upstream.

It shouldn’t come as a surprise that it is hard for developers to
respect the TC’s decisions when we see disrespectful sentences like the
one above from some of its members.

It strikes me that in general, the init system discussion has been
polluted by hateful comments directed towards the GNOME, freedesktop and
systemd projects and the Debian developers involved, by people who don’t
even try to understand the technicalities behind. Too many people have
let this happen, and too many developers have been hurt and demotivated.
Do not expect a TC decision that would rely on political arguments
instead of technical ones to improve that situation.
Post by Steve Langasek
If the team demonstrated they were open to
contributions of the kind you described, volunteers to do the work would not
be hard to come by.
You don’t just need volunteers to dig patches in the Ubuntu patch
tracker and email them. You need volunteers to maintain the resulting
packages.

In other words, if you disagree with the ways the GNOME team maintains
their packages, and appeal to the TC in order to change these ways,
you’d better discuss with all current maintainers whether they are ready
to abide by your rules, and have a proposal for what to put in the
Uploaders: field of the resulting packages. I am pretty sure the same
holds for a systemd package that would be required to remain at version
204, stripped of stuff that doesn’t work with upstart.
--
.''`. Josselin Mouette
: :' :
`. `'
`-
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Bdale Garbee
2014-01-02 16:50:58 UTC
Permalink
It shouldn’t come as a surprise that it is hard for developers to
respect the TC’s decisions when we see disrespectful sentences like the
one above from some of its members.
I agree.

We are of course each entitled to hold opinions about such things, but I
would strongly prefer if we could all try *very* hard to keep such
assertions out of TC discussion. They have no place here.

Bdale
Steve Langasek
2014-01-02 22:27:14 UTC
Permalink
Post by Bdale Garbee
It shouldn’t come as a surprise that it is hard for developers to
respect the TC’s decisions when we see disrespectful sentences like the
one above from some of its members.
I agree.
We are of course each entitled to hold opinions about such things, but I
would strongly prefer if we could all try *very* hard to keep such
assertions out of TC discussion. They have no place here.
I recognize that we as TC members have a moral duty to not gratuitously
demotivate Debian contributors. However, the duty to not alienate
contributors is secondary to our duty of defending the technical integrity
of Debian, and so I stand behind that comment and am going to elaborate with
reference to an example so that the other members of the TC, and the GNOME
team, understand exactly where I'm coming from. (The example is from a
question that was referred to the TC in July 2012 - bug #681687 - so it may
ring a bell for some here.)

For several years the GNOME Team ignored section 9.7 of Policy, concerning
integration with the MIME handling system. They did this in favor of
implementing the related freedesktop.org on the grounds that the fd.o
standard is technically superior (and less work, since it was already
implemented upstream).

As it happens, I think the fd.o standard *is* technically superior (and I
think any other member of the TC who looked at the question would agree).
However, "my way is technically superior" is not a valid justification for
ignoring Debian Policy. Policy is not *just* about being technically
better, it's *also* about having consistent behavior that all packages in
the archive can coordinate around. No single upstream, no matter how large
or prominent they might be, has any business dictating behavior that
contradicts Debian Policy; Debian exists as a distribution to provide a
coherent, integrated OS, not to deliver half a dozen incompatible upstream
experiences to our users, and when Policy needs to be changed it needs to be
changed with transition handling in mind.

Each Debian maintainer has an obligation to ensure their packages comply
with Debian Policy, regardless of what direction upstream is headed in.
Sometimes this means writing compatibility code that's Debian specific;
sometimes it means getting Policy changed so that packages have new, better
rules guiding their integration. Never does it mean silently ignoring the
issue as The Other Maintainer's Problem.

In case anyone missed it at the time,
https://lists.debian.org/debian-devel/2012/01/msg00830.html is the start of
a very long thread on this topic two years ago, when it came to the
attention of the wider project that Josselin had dropped support for mailcap
from evince, the single pdf reader that many Debian users had installed on
their desktop. (Other packages, such as the eog image viewer, had dropped
their integration long before, but with a lower impact than evince.)

What struck me in that discussion is that at no point did the GNOME
maintainers raise this issue on debian-devel or debian-policy to ask for
help with this integration problem. Instead, they uploaded breakage to the
archive and waited for users to trip over it, apparently in the belief that
if no one had provided a fix by now - /for the bug they had not asked the
developer community for help with/ - that such an automated system was not
important enough to worry about complying with policy for.
(http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=658139#29)

Ultimately, bug #497779 and bug #658139 were resolved by someone
volunteering, in response to that thread, to implement an automated tool for
merging .desktop files into mailcap for backwards-compatibility. This was
the desirable outcome all along; however, it happened in spite of the GNOME
Team, not because of them, and after a fair amount of good will had been
spent on both sides in the list discussions. I completely understand the
GNOME Team not having time to write (or maintain) the tool to do this
automated conversion. What I don't find acceptable is their not bringing
this issue to the project's attention /and asking for help/.

Maybe 'obstructionist' is not the right word. But the GNOME Team has a
pattern of failing to engage constructively with the rest of the project
around crucial integration issues. Josselin claims that a comment from a TC
member calling this out makes it difficult for developers to respect TC
decisions. I counter that the GNOME Team's past handling of such problems
shows an existing lack of respect for project values and procedures, and I'm
merely giving voice to a view widely held among Debian developers who would
in fact be more than happy to contribute to improving GNOME's integration in
Debian, if only they believed this help would be welcomed by the current
package maintainers.

While I stop short of calling for the formal censure of the Debian GNOME
Team as Ian has in the past, I think the GNOME Team should take a hard look
at how their own actions have contributed to any sense they might have that
they lack the resources to comply with Policy. I don't think it's a
coincidence that over the past two years, over a quarter of all the issues
decided by the TC have related to GNOME packages.


To reconnect this with the actual point of this subthread: it is entirely
possible that the TC will decide that systemd is the technically better
solution for Debian to move forward with; and any member who thinks this
should certainly vote accordingly. But if the members of the TC do
*not* think this is true - if, indeed, our collective preference is for
upstart rather than for systemd - then I don't think we should be swayed by
assertions that GNOME upstream is tethering itself to a specific init system
and that the current GNOME maintainers may force the issue by uploading
packages to the archive that have a hard dependency on systemd as PID1.
That's nothing more than hostage taking, especially when there would be no
difficulty getting cycles for the integration bugs with GNOME and whatever
init system Debian standardized on... *provided that* the GNOME maintainers
showed themselves open to this work instead of blocking it. From Joss's
reply to my message, it seems altogether too likely that he *would* block
such work. But that is a bridge we should cross when we come to it, not
something we should allow to drive the future course of Debian... nor
something we should beat the GNOME Team up about before it's actually
happened.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org
Josselin Mouette
2014-01-03 00:17:22 UTC
Permalink
Post by Steve Langasek
For several years the GNOME Team ignored section 9.7 of Policy, concerning
integration with the MIME handling system. They did this in favor of
implementing the related freedesktop.org on the grounds that the fd.o
standard is technically superior (and less work, since it was already
implemented upstream).
[snip]
Post by Steve Langasek
What struck me in that discussion is that at no point did the GNOME
maintainers raise this issue on debian-devel or debian-policy to ask for
help with this integration problem.
You forgot to mention that the actual bug at hand affected only a small,
although quite vocal, number of users – vocal users with a lot of time
to spend on debian-devel (and now debian-ctte) being a recurrent issue
in Debian nowadays.

The maintainer for mime-support had been aware of the problem for more
than three years without any change happening. There was a failure of
Debian as a whole to have let this part of the policy rot for such a
long time, and I’ll admit to my share of responsibility in letting that
happen, but certainly not to the whole of it.

I still stand on the opinion that, after such a long time, aggressive
removal of legacy MIME files was a right course of action.
Post by Steve Langasek
I'm
merely giving voice to a view widely held among Debian developers who would
in fact be more than happy to contribute to improving GNOME's integration in
Debian, if only they believed this help would be welcomed by the current
package maintainers.
Vague, unsubstantiated, false claims. Again.

I do not recall the members of the team rejecting the help proposed by
other contributors, apart from a handful of people who obviously failed
to meet the technical standards to contribute to any Debian package. If
there are any developers who would be happy to contribute to GNOME
packaging but are afraid their help will be refused, any member of the
team will be happy to soothe their fears. We have always been inclusive,
since, as a former DPL said, “what can’t you undo?”

Anyway, I have serious doubts about your allegation that manpower issues
are related to the current team members, unless you want to extend this
criticism to most core packaging teams. I might have to remind you that
the kernel, glibc, KDE, GNOME, Xfce and Xorg maintainers have all
repeatedly and publicly stated their lack of contributors and difficulty
to handle bug reports.
Post by Steve Langasek
I don't think it's a
coincidence that over the past two years, over a quarter of all the issues
decided by the TC have related to GNOME packages.
“Over a quarter” being three issues, two of them being the same. And
let’s not mention some TC member’s behavior regarding the handling of
that one, shall we?
Post by Steve Langasek
That's nothing more than hostage taking, especially when there would be no
difficulty getting cycles for the integration bugs with GNOME and whatever
init system Debian standardized on... *provided that* the GNOME maintainers
showed themselves open to this work instead of blocking it. From Joss's
reply to my message, it seems altogether too likely that he *would* block
such work.
This is not what I wrote. I implied that I would not contribute to it in
any way. I know this is the point of view of some other members of the
Debian GNOME team, but maybe not all of them. You’d have to ask them
individually.

Given the general tone of your message, I might have to remind you that
I am not the GNOME team, especially since I have not been providing much
packaging help during the last months. The reason why I’m the one doing
most of the talking is that other people have been so disinterested,
demotivated, or even disgusted, by the confrontational tone of any
public discussion about GNOME, freedesktop or systemd that they don’t
even want to talk about it anymore. Therefore, you should not think I am
the most likely person to block such changes or abandon the ship while
it is sinking.

This kind of attitude is making Debian the fun topic to talk about among
upstreams, not the major Linux player we should be. Debian as a whole
needs to rethink how it can be more friendly to some important upstream
projects, or we will simply stop being the “universal” operating system.
--
.''`. Josselin Mouette
: :' :
`. `'
`-
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Bill Allombert
2014-02-25 22:40:14 UTC
Permalink
Furthermore, I think the idea of an application "belonging" to one system or
the other is misplaced. The purpose of both the original menu system and
the freedesktop standard is to give users consistent, menu-driven access to
the software installed on the computer. While a given desktop environment
is going to give precedence to software that is integrated with that desktop
environment, users should be able to expect that they can access all
software installed on the system through the GUI, via the appropriate
submenus. Telling maintainers to integrate with one of two different menu
systems does not achieve this.
So if the Debian menu system is insufficiently expressive to meet our needs,
we should be giving clear advice for the use of the fdo menu system.
This is backward: the Debian menu system is more expressive than the fdo menu
system and easier to support by window manager by requiring much less software
support (in particular XDG menu requires XML processing).

Debian menu is supported by much more window managers than the XDG menu draft.

I do not believe Debian should favor desktop environment over lightweight
window managers, especially since Debian users are more likely than other
to favor use lightweight environments.

Cheers,
--
Bill. <***@debian.org>

Imagine a large red swirl here.
Paul Wise
2014-02-26 05:47:01 UTC
Permalink
Post by Bill Allombert
Debian menu is supported by much more window managers than the XDG menu draft.
This issue is addressed by the xdg-menu system from Arch Linux.

https://wiki.archlinux.org/index.php/Xdg-menu

For the awesome window manager there is also a specific addon.

https://github.com/terceiro/awesome-freedesktop
--
bye,
pabs

http://wiki.debian.org/PaulWise
Steve Langasek
2013-12-31 19:05:12 UTC
Permalink
Post by Ansgar Burchardt
And if upstart wants to use parts of systemd, why shouldn't the upstart
maintainer do the work for this? Or they could fork logind which they
suggested before... This would also allow having a newer systemd in
Debian.
upstart requires no part of systemd (unless you count udev as "systemd"
now). It's only the desktop environments which have dependencies on these
dbus interfaces.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org
Uoti Urpala
2013-12-30 23:44:49 UTC
Permalink
Post by Ian Jackson
Also, I get the impression me that the "integration" of much of this
functionality into the systemd source package has been done for
political rather than technical reasons. Indeed to the extent that
there is a problematically tight technical coupling between these
components, I find it difficult to avoid doubting the good faith of
the people making those decisions. At the very least, I worry that
the systemd project as a whole is far too willing to impose decisions
of all kinds on its downstreams.
Your own expressed preference for upstart appeared to be very much
driven by political rather than technical considerations. Using the same
terminology you do, would it not be entirely fair to say that your
decision to support upstart was made in bad faith?
Post by Ian Jackson
Post by Russ Allbery
3.3. Project Momentum
I don't see the outlook here the same way as you do.
Furthermore, I am much less worried about Debian going it alone
(although, of course, it's not alone) than you seem to be. We have
historically been entirely unafraid to do our own better things, even
if it is more work and takes us longer.
I felt that way when Debian was very much a minority interest. That's
far from the case nowadays; I've heard it asserted that Debian-based
distros now account for a majority of traditional distro installs. I
guess numbers on that are all speculative but it is certainly true
that we have a thriving ecosystem of our own.
We have got where we are by doing things the way we think is best, not
by simply following the herd.
Who would actually do the work? Getting the amount of development there
is for systemd is not easy. Do you really believe that a Debian decision
in favor of upstart would create that many interested developers working
on it, when that has not happened while it's been used in Ubuntu?

Working on things that they believe to be better than existing ones does
motivate people. But how many Debian developers actually share your
extreme views about portability to the extent that they would be happy
to work on another system motivated by that, when systemd already works
better on Linux? I doubt that group is large enough to create
significant momentum.
Russ Allbery
2013-12-30 21:44:10 UTC
Permalink
This message contains some supplemental information to go with my primary
writeup, and some profound thanks for the people involved in this
investigation.

I apologize for the huge volume of mail, and I know it's going to take a
while to digest. I appreciate people's willingness to read all these
messages in detail. This is a very complex decision-making process. I'm
pretty mentally exhausted by the effort of trying to synthesize all these
pieces, so my apologies in advance for the inevitable errors and omissions
(some of which affected my original writeup).


1. Thanks

Throughout this evaluation process, my interactions with upstart and
systemd upstream developers and Debian packagers have been uniformly
excellent. Bug reports filed against both systemd and upstart have
received excellent and timely response, and all involved have been quite
willing to explain things I've misunderstood, correct my false starts, and
discuss technical and practical aspects of their designs.

I was particularly impressed by the clear effort that the systemd and
upstart maintainers in Debian have put into fully integrating their init
systems in such a way that makes them easy to test and use with existing
Debian packages. This includes but is not limited to update-rc.d support,
invoke-rc.d support, status synchronization with sysvinit, past Policy
discussion, and attention to upgrade paths and init-switching use cases.

I also want to particularly thank the OpenRC upstream development team for
their involvement in this process and their contributions to the
discussion. I personally don't think that package is a god match for
Debian's needs on Linux, but that's through no fault of the people
involved, and I think they would be an excellent upstream if that package
looked like a good fit for the needs of any of Debian's non-Linux ports.

I also want to thank Petter Reinholdtsen, Roger Leigh, and everyone else
who has worked on the sysvinit package over the years, the insserv
conversion to dependency-based boot, and the inclusion of LSB support. If
it weren't for their hard work over the years, we would be in a far worse
position than we are today. It's often hard to see people discussing the
inadequacies of something into which you put years of hard work. I want
to call attention to their long-term contributions to the distribution and
the number of Debian systems that have booted through their efforts
through the years.

I am carefully keeping this discussion to the Technical Committee bug
report so that all of my comments stay in context with their rebuttals,
but this one section I think is unrelated to the rest of the discussion
and should be more widely posted, so I will also be posting the above to
my blog and Planet Debian.


2. upstart vs. systemd: The Equivalencies

In doing this evaluation, I looked at quite a few different things. In a
great number of cases, I found both upstart and systemd to be at an
equally high level of quality. There are too many of those to list here,
but I wanted to mention a few points that had been the topic of wider
discussion.

* The documentation of both systems was uniformly excellent. Both systems
had complete reference manuals plus guidance to packagers for how to
integrate with them. systemd also had excellent guidance for upstream
developers on how to add systemd support relevant to upstream packages.

* Both packages preserve traditional logging behavior, continue to
properly utilize syslog, and retain logs in the places where I expected
them. This is obviously not surprising for upstart, but I wanted to
mention it explicitly since those unfamiliar with the systemd journal
may be afraid that it would migrate logs into a separate, unexpected
store. This is not the case; systemd logs are available in the normal
places and can continue to be managed through syslog configuration. The
journal is supplementary and enables some additional features discussed
elsewhere.

* Both packages move to configuration, from code, many of the annoyances
and fiddly pieces involved in starting daemons. Both support
non-forking daemon models and proper PID tracking in their own ways.
Both directly support such routine needs as setting user and group, OOM
score adjustment, resource limits, and so forth.

* Both packages provide MAC integration, which is something that I think
will become more important for Debian in the future. (systemd provides
some additional features around Smack, but at least given what I know
right now, I don't think this is a significant differentiation.)

There was another point that I am calling equivalent since I didn't
evaluate it either way. It's possible that this would convert into an
advantage for one side or the other after further investigation:

* Both systemd and upstart provide a way to run the system as a user
process to manage user jobs in a similar fashion to how they manage
system jobs. I believe this is already used extensively in Ubuntu to
manage user desktop sessions. I don't where the systemd equivalent has
been used.

Finally, my interaction with both upstreams has been excellent, as I
mentioned above in the Thanks section. I believe both maintenance teams
would make excellent partners and upstreams for Debian going forward.


3. upstart: The Minor Advantages

The following points did not make the cut into my main analysis. I
consider them minor advantages that don't carry the same weight as the
things that I commented on. However, I did want to mention some that had
been the topic of discussion.

* The upstart and libnih code bases are beautiful pieces of work. I was
quite impressed by the code quality and documentation level, and more
impressed by the extensive test suite. upstart and libnih follow the
gold standard, commonly advocated but rarely met, of having around half
of the code in the package be test cases. These are clearly code bases
that have seen a great deal of love, care, and consistent development
standards. The systemd code base also struck me as solid and at the
standard that we would expect for a critical package, but the testing
model is not as comprehensive or as integrated with the code, and it
didn't impress me the same way that the upstart code did.

* Both upstart and systemd are used by other major distribution projects,
but upstart is used by Ubuntu, with which Debian has a special
relationship. Debian collaborates more extensively with Ubuntu than
with any other project, including largely sharing packaging between the
projects and benefiting greatly from each other's testing and
integrations. With upstart, Debian would have the immediate use of many
upstart configurations that are either already in the archive or already
available in Ubuntu, and which already fit Debian packaging standards
and tools. I consider this a minor advantage for upstart.

* upstart provides a more straightforward escape to shell scripting for
some difficult initialization situations, whereas the systemd syntax for
doing so is rather awkward. This cuts both ways, so you'll find that
this point makes an appearance in the section below as well. But I
think that, in some situations, this is a readability and ease of
integration advantage for upstart.


4. systemd: The Minor Advantages

Similarly, during this evaluation, I found several things that I preferred
in systemd, but which didn't make the significance cut in my final
evaluation. Here are some of them for the record.

* systemd does not require a CLA, whereas upstart does. The Canonical
Contributor Licensing Agreement has been much-discussed in these
threads, and some people find it quite intrusive and unacceptable. The
upstart package maintainers have committed to carrying non-CLA-covered
contributions as local patches, which does a lot to ameliorate this
concern, but I still consider this a minor advantage for systemd.

* systemd provides really nice command-line tools for understanding the
state of the system and the relationships between the unit files. I
don't believe upstart has an equivalent of systemctl list-dependencies,
for example. (systemctl status got special mention in my main
evaluation.)

* Both upstart and systemd, due to their more declarative nature, allow
for daemon configurations that are more portable between different
systems running the same init system. systemd has taken this farther
into comprehensive and useful advice to upstream daemon authors for how
to incorporate systemd support into a package, and also provides
step-by-step instructions for how to install systemd unit files during
package installation. While I don't believe that full portability of
unit files will be achievable, I still think Debian would benefit from
this when integrating systemd, as upstream unit files will often be
usable as-is or with some minor patching.

* systemd puts a lot of work into providing all the configuration
directives required to express a huge variety of use cases without
having to write complex startup commands or escape out to shell. This
is the flipside, in some ways, to upstart's easy escape to shell. I
particularly noted the wide variety of Condition* settings to control
whether a unit should start. This is valuable for distribution
packaging cases, but I think it's even more valuable for users of
systemd-controlled systems in setting up local overrides and conditions
for when they want to run a particular service.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Steve Langasek
2013-12-30 23:42:47 UTC
Permalink
Post by Russ Allbery
* systemd provides really nice command-line tools for understanding the
state of the system and the relationships between the unit files. I
don't believe upstart has an equivalent of systemctl list-dependencies,
for example.
I think the relevant tools on the upstart side are 'initctl show-config -e'
and 'initctl2dot'. This doesn't render in a tree format the way 'systemctl
list-dependencies' does, because job relationships are a DAG, not a tree;
but I can see there being room for a simplified view of jobs that works from
a terminal.

Cheers,
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
***@ubuntu.com ***@debian.org
Russ Allbery
2014-01-02 17:25:46 UTC
Permalink
As I mentioned in some of my previous notes, I was unable to evaluate
OpenRC in a meaningful way during my general experiments for a few
reasons. My impression was formed based on previous discussion and what
documentation I could find, which was fairly minimal.

Thomas Goirand sent me considerably more information, including some
details about OpenRC project goals that corrects information in my
original writeup. With his permission, I'm including that here for the
benefit of everyone else who is following this debate.

Thomas, please follow up with anything that I left out or anything else
that you think people should be aware of.
Post by Russ Allbery
If we're going to the effort of
replacing init systems and changing our startup scripts, a bare
minimum requirement for me is that we at least address the known
* Lack of integration with kernel-level events to properly order startup.
* No mechanism for process monitoring and restarting beyond inittab.
* Heavy reliance on shell scripting rather than declarative syntax.
* A fork and exit with PID file model for daemon startup.
My impression of OpenRC is that it is not attempting to solve these
issues in the same way that systemd and upstart are.
I'm not sure how I should take this. Yes, OpenRC isn't doing "the same
way", though it's addressing all of the above points. Did you really try
OpenRC?
FYI, OpenRC does understand and use cgroups. So that's for the last
point above. OpenRC does use runscript so this address the "reliance on
shell scripting rather than declarative syntax". OpenRC is currently
implementing the use of "monit" so it can monitor and restart processes,
without reinventing the wheel (monit has been around for years...). As
much as I know, that part is nearly done, though it needs to have a few
issues solved (that's what I heard from upstream). I think this is an
awesome approach to use monit. If you haven't used monit, I would
recommend you to try it. It's modular and checks for the actual service
to be running rather than a process running.
The only thing which it might lack support of would be kernel-level
events, but I think this is really overrated in the current discussion,
and also it can be handled most of the time with some very easy to
implement loops (there's some examples in some of the Gentoo runscripts).
I think you should re-evaluate OpenRC. Yes, it's not as big as Upstart
and Systemd (and for me, that's a *good* point, not a bad point), though
it's not as bad as you describe it.
If the tech committee was to decide to switch to systemd or upstart, I
think one of the key decision that could come with it would be to
deprecate sysv-rc in the favor of OpenRC (with or without mandatory
support in linux versions of Debian, that's up to the tech-ctte to
decide...).
FYI, I have just uploaded a new version of OpenRC to the new queue. It's
IMO back into shape again after fixing the /sbin/rc and /sbin/runscript
renames, though there's still some corner cases to fix, like "apt-get
install --reinstall initscripts" fails because there's no runlevels in
the LSB headers of these scripts (I don't understand why yet though). I
don't think it'd be hard to fix that one though...
git clone ssh://git.debian.org/git/collab-maint/openrc.git
cd openrc
./debian/rules make-orig-file
git-buildpackage
Then simply dpkg -i the resulting libs and openrc itself. This should
normally work on both Wheezy and Sid, and in all ports (I tested it on
kFreeBSD and amd64).
A quick run-through now. Once you've installed it, it wont know about
started stuff, so the first reboot will not shutdown correctly. I'm not
really sure how to fix the first reboot case currently. Though after the
first reboot, you can try "rc-status" which is a nice stuff.
I couldn't change "service" utility since it's owned by sysv-utils and
not OpenRC, though I have updated invoke-rc.d so that it continues to
update the symlinks in /run/openrc/started (so rc-status will work as
expected). Unless the sheebang is changed from /bin/sh to
/sbin/openrc-run in scripts in /etc/init.d, starting the init scripts
"by hand" with /etc/init.d/<whatever> start/stop will not update the
status either. All this is fine since everything should be using
update-rc.d and invoke-rc.d, it's just nice to other ways to start/stop
daemons work with rc-status support as well for our users.
I would recommend that you also have a look into /etc/rc.conf, and play
a bit with the options. Note that "rc_parrallel" does work, even though
it has an ugly output.
--
Russ Allbery (***@debian.org) <http://www.eyrie.org/~eagle/>
Lisandro Damián Nicanor Pérez Meyer
2014-02-26 00:32:17 UTC
Permalink
On Tuesday 25 February 2014 23:40:14 Bill Allombert wrote:
[snip]
Post by Bill Allombert
I do not believe Debian should favor desktop environment over lightweight
window managers, especially since Debian users are more likely than other
to favor use lightweight environments.
If you are going to do that statement, please add the supporting data for it.
--
6: Cual es el boton del mouse que permite acceder a las acciones mas
comunes del manejo de archivos
* Depende el tipo de accion mas comun
Damian Nadales
http://mx.grulic.org.ar/lurker/message/20080307.141449.a70fb2fc.es.html

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/
Josh Triplett
2013-12-31 05:10:53 UTC
Permalink
Post by Steve Langasek
Post by Russ Allbery
Please recall the context here: this whole aside started with an objection
to my contention that adopting upstart requires disassembly and redoing of
an integration that we would otherwise not have to disassemble. Nowhere
in my message did I say that we would or could not do that disassembly if
we adopted upstart; I said that it was work that we otherwise wouldn't
have to do.
That's the intended context of my point above: I don't think we're going
to port GNOME to a non-systemd infrastructure, in the sense of carrying
significant patches to GNOME to adopt it to, say, not using logind. I
think GNOME will continue to use systemd APIs heavily, which makes GNOME
less portable. That means that systems that are not capable of running
those systemd components will need to either port them or develop
alternatives.
I don't consider this wailing or gnashing of teeth, but rather a realistic
look at what efforts the project is talking about committing to, as
opposed to supporting people working on if they so choose.
Ok. I think our core point of disagreement here, then, is in our assessment
of how much work we think this actually is (for the Linux case, not for the
non-Linux case). I think the actual package maintenance to make this happen
is not even a weekend's worth of free time, and therefore represents a
negligible committment of resources on Debian's part, given that this
dissassembly/integration has already been done in Ubuntu.
I'm making the assumption, here, that the work you're talking about is
making logind and other such services run without systemd, rather than
attempting to make GNOME and other desktop environments run without
those services.

I think you're underestimating the amount of *ongoing* effort required
here. I'd point out that systemd in Debian is still stuck at version
204, despite the very nice features available in 205 and newer,
specifically because logind dared to make use of those features. I
fully expect systemd to continue producing new and interesting features
and *using* them, requiring alternative implementations to either
reimplement more of systemd or create an increasingly invasive fork of
it. And while I think it's *possible* to continue doing so on an
ongoing basis, that's work that could be spent on other productive tasks
that don't involve reimplementation.

In any case, I sincerely hope that the cost of doing that work is borne
entirely by people who find it a worthwhile activity rather than a
monumental waste of time. And I furthermore hope that an unmangled and
unforked version of systemd continues to be available in Debian for
folks who want to run the init system that continues to create
functionality so useful that the proponents of upstart are willing to do
a huge amount of work in order to adopt most of it other than the init
system itself.

- Josh Triplett
Ansgar Burchardt
2013-12-31 14:26:26 UTC
Permalink
Package: upstart

Upstart should provide additional security features, such as isolating
processes from the network using network namespaces, restricting file
system access, preventing the processes to regain privileges, system
call filters, private /tmp and limiting device access.
Post by Russ Allbery
* Security defense in depth. Both upstart and systemd support the basics
(setting the user and group, process limits, and so forth). However,
systemd adds a multitude of additional defense in depth features,
ranging from capability limits to private namespaces or the ability to
deny a job access to the network. This is just a simple matter of
programming on the upstart side, but it still contributes to the general
feature deficit; the capabilities in systemd exist today. I'm sure I'm
not the only systems administrator who is expecting security features
and this sort of defense in depth to become increasingly important over
the next few years.
Here again, I think we have an opportunity for Debian to be more
innovative and forward-looking in what we attempt to accomplish in the
archive by adopting frameworks that let us incorporate the principles of
least privilege and defense in depth into our standard daemon
configurations.
Ansgar
Dmitry Yu Okunev
2013-12-31 14:19:36 UTC
Permalink
Hello.

I'm writing to you to request to do not use no systemd, nor upstart.

I can guess that you have very important reasons to discuss this
possibilities, but…

IMHO, "systemd" doesn't even fit to UNIX philosophy [1].

[1] http://en.wikipedia.org/wiki/Unix_philosophy

Sorry if I'm wrong.

"upstart" is more satisfactorily. I have a lot of problems with it in
LXC and other tasks*. So, I can work with "upstart", but would prefer
not to do that. I didn't try "systemd" in LXC, but I can guess that I'll
catch much more problems with it.

* I don't rule out the likelihood of that my hands were too curves.

Also, I can also guess, that my opinion doesn't worth anything here. But
at the moment the best Linux (meta-)distributions for me (and under my
tasks) is Debian and Gentoo. With moving to "systemd" you will force me
to choose another distribution as replacement for Debian.

Please, don't do that.


P.S.: It would be great to see "OpenRC" as default init-system for Debian :)
--
Best regards, Dmitry,
head of UNIX-tech department NRNU MEPhI,
tel. 8 (495) 788-56-99, add. 8255
Continue reading on narkive:
Loading...