Bug#727708: loose ends for init system decision

[Ian Jackson]

We have been asked to decide the default init system for jessie. As I
have just said, my conclusion is that we should choose upstart.

However, we also need to decide whether we intend to allow users to
choose otherwise. So we need to say what we expect of package
maintainers in terms of support for other init system(s).

It seems to me that the benefits of upstart integration are such that
we should encourage transition to upstart jobs, subject to my concerns
about the need for a new readiness protocol.


I am not comfortable with the idea of /mandating/ use of upstart. We
may discover that despite our adoption, upstart loses the war, or we
may find that we want to switch to openrc or another contender. And,
given the strong opinions on this topic, I think it is at the very
least premature to tell our users and downstreams that they are on
their own if they want to use systemd.

At the moment, there is no meaningful compatibility layer between all
these systems other than sysvinit scripts. This is unfortunate, but
given the current state of development of both systems I think this is
to be expected.

So I'm sorry to say that I think we need to continue to maintain
sysvinit scripts in jessie. The alternative would be to permit a
package to drop sysvinit support if both systemd and upstart
configuration were provided, but I'm not happy at this stage to burn
our bridges like that.

It would be worth researching some kind of compatibility options.
Perhaps simple daemons with very formulaic upstart job files could
have synthetic systemd units created. Or something.


I think we should give package maintainers some guidance on what kinds
of upstart or systemd patches should be accepted. Without this, it's
likely we'll find ourselves adjudicating disputes that ought to have
been settled in principle much earlier.

I think that patches for either should:
- use a non-forking startup method
- avoid significant amounts of open-coded AF_UNIX socketry
- avoid embedded copies of daemon library code
- avoid extra build-dependencies (eg on libsystemd)
- avoid extra runtime dependencies (eg on deb-systemd-helper)
(If the current state of the art in readiness protocols persists that
means that upstart patches using raise(SIGSTOP) ought to be accepted,
but systemd ones rejected unless they can use socket activation.)

We should recommend:
- daemon authors and Debian maintainers should prefer command-line
options to environment variables
- whatever environment variables and fds are used, measures should be
taken to avoid them being inherited by daemons' arbitrary children

IMO we should treat native non-forking upstart support as we have in
the past done for release goals, ie with a low threshold NMU policy of
some kind.


But as I say I would like to see us try to get agreement or
near-agreement on an improved startup protocol. I would suggest that
we allow (say) a month for this process, starting now. Before this is
settled we shouldn't go around converting a whole bunch of daemons.


And, there are two loose ends about systemd in particular:
- There doesn't seem to be any Debian policy about how to
provide systemd units in Debian;
- systemd upstream and the Debian maintainers like to put much of the
configuration in /lib; this is controversial and someone needs to
decide whether this is appropriate.

Ian.

[Russ Allbery]

This message is about a transition plan for an init system replacement and
about how to handle portability to our non-Linux ports. I'm keeping the
same subject as Ian's message on the same basic topics and attaching it to
the same thread, but this is more of a separate writeup than a reply.
I'll reply to Ian's message separately.

I've expressed my opinion separately about which init system we should
choose. This message is written from the assumption that we will choose
either upstart or systemd as the init system for the non-Linux ports. I
believe either system would pose basically the same concerns.


1. Role of Non-Linux Ports in Debian

There has been a lot of discussion in various places, including the Debian
mailing lists, about the utility of the non-Linux ports and about how much
they should play a role in project decisions. I think this deserves to be
addressed directly here.

One of the things that I love about Debian, and one of the reasons why I
am involved in this project as opposed to a different distribution, is
that it is a labor of love. Debian is not driven by market forces; we
have users and we want the distribution to work for them, but we're not
competing for market share. Debian does not make decisions based on
simple popularity, or on customer counts. Rather, Debian is a project of
mutual cooperation among Debian's contributors. We work on what we care
about, regardless of whether that makes economic sense.

It is not, in general, necessary to justify what you want to do in Debian.
It doesn't matter if it's going to be used by thousands of people or two
people. If you can do your work to the standards that we expect to be
maintained across the archive, and without negative impact on Debian's
other contributors, you can work on whatever you love. And, furthermore,
we all support each other in our passions. Debian is built on a culture
of deference to other people's work, and reasonable accomodation to the
projects that other people want to work on.

Now, there is a fine balance here. Deference to other people's work does
not mean a requirement to join their work. Reasonable accomodation does
not mean that every Debian developer is required to test their software on
non-Linux ports. The goal is that all of us should be empowered to work
on the things we are passionate about, which implicitly includes being
empowered to not work on the things that are not of interest to us.
Therefore, for some efforts, and specifically for the non-Linux port
efforts, the work is mostly born by the porters. They're expected to
test, diagnose problems, and submit patches. The deference and reasonable
accomodation that we expect of Debian contributors is to merge those
patches when they are reasonable, to not undermine the porting effort when
there is a reasonable approach that preserves it, and to be aware of the
implications of their packaging for those ports in the broad sense (such
as qualifying build-dependencies with [linux-any] where appropriate).

We do not expect Debian contributors to reject significant new upstream
versions or significant new integrations because they will affect the
non-Linux ports, or, for that matter, other projects in Debian. We do
expect those changes to follow the standards that we expect of Debian as a
whole, and that porting efforts will be treated with deference and
reasonable accomodation.

I think this status applies to both our Hurd port and our kFreeBSD port.
Those ports have different challenges, and arguably different levels of
utility to general users, but as mentioned, I don't consider popularity to
be a useful metric here. It doesn't make any difference to me if the port
is used by thousands of people or if it's only used by the people actively
working on it: the same principles of deference and reasonable
accomodation apply.

I believe strongly that this is something that defines Debian as a project
and is worth preserving.

It's also worth saying here that I don't believe any of the above is
particularly controversial within the project. We have wide-ranging
arguments about it in the abstract, and quite a few disparaging comments
have been thrown around about the non-Linux ports in the init system
discussion, which makes me sad. But when it comes to the day-to-day work
in the project, nearly all maintainers are quite good (and have been for
years) about merging required patches for the Hurd and pushing them
upstream, responding to bug reports on systems they don't use, and making
a reasonable effort to accomodate other people's projects.

Similarly, our non-Linux porters have been exemplary. Neither the Hurd
nor the kFreeBSD ports have expected every Debian contributor to directly
support their platforms. They track down problems, develop patches, and
submit tested patches to the BTS. They have developed their own solutions
for packages that are not portable and worked out how to integrate them
with the archive.

Despite the often off-putting public arguments, and despite occasional
tensions and counterexamples, by and large we do a good job at this. And
we should be proud of that.

This is, of course, directly applicable to the init system discussion
since neither systemd nor upstart are currently portable to either of our
non-Linux ports. It will continue to be directly applicable to this
discussion even if upstart is ported to kFreeBSD until such time as it's
also ported to the Hurd (or the Hurd porting group decides they're no
longer interested in working on the port, but I don't expect that to
happen).


2. Impact of Multiple Init Systems

Obviously, the ideal situation for project-wide integration and support,
and for Debian's ability to make full use of the capabilities of new init
systems, is for all of Debian's ports to be running the same init system.
Attempting to support multiple init systems has several obvious drawbacks:

* Packages will either be limited by the functionality of the least
capable init system or will behave differently (and have to be
configured differently) on different ports.

* Nearly all Debian contributors are personally only going to be running
the default init system on the amd64 or i386 ports. It's not viable to
ask them to test all packages on non-Linux ports. This, plus the
distribution of our user base, means that the configuration for the
default init system is the one that will be tested, and configurations
for any other init system will tend to bitrot.

* Related, Debian contributors will normally not be in a position to test
patches for daemons for non-Linux ports, and will be entirely reliant on
contributed patches from porters or users of those ports, which means
the level of quality we can maintain for those configurations will be
lower.

* If we're going to support init system switching, we will have to
continue to externalize daemon configuration in files that are
compatible with the sysvinit system. Both systemd and upstart have
better ways to handle local configuration than /etc/default files, but
their methods are incompatible (since they involve overriding the
init-system-specific configuration files in their own ways). If we
fully adopt one of those init systems, we can largely drop the somewhat
clunky /etc/default machinery in favor of much simpler overrides or
direct conffile modifications.

* Upstreams may, over time, drop support for init systems that are rarely
used outside of Debian's non-Linux ports. I don't think this is likely
to happen soon, but I can anticipate a world in the future (particularly
if upstart adds support for the systemd socket activation protocol)
where some upstreams will strip support for fork and exit daemon startup
methods from their daemons, or require use of socket activation and
remove socket binding support. In some cases, we already have tools to
deal with this; handling daemons that don't support fork and exit is
fairly easy, for example. But in some cases we may not, and I think
maintaining full code for socket binding and setup as a patch that
upstream does not want is too much to ask from all Debian contributors.

However, there are also some counter-balancing points:

* All packages in Debian are already ported to sysvinit, which means that
the init scripts and related machinery already exist. Furthermore,
maintaining Debian's normal support for partial upgrades and loose
upgrade ordering between stable releases requires that we continue to
support sysvinit scripts at least through the release of jessie. In the
short term, we cannot avoid supporting two init systems if we're going
to provide robust upgrade support, which gives us short-term support of
the non-Linux ports for "free."

* Shell init scripts are obnoxious to write and often have buggy corner
cases, but once written, they mostly keep working in their buggy glory
with some minor maintenance. In other words, they don't do as good of a
job, but regressions are relatively uncommon. This means that it is
possibly viable for porters to provide patches and continued maintenance
of init scripts for at least the services that are considered most
critical even if the primary maintainer cannot easily test. Packages
with extremely complex init scripts will require someone do special
testing, but the number of such packages is relatively low. This will
fail if upstreams start dropping required support for sysvinit-style
startup, but I don't think this will happen soon.

* Maintaining init scripts is not, in the grand scheme of Debian work, one
of the harder things we do. I think asking Debian contributors to
maintain sysvinit scripts when Debian's default init system is something
else falls outside the boundary of reasonable accomodation, but that's
mostly because of the testing requirement. I do not think that leaving
init scripts in the package where they already exist and applying
patches from porters falls outside those boundaries.


3. systemd and upstart As Multiple Systems

I said a great deal above about Debian's non-Linux ports. It's worth
considering that the same statements potentially apply to multiple
next-generation init systems.

As we've seen from this debate, both upstart and systemd inspire
passionate loyalties and preferences. Having looked at both of them, I
fully understand the feeling. I have a strong personal preference for
systemd, but upstart is a beautiful piece of code that would be a delight
to work on. It's clean, well-documented, consistent, and has an excellent
test suite. Both projects are working hard at writing something they
believe in.

Given that, I don't believe a Technical Committee choice of a default init
system is going to make either the systemd or the upstart maintainers want
to stop maintaining their packages. For upstart, there is also the
ongoing fact that Ubuntu uses upstart, and Ubuntu is one of our major
downstreams and a collaborative project.

I therefore think that, regardless of which init system we pick, we should
keep in mind the above principle of Debian's deference and reasonable
accomodation to other people's projects and apply that to systemd and
upstart as well as to the non-Linux ports. Obviously, this also has the
same issues mentioned above: Debian contributors can only be expected to
test on the primary init system, other configurations will tend to bitrot
without active porter attention, and so forth. But if people want to take
on the work, that deserves our respect.


4. Conclusions

I previously argued that much of the benefit of a new init system comes
from when we can stop maintaining init scripts. I still believe that, but
after thinking more about the cultural and project issues at stake here,
as well as the immediate needs for a clean upgrade path, I ended up at
more of a compromise position than I expected.

I believe Debian should take this path forward:

1. We should select a new init system for jessie, either systemd or
upstart. Support for that init system should be release-critical, but
only in the sense that the daemon works properly under that init
system. In other words, use of the sysvinit compatibility of either
init system is acceptable support for jessie.

2. All packages providing init scripts must continue to support sysvinit
scripts through the jessie release. Such support will continue to be
release-critical. This is going to be painful for packages that want
to do an early conversion, since it means testing two different init
systems for this release cycle, but I think this is the right thing to
do regardless for a clean upgrade path and Debian's normal robust
committment to upgrades. Here it has the additional advantage of
giving the non-Linux ports some breathing space to strategize.

3. Related, up through the jessie release, packages must (where possible;
it's possible there will be cases where this is simply impossible)
support switching back and forth between the new default init system
and sysvinit. This means that configurations should not be moved out
of /etc/default and that startup files for the new init system should
read the same configuration that the existing sysvinit scripts use (or
both should be modified compatibly).

4. Post-jessie, support for sysvinit will no longer be release-critical,
and package maintainers will no longer be expected to ensure that it
continues working. However, for as long as Debian has accepted
non-Linux ports using a different init system, package maintainers
should continue to ship init scripts if they work and should apply
patches and other reasonable fixes from porters for those init scripts.
In other words, this should be treated the same as merging patches for
the Hurd to remove hard-coded constant assumptions: if the change is
reasonable and doesn't break Linux ports (and this should be fairly
easy to handle for nearly all cases with init scripts), the package
maintainer should merge it.

5. Support for the other init system that was not chosen should be handled
in the same fashion, should a team choose to pursue it. If we select
systemd, package maintainers should still be willing to merge
contributed upstart configuration, with the understanding that they
can't test it and any support is on a best-effort basis only.
Similarly, if we select upstart, package maintainers should be willing
to merge systemd unit files and to enable upstream systemd support
where requested and where it doesn't interfere with the operation of
the daemon under upstart, with the understanding that the package
maintainer is not committing to testing or directly supporting this
configuration.

6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use. The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use. I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.

7. After jessie, functionality between systems running the primary Linux
init system and other init systems (including non-Linux ports) should
be allowed to drift. In other words, there will be cases where
features will only be available with the primary init system. Possible
examples include security hardening, socket activation, automatic
daemon restarts, and so forth. Packagers are under no obligation to
port those features to other init systems, but should welcome and merge
patches that do so. After jessie, packagers will no longer be required
to preserve daemon configuration when the init system is switched, so
use of such facilities as modification of upstart configuration files
or systemd overrides may be used.

We should revisit this decision again after the jessie release in case the
situation has substantially changed.

This is all rather unsatisfying all around, I realize. Debian is going to
miss out on some opportunities to pursue new init system features
aggressively and consistently across the archive, particularly for the
jessie release where we're maintaining full sysvinit compatibility, if we
take this posture to both upgrade compatibility and deference to porting
efforts. And this is also going to increase the porting work required for
non-Linux ports substantially, since the init scripts will have to be
included in that and porting won't be as easily measured by buildd logs.

However, I think it's the best available approach that balances our ideals
as a project against the opportunities offered by a new init system. This
approach does permit full use of new init system features for jessie
except for eliminating /etc/default files (which I doubt we'd successfully
do quickly anyway), and opens up the full spectrum of use cases after
jessie. The cost is that packagers should merge contributed patches to
the init systems that they don't use. I don't think this is too much to
ask, nor do I think it will have serious effects on package complexity
based on my own experience configuring a package to work under all three
init systems I considered.

[Ian Jackson]

Post by Russ Allbery
1. Role of Non-Linux Ports in Debian

I agree with most of this.

Post by Russ Allbery
2. Impact of Multiple Init Systems

I don't want to do a blow-by-blow quote/rebuttal of this.

Post by Russ Allbery
3. systemd and upstart As Multiple Systems

..

Post by Russ Allbery
I therefore think that, regardless of which init system we pick, we should
keep in mind the above principle of Debian's deference and reasonable
accomodation to other people's projects and apply that to systemd and
upstart as well as to the non-Linux ports. Obviously, this also has the
same issues mentioned above: Debian contributors can only be expected to
test on the primary init system, other configurations will tend to bitrot
without active porter attention, and so forth. But if people want to take
on the work, that deserves our respect.

Right.

Post by Russ Allbery
4. Conclusions

...

Post by Russ Allbery
1. We should select a new init system for jessie, either systemd or
upstart. Support for that init system should be release-critical, but
only in the sense that the daemon works properly under that init
system. In other words, use of the sysvinit compatibility of either
init system is acceptable support for jessie.

I agree.

Post by Russ Allbery
2. All packages providing init scripts must continue to support sysvinit
scripts through the jessie release. Such support will continue to be
release-critical. This is going to be painful for packages that want
to do an early conversion, since it means testing two different init
systems for this release cycle, but I think this is the right thing to
do regardless for a clean upgrade path and Debian's normal robust
committment to upgrades. Here it has the additional advantage of
giving the non-Linux ports some breathing space to strategize.

Yes.

Post by Russ Allbery
3. Related, up through the jessie release, packages must (where possible;
it's possible there will be cases where this is simply impossible)
support switching back and forth between the new default init system
and sysvinit. This means that configurations should not be moved out
of /etc/default and that startup files for the new init system should
read the same configuration that the existing sysvinit scripts use (or
both should be modified compatibly).

Yes.

Post by Russ Allbery
4. Post-jessie, support for sysvinit will no longer be release-critical,
and package maintainers will no longer be expected to ensure that it
continues working. [...]
5. Support for the other init system that was not chosen should be handled
in the same fashion, should a team choose to pursue it. [...]

I think it is probably premature for us to drop support for the other
system in jessie+1.

But we don't need to make this decision now.

Post by Russ Allbery
6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use. The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use. I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.

Is there some difficulty expected with upstart on hurd ?

Post by Russ Allbery
We should revisit this decision again after the jessie release in case the
situation has substantially changed.

I would be very reluctant to write now that the support for sysvinit,
or the non-default new system, may be dropped in jessie+1. That will
result in premature rot and removal.

It's also possible that some kind of compatibility mechanism will
become available.

I would like to leave this decision to the policy maintainers, with
the expectation that the TC will probably need to decide.

Thanks,
Ian.

[Russ Allbery]

Post by Ian Jackson

Post by Russ Allbery
6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use. The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use. I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.

Is there some difficulty expected with upstart on hurd ?

I don't know if anyone has looked at it, and in the absence of any
practical experience, I think it's fair to expect some challenges. The
blog post on kFreeBSD porting indicated that the porting effort to date
required eglibc and FreeBSD kernel changes. It seems like a reasonable
assumption that at least a similar level of effort will be required on
Hurd, and I don't know if the Hurd has as mature (if incompatible)
capabilities such as kqueue on FreeBSD to use to implement such things as
inotify or epoll.

I know very little about the Hurd, so basically I just don't know.

Post by Ian Jackson
I would be very reluctant to write now that the support for sysvinit, or
the non-default new system, may be dropped in jessie+1. That will
result in premature rot and removal.
It's also possible that some kind of compatibility mechanism will become
available.

That's a reasonable point. I have no objections to deferring that
decision until after the jessie release.

Post by Ian Jackson
I would like to leave this decision to the policy maintainers, with the
expectation that the TC will probably need to decide.

Yeah, with my Policy hat on, I don't want any piece of that one. I would
be inclined to immediately escalate to the TC. I don't believe that
Policy's conservative consensus approach is going to work here, and I
expect trying to be painful.

[Tollef Fog Heen]

]] Russ Allbery

Post by Russ Allbery
Given that, I don't believe a Technical Committee choice of a default init
system is going to make either the systemd or the upstart maintainers want
to stop maintaining their packages.

Given what you're basically deciding between is «upstart + castrated
systemd» or «systemd» and I think I've pretty clearly expressed my
thoughts on splitting up systemd, I don't know where that conclusion
comes from. Were you to choose upstart, I'd be seriously thinking about
stopping maintaining systemd. (This is meant as a data point, not as a
threat or anything like that.) My ideas for the package align pretty
well with upstream (which I'm a sometimes-active part of) where the
different services are free to assume that they run with systemd as pid
1. Continously porting new upstream versions to an environment
unsupported by upstream is not what I consider reasonable or fun. I'm
speaking for myself here, not the other systemd maintainers.

In addition, as per my other message, were you to choose upstart as a
default, I'd be inclined to remove the init functionality from systemd.
I think trying to support multiple init systems is crazy and a recipe
for disaster and I don't want to do that. (Doing it as part of a
transition is a necessary pain, that much I can agree with, but ongoing?
No thanks.)

Post by Russ Allbery
6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use. The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use. I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.

I think allowing them to use a compatible init system should be ok too,
if somebody wants to do that.

[Russ Allbery]

Post by Tollef Fog Heen
]] Russ Allbery

Post by Russ Allbery
Given that, I don't believe a Technical Committee choice of a default
init system is going to make either the systemd or the upstart
maintainers want to stop maintaining their packages.

Given what you're basically deciding between is «upstart + castrated
systemd» or «systemd» and I think I've pretty clearly expressed my
thoughts on splitting up systemd, I don't know where that conclusion
comes from.

I have to admit that I didn't give it a whole lot of thought. It was an
assumption based on the presence of both in the archive for some time now
as non-default init systems under sysvinit. In that sense, things don't
change that horribly much, at least up to the jessie release, if the
default changes from one non-systemd thing to another non-systemd thing.

That being said, obviously you should speak for yourself, and I stand
corrected. Apologies for misprepresenting your feelings here.

Post by Tollef Fog Heen

Post by Russ Allbery
6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use. The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use. I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.

I think allowing them to use a compatible init system should be ok too,
if somebody wants to do that.

Oh, yes, good point. I was thinking more in terms of two different init
systems with different preferred configuration files.

[Michael Gilbert]

Post by Russ Allbery
4. Conclusions
I previously argued that much of the benefit of a new init system comes
from when we can stop maintaining init scripts. I still believe that, but
after thinking more about the cultural and project issues at stake here,
as well as the immediate needs for a clean upgrade path, I ended up at
more of a compromise position than I expected.
1. We should select a new init system for jessie, either systemd or
upstart. Support for that init system should be release-critical, but
only in the sense that the daemon works properly under that init
system. In other words, use of the sysvinit compatibility of either
init system is acceptable support for jessie.
2. All packages providing init scripts must continue to support sysvinit
scripts through the jessie release. Such support will continue to be
release-critical. This is going to be painful for packages that want
to do an early conversion, since it means testing two different init
systems for this release cycle, but I think this is the right thing to
do regardless for a clean upgrade path and Debian's normal robust
committment to upgrades. Here it has the additional advantage of
giving the non-Linux ports some breathing space to strategize.
3. Related, up through the jessie release, packages must (where possible;
it's possible there will be cases where this is simply impossible)
support switching back and forth between the new default init system
and sysvinit. This means that configurations should not be moved out
of /etc/default and that startup files for the new init system should
read the same configuration that the existing sysvinit scripts use (or
both should be modified compatibly).
4. Post-jessie, support for sysvinit will no longer be release-critical,
and package maintainers will no longer be expected to ensure that it
continues working. However, for as long as Debian has accepted
non-Linux ports using a different init system, package maintainers
should continue to ship init scripts if they work and should apply
patches and other reasonable fixes from porters for those init scripts.
In other words, this should be treated the same as merging patches for
the Hurd to remove hard-coded constant assumptions: if the change is
reasonable and doesn't break Linux ports (and this should be fairly
easy to handle for nearly all cases with init scripts), the package
maintainer should merge it.
5. Support for the other init system that was not chosen should be handled
in the same fashion, should a team choose to pursue it. If we select
systemd, package maintainers should still be willing to merge
contributed upstart configuration, with the understanding that they
can't test it and any support is on a best-effort basis only.
Similarly, if we select upstart, package maintainers should be willing
to merge systemd unit files and to enable upstream systemd support
where requested and where it doesn't interfere with the operation of
the daemon under upstart, with the understanding that the package
maintainer is not committing to testing or directly supporting this
configuration.
6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use. The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use. I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.
7. After jessie, functionality between systems running the primary Linux
init system and other init systems (including non-Linux ports) should
be allowed to drift. In other words, there will be cases where
features will only be available with the primary init system. Possible
examples include security hardening, socket activation, automatic
daemon restarts, and so forth. Packagers are under no obligation to
port those features to other init systems, but should welcome and merge
patches that do so. After jessie, packagers will no longer be required
to preserve daemon configuration when the init system is switched, so
use of such facilities as modification of upstart configuration files
or systemd overrides may be used.
We should revisit this decision again after the jessie release in case the
situation has substantially changed.

Doesn't a TC mandate on the default init system in some sense violate
Debian's spirit of meritocracy? May I suggest something like the
following (this isn't meant to be definitive) as a more meritocratic
approach:

1. The TC encourages a team (probably debian-boot) to provide a
package (similar to tasksel), let's call it initsel, that prompts
users during an expert install (and dpkg-reconfigure) time to make an
init system selection (with sysvinit as the default to begin with)

2. The TC recommends approving jessie release goals to support all of
the viable init systems (concurrently maintaining full support for
sysvinit).

3. The TC recommends that initsel support selection among any of the
init systems that are sufficiently capable on the user's current
architecture.

4. When the init systems sufficiently support all of the planned
release architectures, which would include kfreebsd-* (at least for
now) and possible hurd if it does become an officially supported arch,
the TC recommends that the team supporting initsel evaluate changing
the default to their best choice among all of those init systems that
do support all of the release architectures.

5. At some future point deprecate sysvinit and any of the other init
systems that are clearly not worth continuing, and encourage
maintainers to remove support for those.

Doing something like this, the best init system can win based truly on
merit (if/when the work gets done), rather than as a fuzzy upfront
judgement call.

Best wishes,
Mike

[Russ Allbery]

Post by Michael Gilbert
Doesn't a TC mandate on the default init system in some sense violate
Debian's spirit of meritocracy?

I believe that we have enough information to make an informed choice
already, and that the sides are fairly well-defined and hardened in their
opinions. That means that this dispute falls under section 6.1.2 of the
constitution:

Decide any technical matter where Developers' jurisdictions overlap.

In cases where Developers need to implement compatible technical
policies or stances (for example, if they disagree about the
priorities of conflicting packages, or about ownership of a command
name, or about which package is responsible for a bug that both
maintainers agree is a bug, or about who should be the maintainer for
a package) the technical committee may decide the matter.

Regardless of how we structure the installer, we need to have a default
init system (unless we plan on making every user choose, which I would
dismiss out of hand as a horrible UI experience for the average user, who
really doesn't care). We have to mandate support for at least that
default init system. Realistically, the ability of the Debian developer
community to support more than one init system is limited, since any given
system is generally only going to run one. That means the level of
quality of integration is going to drop off significantly relative to the
default init system, particularly over time.

Init systems are not like desktop environments: they require work by a
huge swath of the developer community, and the average user does not
generally switch from one to the other. The reality is that either
systemd or upstart is fully capable of everything the typical user is
going to need (for that matter, sysvinit is capable of most of what the
typical user needs); there isn't the same driving force of user preference
that leads users to switch between desktop environments, and quite a bit
more integration support is required for the init system.

[Michael Gilbert]

Post by Russ Allbery

Post by Michael Gilbert
Doesn't a TC mandate on the default init system in some sense violate
Debian's spirit of meritocracy?

I believe that we have enough information to make an informed choice
already, and that the sides are fairly well-defined and hardened in their
opinions. That means that this dispute falls under section 6.1.2 of the

I entirely concur that the social problem resides rightly within the
jurisdiction of the TC. With that said, however, it is worth
considering whether the role of the TC may be more effective if
directed at the root (the social), rather than the branches (the
technical choice), of the problem. The key, I think, is for the TC to
provide a reasonable path for those currently identifying with any of
the hardened camps to redirect their negative energy away from
argument and toward something more positive: technical work and actual
code.

Post by Russ Allbery
Regardless of how we structure the installer, we need to have a default
init system (unless we plan on making every user choose, which I would
dismiss out of hand as a horrible UI experience for the average user, who
really doesn't care).

As stated in my suggestion, initsel would of course always have a
default, and only "expert" users would be empowered to travel the road
less traveled. The no default idea that gets thrown about a lot is,
of course, infeasible as a matter of practicality.

Post by Russ Allbery
Init systems are not like desktop environments: they require work by a
huge swath of the developer community, and the average user does not
generally switch from one to the other.

I think, on the contrary, the nmu procedure is quite effective at
enabling a small subset of the developer community (those that have a
strong shared interest) to effect large changes across the entire
archive (of course when maintainers stay out of the way,).

This process has been demonstrated many times (although slow-going)
for lots of other sweeping changes (e.g. buildflags, usr/share/doc,
etc.).

Best wishes,
Mike

[Russ Allbery]

Post by Michael Gilbert

Post by Russ Allbery
I believe that we have enough information to make an informed choice
already, and that the sides are fairly well-defined and hardened in
their opinions. That means that this dispute falls under section 6.1.2

I entirely concur that the social problem resides rightly within the
jurisdiction of the TC. With that said, however, it is worth
considering whether the role of the TC may be more effective if directed
at the root (the social), rather than the branches (the technical
choice), of the problem. The key, I think, is for the TC to provide a
reasonable path for those currently identifying with any of the hardened
camps to redirect their negative energy away from argument and toward
something more positive: technical work and actual code.

Well, I think it's worth pointing out that my transition plan looks
somewhat similar to your plan, as far as the jessie release. (Then it
starts diverging.) Part of my goal in writing up that plan was, as you
say, to try to provide a means for people who are committed to one system
or the other to continue to work on what they're passionate about even if
it's not chosen as the default init system.

Whether they do so or not is up to them, of course, as it should be.

However, I don't want to understate the amount of effort required to
integrate with the init system across the distribution. I'm less
pessimistic than Steve, but he's not wrong that the choice of a default
init system will have sweeping consequences for what will work and what
won't. This will particularly be the case if that init system supports
capabilities beyond the sysvinit set.

I do think it would be possible to maintain package compatibility with
both systemd and upstart. That was something I was curious about and
therefore explicitly tested as part of my evaluation, and was able to do
so to my satisfaction. That said, I tackled a fairly simple daemon, and
something like NFS support would require people with deep knowledge of
each supported init system to maintain that support.

I don't think it's a good idea to ask everyone to pursue all paths in
parallel. I think Debian does a bit too much of that right now. We
should pick a winner that we believe is technically superior and focus the
mandatory, archive-wide effort on it. We should then *not get in the way*
of people who also want to pursue alternative paths, but not assume that
they will necessarily be successful, and not require that everyone get
involved in that effort beyond the level of integrating patches that we
currently expect for, say, the Hurd port.

But, anyway, I don't think our positions are really that different. The
main difference is that I think we should pick a default init system for
jessie now, and you feel like we should do effectively an archive-wide
bake-off and then go with whatever one achieves the best integration. I
have to admit to a deep personal dislike of "contests" like that, since I
find them stressful and demotivating and think they make the process of
free software development less fun. I'd rather decide on our default and
on the mandatory work now, so that everyone knows where they stand, and
then let people make their own decisions about what they want to spend
time on beyond the required minimum.

[Michael Gilbert]

Post by Russ Allbery

Post by Michael Gilbert

Post by Russ Allbery
I believe that we have enough information to make an informed choice
already, and that the sides are fairly well-defined and hardened in
their opinions. That means that this dispute falls under section 6.1.2

I entirely concur that the social problem resides rightly within the
jurisdiction of the TC. With that said, however, it is worth
considering whether the role of the TC may be more effective if directed
at the root (the social), rather than the branches (the technical
choice), of the problem. The key, I think, is for the TC to provide a
reasonable path for those currently identifying with any of the hardened
camps to redirect their negative energy away from argument and toward
something more positive: technical work and actual code.

Well, I think it's worth pointing out that my transition plan looks
somewhat similar to your plan, as far as the jessie release. (Then it
starts diverging.)

The key differences are that it is more succinct, roles and
requirements are defined, no init system is outright rejected, and the
default is selected on demonstrable merit.

Post by Russ Allbery
Part of my goal in writing up that plan was, as you
say, to try to provide a means for people who are committed to one system
or the other to continue to work on what they're passionate about even if
it's not chosen as the default init system.

Unfortunately at least two camps will be entirely dejected by any TC
mandate here.

Post by Russ Allbery
Whether they do so or not is up to them, of course, as it should be.
However, I don't want to understate the amount of effort required to
integrate with the init system across the distribution. I'm less
pessimistic than Steve, but he's not wrong that the choice of a default
init system will have sweeping consequences for what will work and what
won't. This will particularly be the case if that init system supports
capabilities beyond the sysvinit set.
I do think it would be possible to maintain package compatibility with
both systemd and upstart. That was something I was curious about and
therefore explicitly tested as part of my evaluation, and was able to do
so to my satisfaction. That said, I tackled a fairly simple daemon, and
something like NFS support would require people with deep knowledge of
each supported init system to maintain that support.
I don't think it's a good idea to ask everyone to pursue all paths in
parallel. I think Debian does a bit too much of that right now. We
should pick a winner that we believe is technically superior and focus the
mandatory, archive-wide effort on it. We should then *not get in the way*
of people who also want to pursue alternative paths, but not assume that
they will necessarily be successful, and not require that everyone get
involved in that effort beyond the level of integrating patches that we
currently expect for, say, the Hurd port.

I don

Post by Russ Allbery
But, anyway, I don't think our positions are really that different. The
main difference is that I think we should pick a default init system for
jessie now, and you feel like we should do effectively an archive-wide
bake-off and then go with whatever one achieves the best integration.

And Debian ends up with not only apple pie, but pumpkin and blueberry
pies, and of in a corner there will be others thinking about
cinnamon-spiced apple and blueberry-raspberry and other
never-before-seen flavors. What a wonderful bake-off that would be
:)

Think about how it would go over if the directors of that metaphorical
bake-off forced all the participants to produce the exact same pie.
No one would really want to participate, and winning would be entirely
meaningless. It wouldn't be a bake-off at all. It would be more like
a mass-production assembly line.

Best wishes,
Mike

[Steve Langasek]

Post by Michael Gilbert

Post by Russ Allbery
Part of my goal in writing up that plan was, as you
say, to try to provide a means for people who are committed to one system
or the other to continue to work on what they're passionate about even if
it's not chosen as the default init system.

Unfortunately at least two camps will be entirely dejected by any TC
mandate here.

I don't agree with that conclusion.

When it comes to technology choices, you win some and you lose some. If
upstart wins, I will be happy. If systemd wins, I will also be happy,
because it's long overdue that Debian *make a decision*; and for all that
there are aspects of systemd that make me uncomfortable, it will still be
far better than the status quo.

Your proposal smells like the status quo. Namely, instead of the project
making a decision and being able to all pull together in the same direction
to provide the best possible OS, we will continue to coast, squandering
efforts on preserving users' ability to make choices about things that no
user should ever be asked to care about.

[Russ Allbery]

Post by Steve Langasek
When it comes to technology choices, you win some and you lose some. If
upstart wins, I will be happy. If systemd wins, I will also be happy,
because it's long overdue that Debian *make a decision*; and for all
that there are aspects of systemd that make me uncomfortable, it will
still be far better than the status quo.

I concur with this.

I freely admit that I fell in love with systemd while investigating both
systems, but if we agree to use upstart, I will happily go add upstart
configurations to all of my packages and make use of those features. It's
still a massive improvement over what we have now.

Furthermore, and more philosophically, Debian has to learn how to respect
a governance process and make decisions. We spent way too much time and
effort not making decisions in this project, which results in big
flamewars and discomfort and everyone sniping at each other while little
productive happens. Frequently, I think this is like tearing off bandages
over the course of months. The cumulative pain is much higher than if we
just made a decision and everyone knew where they stood and what reality
they needed to adjust to.

Also, I too am happy when we can successfully pursue all courses of action
at the same time, but at the end of the day, we're trying to create a
distribution, and that means integrating components, and that means
deciding on integration protocols. It's not useful to try to integrate
everything with everything else in every possible way at the same time.
You just create an unmanageable combinatoric explosion of possibilities,
and then someone who just wants to run a Debian system doesn't know which
paths are supported and tested, and which paths have only been touched by
one developer if that.

We need to pick a winner. That doesn't mean we need to pick losers. It
does mean that one solution is going to get better support than everything
else, because that's what integration *means*. It doesn't mean, or
shouldn't mean, that other solutions are impossible if someone wants to
make them work; it just means that we're not going to *require* that they
work in order to get one's software into Debian.

We probably should have had this discussion for wheezy, to be honest. But
regardless, we should pick a default, supported init system for jessie.

[Michael Gilbert]

Post by Steve Langasek

Post by Michael Gilbert

Post by Russ Allbery
Part of my goal in writing up that plan was, as you
say, to try to provide a means for people who are committed to one system
or the other to continue to work on what they're passionate about even if
it's not chosen as the default init system.

Unfortunately at least two camps will be entirely dejected by any TC
mandate here.

I don't agree with that conclusion.

That no unhappiness will arise out of this decision is, I think, a
quite unlikely outcome, so this statement is quite surprising. As
Russ eloquently states today [1]

The degree to which this all should affect our decision-making
process is limited. That's particularly true in this case, which
is one of those decisions where, whatever we choose, some
group of people in Debian who have put a ton of work into
something they care about are going to be at least somewhat
unhappy. That may be the systemd maintainers, the GNOME
team, the Hurd porters, the kFreeBSD porters, the upstart
maintainers, those who care deeply about tight Debian and
Ubuntu integration, or any number of other groups. It will
probably be several of those.

The TC needs to find a path that minimizes unintended social
consequences. I have been trying to make the case in this thread that
the only reasonable solution that does so is one which empowers all
camps to do work to further their cause without feeling excluded.
Eventually, and this may take sometime but everything worth doing
does, a clearly superior solution will emerge that can, when ready,
become the default.

Post by Steve Langasek
When it comes to technology choices, you win some and you lose some.

The fact that there may be winners and losers is not the issue at
hand. It is the specter of being disqualified before the game even
starts.

Post by Steve Langasek
Your proposal smells like the status quo.

That is only true if none of the systemd, openrc, and upstart teams
get to the point where their init is usable, then we're stuck with
sysvinit. This is probably an unlikely circumstance, but sysvinit
does work.

Post by Steve Langasek
Namely, instead of the project
making a decision and being able to all pull together in the same direction
to provide the best possible OS, we will continue to coast, squandering
efforts on preserving users' ability to make choices about things that no
user should ever be asked to care about

The project never goes in the same direction. 1,000 people go in
their own direction, creative solutions eventually emerge, one often
gains the most traction, but others remain as alternatives (think
debhelper/cdbs/etc), and choice and freedom remain. It will be a
major change in project governance to select any resolution prior to
emergence.

In my opinion, the only reasonable TC decision is one that requests
project members feel empowered to work toward the solutions that they
are interested in, while being open-minded and non-obstructive to all
of the other competing solutions.

Best wishes,
Mike

[Cory]

Post by Michael Gilbert

Post by Russ Allbery
4. Conclusions
I previously argued that much of the benefit of a new init system comes
from when we can stop maintaining init scripts. I still believe that, but
after thinking more about the cultural and project issues at stake here,
as well as the immediate needs for a clean upgrade path, I ended up at
more of a compromise position than I expected.
1. We should select a new init system for jessie, either systemd or
upstart. Support for that init system should be release-critical, but
only in the sense that the daemon works properly under that init
system. In other words, use of the sysvinit compatibility of either
init system is acceptable support for jessie.
2. All packages providing init scripts must continue to support sysvinit
scripts through the jessie release. Such support will continue to be
release-critical. This is going to be painful for packages that want
to do an early conversion, since it means testing two different init
systems for this release cycle, but I think this is the right thing to
do regardless for a clean upgrade path and Debian's normal robust
committment to upgrades. Here it has the additional advantage of
giving the non-Linux ports some breathing space to strategize.
3. Related, up through the jessie release, packages must (where possible;
it's possible there will be cases where this is simply impossible)
support switching back and forth between the new default init system
and sysvinit. This means that configurations should not be moved out
of /etc/default and that startup files for the new init system should
read the same configuration that the existing sysvinit scripts use (or
both should be modified compatibly).
4. Post-jessie, support for sysvinit will no longer be release-critical,
and package maintainers will no longer be expected to ensure that it
continues working. However, for as long as Debian has accepted
non-Linux ports using a different init system, package maintainers
should continue to ship init scripts if they work and should apply
patches and other reasonable fixes from porters for those init scripts.
In other words, this should be treated the same as merging patches for
the Hurd to remove hard-coded constant assumptions: if the change is
reasonable and doesn't break Linux ports (and this should be fairly
easy to handle for nearly all cases with init scripts), the package
maintainer should merge it.
5. Support for the other init system that was not chosen should be handled
in the same fashion, should a team choose to pursue it. If we select
systemd, package maintainers should still be willing to merge
contributed upstart configuration, with the understanding that they
can't test it and any support is on a best-effort basis only.
Similarly, if we select upstart, package maintainers should be willing
to merge systemd unit files and to enable upstream systemd support
where requested and where it doesn't interfere with the operation of
the daemon under upstart, with the understanding that the package
maintainer is not committing to testing or directly supporting this
configuration.
6. Debian's non-Linux ports should either use the same init system as
Debian's Linux ports or agree on an init system that they're both going
to use. The porting work is going to be hard enough without the ports
going in different directions on which secondary init system they want
to use. I prefer to leave it up to the porters to decide which init
system to choose, but I do think OpenRC would be a strong contender.
7. After jessie, functionality between systems running the primary Linux
init system and other init systems (including non-Linux ports) should
be allowed to drift. In other words, there will be cases where
features will only be available with the primary init system. Possible
examples include security hardening, socket activation, automatic
daemon restarts, and so forth. Packagers are under no obligation to
port those features to other init systems, but should welcome and merge
patches that do so. After jessie, packagers will no longer be required
to preserve daemon configuration when the init system is switched, so
use of such facilities as modification of upstart configuration files
or systemd overrides may be used.
We should revisit this decision again after the jessie release in case the
situation has substantially changed.

Doesn't a TC mandate on the default init system in some sense violate
Debian's spirit of meritocracy? May I suggest something like the
following (this isn't meant to be definitive) as a more meritocratic
1. The TC encourages a team (probably debian-boot) to provide a
package (similar to tasksel), let's call it initsel, that prompts
users during an expert install (and dpkg-reconfigure) time to make an
init system selection (with sysvinit as the default to begin with)
2. The TC recommends approving jessie release goals to support all of
the viable init systems (concurrently maintaining full support for
sysvinit).
3. The TC recommends that initsel support selection among any of the
init systems that are sufficiently capable on the user's current
architecture.
4. When the init systems sufficiently support all of the planned
release architectures, which would include kfreebsd-* (at least for
now) and possible hurd if it does become an officially supported arch,
the TC recommends that the team supporting initsel evaluate changing
the default to their best choice among all of those init systems that
do support all of the release architectures.
5. At some future point deprecate sysvinit and any of the other init
systems that are clearly not worth continuing, and encourage
maintainers to remove support for those.
Doing something like this, the best init system can win based truly on
merit (if/when the work gets done), rather than as a fuzzy upfront
judgement call.
Best wishes,
Mike

1
i really don't see any "point" at all of talking about debian BSD init
systems they're porting there own it's called open-launchd

https://github.com/rtyler/openlaunchd

2
having used Upstart OpenRC and systemd and i can tell you systemd works
the best out of the 3

3
if debian does go with Upstart will this let all of debians down streams
replace Upstart with systemd as most of debian's down streams who don't
use the default inti system use systemd if not all of them

4
now talking about down streams even SteamOS looks to use systemd
http://repo.steampowered.com/steamos/pool/main/s/systemd/

5
3 DE's use logind Gnome, KDE, MATE,

6
consolekit is dead logind is way better

[Vincent Bernat]

Post by Michael Gilbert
Doing something like this, the best init system can win based truly on
merit (if/when the work gets done), rather than as a fuzzy upfront
judgement call.

Unfortunately, being the best init is the not only the matter of its
maintainers. A good integration implies to modify some packages whose
maintainer may be hostile to some init and may consider that their
package work well enough to not enable some feature needed by some init.

I am pretty vague, by I believe that the TC has currently a case for
such an example.

Only once (and if) systemd gets to be the default init, we will be able
to leverage all its abilities by full integration into Debian.

[Michael Gilbert]

Post by Vincent Bernat

Post by Michael Gilbert
Doing something like this, the best init system can win based truly on
merit (if/when the work gets done), rather than as a fuzzy upfront
judgement call.

Unfortunately, being the best init is the not only the matter of its
maintainers. A good integration implies to modify some packages whose
maintainer may be hostile to some init and may consider that their
package work well enough to not enable some feature needed by some init.

That is a hypothetical, of course, but it is a very real possibility
for any path that the TC decides here. However, selectable inits may
(possibly counter-intuitively) reduce this likely-hood. Since that
maintainer in the way also gets his init of choice, he may be less
likely to oppose an alternative that doesn't in any real sense get in
his way.

However, if/when this does happen, it will be an interesting test of
whether the TC has the political will to override an indignant
maintainer with their 3-to-1 majority power.

Best wishes,
Mike

[Ian Jackson]

Post by Michael Gilbert

Post by Vincent Bernat
Unfortunately, being the best init is the not only the matter of its
maintainers. A good integration implies to modify some packages whose
maintainer may be hostile to some init and may consider that their
package work well enough to not enable some feature needed by some init.

That is a hypothetical, of course, but it is a very real possibility
for any path that the TC decides here. However, selectable inits may
(possibly counter-intuitively) reduce this likely-hood. Since that
maintainer in the way also gets his init of choice, he may be less
likely to oppose an alternative that doesn't in any real sense get in
his way.
However, if/when this does happen, it will be an interesting test of
whether the TC has the political will to override an indignant
maintainer with their 3-to-1 majority power.

This is why I think the TC resolution should explain what kinds of
integration package maintainers shuuld accept. Maintainers who flout
that advice will indeed get overruled.

Ian.

[Anthony Towns]

Okay, let's see how replying to a mail on my phone while in flight mode
goes. Apologies in advance for formatting, quoting and vocabulary issues.

Post by Russ Allbery
2. Impact of Multiple Init Systems
Obviously, the ideal situation for project-wide integration and support,
and for Debian's ability to make full use of the capabilities of new init
systems, is for all of Debian's ports to be running the same init system.

So, reading this (and trimming loss of stuff that makes sense) I wonder if
it's worth stepping back and considering what happens when a package
doesn't support /any/ init system. The answer I think is that the sysadmin
sighs, adds their own configuration, and maybe thinks "well at least I
didn't have to compile it, I guess."

That still sucks compared to the alternative of typing "apt-get install"
and having it just start working, of course.

Post by Russ Allbery
* Packages will either be limited by the functionality of the least
capable init system or will behave differently (and have to be
configured differently) on different ports.

I think the "Debian" way of dealing with multiple init systems would be to
provide a compatibility layer for the most common packaging and admin
tasks, and allowing packages to provide fancier integration where
appropriate. I'm thinking of things like newaliases and sendmail, and
cgi-bin and apache modules, I guess. Probably that's already covered for
init systems via sysvinit compatibility and update-rc.d. Maybe there's a
missing tool that could be written to let you set init specific
configuration somehow, which would change /etc/default for sysv and other
files for upstart/systemd.

It seems like there's maybe three separate questions then: what init system
gets used by default, what work gets done to make that experience
different/better than everything just having upstart or systemd pretending
to be sysvinit, and what's the experience of maintaining packages to
support secondary init systems and using secondary init systems on a Debian
system?

(Personally, I think a gr would be better for choosing the default then
having the tech ctte decide that issue, but whatever)

Based on what I've read, it seems like the ideas floating around amount to:

- if systemd is default: there would be a release goal to include systemd
configs added to packages and to get daemons converted to support socket
activation. Maybe other stuff too? As far as maintaining sysvinit, openrc
or upstart systemd goes, you'd just have to get upstart configs written and
packaged, and accept that there's an unused systemd library on your system.
Multiple inits must be supported to some extent, since systemd isn't
available on ports and that isn't likely to change apparently.

- upstart is default, other inits are supported: pull in Ubuntu configs for
upstart for various packages. If systemd socket stuff is allowed, dummy
library will probably be on most systems, if not, systemd in Debian won't
be very interesting.

- upstart is default and only init supported by Debian. Same support work
for Jessie, except any ports that want to stick around need to get upstart
enabled.

I don't think the latter is really the Debian way - better to have the
choice left in the users' hand if feasible, but it's likely a lot easier to
get some impressive results that way. I think the ideal page for tradeoffs
like that is in derivatives like Ubuntu personally.

Post by Russ Allbery
1. We should select a new init system for jessie, either systemd or
upstart. Support for that init system should be release-critical, but
only in the sense that the daemon works properly under that init
system. In other words, use of the sysvinit compatibility of either
init system is acceptable support for jessie.

So that would mean switching the init system, and decorating anything that
breaks as a result RC buggy. Seems sensible.

Post by Russ Allbery
2. All packages providing init scripts must continue to support sysvinit
scripts through the jessie release. Such support will continue to be
release-critical.

I'm not sure this makes sense, though. If someone packages a new daemon
that works fine with systemd and upstart, I don't see why it shouldn't get
released just because it doesn't work with two secondary init systems.
Filling a wishlist bug with a patch and doing an NMU seem like they'd be
sufficient here.

As far as upgrades go, if the idea is that systemd is the way to go for
Jessie it seems to me that an update would by default switch you from
sysvinit to systemd (likewise for upstart) - I don't think it makes much
sense to do systemd/upstart for new installs but leave upgrades with sysv
until Jessie+1.

Post by Russ Allbery
7. After jessie, functionality between systems running the primary Linux
init system and other init systems (including non-Linux ports) should
be allowed to drift. In other words, there will be cases where
features will only be available with the primary init system. Possible
examples include security hardening, socket activation, automatic
daemon restarts, and so forth. Packagers are under no obligation to
port those features to other init systems, but should welcome and merge
patches that do so. After jessie, packagers will no longer be required
to preserve daemon configuration when the init system is switched, so
use of such facilities as modification of upstart configuration files
or systemd overrides may be used.

The only way maintainers are required to do these things is that either a)
someone else will do it and nmu their package, or b) the release team will
drop it from Jessie prior to release, no? Personally I'd be inclined to
encourage patches and nmus here, and limit the rc stuff to actual belated
with whatever Jessie's default end up being. I don't see any reason that
wouldn't suffice to allow comparability and an aggressive approach to
making use of the new default init's extra capabilities.

(I wonder if it might be interesting for the tech ctte to track the bugs
for resulting from changing the init system rather than leaving it to the
release team as either rc issues or release goals... Might be a good
approach for some high impact improvements)

(Fwiw, I'm employed by Red Hat, but have no particular stake in systemd vs
upstart, nor much knowledge about either than what I've read here)

Cheers,
aj

[Neil McGovern]

Post by Russ Allbery
However, I think it's the best available approach that balances our ideals
as a project against the opportunities offered by a new init system. This
approach does permit full use of new init system features for jessie
except for eliminating /etc/default files (which I doubt we'd successfully
do quickly anyway), and opens up the full spectrum of use cases after
jessie. The cost is that packagers should merge contributed patches to
the init systems that they don't use. I don't think this is too much to
ask, nor do I think it will have serious effects on package complexity
based on my own experience configuring a package to work under all three
init systems I considered.

I’m no longer a RM, but I would echo Russ’ comments here - an ordered transition is very important for Debian releases - and the size of this transition would be something that (in my opinion) would be very hard to achieve in one release cycle.

Neil

[Russ Allbery]

Post by Ian Jackson
I think we should give package maintainers some guidance on what kinds
of upstart or systemd patches should be accepted. Without this, it's
likely we'll find ourselves adjudicating disputes that ought to have
been settled in principle much earlier.
- use a non-forking startup method
- avoid significant amounts of open-coded AF_UNIX socketry
- avoid embedded copies of daemon library code

These all seem reasonable to me, with the caveat that if *upstream*
includes open-coded AF_UNIX socketry or embedded copies of daemon library
code, I don't see any need for Debian package maintainers to remove that
code.

Post by Ian Jackson
- avoid extra build-dependencies (eg on libsystemd)
- avoid extra runtime dependencies (eg on deb-systemd-helper)

These requirements, on the other hand, I find completely baffling.

This approach seems completely contrary to Debian best practices. Our
standard practice with all packages is to fully enable all available
features that make sense on Debian and don't pose some concrete risk.
This means that, for example, I have CUPS libraries on this system despite
the fact that I never print, Avahi libraries depsite the fact that I will
never use that software, and SELinux libraries deeply embedded in core
code despite the fact that few Debian systems currently run SELinux.

These requirements seem like would fit with Gentoo, where much emphasis is
placed on letting people build custom-crafted binaries to their local
requirements, not Debian's approach of providing a full-featured and
uniform distribution.

Needless to say, I don't think we should avoid dependencies on
libsystemd-daemon or init-system-helpers, and I think it would be wholly
inappropriate for the Technical Committee to say anything of the sort.

Post by Ian Jackson
(If the current state of the art in readiness protocols persists that
means that upstart patches using raise(SIGSTOP) ought to be accepted,
but systemd ones rejected unless they can use socket activation.)

So, as mentioned above, I don't agree with this, although of course I
think using socket activation is, in general, better than using the
readiness protocol.

Post by Ian Jackson
- daemon authors and Debian maintainers should prefer command-line
options to environment variables

I disagree with including this in a statement. I think it's outside the
scope of what we were asked to resolve, and I don't think it's the place
of the TC to offer this sort of general advise to upstreams.

Post by Ian Jackson
- whatever environment variables and fds are used, measures should be
taken to avoid them being inherited by daemons' arbitrary children

I agree with this as good practice for daemons, but again, I don't think
it's the role of the TC to issue this sort of general advice that is not
directly related to a question put before us.

If we're going to offer specific advice, we should limit it to the
specific integrations that we're asked to consider. But I think we should
be mindful of the restriction on the TC not doing design work and let
Policy for packaging support of upstart and/or systemd be hashed out
through the regular Policy process.

[Ian Jackson]

Post by Russ Allbery

Post by Ian Jackson
- avoid extra build-dependencies (eg on libsystemd)
- avoid extra runtime dependencies (eg on deb-systemd-helper)

These requirements, on the other hand, I find completely baffling.
This approach seems completely contrary to Debian best practices. Our
standard practice with all packages is to fully enable all available
features that make sense on Debian and don't pose some concrete risk.

The case in point is a little different. It's one thing to say that
mail daemons should come with ldap support enabled, or whatever (and
even then we sometimes have two versions e.g. "heavy" and "light").

For me it's a different proposition to suppose that _every_ daemon
should bring in these kind of dependencies. This is particularly the
case for build-dependencies on an avowedly and intentionally
non-portable program. Of course this can be made conditional, but
this is IMO too fiddly.

Post by Russ Allbery

Post by Ian Jackson
- daemon authors and Debian maintainers should prefer command-line
options to environment variables

I disagree with including this in a statement. I think it's outside the
scope of what we were asked to resolve, and I don't think it's the place
of the TC to offer this sort of general advise to upstreams.

It is very likely that Debian contributors will be implementing native
support for whatever readiness protocol, in the upstream parts of the
codebase. It is IMO appropriate for those contributors to get advice
on how to do this. Policy already gives advice on this kind of
thing. Given the context, and the fact that this advice is going to
be read by a lot of people as part of a big enhancement project, I
think it's appropriate for the TC to answer this question.

Post by Russ Allbery
If we're going to offer specific advice, we should limit it to the
specific integrations that we're asked to consider. But I think we should
be mindful of the restriction on the TC not doing design work and let
Policy for packaging support of upstart and/or systemd be hashed out
through the regular Policy process.

Are you proposing then that the TC should decide the default init
system, but asmk people NOT to start converting packages until the
policy questions are settled ?

Ian.

[Russ Allbery]

Post by Ian Jackson

Post by Russ Allbery
These requirements, on the other hand, I find completely baffling.
This approach seems completely contrary to Debian best practices. Our
standard practice with all packages is to fully enable all available
features that make sense on Debian and don't pose some concrete risk.

The case in point is a little different. It's one thing to say that
mail daemons should come with ldap support enabled, or whatever (and
even then we sometimes have two versions e.g. "heavy" and "light").

I'm aware of only one case of that in the archive. It's certainly not
what we normally do.

Post by Ian Jackson
For me it's a different proposition to suppose that _every_ daemon
should bring in these kind of dependencies.

It's only going to be *every* daemon if we select systemd as the default
init system, in which case we should be doing this in many daemons for
either socket activation or for readiness notification as part of a proper
systemd integration. If we do not select systemd as the default init
system, the only maintainers doing this will be those who want their
packages to work well with systemd or whose daemons are of particular
interest to people who want to run systemd as a non-default init system.
In which case, what's the harm? It's going to be very difficult to even
notice this dependency is there.

I think the position you're taking here is directly contrary to Debian's
position of deference and reasonable accomodation to things that people
want to work on.

Post by Ian Jackson
This is particularly the case for build-dependencies on an avowedly and
intentionally non-portable program. Of course this can be made
conditional, but this is IMO too fiddly.

Adding [linux-any] to the Build-Depends line is not too fiddly, and if the
goal is to enable optional upstream support for systemd, that will be
sufficient. Besides, in general, it's up to the packager to decide what
is or is not too fiddly in how they want to package software.

Obviously, we should not add an unconditional build dependency for a
library that isn't available on some of our ports to software that can
work on those ports. But I think that's obvious. I'm happy to have us
say that explicitly if if helps.

Post by Ian Jackson

Post by Russ Allbery

Post by Ian Jackson
- daemon authors and Debian maintainers should prefer command-line
options to environment variables

I disagree with including this in a statement. I think it's outside
the scope of what we were asked to resolve, and I don't think it's the
place of the TC to offer this sort of general advise to upstreams.

It is very likely that Debian contributors will be implementing native
support for whatever readiness protocol, in the upstream parts of the
codebase. It is IMO appropriate for those contributors to get advice on
how to do this. Policy already gives advice on this kind of thing.
Given the context, and the fact that this advice is going to be read by
a lot of people as part of a big enhancement project, I think it's
appropriate for the TC to answer this question.

If you feel like Policy should give advice on this, you can bring it up in
the context of Policy. I don't think the exact mechanism of integration
is important enough for the TC to explicitly favor a particular mechanism,
and different upstreams may have different approaches. I'm not seeing the
important gain to Debian here in trying to standardize exactly how one
tells a daemon to use socket activation or readiness notification.

I also, for what it's worth, directly disagree with this advice with
respect to systemd because I think compatibility with how systemd is used
elsewhere is more important than any marginal gain from using a
configuration method that's not inherited by subprocesses by default,
particularly given that the standard systemd APIs include environment
sanitizing. But I also don't see any point in arguing about it here,
since I don't think this dispute is ripe (in the legal sense).

If you do bring it up in the context of Policy and we can't resolve the
debate there, it can get appealed to the TC and we can decide it here
separately, but I think this is all premature, given that the whole point
becomes basically moot if we don't pick systemd anyway.

If you're worried about programs configured with environment variables in
the archive, you've got quite a lot of work to do, starting with nearly
every Perl program in existence.

Post by Ian Jackson

Post by Russ Allbery
If we're going to offer specific advice, we should limit it to the
specific integrations that we're asked to consider. But I think we
should be mindful of the restriction on the TC not doing design work
and let Policy for packaging support of upstart and/or systemd be
hashed out through the regular Policy process.

Are you proposing then that the TC should decide the default init
system, but asmk people NOT to start converting packages until the
policy questions are settled ?

Sure. I think that's what's going to happen anyway. Again, it's not the
place of the TC (explicitly, per the constitution) to do design. Once we
decide on a direction, we should let the rest of the project work out the
details using our normal procedures. We have a fair bit of time left
before the jessie release, and I expect integration policy to be fairly
well-resolved by the end of February using our normal process. And even
if it's not, given that we need to support sysvinit for jessie anyway, I
expect most packages will not be in a rush to convert.

[Ian Jackson]

Post by Russ Allbery

Post by Ian Jackson
For me it's a different proposition to suppose that _every_ daemon
should bring in these kind of dependencies.

It's only going to be *every* daemon if we select systemd as the default
init system, in which case we should be doing this in many daemons for
either socket activation or for readiness notification as part of a proper
systemd integration. [...]

Well, no. I think that even if we select upstart as the default, we
should enable the systemd community to provide as complete a set of
integration in Debian as they care to put the work in for.

That translates directly to an expectation that the maintainer of any
daemon package must be willing to accept reasonable systemd
integration patches.

If the systemd community is very dynamic (and certainly we can't fault
their dynamism so far) this may well translate to all or almost all
daemons. Certainly it could be expected to include most of the core
daemons where the extra dependencies are most troublesome.

So I think we need to say what we regard as "reasonable" patches, in
advance. As the Debian maintainer for uservd (for example), am I
entitled to decline to incorporate systemd integration into my package
on the grounds that the patch involves additional build- and runtime
dependencies which I consider undesirable ?

I don't consider it desirable to answer this question differently for
different daemons. We need to set out a clear rule for maintainers to
follow or we'll end up adjudicating a gazillion individual disputes.

And if the answer to this question is, in general, "the maintainer
must include the patch" then the logical conclusion is that it is OK
for every daemon in Debian to acquire these additional build- and
runtime dependencies. We would be telling non-Linux ports that they
need to arrange for these dependencies to be somehow provided despite
the lack of systemd on their architecture, or to have conditional
compilation of the systemd support in every daemon package.

If the TC's answer is "the maintainer may reject the patch", we would
be telling the systemd community that they need to improve and
simplify their integration.

I think the latter is the right approach, for two reasons. Firstly,
it is less work overall. Secondly, it is proper that the maintainers
of an init system should be encouraged to provide as simple and
nonintrusive an integration interface as possible.

Post by Russ Allbery

Post by Ian Jackson
This is particularly the case for build-dependencies on an avowedly and
intentionally non-portable program. Of course this can be made
conditional, but this is IMO too fiddly.

Adding [linux-any] to the Build-Depends line is not too fiddly, and if the
goal is to enable optional upstream support for systemd, that will be
sufficient. Besides, in general, it's up to the packager to decide what
is or is not too fiddly in how they want to package software.

Adding [linux-any] to the Build-Depends is not sufficient.

It is also necessary to make the actual source code conditionally use
sd_notify, using #ifdefs or other similar techniques. The conditional
compilation will have to be automated (perhaps with autoconf if the
package has it - an then we need new --with[out]-systemd configuration
options to avoid miscompilation; or perhaps in an ad hoc way if the
daemon doesn't use autoconf). The fact that the systemd-specific
command line option may or may not be compiled in needs to be
mentioned in the documentation, along with text which allows the user
to know whether it's included in the version they actually have.

This turns what could be a simple change into a multi-part epic
involving all of the components of the package: primary source code,
upstream build system and portability layer, Debian build system,
documentation, and so on.

And it might come about later that someone provides a portable
sd_notify so half (but just the right half) of the above would have to
be undone.

Post by Russ Allbery
Obviously, we should not add an unconditional build dependency for a
library that isn't available on some of our ports to software that can
work on those ports. But I think that's obvious. I'm happy to have us
say that explicitly if if helps.

I think maintainers should not be required to introduce the additional
build- and runtime dependencies, if they do not wish to. Obviously if
they want to then that's fine by me.

Post by Russ Allbery

Post by Ian Jackson
It is very likely that Debian contributors will be implementing native
support for whatever readiness protocol, in the upstream parts of the
codebase. It is IMO appropriate for those contributors to get advice on
how to do this. Policy already gives advice on this kind of thing.
Given the context, and the fact that this advice is going to be read by
a lot of people as part of a big enhancement project, I think it's
appropriate for the TC to answer this question.

If you feel like Policy should give advice on this, you can bring it up in
the context of Policy.

The TC's mandate includes deciding on policy questions.

Post by Russ Allbery
I don't think the exact mechanism of integration is important
enough for the TC to explicitly favor a particular mechanism, and
different upstreams may have different approaches. I'm not seeing
the important gain to Debian here in trying to standardize exactly
how one tells a daemon to use socket activation or readiness
notification.

I'm not saying it should be standardised.

I'm trying to give guidance to Debian contributors who will be doing
this work on many existing daemons. We would like them to do the best
thing and giving them non-binding guidance is correct.

Post by Russ Allbery
I also, for what it's worth, directly disagree with this advice with
respect to systemd because I think compatibility with how systemd is used
elsewhere is more important than any marginal gain from using a
configuration method that's not inherited by subprocesses by default,
particularly given that the standard systemd APIs include environment
sanitizing. But I also don't see any point in arguing about it here,
since I don't think this dispute is ripe (in the legal sense).

I'm afraid to say that this is quite a frustrating approach.

If it will help I can go through the motions of filing a policy bug,
having everyone point out that this is entangled with the whole init
system debate, and having you reject my proposal.

But perhaps we could just pretend I had done that ?

Post by Russ Allbery
If you do bring it up in the context of Policy and we can't resolve the
debate there, it can get appealed to the TC and we can decide it here
separately, but I think this is all premature, given that the whole point
becomes basically moot if we don't pick systemd anyway.

No, it doesn't become moot. If we pick upstart then we still need to
tell maintainers what kind of systemd patches to accept.

If anything, it becomes moot if we pick systemd. It seems unlikely
that there would be a majority in the TC for picking systemd, and
separately a majority in the TC for overruling the systemd maintainers'
refusal to implement a simpler readiness protocol.

So a decision to pick systemd automatically comes with the expectation
that all daemons will get the new build- and runtime dependencies, and
maintainers will be expected to accept those patches.

Post by Russ Allbery

Post by Ian Jackson
Are you proposing then that the TC should decide the default init
system, but asmk people NOT to start converting packages until the
policy questions are settled ?

Sure. I think that's what's going to happen anyway.

I don't think that's likely, unless we state it explicitly.

And I want the conversion work to be able to start as soon as
possible.

These questions are all entangled with the init system choice. Having
dealt with the primary issue the TC members will be in a good position
to dispose of these ancillary issues.

I think remanding contentious consequential issues back to the policy
process is a very unattractive option. It will just incur delay.

I'm happy with leaving uncontentious details to the policy process.

Ian.

[Michael Stapelberg]

Hi Ian,

Post by Ian Jackson

Post by Russ Allbery

Post by Ian Jackson
This is particularly the case for build-dependencies on an avowedly and
intentionally non-portable program. Of course this can be made
conditional, but this is IMO too fiddly.

Adding [linux-any] to the Build-Depends line is not too fiddly, and if the
goal is to enable optional upstream support for systemd, that will be
sufficient. Besides, in general, it's up to the packager to decide what
is or is not too fiddly in how they want to package software.

Adding [linux-any] to the Build-Depends is not sufficient.
It is also necessary to make the actual source code conditionally use
sd_notify, using #ifdefs or other similar techniques. The conditional
compilation will have to be automated (perhaps with autoconf if the
package has it - an then we need new --with[out]-systemd configuration
options to avoid miscompilation; or perhaps in an ad hoc way if the
daemon doesn't use autoconf). The fact that the systemd-specific
command line option may or may not be compiled in needs to be
mentioned in the documentation, along with text which allows the user
to know whether it's included in the version they actually have.
This turns what could be a simple change into a multi-part epic
involving all of the components of the package: primary source code,
upstream build system and portability layer, Debian build system,
documentation, and so on.

No, this is all not true :).
libsystemd-daemon already contains the conditionals necessary to be a
noop on non-linux:
http://sources.debian.net/src/systemd/204-5/src/libsystemd-daemon/sd-daemon.c

Therefore, you can just use it, and don’t need to use any conditionals
on your end, neither in the compilation process, nor in the code itself.

That being said, I just checked and realized the package is not
available on non-linux. I’ll look into that now, since intuitively there
is no reason for this.

[Russ Allbery]

Post by Michael Stapelberg
That being said, I just checked and realized the package is not
available on non-linux. I’ll look into that now, since intuitively there
is no reason for this.

Thank you, Michael. That would indeed make things much easier. I think
Ian is being excessively dramatic about very simple conditional
integrations, but still, avoiding the conditional entirely is useful.

One approach that would, I hope, minimize Ian's concerns is to provide a
libsystemd-daemon-dev on non-Linux ports that just stubs out the calls,
and that, on those architectures, provides some sort of dummy empty
library that does nothing and adds no dependency. (There may be an
elegant way to do this with a linker script.) That way, the exact same
build process can be used everywhere, and everything just quietly
disappears on non-Linux ports where systemd isn't available, without even
adding a dependency on an (on that platform) useless shared library.

What I used for lbcd is:

#ifdef HAVE_SD_NOTIFY
# include <systemd/sd-daemon.h>
#else
# define SD_LISTEN_FDS_START 3
# define sd_listen_fds(u) 0
# define sd_notify(u, s) 0
#endif

which of course only stubs out the two major calls and doesn't address the
rest of the API. That would need to be expanded to at least account for
sd_notifyf (which will require a variadic macro, but that shouldn't be a
problem in Debian) and sd_booted.

I'm not sure what the best thing to do about the sd_is_* calls would be.
One possible approach would be to just define them all to -ENOSYS, since
they generally wouldn't be called when sd_listen_fds isn't available,
although that would be a problem if any package ever used those APIs
outside of a systemd context. (But I'm not sure why one would do that.)

[Tollef Fog Heen]

]] Ian Jackson

Post by Ian Jackson
So I think we need to say what we regard as "reasonable" patches, in
advance. As the Debian maintainer for uservd (for example), am I
entitled to decline to incorporate systemd integration into my package
on the grounds that the patch involves additional build- and runtime
dependencies which I consider undesirable ?

If find it incredible that you consider adding 18 lines of
unconditional code to your daemon unreasonable, while forcing the
systemd maintainers to split the package, add explicitly rejected
upstream-incompatible features reasonable.

[Russ Allbery]

Post by Ian Jackson
Well, no. I think that even if we select upstart as the default, we
should enable the systemd community to provide as complete a set of
integration in Debian as they care to put the work in for.
That translates directly to an expectation that the maintainer of any
daemon package must be willing to accept reasonable systemd integration
patches.
If the systemd community is very dynamic (and certainly we can't fault
their dynamism so far) this may well translate to all or almost all
daemons. Certainly it could be expected to include most of the core
daemons where the extra dependencies are most troublesome.
So I think we need to say what we regard as "reasonable" patches, in
advance. As the Debian maintainer for uservd (for example), am I
entitled to decline to incorporate systemd integration into my package
on the grounds that the patch involves additional build- and runtime
dependencies which I consider undesirable ?

Ah! Okay, now I understand why you think this is linked and something we
should decide here. Thank you.

Okay, so let me see if I understand what we agree on and what we would
need to debate. Would you agree with both of these statements?

* If we adopt systemd, then obviously integration with systemd would be
desirable and should be accepted, including build and runtime
dependencies as is appropriate for how systemd is integrated into the
distribution as a whole.

* If a maintainer wishes to integrate with systemd via build and runtime
dependencies in their daemon, that would be their call, not something
the TC would decide. (I think you do agree with this given the
statement below, but leaving this in as summary.)

If so, then the problem reduces to what to do in the case where someone
requests integration with systemd via a bug report and the maintainer
doesn't want to provide that integration, at least with build or runtime
dependencies or both.

Let's put aside the question of how to handle this on the ports that don't
have systemd for the moment, and hence the conditional build integration
parts, since that was discussed in the other thread and I think there are
possible lightweight schemes that are specific to the case where we just
want to stub out the calls.

I assume that the reason why you're bringing this up, per your original
message, is that you don't want to introduce either build or runtime
dependencies. I don't, however, understand why; the logic doesn't make
any sense to me. Both of those dependencies are so lightweight that I
have a hard time seeing how anyone would notice they exist, beyond having
a couple of packages installed on the system that wouldn't otherwise be.

I'm inclined to say that asking maintainers to add those dependencies is
appropriate. I'm not aware of a precedent for this for something that
we're not committing to supporting for the archive, but it certainly
parallels other deployments in Debian, such as SELinux and multi-arch.

Post by Ian Jackson
I think the latter is the right approach, for two reasons. Firstly, it
is less work overall. Secondly, it is proper that the maintainers of an
init system should be encouraged to provide as simple and nonintrusive
an integration interface as possible.

I believe that systemd has already provided this, and I find your belief
that it hasn't to be strange. I certainly don't think that we will
succeed, or *should* succeed, in convincing systemd to adopt a new API
that offers no clear benefit over their existing API and is incompatible
with it. If I were them, I'd reject it out of hand as a waste of time and
effort.

Incidentally, the tools in the init-system-helpers package are basically
playing the same role as tools that are in the sysv-rc package now. I
think one could make a pretty good argument that, were we designing this
from scratch today without worrying about history or backward
compatibility, update-rc.d and invoke-rc.d would go into
init-system-helpers, since they're (now) general tools that support a
variety of underlying init systems.

If the dependency on that package is really such a burden, one thing we
could do is just move the two scripts in it into sysv-rc and use sysv-rc
as the package that holds those tools. I don't really care, and I don't
mind having two packages, but looking at the current contents of both,
they're really quite close to the same thing. (sysv-rc also continues to
provide a small amount of additional infrastructure for sysvinit, but it's
pretty limited.)

Post by Ian Jackson
I think maintainers should not be required to introduce the additional
build- and runtime dependencies, if they do not wish to. Obviously if
they want to then that's fine by me.

Ah, good, okay. This is less of a conflict than at first I thought we
had.

I definitely understand the desire to not impose requirements on package
maintainers that they don't like, and to minimize the impact of such
requirements.

Post by Ian Jackson

Post by Russ Allbery
If you feel like Policy should give advice on this, you can bring it up
in the context of Policy.

The TC's mandate includes deciding on policy questions.

I know, but it also contains the following statement:

Technical Committee makes decisions only as last resort.

The Technical Committee does not make a technical decision until
efforts to resolve it via consensus have been tried and failed, unless
it has been asked to make a decision by the person or body who would
normally be responsible for it.

Basically, I'm really uncomfortable having this sort of conversation here
among a rather small group of people. I think it's likely that we're
going to miss implications or impacts, and in some cases it's quite
valuable to have more people available to provide a reality check.

Post by Ian Jackson
I'm afraid to say that this is quite a frustrating approach.
If it will help I can go through the motions of filing a policy bug,
having everyone point out that this is entangled with the whole init
system debate, and having you reject my proposal.
But perhaps we could just pretend I had done that ?

If we're going to discuss it here and now, then my recommendation for
systemd integration is to do what I did with lbcd: use the presence of the
LISTEN_FDS and LISTEN_PID environment variables to trigger socket
activation code, and use the presence of the NOTIFY_SOCKET environment
variable to trigger status notification. In both cases, clear the
environment variables after recovering the data (unless you're using the
watchdog support) so that they aren't inherited by children.

I think this provides the most natural integration with systemd and
reduces the amount of fiddling that the packager has to do in setting up
the daemon configuration. It has the advantage of letting the daemon work
either with socket activation or without, using the same unit
configuration, and continuing to work if one cuts and pastes the same
command line into the shell (to test something, for example).

It doesn't hurt anything if someone wanted to add another flag to the
daemon to tell it to look at those environment variables (as long as it
keeps working when that flag is given but they're not set -- that's
important for being able to switch from socket activation to not), but I
don't see any need to do so.

For upstart readiness, obviously one needs some sort of explicit flag or
trigger to enable the raise(SIGSTOP) behavior, since that will otherwise
cause rather obvious problems in getting the daemon to work outside of
upstart. I prefer to use a command-line flag for this (-Z, per your
recommendation) since it has to be explicitly enabled and there isn't the
same sort of safe fallback if it's missing the way that there is for the
systemd protocol. I don't see any real problem with using an environment
variable, though, as long as it's unset afterwards so that it's not
inherited by children.

[Colin Watson]

Post by Russ Allbery
For upstart readiness, obviously one needs some sort of explicit flag or
trigger to enable the raise(SIGSTOP) behavior, since that will otherwise
cause rather obvious problems in getting the daemon to work outside of
upstart. I prefer to use a command-line flag for this (-Z, per your
recommendation) since it has to be explicitly enabled and there isn't the
same sort of safe fallback if it's missing the way that there is for the
systemd protocol. I don't see any real problem with using an environment
variable, though, as long as it's unset afterwards so that it's not
inherited by children.

Bear in mind that this may often be being done by the Debian maintainer
in advance of upstream, and adding a one-character command-line option
is problematic since that stands a decent chance of clashing.

[Nikolaus Rath]

Post by Russ Allbery
This message is about a transition plan for an init system replacement and
about how to handle portability to our non-Linux ports. I'm keeping the
same subject as Ian's message on the same basic topics and attaching it to
the same thread, but this is more of a separate writeup than a reply.

[...]

Post by Russ Allbery
5. Support for the other init system that was not chosen should be handled
in the same fashion, should a team choose to pursue it. If we select
systemd, package maintainers should still be willing to merge
contributed upstart configuration, with the understanding that they
can't test it and any support is on a best-effort basis only.
Similarly, if we select upstart, package maintainers should be willing
to merge systemd unit files and to enable upstream systemd support
where requested and where it doesn't interfere with the operation of
the daemon under upstart, with the understanding that the package
maintainer is not committing to testing or directly supporting this
configuration.

Thanks all a lot for all your detailed writeups Russ. It is much
appreciated.


I think there is one additional questions that will probably need to be
decided by the tc but hasn't really been discussed yet:

Will packages that explicity depend on a (non-default) init system be
allowed in Debian?


For example, if I want to package a program that depends on a feature
available only in upstart, but systemd was chosen as the default init
system, can I upload this package with a depends on upstart, or will
this be rejected?

If such packages will not be allowed in the archive, does the burden of
making them work with the default init lie on the maintainers of the
default init (to add the missing feature), or the package maintainer (to
work around the features absence if he wants the package in Debian)?


The specific situation that I have in mind is of course when upstart
becomes the default, and gnome becomes dependent on systemd, but I think
the question is larger than just this example.


Best,
-Nikolaus

[Ian Jackson]

Post by Nikolaus Rath
I think there is one additional questions that will probably need to be
Will packages that explicity depend on a (non-default) init system be
allowed in Debian?

My answer to this is "no".

So, firstly, I would say that all packages must, in jessie at least,
continue to support sysvinit. Russ (from the other side of the
upstart/systemd fence) agrees. Failure to support sysvinit would be
an RC bug.

And since all the proposed replacement inits have a compatibility
mode, that naturally means they'll work.

Contributors who support the non-default new init system will be able
to supply patches for native support and should have them accepted.

Post by Nikolaus Rath
If such packages will not be allowed in the archive, does the burden of
making them work with the default init lie on the maintainers of the
default init (to add the missing feature), or the package maintainer (to
work around the features absence if he wants the package in Debian)?

The latter.

Post by Nikolaus Rath
The specific situation that I have in mind is of course when upstart
becomes the default, and gnome becomes dependent on systemd, but I think
the question is larger than just this example.

Such a decision by Gnome (implying ditching all non-Linux
architectures, too) would be very disappointing IMO. I think this is
a bridge we should cross if and when we come to it.

Ian.

[Nikolaus Rath]

Post by Ian Jackson

Post by Nikolaus Rath
I think there is one additional questions that will probably need to be
Will packages that explicity depend on a (non-default) init system be
allowed in Debian?

My answer to this is "no".
So, firstly, I would say that all packages must, in jessie at least,
continue to support sysvinit. Russ (from the other side of the
upstart/systemd fence) agrees. Failure to support sysvinit would be
an RC bug.
And since all the proposed replacement inits have a compatibility
mode, that naturally means they'll work.
Contributors who support the non-default new init system will be able
to supply patches for native support and should have them accepted.

Post by Nikolaus Rath
If such packages will not be allowed in the archive, does the burden of
making them work with the default init lie on the maintainers of the
default init (to add the missing feature), or the package maintainer (to
work around the features absence if he wants the package in Debian)?

The latter.

Just to make sure that I expressed myself correctly: I was not thinking
about a package that depends on a specific init system to start or stop,
but about a program that is really specific to the *new* features of
upstart or systemd.


For example, a hypothetical future program to interactively adjust
program cgroups cannot be sysvinit compatible in any meaningful sense,
because it does not need to be supervised, started, or stopped. However,
this program would depend on the cgroups API offered by systemd. So this
program would not be allowed in Debian, unless its maintainer adds
support for whatever cgroups managed we would eventually use with upstart?


Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

»Time flies like an arrow, fruit flies like a Banana.«

[Ian Jackson]

Post by Nikolaus Rath
For example, a hypothetical future program to interactively adjust
program cgroups cannot be sysvinit compatible in any meaningful sense,
because it does not need to be supervised, started, or stopped. However,
this program would depend on the cgroups API offered by systemd. So this
program would not be allowed in Debian, unless its maintainer adds
support for whatever cgroups managed we would eventually use with upstart?

I would hope that we can standardise on a single API to the system's
single cgroup writer.

Ian.

[Nikolaus Rath]

Post by Ian Jackson

Post by Nikolaus Rath
For example, a hypothetical future program to interactively adjust
program cgroups cannot be sysvinit compatible in any meaningful sense,
because it does not need to be supervised, started, or stopped. However,
this program would depend on the cgroups API offered by systemd. So this
program would not be allowed in Debian, unless its maintainer adds
support for whatever cgroups managed we would eventually use with upstart?

I would hope that we can standardise on a single API to the system's
single cgroup writer.

I certainly hope so too, but I think the question of how such a
situation would be handled needs to be answered. Even if we end up with
a standardized cgroup API, it may show up in a different disguise.


Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C

»Time flies like an arrow, fruit flies like a Banana.«

[Josselin Mouette]

Post by Ian Jackson
I would hope that we can standardise on a single API to the system's
single cgroup writer.

I have already explained why this is not going to happen. The cgroups
API in systemd is already part of the core systemd interface and subject
to the stability promise. The only other existing implementation
(cgmanager) is merely wrapping in D-Bus the existing kernel API, which
is going to die when a single writer becomes necessary.
--
.''`. Josselin Mouette
: :' :
`. `'
`-

[Ian Jackson]

Post by Josselin Mouette

Post by Ian Jackson
I would hope that we can standardise on a single API to the system's
single cgroup writer.

I have already explained why this is not going to happen. The cgroups
API in systemd is already part of the core systemd interface and subject
to the stability promise. The only other existing implementation
(cgmanager) is merely wrapping in D-Bus the existing kernel API, which
is going to die when a single writer becomes necessary.

Err, I'm not sure I see what is stopping the provision of the systemd
cgroup manager API by a program other than systemd. But I haven't
looked at this in detail.

Ian.

[Cameron Norman]

Post by Josselin Mouette

Post by Ian Jackson
I would hope that we can standardise on a single API to the system's
single cgroup writer.

I have already explained why this is not going to happen. The cgroups
API in systemd is already part of the core systemd interface and subject
to the stability promise. The only other existing implementation
(cgmanager) is merely wrapping in D-Bus the existing kernel API, which
is going to die when a single writer becomes necessary.

It seems like you just explained how it has already happened. systemd's
cgroup interface is covered under the stability promise, and seems like a
good fit for a standard interface to the single cgroup arbitrator.

Cameron

[Tollef Fog Heen]

]] Ian Jackson

Post by Ian Jackson

Post by Nikolaus Rath
I think there is one additional questions that will probably need to be
Will packages that explicity depend on a (non-default) init system be
allowed in Debian?

My answer to this is "no".
So, firstly, I would say that all packages must, in jessie at least,
continue to support sysvinit. Russ (from the other side of the
upstart/systemd fence) agrees. Failure to support sysvinit would be
an RC bug.

I think it would be unwise to require upstart and systemd to continue to
support sysvinit. I'm not even sure what that would mean, in particular
in the case of systemd-sysv whose sole purpose is to replace sysvinit.

[Ian Jackson]

Post by Tollef Fog Heen

Post by Ian Jackson
So, firstly, I would say that all packages must, in jessie at least,
continue to support sysvinit. Russ (from the other side of the
upstart/systemd fence) agrees. Failure to support sysvinit would be
an RC bug.

I think it would be unwise to require upstart and systemd to continue to
support sysvinit. I'm not even sure what that would mean, in particular
in the case of systemd-sysv whose sole purpose is to replace sysvinit.

I was speaking looaely. I meant, still loosely speaking but rather
less so, all packages which contain daemons which are to be started by
the init system.

In my draft resolution I gave an even longer and more precise
definition which proves still yet to have edge cases people aren't
happy with. But I hope you understand roughly what I'm getting at.

For the avoidance of doubt, I mean to include things like inetd and
apache and the system dbus and udev and the nameserver, but to exclude
things like systemd-sysv and upstart-socket-bridge.

The wording in my draft resolution is designed to tolerate (although
obviously not encourage) a hypothetical daemon whose bare bones
packaging doesn't arrange for the daemon to be started at all,
regardless of the init system in use.

Ian.