Discussion:
Bug#815793: IPv6 code ignores unsolicited router advertisements
(too old to reply)
Martin Pitt
2016-02-28 21:20:22 UTC
Permalink
Hello Marc,
systemd 229 seems to ignore unsolicited router advertisements. This
breaks, for example, prefix changes. And it prevents the self-healing
characteristics of lost router solicitations.
Can you please report this and also #815884 upstream? I don't know
much about IPv6 routing I'm afraid, so I think you are in a better
position to explain/discuss this with Tom. Some regressions are
already covered by https://github.com/systemd/systemd/issues/2572
but not these two.

In the meantime, I reverted the networkd handling of RA to go back to
the pre-229 behaviour.

Many thanks!

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Marc Haber
2016-02-29 09:20:19 UTC
Permalink
Hi Martin,
Post by Martin Pitt
systemd 229 seems to ignore unsolicited router advertisements. This
breaks, for example, prefix changes. And it prevents the self-healing
characteristics of lost router solicitations.
Can you please report this and also #815884 upstream? I don't know
much about IPv6 routing I'm afraid, so I think you are in a better
position to explain/discuss this with Tom. Some regressions are
already covered by https://github.com/systemd/systemd/issues/2572
but not these two.
I think that it would not be productive if I tried communicating with
systemd upstream. We both consider each other toxic, and I'd rather
not mess with my sanity trying to be nice to a <censored>.

I lost my temper on systemd-devel in december and cannot write there
any more due to policy decision of the list owner. I doubt that a bug
report filed by me would get any helpful upstream attention.

So, I regret that in this case my answer is thanks, but no thanks.

Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Martin Pitt
2016-07-06 06:57:43 UTC
Permalink
Control: reopen -1
Control: tag -1 moreinfo

Hello Marc,

while we reverted the change in 229, we don't want to carry the
reversion forever. Also, some problems were fixed already in 230, like
[1]. I forwarded this upstream to
https://github.com/systemd/systemd/issues/3663, and will now play
relay :-)
systemd 229 seems to ignore unsolicited router advertisements. This
breaks, for example, prefix changes. And it prevents the self-healing
characteristics of lost router solicitations.
14:41:40.062484 7e:79:61:31:55:28 > 33:33:00:00:00:01, ethertype IPv6 (0x86dd), length 294: fe80::1 > ff02::1: ICMP6, router advertisement, length 240
Do you have some instructions how this can be reproduced? It sounds
like it should be possible to test this using a veth pair, possibly a
dnsmasq on one end, and some ip commands. Then I'd like to write a
test case for [2].

If this is hardware specific, can you please try this with 230 with
the patch reverted? I build packages for amd64 here:
https://people.debian.org/~mpitt/tmp/systemd-userspace-ndisc/
(there's also a Packages.gz so this can be used as an apt deb line)
There were a lot of changes to the RA handling in 230.

If it still happens, can you please run it in debugging mode with

systemd-analyze set-log-level debug
systemctl restart systemd-networkd

then reproduce the bug, and finally

systemd-analyze set-log-level info
journalctl -u systemd-networkd.service > /tmp/networkd.log

and append the log file here?

Thanks!

[1] https://github.com/systemd/systemd/issues/2572
[2] https://github.com/systemd/systemd/blob/master/test/networkd-test.py
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Marc Haber
2016-07-06 07:03:04 UTC
Permalink
Hi Martin,
Post by Martin Pitt
while we reverted the change in 229, we don't want to carry the
reversion forever. Also, some problems were fixed already in 230, like
[1]. I forwarded this upstream to
https://github.com/systemd/systemd/issues/3663, and will now play
relay :-)
Thanks for your consideration.
Post by Martin Pitt
Do you have some instructions how this can be reproduced? It sounds
like it should be possible to test this using a veth pair, possibly a
dnsmasq on one end, and some ip commands. Then I'd like to write a
test case for [2].
If this is hardware specific, can you please try this with 230 with
https://people.debian.org/~mpitt/tmp/systemd-userspace-ndisc/
(there's also a Packages.gz so this can be used as an apt deb line)
There were a lot of changes to the RA handling in 230.
I will try this, but since my current project is in a critical phase I
do not have too much time. I am confident that I can produce results
during july, but most probably not next weekend. I will keep you posted.

I recently bought new hardware for virtualization and will use
building the new system as test lab, so your request comes just at the
right time since I can now experiment without breaking productive
systems.

Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Martin Pitt
2016-07-06 07:29:35 UTC
Permalink
Hey Marc,
Post by Marc Haber
Post by Martin Pitt
Do you have some instructions how this can be reproduced? It sounds
like it should be possible to test this using a veth pair, possibly a
dnsmasq on one end, and some ip commands. Then I'd like to write a
test case for [2].
If this is hardware specific, can you please try this with 230 with
https://people.debian.org/~mpitt/tmp/systemd-userspace-ndisc/
(there's also a Packages.gz so this can be used as an apt deb line)
There were a lot of changes to the RA handling in 230.
I will try this, but since my current project is in a critical phase I
do not have too much time. I am confident that I can produce results
during july, but most probably not next weekend. I will keep you posted.
Thanks. Testing the above packages will indeed take some time, but I
wondered if you could give me a hint whether this can be reproduced
with something like "set up a normal RA server/IPv6 networkd client"
(I know how to do that of course) and then change foo/run blah to
trigger this".

Thanks,

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Marc Haber
2016-07-06 07:42:33 UTC
Permalink
Post by Martin Pitt
Post by Marc Haber
Post by Martin Pitt
Do you have some instructions how this can be reproduced? It sounds
like it should be possible to test this using a veth pair, possibly a
dnsmasq on one end, and some ip commands. Then I'd like to write a
test case for [2].
If this is hardware specific, can you please try this with 230 with
https://people.debian.org/~mpitt/tmp/systemd-userspace-ndisc/
(there's also a Packages.gz so this can be used as an apt deb line)
There were a lot of changes to the RA handling in 230.
I will try this, but since my current project is in a critical phase I
do not have too much time. I am confident that I can produce results
during july, but most probably not next weekend. I will keep you posted.
Thanks. Testing the above packages will indeed take some time, but I
wondered if you could give me a hint whether this can be reproduced
with something like "set up a normal RA server/IPv6 networkd client"
(I know how to do that of course) and then change foo/run blah to
trigger this".
I think this particular issue can be triggered by:

- set up a router running radvd
- start up a client with networkd and systemd IPv6 implementation
- manually remove any IPv6 address and IPv6 route from the client
- restart the radvd on the router

The client should pick up the prefix and route that the newly started
radvd announces. At least this would be my first test.

I _think_ that I stumbled upon this issue because the systemd IPv6
code sent out its _single_ router solicitation so early that the
network link was not yet up (thus, no IPv6 on the client), and then
proceeded to ignore the regular route updates sent out by the radvd
because it didn't ask for them.

Ignoring unsolicited announcements is state of the art for ARP and
DHCP to protect a system's caches against poisioning attacks, which is
probably the reason why this was implemented in systemd IPv6 code as
well. But, IPv6 works differently and it is necessary for network
functionality to listen and to act even on unsolicited router
announcements.

At least this is how I explained to myself why the code was written
this way. Too bad they didn't test it in reality but instead released
it.

Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Martin Pitt
2016-07-06 09:49:17 UTC
Permalink
Hello Marc,
Post by Marc Haber
- set up a router running radvd
- start up a client with networkd and systemd IPv6 implementation
- manually remove any IPv6 address and IPv6 route from the client
- restart the radvd on the router
The client should pick up the prefix and route that the newly started
radvd announces. At least this would be my first test.
I did that now:

| ip link add name enc type veth peer name ens
| ip a add 2600::1/64 dev ens
| ip link set ens up
|
| cat <<EOF > /tmp/ra.conf
| interface ens
| {
| AdvSendAdvert on;
| prefix 2600::1/64
| {};
| };
| EOF
|
| radvd -C /tmp/ra.conf -n -m stderr -d2

Other terminal:

| mkdir -p /run/systemd/network
| cat <<EOF > /run/systemd/network/enc.network
| [Match]
| Name=enc
|
| [Network]
| IPv6AcceptRouterAdvertisements=yes
| EOF
|
| SYSTEMD_LOG_LEVEL=debug /lib/systemd/systemd-networkd

I also ran "tcpdump -i ens" in parallel, as before. This does the
solicitation and picks up radvd's RA and I get some random 2600:*
address:

| Disc CLIENT: Received Router Advertisement: flags none preference medium lifetime 1800 sec
| NDisc CLIENT: Update prefix 2600:0000:0000:0000:0000:0000:0000:0001/64 lifetime 86400 expires in 1d
| ens: Updating address: 2600::7852:45ff:fed7:9ab4/64 (valid for 1d)

I now killed radvd, then "ip a del 2600:[..] dev enc", and restart
radvd. And once again networkd picks this up and adds an IPv6 address.

This looks the same for 229, 230/upstream and 230/sid (with the
reverted patch). Thus I cannot confirm that networkd ignores these
unsolicited router advertisements.

So re-trying with my test packages and getting a networkd debug log
would be appreciated. Thanks!

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Marc Haber
2016-07-06 09:52:50 UTC
Permalink
Post by Martin Pitt
This looks the same for 229, 230/upstream and 230/sid (with the
reverted patch). Thus I cannot confirm that networkd ignores these
unsolicited router advertisements.
Can you, just for reference, try with systemd 229-1, just to see
wheteher your setup triggers the bug in the same software version that
I reported it with?
Post by Martin Pitt
So re-trying with my test packages and getting a networkd debug log
would be appreciated. Thanks!
Will try, but probably only later this or even next week.

Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Martin Pitt
2016-07-06 09:55:19 UTC
Permalink
Hello Marc,
Post by Marc Haber
Post by Martin Pitt
This looks the same for 229, 230/upstream and 230/sid (with the
reverted patch). Thus I cannot confirm that networkd ignores these
unsolicited router advertisements.
Can you, just for reference, try with systemd 229-1
I did (that was the "229" above).

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)
Marc Haber
2016-07-06 11:15:53 UTC
Permalink
Post by Martin Pitt
Post by Marc Haber
Post by Martin Pitt
This looks the same for 229, 230/upstream and 230/sid (with the
reverted patch). Thus I cannot confirm that networkd ignores these
unsolicited router advertisements.
Can you, just for reference, try with systemd 229-1
I did (that was the "229" above).
Ah, I thought you meant 229 with the ipv6 code disabled. I will try to
reproduce things.

Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Marc Haber
2016-07-15 19:04:59 UTC
Permalink
Hi Martin,
Post by Martin Pitt
while we reverted the change in 229, we don't want to carry the
reversion forever. Also, some problems were fixed already in 230, like
[1]. I forwarded this upstream to
https://github.com/systemd/systemd/issues/3663, and will now play
relay :-)
If this is hardware specific, can you please try this with 230 with
https://people.debian.org/~mpitt/tmp/systemd-userspace-ndisc/
(there's also a Packages.gz so this can be used as an apt deb line)
There were a lot of changes to the RA handling in 230.
Unfortunately the system I have been seeing this on is a Banana Pi,
thus armhf, and I cannot run your test packages on this system. This
bug does not, however, show itself on a reference system running amd64
with 230-5pitti1.

I have found another showstopper IPv6 bug in 230-5pitti1 and have duly
reported it.

Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Marc Haber
2016-07-16 12:16:25 UTC
Permalink
Hi Martin,
Post by Marc Haber
Post by Martin Pitt
while we reverted the change in 229, we don't want to carry the
reversion forever. Also, some problems were fixed already in 230, like
[1]. I forwarded this upstream to
https://github.com/systemd/systemd/issues/3663, and will now play
relay :-)
If this is hardware specific, can you please try this with 230 with
https://people.debian.org/~mpitt/tmp/systemd-userspace-ndisc/
(there's also a Packages.gz so this can be used as an apt deb line)
There were a lot of changes to the RA handling in 230.
Unfortunately the system I have been seeing this on is a Banana Pi,
thus armhf, and I cannot run your test packages on this system. This
bug does not, however, show itself on a reference system running amd64
with 230-5pitti1.
I regret to inform you that we have a heisenbug here. When
reconfiguring and restarting radvd, systemd sometimes decides to act
on the new announcement, and sometimes doesn't.

For example, after receiving this announcement:

13:40:23.075889 IP6 (flowlabel 0x4717d, hlim 255, next-header ICMPv6 (58) payload length: 240) fe80::1 > ff02::1: [icmp6 sum ok] ICMP6,
hop limit 64, Flags [none], pref medium, router lifetime 1800s, reachable time 0s, retrans time 0s
prefix info option (3), length 32 (4): 2a01:db8:0::3282::/64, Flags [onlink, auto], valid time 86400s, pref. time 14400s
route info option (24), length 24 (3): ::/0, pref=high, lifetime=1800s
route info option (24), length 24 (3): 2000::/3, pref=high, lifetime=1800s
route info option (24), length 24 (3): 2a01:db8:0::3280::/60, pref=high, lifetime=1800s
route info option (24), length 24 (3): 2a01:db8:0::32b0::/60, pref=high, lifetime=1800s
rdnss option (25), length 40 (5): lifetime 600s, addr: 2a01:db8:0::3281::35:100 addr: 2a01:db8:0::328e::35:100
dnssl option (31), length 48 (6): lifetime 600s, domain(s): zugschlus.de. ka51.zugschlus.de.
source link-address option (1), length 8 (1): 7e:79:61:31:55:28

systemd didn't configure an SLAAC IPv6 address on the interface. I
guess that this depends on whether and how many other IP addresses are
on the interface. It seems to work more reliably if the interface
doesn't already have an address from this prefix. If this is the case,
it is another proof of the gross misunderstanding of IPv6 that the
person writing this piece of the code has.

Greetings
Marc
--
-----------------------------------------------------------------------------
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany | lose things." Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature | How to make an American Quilt | Fax: *49 6224 1600421
Continue reading on narkive:
Loading...