Important!
This article is
THE most-viewed on The Z-Issue, and is sometimes read tens of thousands of times per day. If it has helped you, please consider a small donation to
The Parker Fund by using the top widget at the right. Thanks!
Recently I ran into a problem with RHEL 6 (and any derivatives, like CentOS 6 or Scientific Linux 6) where having two NICs (network interfaces) in the same subnet resulted in strange behaviour. In RHEL ≤5 (or CentOS ≤5), one could have two interfaces with IPs in the same subnet and there weren’t any problems (besides the obvious question of why one would set it up this way instead of just bonding the interfaces). However, in RHEL 6 (or CentOS 6), having two interfaces with IPs in the same subnet results in the primary one pinging but the secondary one not responding.
The cause of this problem is that the rp_filter settings changed between these kernels (2.6.18 in RHEL 5 and 2.6.32 in RHEL 6). In RHEL 5, the rp_filter setting was a boolean where 1 meant that source validation was done by reversed path (as in RFC1812), and 0 meant no source validation. However, in RHEL 6, this setting changed to an integer with the following settings:
*****
0 – No source validation
1 – Strict Reverse Path validation (RFC3704) – Packets are checked against the FIB (Forwarding Information Base), and only the best ones succeed
2 – Loose Reverse Path validation (RFC3704) – Packets are checked against the FIB, but only non-reachable BY ANY INTERFACE will fail
*****
So, though the default setting is still 1, it now has a different meaning. In order to get these two network interfaces with IPs in the same subnet to both respond, I needed to make two changes in /etc/sysctl.conf:
- Change
net.ipv4.conf.default.rp_filter
from ‘1’ to ‘2’
- Add the line
net.ipv4.conf.all.rp_filter = 2
To better illustrate the changes, here are the differences:
DEFAULT SETTINGS:
# grep '.rp_filter' /etc/sysctl.conf
net.ipv4.conf.default.rp_filter = 1
REQUIRED SETTINGS:
# grep '.rp_filter' /etc/sysctl.conf
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.rp_filter = 2
====================
Update – 19 April 2017: (Big thanks to Henry Butterworth for making me aware of this caveat)
In some distributions, changing the global settings for default
and/or all
is not sufficient. In those cases, you also need to specify the rp_filter
setting for each ethernet interface individually. For example, if you have eth0
and eth1
to configure, you will need the following lines in your /etc/sysctl.conf
:
net.ipv4.conf.eth0.rp_filter = 2
net.ipv4.conf.eth1.rp_filter = 2
====================
In order to make these changes effective immediately, you can reload the configuration with:
# sysctl -p
Ultimately, the new defaults make it so that the kernel discards packets when the route for outbound traffic differs from the route of incoming traffic. Changing these settings as mentioned above will make the kernel handle those packets like it did before 2.6.32. That way, having two or more interfaces with IPs in the same subnet will function as intended. Also, these changes aren’t limited to just RHEL 6 and derivatives, but also to any distribution with ≥kernel-2.6.32 in which the defaults were not changed.
Cheers,
Zach