Linux – RHEL 6 / CentOS 6 two NICs in the same subnet, but secondary doesn’t ping

Recently I ran into a problem with RHEL 6 (and any derivatives, like CentOS 6 or Scientific Linux 6) where having two NICs (network interfaces) in the same subnet resulted in strange behaviour. In RHEL ≤5 (or CentOS ≤5), one could have two interfaces with IPs in the same subnet and there weren’t any problems (besides the obvious question of why one would set it up this way instead of just bonding the interfaces). However, in RHEL 6 (or CentOS 6), having two interfaces with IPs in the same subnet results in the primary one pinging but the secondary one not responding.

The cause of this problem is that the rp_filter settings changed between these kernels (2.6.18 in RHEL 5 and 2.6.32 in RHEL 6). In RHEL 5, the rp_filter setting was a boolean where 1 meant that source validation was done by reversed path (as in RFC1812), and 0 meant no source validation. However, in RHEL 6, this setting changed to an integer with the following settings:

*****
0 – No source validation

1 – Strict Reverse Path validation (RFC3704) – Packets are checked against the FIB (Forwarding Information Base), and only the best ones succeed

2 – Loose Reverse Path validation (RFC3704) – Packets are checked against the FIB, but only non-reachable BY ANY INTERFACE will fail
*****

So, though the default setting is still 1, it now has a different meaning. In order to get these two network interfaces with IPs in the same subnet to both respond, I needed to make two changes in /etc/sysctl.conf:

  • Change net.ipv4.conf.default.rp_filter from ’1′ to ’2′
  • Add the line net.ipv4.conf.all.rp_filter = 2

To better illustrate the changes, here are the differences:

DEFAULT SETTINGS:
# grep '.rp_filter' /etc/sysctl.conf
net.ipv4.conf.default.rp_filter = 1

REQUIRED SETTINGS:
# grep '.rp_filter' /etc/sysctl.conf
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.rp_filter = 2

In order to make these changes effective immediately, you can reload the configuration with:

# sysctl -p

Ultimately, the new defaults make it so that the kernel discards packets when the route for outbound traffic differs from the route of incoming traffic. Changing these settings as mentioned above will make the kernel handle those packets like it did before 2.6.32. That way, having two or more interfaces with IPs in the same subnet will function as intended. Also, these changes aren’t limited to just RHEL 6 and derivatives, but also to any distribution with ≥kernel-2.6.32 in which the defaults were not changed.

Cheers,
Zach

18 thoughts on “Linux – RHEL 6 / CentOS 6 two NICs in the same subnet, but secondary doesn’t ping

    • Hi Andy,

      Really glad that you liked the article. I fought with this one for far too long, so I thought that it might spare someone else the pain. :)

      Cheers,
      Zach

  1. Great post, thanks for sharing!

    It sounds like this could impact home users whose wired and wireless connections are in the same subnet and, for some reason, wish to use the wireless interface.

    An unusual case might be a technician using his wireless access to troubleshoot an Ethernet switch in the same subnet, although this very case is an argument to segment the network into separate VLANs.

  2. Hi Zach,

    Thank you for posting this excellent article. You helped me resolve an issue I’d been stuck on for over a week.

    Cheers,
    Robert

  3. Zach,

    You saved my life. Thank you so very much for this article.
    I was stuck and god only know what not I tried to get it working.

    Thanks,
    Krishna

  4. BINGO!!! Saved me a bunch of headaches. I started to suspect infrastructure routers/switches (it’s new). But also wondered if it was a RHEL change from 5 to 6.

Leave a Reply

Your email address will not be published. Required fields are marked *


*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>