Performance of Linux IP Aliased Network Interfaces: Lessons Learnt
posted at June 4, 2012 with tags kernel, linux, networking

In an earlier post, I had put together a set of benchmarks to measure the performance of IP aliased network interfaces in Linux. While I was expecting a performance degredation due to IP aliasing related book keeping at the kernel level, suprisingly results were pointing out that the overall network throughput measured over the used physical interface increases proportional to the number of aliased interfaces. In this post, I will re-investigate the subject with the lessons I learnt from my previous try.

To begin with, I want to address Chris Siebenmann’s concerns regarding the raw performance of the used network interface card. For this purpose, I first replaced the NIC with a real one (RTL8111/8168B) that is capable of achieving gigabit rates. Then, I started playing with iperf parameters. After a couple of tries, I figured out that the game changer is TCP MSS (maximum segment size) for me. Setting MSS to 1448 helped me to boost my speed to gigabit rates. Other configurations (e.g., disabling TCP delays, i.e., Nagle’s algorithm) did not change the results that much, but I present them hereby anyway.

New results are as follows. (Each pass is set to run for 60 seconds.)

# of alias 1 2 4 8 16 32 64
Kbits/sec (tcp) 920,462 921,468 921,311 920,909 920,679 920,770 922,861
Kbits/sec (nodelay) 921,261 921,184 921,179 920,832 920,629 920,732 921,521
# of alias 96 128 160 192 224 254  
Kbits/sec (tcp) 926,282 969,967 949,506 952,449 986,355 990,832  
Kbits/sec (nodelay) 925,444 945,879 949,204 953,509 973,846 993,143  

Results

Ok, the results are cool. But how do we explain them? For this purpose, by finding his name from Wikipedia IP Aliasing page, I got in touch with Juan José Ciarlante – the author of first IP aliasing support in Linux kernel in 1995 – and he kindly replied my questions. Below, I directly quote from his own words.

you may be “just” exploiting the fact that you’re using more TCP connections (==N_WORKERS) as number of aliases increases, and thus increasing the “parallelism of your transferring pipes”, much in a way p2p networks do (spread the downloading between zillion of connections), suggest reading about TCP congestion window control semantics.

Finally, we have a valid answer!