Software Performance Tuning: Follow the Data, or the Buzzwords?
== Software Performance Tuning: Follow the Data, or the Buzzwords?
Many people cannot discuss performance without bringing up zero copy. Consider Wikipedia’s definition:
“Zero-copy” describes computer operations in which the CPU does not perform the task of copying data from one memory area to another. This is frequently used to save CPU cycles and memory bandwidth when transmitting a file over a network.
Clearly, zero-copy means that when data must be copied from point A in memory to point B, the CPU does not participate in the process—that is, the copy is not performed by calling memcpy(). This process is usually handled by a device driver, which uses DMA (direct memory access) to bypass the CPU and shunt data between a peripheral (typically a NIC, i.e., network interface card) and main memory. Here’s a diagram:
How DMA works, how it is initiated, how it notifies the CPU, and how cache coherence is handled are beyond the scope of this article.
Of course, this is zero copy in the narrow sense. In the broader sense, zero copy also covers the entire lifetime of the data after DMA finishes: copy the data as little as possible (definitely not zero copies!).
Let’s look at how the kernel receives packets (simplified):
- When the driver initializes, it allocates a ring buffer for the NIC.
- When the NIC receives a packet, it finds the next free buffer, DMA-copies the packet into the memory specified by that buffer, and raises an interrupt to notify the kernel. (First zero copy.)
- The kernel handles the interrupt, takes ownership of that buffer for further processing, then allocates a new buffer and returns it to the driver. (Second zero copy.)
- Next, the data is handed to the TCP stack (since we’re talking about server development, usually a TCP server). TCP is a stream-based protocol. Here, packet ordering, reassembly (stripping TCP headers and concatenating payloads for the application), and so on may require at least one copy depending on the implementation. In addition, if we are the sender, because potential retransmission must be considered, a copy is usually made into the retransmit queue. (Special network-device optimizations are outside the scope of this discussion.)
- After that, the data is handed to the application, which involves switching from kernel space to user space. Mapping the application’s buffer to the kernel’s buffer is not simple, so there is generally another copy here.
Of course, there are now technologies that skip the kernel stack and DMA directly from the NIC into user space, but that is usually the kind of thing network vendors do. I think when we build server software, we should not steal the TCP stack’s business; otherwise you’ll get yourself killed.
Basically, these actions all happen in the kernel, beyond the application’s control. The benefit is: when you call recv(), you get a reassembled, linearly readable buffer containing only the TCP payload. You only need to care about application-level logic. All you can control is how little you copy that buffer afterward. After this long detour, we finally return to the earlier question: if we don’t talk about zero copy, what is there to say about server performance or optimizing system calls!
Unfortunately, we can only control application-level zero copy.
Suppose we write an HTTP server. Most HTTP request headers are not that large, unless we’re posting a file or the application server sets enormous cookies. Let’s assume that on average the whole request is 4 KB (this assumption is worthless; it’s only for comparison), and that our server is slightly daft: in addition to the copies in normal processing, it also memmcpy()s the entire request once. To be fair, we won’t compare against system calls that take a relatively long time; we’ll just compare against gettimeofday(). The main code is as follows:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/time.h>
#define ROUNDS 10000000
#define BUF_SIZE 4096
void get_time() {
struct timeval tv;
struct timezone tz;
int i;
for (i=0; i<ROUNDS; i++) {
gettimeofday(&tv, &tz);
}
}
void buf_cpy(void *src, size_t n) {
void *data = malloc(n);
int i;
for (i=0; i<ROUNDS; i++) {
memcpy(data, src, n);
}
free(data);
}CC=cc
PROF=gprof
SRCS=test.c
OBJS=$(SRCS:.c=.o)
TARGET=test
build:
@$(CC) -g -c $(SRCS) -pg
@$(CC) -o $(TARGET) $(OBJS) -pg
@echo "DONE!"
prof:
@time ./$(TARGET) && $(PROF) $(TARGET) -pCompile and run with gprof, then inspect the results:
(Note: you’d better write code and test this yourself; results may vary across environments. I used the smallest DigitalOcean instance, Ubuntu 14.04.)
Ah, it seems memcpy() is not as slow as I imagined—just a little slower than gettimeofday().
So here’s the question: will you go to great lengths to optimize the tiny, scattered, basically unavoidable copies, or will you call gettimeofday() once per second instead of once per packet, eliminating 99.99% of the calls?
Will you merge the two write() calls that send the header and body separately into one, reducing the tens-of-milliseconds network round-trip delay, or will you go to great lengths to optimize scattered copies?
Will you use tools such as strace, gprof, and systemtap to trace the true performance bottleneck and prescribe the right remedy, or will you jump up without thinking and declare: any server software that discusses performance without mentioning zero-copy is just hooliganism!
When you look down on the overhead of system calls, do you also know that while you are doggedly pursuing zero-copy, the kernel has been doing its best to provide various extended system calls to make certain application scenarios as fast as possible? For example, sendfile(). If your response is a static file, you can easily achieve zero-copy through this system call.
Writing all this is not to prove how right I am; my knowledge may also be wrong. It is only to ask: when we encounter a problem, do we truly measure, or do we follow the crowd, spit out a few buzzwords, and convince ourselves the problem is solved?
