I spent a month optimizing my epoll based HTTP server from 15k req/sec to 125k req/sec

Greetings to my fellow nerds.

A month ago, I had zero network programming experience. So I decided to fix that by building an epoll based HTTP server from scratch and benchmarked every major architectural change along the way.

# Performance Benchmarks :

* **Benchmark command** : `wrk -t4 -c10000 -d10s http://127.0.0.1:8080/`
* **Request**: `GET /index.html`
* **Response**: Static HTML file (\~1500 bytes)
* **CPU**: Intel i5-13420H (13th Gen)
* **Compiler**: Clang (O3)

|Architecture|Throughput (req/sec)|Description|
|:-|:-|:-|
|**Blocking**|\~15k|Single threaded blocking accept/read/write|
|**Epoll (LT)**|\~34k|Single threaded event loop utilizing non blocking I/O multiplexing|
|**Epoll (LT, keep alive)**|\~37.5k|Single threaded event loop with persistent connections|
|**Epoll (LT, keep alive, sendfile)**|\~41k|Single threaded event loop with persistent connections and zero copy file serving|
|**Epoll (LT, keep alive, sendfile, multithreading)**|\~125k|Multithreaded architecture running 4 concurrent epoll loops (optimal on test machine)|

# Some Surprising Observations :

* sendfile mattered less than I expected... for a server whose entire purpose is to serve files, I was expecting a bigger gain but maybe because my file was only \~1.5KB, it did not help much.
* More threads made things worse :

|Worker Threads|Throughput (req/sec)|
|:-|:-|
|1|\~40k|
|2|\~95k|
|3|\~115k|
|4|\~125k|
|5|\~90k|
|6|\~90k|
|8|\~75k|
|10|\~70k|
|12|\~65k|

My CPU has 6 physical cores and 12 logical processors, I suspect that the cost of all the syscalls for every loop, context switching, and lock contention on shared kernel objects, dominated on higher thread counts. Though I havent fully investigated it yet.

# Profiling with perf :

|Function|Approx. CPU Samples|
|:-|:-|
|readSock()|\~22%|
|writeSock()|\~16%|
|parse()|\~8%|
|std::format()|\~7%|
|open()|\~3%|
|sendfile()|\~2.5%|

Turns out Im still spending more time reading and parsing requests than sending responses, meaning there might still be room for batched reads or buffer pooling in a future iteration...

# Final Thoughts :

I could hunt for possible micro optimizations or even experiment with an edge triggered architecture but im kinda burnt out at this point and this feels like a great point to end this project...

The codebase is pretty small (\~1k LOC), so if anyone's interested in taking a look : [https://github.com/Raju1173/epoll-http-server](https://github.com/Raju1173/epoll-http-server)

https://redd.it/1twh4e1
@r_cpp