class: center, middle # Using EPoll with Multiple Threads ### [http://goo.gl/KX6eZF](http://goo.gl/KX6eZF) ### Ross Delinger --- class: center, middle # Who am I? ### Computer Science House Member ### Distributed Systems Geek ### Obsessed with High Performance Servers --- class: center, middle # What is EPoll? --- # Its Pretty Awesome ### Scalable file descriptor event notification ### Have the kernel tell you when your data is ready ### Used for all that fancy 'event loop' stuff like node.js has ### [libuv](https://github.com/joyent/libuv) uses epoll as a backend on linux --- class: center, middle # Why use EPoll --- # Radical Performance! ### No seriously, EPoll and its equivalents such as kqueue are super fast. ### Used for many projects that need absurd performance * libuv (Used in Node js) * Nginx * Gevent (python), uses libev, which is built on epoll/kqueue --- # Why use EPoll with multiple threads? ### You need to handle a *LOT* of connections ### DDoS ### Surges of legit traffic * Like being slashdotted (reddit hug of death, HN Effect, etc) ### In my case the [Ultimate Victory Battle](https://github.com/rossdylan/uvb-server) ### UVB is a game, someone makes a server, everyone else breaks it --- # How it works ### You ask the kernel to update you when a file descriptor changes ### You can make a new EPoll instance like so ```c #include
#include
int epoll_fd; if((epoll_fd = epoll_create1(0)) == -1) { perror("epoll_create"); exit(EXIT_FAILURE); } ``` ### This returns a file descriptor which is used to control epoll --- # Adding File Descriptors to an EPoll set ```c typedef struct { int fd; int otherThing; } Data; struct epoll_event event; // event.data.ptr how you attach some persistent data to a fd in epoll if((event.data.ptr = malloc(sizeof(Data))) == NULL) { perror("malloc"); exit(EXIT_FAILURE); } // we are going to assume that socket_fd is an open fd event.events = EPOLLIN | EPOLLET; if((epoll_ctl(epoll_fd, EPOLL_CTL_ADD, socket_fd, &event)) == -1) { perror("epoll_create"); exit(EXIT_FAILURE); } ``` --- # Wait? What did we just do? ### Add some custom data to the event, How we identify a connection inside the server ``` event.data.ptr = malloc(sizeof(Data)) ``` ### Tell epoll what events we want, and how we want to operate ``` event.events = EPOLLIN | EPOLLET; ``` ### Tell epoll to link a socket file descriptor to our new event ``` epoll_ctl(epoll_fd, EPOLL_CTL_ADD, socket_fd, &event) ``` --- # What about EPOLLIN | EPOLLET? ### Tell epoll to notify us when there is data to read * EPOLLOUT is for writing data ### EPOLLET stands for EPoll Edge Triggered * This changes EPoll to return from epoll_wait whenever a fd changes status. * Must be used in conjunction with nonblocking sockets --- # Actually using epoll ```c // An array of events struct epoll_event *events; // A single event we use as scratch space struct epoll_event event; events = calloc(MAXEVENTS, sizeof(struct epoll_event)) int n; while (1) { n = epoll_wait(kepoll_fd, events, MAXEVENTS, -1); for(int i = 0; i < n; i++) { Data dat = (Data *)events[i].data.ptr; if(events[i].events & EPOLLIN) { /** Read from dat->fd */ } } } ``` --- # Wat? ### Make an array for the events the kernel has given us ```c events = calloc(MAXEVENTS, sizeof(struct epoll_event)) ``` ### Grab the data we attached to the event ```c Data dat = (Data *)events[i].data.ptr; ``` ### Check if there is actually data to be read ```c if(events[i].events & EPOLLIN) ``` ### And then you can treat it like any other socket --- class: center, middle # Lets talk about Multithreading --- # Threading is hard ### Need to worry about: * Race conditions * Deadlocks * General data synchronization. --- # We need to change a few things ### First change our epoll flags ```c event.events = EPOLLIN | EPOLLET | EPOLLONESHOT; ``` --- # What is EPOLLONESHOT ### When an event/fd is handed to a thread disable it within EPoll ### This ensures only one thread handles an event. --- # When the thread is done with the event re-enable it ```c // Reset what we are watching for events[i].events = EPOLLIN | EPOLLET | EPOLLONESHOT; // tell epoll to re-enable this fd. epoll_ctl(epoll_fd, EPOLL_CTL_MOD, dat->fd, &events[i]) ``` --- # What does this give me? ### EPoll on a single CPU is fantastic ### Even better on multiple ### Most servers have more than one CPU ### Squeeze out as much performance as possible --- # Other Concerns ### Keeping your data in sync ### Cleaning up after yourself. ### Tuning Linux * Increase max file descriptors * Tune TCP for high performance * I'm still getting problems --- # Demo Time! ### http://8ball.csh.rit.edu:8000/stats ### http://8ball.csh.rit.edu:8000/foo ### My code (https://github.com/rossdylan/uvb-server) is running on 8ball --- class: center, middle # Questions? Comments? --- # Links ### ~❯ [man epoll](http://linux.die.net/man/4/epoll) ### ~❯ [man epoll_ctl](http://linux.die.net/man/4/epoll_ctl) ### [C10K Problem](http://www.kegel.com/c10k.html) * Ways to handle >= 10k connections at once * [Ultimate Victory Battle Server](https://github.com/rossdylan/uvb-server) --- class: center, middle # Thanks!