gevent_uwsgi connection limit

[youyueyinfeng]

Hi, I found if almost more than 1024 clients connect to the server running using gevent_uwsgi, the extra clients will connect failed. After debugging, the below code will raise error since the connection_fd() return a value bigger than 1024.

# spawn a select greenlet
def select_greenlet_runner(fd, event):
    """Sets event when data becomes available to read on fd."""
    while True:
        event.set()
        try:
            select([fd], [], [])[0]
        except ValueError:
            import traceback
            print(traceback.format_exc())
            break
self._select_greenlet = gevent.spawn(select_greenlet_runner, self._sock, self._event)

In this case, gevent.select() doesn't work, server has to recv the msg after event.wait(timeout=3). If I pass a wait_timeout > 3 to client.connect(), the client can work, while all the msgs are processed 3s latter.

So I wonder if there's a way to advance this scene.

[miguelgrinberg]

How did you start uwsgi? What options did you use?

[youyueyinfeng]

Run uwsgi like this:

uwsgi --http-socket :10086 --gevent 10000 --http-websockets --wsgi test --callable app

I changed the linux open file limit

$ ulimit -a
core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 255974
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 65535
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 512366
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

[youyueyinfeng]

The test code if you need:
Server:

import socketio
sio = socketio.Server()
app = socketio.WSGIApp(sio)

Clients:

from gevent import monkey
monkey.patch_all()

import gevent
import socketio

ALL_USER = 1030
SPAWN_USER = 10

num = 0
for i in range(int(ALL_USER/SPAWN_USER)):
    for j in range(SPAWN_USER):
        gevent.spawn(socketio.Client().connect(
            'http://127.0.0.1:10086', transports=['websocket'], wait_timeout=1))
    num += SPAWN_USER
    print(num)
    gevent.sleep(1)

while True:
    gevent.sleep(1)

pip requirement.txt

gevent==21.1.2
greenlet==1.1.0
uWSGI==2.0.19.1
six==1.12.0
python-socketio==5.2.1
python-engineio==4.1.0
bidict==0.21.2
websocket_client

[miguelgrinberg]

This is a question that you need to ask someone familiar with the uWSGI project. As far as I can see your configuration from the Flask-SocketIO side is correct.

[youyueyinfeng]

It is not problem about uwsgi. The major problem is gevent.select used in python-engineio/async_drivers/gevent_uwsgi.py

[miguelgrinberg]

Okay, sorry, I do understand now. This is very old code that I haven't seen in many years, I thought you were copying code from uWSGI.

I have changed the use of select to the selectors module. You will need to install python-engineio from the main branch to try this out. Let me know if this addresses the issue.

[youyueyinfeng]

Yeah, the new code in the main branch fix the problem. The DefaultSelector choose EpollSelector on my machine. Thanks a lot~