TypeError: <engineio.packet.Packet object at 0x7f2243ec4668> is not a byte

[sovaa]

I've getting a bunch of these at regular intervals, not sure where it comes from, or if it's anything serious. I'm using python-engineio 2.0.0 and gunicorn 19.7.0

[2018-07-16 05:43:47 +0200] [11440] [ERROR] Error handling request
Traceback (most recent call last):
  File "/home/dino/miniconda3/envs/wioprod/lib/python3.5/site-packages/gunicorn/workers/async.py", line 115, in handle_request
    resp.write(item)
  File "/home/dino/miniconda3/envs/wioprod/lib/python3.5/site-packages/gunicorn/http/wsgi.py", line 344, in write
    raise TypeError('%r is not a byte' % arg)
TypeError: <engineio.packet.Packet object at 0x7f2243ec4668> is not a byte

[miguelgrinberg]

Hmm, this is an odd error that I haven't seen before. Does it happen constantly, or only occasionally? Were you able to determine if this error occurs when a client is connecting, or during an active connection?

[sovaa]

I've had an issue for a while that under heavy load no new socketio connections are being accepted; clients get 500 until the processes have been restarted. I usually don't see this error on low load. As for whether or not it happens while connecting or during an active one I can't really tell you, since there's a bit too much traffic to be sure. I'll see if I can add further details later.

[sovaa]

The y-axis is the amount of said errors aggregated over 30min periods:

Looks pretty correlated with the amount of load on the system:

The big dip in the middle is from when the system stopped accepting new connections (midnight) until it was restarted (6am).

Not sure if I can find anything more useful than this.

Edit: The time zones are different in the two graphs; the first graph is 6h behind the second graph, so midnight in the first graph is 6am in the second graph.

[miguelgrinberg]

You should take a look at your system logs, to see if anything there was reported that coincides with the outage.

[miguelgrinberg]

Closing due to inactivity. If this is still a problem please reopen and we can resume this discussion.

[taras42]

I was able to reproduce this error locally, using [email protected] and [email protected].
It happens when the transport is set to websockets, but there's no http_upgrade header
(for example, cloudflare didn't support websockets in free plan, so it would cut the header from the request.
The same thing can happen behind any proxy or firewall)

In my case I had the client-side code set up like this:

import socketClient from 'socket.io-client';

const socket = socketClient('/websocket', {
  transports: ['websocket'] // no polling specified
});

It gets better if the connection is established with polling first and then it tries to upgrade, but the error still occurs sometimes.

If you set up a simple socket server (gunicorn+python-sockeio) it's easy to reproduce locally using curl:

curl http://localhost:8000/socket.io/?EIO=3&transport=websocket // (without upgrade header)

You should see TypeError: <engineio.packet.Packet object at 0x7f2243ec4668> is not a byte in the server's log.

I debugged the issue locally a bit, it seems like it should be somewhere here:

https://github.com/miguelgrinberg/python-engineio/blob/master/engineio/server.py, _handle_connect, line 533.

Then it goes to socket.handle_get_request and after that engineio packet ends up in gunicorn.

[miguelgrinberg]

@taras42 Thanks. I have added a check to prevent this error and instead return a 400 bad request response.