Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

idiomatic twisted.web.template.renderElement usage could have greatly lower costs and resulting response size #10348

Closed
twisted-trac opened this issue Jun 3, 2022 · 3 comments
Closed

idiomatic twisted.web.template.renderElement usage could have greatly lower costs and resulting response size #10348

twisted-trac opened this issue Jun 3, 2022 · 3 comments
Assignees
@exarkun
Labels
enhancement fixed priority-normal web

Comments

@twisted-trac
Copy link

exarkun's avatar @exarkun reported
Trac ID trac#10348
Type enhancement
Created 2022-06-03 20:17:21Z

Consider this behavior of renderElement:

>>> from twisted.web.template import renderElement, XMLString
>>> class Req:
...   def write(self, bs): chunks.append(bs)
...   def finish(self): pass
... 
>>> chunks = []
>>> renderElement(Req(), XMLString("<table><td><tr>Hi</tr></td></table>").load())
1
>>> chunks
[b'<!DOCTYPE html>', b'\n', b'<', b'table', b'>', b'<', b'td', b'>', b'<', b'tr', b'>', b'Hi', b'</tr>', b'</td>', b'</table>']

If the response is using chunked transfer encoding then each of these writes incurs 4 bytes of overhead (\r\n\r\n) plus 1 (or sometimes possibly 2 or 3) more for the chunk length. Since these strings are quite short (many are only 1 byte themselves) this ends up being quite a lot of overhead - as much as 5x for the shortest strings. This is overhead on the size of the response - this data has to be copied around in memory, sent over the network, and interpreted by the client.

In the case of this simple example, the whole response is known synchronously (ie, before renderElement even returns) and there's no reason it couldn't all end up in a single chunk (ie, a single call to Request.write).

For elements containing a Deferred, at least the strings between Deferreds could be combined which still greatly reduces the number of small chunks written (even with every single string coming from a Deferred, it's no worse than the current behavior).

Searchable metadata 
trac-id__10348 10348
type__enhancement enhancement
reporter__exarkun exarkun
priority__normal normal
milestone__None None
branch__ 
branch_author__ 
status__closed closed
resolution__fixed fixed
component__web web
keywords__review review
time__1654287441819996 1654287441819996
changetime__1655256998115097 1655256998115097
version__None None
owner__exarkun exarkun
@twisted-trac
Copy link
Author

exarkun's avatar @exarkun set owner to @exarkun

@twisted-trac
Copy link
Author

exarkun's avatar @exarkun commented

#1736

@twisted-trac
Copy link
Author

exarkun's avatar @exarkun set status to closed

Fixed by 6f8774c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@exarkun exarkun

Labels
enhancement fixed priority-normal web
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@exarkun @twisted-trac