Consider decoupling API from implementation?

[akesling]

I'm in a position where I need an ergonomic Rust HTTP client embedded in a WebAssembly plugin. This plugin, however, runs inside of a host that provides its own custom outbound HTTP API. Being a fan of the reqwest interface, I would like to use it as the client I use, but I can't use the existing hyper- or wasm-bindgen-based backends.

Would this project be open to making the HTTP request fulfillment backend "pluggable"? This seems especially relevant given that A) Wasm has a somewhat fragmented way of interfacing with the outside world (see wasm-bindgen's wholly custom way of interfacing with the browser, WASI Preview 2, WASIX, WebAssembly Components-based WASI (coming soon), any number of custom plugin interfaces for wasm32-uknown-unknown compiled modules, etc.), B) there's currently no easy way to test Reqwest-based code without access to open a port and host a server, and C) any number of crazy ways someone might want to use an HTTP client I can't imagine.

There's some precedent for this type of API decoupling in the Python ecosystem with Requests via the Transport Adapter paradigm.

This could push demands like adding a particular WASI HTTP implementation to a separate crate that can be composed instead of maintaining it herein.

It would also open the door to an HTTP testing library which wouldn't require spinning up a local web server in a test if it didn't make sense for that usage.

If this is a thing you would be open to, I'd be happy to write up a proposal / provide a strawman PR.

[seanmonstar]

This actually drives pretty close to something I want of reqwest: refactor reqwest so that each of its features are just tower middleware. I couldn't find an issue describing this, just a few sentences in this blog post:

I’ve also kicked around the idea loosely about cracking open the reqwest crate, and turning its various features into tower middleware. Then, reqwest is just a single opinionated way to build up a client stack. The community would be more empowered to customize the order of layers, adding or removing or swapping, and still have the power that they come to expect from using reqwest.

I think this would be a larger undertaking, but would eventually be a nice benefit for the community at-large.

[akesling]

So, at a high level, were you thinking of the reqwest Client eventually become a wrapper around some form of tower::Service<http::Request, Response=http::Response, ...> with a default implementation using hyper?

I'm trying to wrap my head around where/how things might be wired in this world... so bear with me :).

If that's the case, would it make sense for reqwest::ClientBuilder to be split into a hyper "configurator"-side which returns a tower::Service with the connection/transport layer bits bound, and a separate builder for managing the default population of the http::Request object? I'd naively imagine that this new (simpler) reqwest::ClientBuilder would take the tower::Service provided by that now-separate builder. The default in this world could be this new hyper-based builder, but allow the library user to inject whatever tower::Service they want which maps from http::Request to http::Response. Maybe the hyper impl could move out to a reqwest-hyper crate or something?

Is that inline with your thoughts? What am I missing? What should I go read to be better educated on the subject?

[andytesti]

I'm working on a framework for a particular WASM ABI that does not rely on WASI or the browser. We have an async ad-hoc HttpClient based on this ABI that largely mimics what the reqwest API does, because we can't depend on reqwest itself. We have to provide this framework to our customers and we have to work hard to maintain and document an API. It would be great if we can onboard reqwest. Looking forward for these new ideas landing on this great library.