GetConn waiting queue as FIFO queue

[cloneable]

Hi, we noticed that under higher load that get_conn() requests get served out-of-order resulting in high tail latencies. This happens when a queued waker gets called, but before the GetConn is polled a newer GetConn grabs the available db connection. On top of that, the previously queued GetConn gets pushed to the back of the waiting queue, adding more latency.

Would you consider turning the waiting queue in Exchange into a FIFO queue? I have a patch that we tested in prod and it shows much lower tail latencies for us. See #213.

[cloneable]

I wonder if this could be done more easily by marking connections as reserved. Because the main problem is that woken GetConns don't get the expected connections. So maybe we can simply move a connection from available: Vec to a new reserved: Vec and woken GetConns from there.

[blackbeam]

Hi. Hmm, hard to say, really.. Reserved connection may end up being dead, this could add to complexity..
Btw, I've updated your PR, could you please look at it? The main idea is to keep queue position for caller whose connection end up being dead and to tidy up the Future impl.

[cloneable]

Yes, I looked at adding a reserved: Vec to Exchange earlier, but keeping track of where things are makes this pretty complex, true.

I looked at your commit. Looks great! Much simpler and covers the case when a connection is bad.

What are the next steps? Do you need me to do anything?