Now that squee cooperates with suspendable ports, this is unnecessary. Use a
connection pool to still support running queries in parallel using multiple
connections.
In to two thread pools, a default one, and one reserved for essential
functionality.
There are some pages that use slow queries, so this should help stop those
pages block other operations.
I think the idle connections associated with idle threads are still taking up
memory, so especially now that you can configure an arbitrary number of
threads (and thus connections), I think it's good to close them regularly.
Previously it would compute a long list of strings, potentially more than
100,000 elements long, then split this string up and insert it in chunks. Only
then could memory be freed.
This new approach builds the strings in batches for the insertion query, then
moves on to the next batch. This should mean that more memory can be freed and
reused along the way.
Previously, if an exception occurred during the processing of any but the last
letpar& expression, the replies for the other expressions would never be
fetched, resulting in that thread in the pool just waiting for a receiver for
the message.
To avoid this, make sure to read all the replies before raising any
exceptions.
Channels don't represent some channel on which messages travel, at least not a
very long one because it can't accommodate any messages. They simply represent
a direct exchange of the message between a sender and receiver. Because of
this, put-message blocks the fiber, and if all the threads on the other end
are waiting for replies to be received, then you have a deadlock.
To avoid this situation, spawn new fibers to send the messages. I think this
works at least, although I'm unsure how sensible it is.
In to a generic thing more like (ice-9 futures). Including copying some bits
from the (ice-9 threads) module and adapting them to work with this fibers
approach, rather than futures. The advantage being that using fibers channels
doesn't block the threads being used by fibers, whereas futures would.
