Commit graph

28 commits

Author SHA1 Message Date
b128e9bd7a Improve null handling 2024-12-01 22:56:12 +00:00
1dc293c570 Switch to spawn for running sqitch
As system* seems to break sigaction picking up SIGINT.
2024-04-02 12:16:15 +01:00
9fcd615c1f Remove the per-thread PostgreSQL connection code
As this has been replaced by a fibers resource pool.
2024-04-01 22:20:00 +01:00
7d1cc4d325 Attempt to avoid issues with the guix-daemon WAL 2024-03-08 11:29:00 +00:00
10bad53ad5 Support polling git repositories for new branches/revisions
This is mostly a workaround for the occasional problems with the guix-commits
mailing list, as it can break and then the data service doesn't learn about
new revisions until the problem is fixed.

I think it's still a generally good feature though, and allows deploying the
data service without it consuming emails to learn about new revisions, and is
a step towards integrating some kind of way of notifying the data service to
poll.
2023-10-09 22:19:02 +01:00
7251c7d653 Stop using a pool of threads for database operations
Now that squee cooperates with suspendable ports, this is unnecessary. Use a
connection pool to still support running queries in parallel using multiple
connections.
2023-07-10 18:56:31 +01:00
29d49ba31a Detach the database setup from the main guix-data-service process
This will allow restarting them independently, leaving it up to the operator
to ensure that all processes are compatible.
2023-06-09 16:11:06 +01:00
05c437d26a Support instrumenting the number of database connections
Since this is now quite dynamic, it's useful to have a metric for it.
2023-01-01 12:43:06 +00:00
ca1e4819b6 Fix closing thread postgresql connections 2022-11-17 16:32:04 +00:00
d06230fcf4 Close postgresql connections when the thread pool thread is idle
I think the idle connections associated with idle threads are still taking up
memory, so especially now that you can configure an arbitrary number of
threads (and thus connections), I think it's good to close them regularly.
2022-10-23 11:28:37 +01:00
0dc05982cd Try to adapt the PostgreSQL paramstring to use with sqitch 2021-06-16 13:44:00 +01:00
64a4058cce Start to add compatibility with squee returning #f for null values
While maintaining compatibility for older versions of squee.
2021-01-02 10:06:27 +00:00
d2646e7110 Remove some non-existent imports 2020-10-04 13:22:24 +01:00
e2e55c69de Rework the shortlived PostgreSQL specific connection channel
In to a generic thing more like (ice-9 futures). Including copying some bits
from the (ice-9 threads) module and adapting them to work with this fibers
approach, rather than futures. The advantage being that using fibers channels
doesn't block the threads being used by fibers, whereas futures would.
2020-10-03 21:32:46 +01:00
9723a18df4 Add some utilities to work with PostgreSQL connections in threads 2020-10-03 09:20:39 +01:00
1bdc8855ba Extract out opening PostgreSQL connections
So this can be reused.
2020-10-03 08:55:56 +01:00
71afa93981 Make with-postgresql-connection work with multiple values 2020-10-02 20:15:23 +01:00
404f39a9ee Drop default thread count for make-postgresql-connection-channel
At least for data deletion, 4 seems unnecessary.
2020-10-01 19:41:13 +01:00
614f9888a5 Add some utilities to use PostgreSQL/Squee through a channel
To allow for some concurrency.
2020-10-01 19:13:30 +01:00
0cc78b90ae Support setting a statement timeout
To stop queries related to the web server that run for a long time.
2020-04-24 08:59:32 +01:00
8e9f1af89b Use the config options when creating the default paramstring 2020-04-07 17:37:03 +01:00
b337c5fcb2 Add a function to check if the test database is being used
This helps avoid problems when running the tests, and the test database not
being used.
2020-02-24 21:15:43 +00:00
356a5954b2 Add support for using a database URI
As this is useful when working with pg_tmp.
2019-10-08 19:45:41 +01:00
1441942200 Hack better NULL support on to (squee)
PQgetvalue used by squee returns null values as empty strings, which are
ambiguous for string fields. Therefore, use PQgetisnull to implement a
serialiser for squee which checks empty strings to see if they're actually a
NULL value, then returns '() in this case.

exec-query-with-null-handling can be used to access this behaviour.
2019-09-04 12:52:12 +02:00
fde1000cb3 Add a couple of functions for PostgreSQL advisory locks
Use symbol-hash to convert a symbol to the number for the lock. I'm hoping
this is OK, and it seems to be stable.
2019-07-12 19:51:42 +01:00
743cec7cb6 Associate a name with database connections
This helps when working out which connection to the database is doing what.
2019-07-12 19:49:26 +01:00
051962b54d Add a with-postgresql-transaction procedure
To help with running tests.
2019-05-05 13:35:17 +01:00
2e45bb0696 Extract out some database functionality to another module 2019-04-14 10:04:51 +01:00