b22834dae7
Refactor opening store connections when processing jobs
...
And set the #:built-in-builders.
2024-07-18 13:57:50 +01:00
e81c6377bf
Re-work the fibers scheduling
...
Use a single thread for receiving and responding to requests, and delegate the
processing of the requests to a separate set of threads.
I'm hoping this will avoid the processing of requests affecting accepting new
connections, or the sending of responses.
2024-07-16 23:03:15 +01:00
d8e9de4ed6
Use a bigger buffer for requests/responses
...
As I'm hoping this will increase the data throughput for responses.
2024-07-16 20:37:07 +01:00
5f8cd2dc56
Stream the package derivation page since it can be large
2024-07-16 17:16:22 +01:00
bbbcea8ff6
Add more time logging in to insert-missing-derivations
2024-07-16 16:13:17 +01:00
7da355b034
Log more timing infromation about inserting derivations
2024-07-16 16:07:47 +01:00
1754d1a321
Stop inserting missing source file nars
...
This was more an issue several years ago, so this code is not really needed
now.
2024-07-16 16:06:46 +01:00
a61c4baccd
Try and speed up large package derivation comparisions
2024-07-12 13:33:37 +01:00
9032079bda
Fix delete-revisions-from-branch
2024-07-07 11:59:53 +02:00
de98cbfaa3
Remove the statistics page
...
As it's been broken for a long time now, there are too many rows to count.
2024-06-30 11:14:16 +01:00
335715bc72
Properly handle deleting commits that occur on multiple branches
2024-06-29 11:49:45 +01:00
b5fbde5ac8
Delete duplicates when inserting license data
...
As I think this is necessary.
2024-06-25 14:41:16 +01:00
e37eb34db6
Block asyncs when starting inferiors
...
Because this code deals with global state, like environment variables.
2024-06-24 23:02:41 +01:00
31bd2156f7
Support setting environment variables in the inferior
...
When processing jobs, this is mostly to allow setting GUIX_DOWNLOAD_METHODS.
2024-06-24 23:02:14 +01:00
d7103eccc9
Fix the arguments to most-appropriate-mime-type
2024-06-24 09:59:37 +01:00
eeb026b654
Further tweak the package derivation comparison query
2024-06-22 08:55:44 +01:00
e6205e988a
Speed up querying for revision package derivations
...
By splitting it up by system.
2024-06-21 15:29:34 +01:00
4e7c2bcfbf
Don't compare across systems in one query
...
As the query seems to be super slow, and this allows parallelising it as well.
2024-06-21 12:11:48 +01:00
94e66d5b1f
Add a JSON error page
2024-06-21 11:04:57 +01:00
84a2ad5b25
Speed up deleting derivation sources
2024-06-20 17:07:41 +01:00
530f58b59c
Cache the derivations that weren't deleted
...
When deleting derivations, as I think this might reduce the number of queries.
2024-06-20 15:47:21 +01:00
5e88a98c47
Speed up select-build-outputs
2024-06-20 15:33:29 +01:00
a9dbeb5e09
Reduce max-age for the latest-processed-commit page
...
As this shouldn't be cached for a day.
2024-06-12 16:54:35 +01:00
d74422c268
Add more logging around polling git repositories
2024-05-23 09:38:24 +01:00
f7ada4bf1f
Guard against trying to delete an empty list of commits
2024-05-22 11:46:18 +01:00
5d50a0e3e1
Support regexes for included and excluded branches
2024-05-22 11:00:04 +01:00
2043a4ef6f
Fix WAL threshold
...
As it was too small.
2024-05-13 17:20:25 +01:00
f4be647610
Use a separate fiber to send pool stats
...
As the requester might timeout.
2024-05-02 20:41:04 +01:00
957727c51a
Fix package replacement handling on the revision packages page
2024-04-28 22:04:08 +01:00
6201099003
Instrument requests
2024-04-26 09:46:10 +01:00
824e731a5c
Attempt to avoid the WAL growing excessively
2024-04-25 23:33:12 +01:00
4a09be753c
Use non-blocking inferior-eval-with-store in for glibc-locales
2024-04-25 23:29:50 +01:00
993887fe0c
Further change job store connection handling
...
The guix-dameon WAL is still growing excessively, so avoid doing anything with
the long running store connection except registering temporary roots.
2024-04-25 22:01:22 +01:00
c18589249f
Instrument the resource pool checkout timeouts
2024-04-15 13:54:25 +01:00
f1add88867
Add support for tracking resource pool checkout timeouts
2024-04-15 13:53:35 +01:00
88b9d34fb1
Close store connections earlier
...
As I think the temporary roots on the long running store connection should be
sufficient.
2024-04-15 13:43:37 +01:00
7f5f11048b
Add error handling for startup failures
2024-04-02 12:16:27 +01:00
65f92ca701
Guard against logging failing
...
As I've seen this happening where there are errors writing to FD 1.
2024-04-02 12:16:15 +01:00
15d436032a
Move sleep to start of spawn-port-monitoring-fiber
2024-04-02 12:16:15 +01:00
1dc293c570
Switch to spawn for running sqitch
...
As system* seems to break sigaction picking up SIGINT.
2024-04-02 12:16:15 +01:00
6808532338
Close unused PostgreSQL connections
...
As they take up memory, so free some by closing them.
2024-04-01 22:20:46 +01:00
9fcd615c1f
Remove the per-thread PostgreSQL connection code
...
As this has been replaced by a fibers resource pool.
2024-04-01 22:20:00 +01:00
b5f59189e1
Move backfilling in to the server module and use the connection pool
...
To avoid using the old PostgreSQL connection per thread code.
2024-04-01 21:51:29 +01:00
5f5b8fc91d
Stop using a inferior-package-derivations lock
...
As this isn't for correctness reasons, but resource usage. I'm hoping to
manage this differently.
2024-03-31 19:55:46 +01:00
5d0e84bac4
Make changes for the guile-prometheus update
2024-03-30 21:37:16 +00:00
b4583bb177
Fallback when building channel instances
2024-03-13 15:34:48 +00:00
a667db2f5d
Don't start new jobs when there's low disk space
2024-03-11 19:34:20 +00:00
7d1cc4d325
Attempt to avoid issues with the guix-daemon WAL
2024-03-08 11:29:00 +00:00
5f80856ad1
Allow prioritising jobs from specific repositories
...
To help the QA data service prioritise branches over patches.
2024-03-07 10:02:18 +00:00
184d341671
Report the size of the guix database and WAL file
...
Since the Guix daemon doesn't export these metrics, instrument it here.
2024-03-05 11:15:04 +00:00
