b6551842d1
Increase parallelism when loading revisions
2024-10-27 14:39:50 +00:00
f1071cbd4d
Use a dedicated fiber for heap size monitoring
2024-10-27 14:38:50 +00:00
46b86761f5
Fix free-disk-space call
...
As the WAL file might not exist.
2024-10-13 19:12:09 +01:00
c04ea067c2
Make the WAL threshold more flexible
...
So that the WAL can grow more when there's sufficient space. When the
inferiors are closed it takes time to restart them, so doing this less should
speed up processing revisions.
2024-10-10 18:26:35 +01:00
bb5b3731c8
Vacuum some tables when deleting derivations
2024-10-04 08:14:38 +01:00
53ddb26300
Don't use the reserved pool for HTML/JSON responses
...
Just use it for providing substitutes.
2024-09-08 12:58:39 +01:00
395d0b2bd0
Include background pool metrics
2024-09-08 12:58:24 +01:00
8253a21196
Increase the resource pool default timeout a little
2024-09-08 11:54:22 +01:00
eac37b9e8a
Speed up package-description-and-synopsis-locale-options-guix-revision
2024-09-08 11:19:44 +01:00
ea03ca7230
Don't timeout waiting for background pool connections
2024-09-08 10:23:16 +01:00
aca5f8ca05
Add a new background connection pool
...
Mostly to constrain the build server event handlers.
2024-09-08 10:20:21 +01:00
b7ce9fe008
Speed up select-builds-with-context-by-derivation-output
2024-09-07 21:03:57 +01:00
4ecd20b973
Add an index on builds.build_server_build_id
...
As this is used when rendering builds.
2024-09-07 18:45:26 +01:00
d73d20c985
Timeout when trying to process build events
...
As if they relate to derivations that are currently being inserted, they'll
have to wait until the job is processed.
2024-09-03 13:38:56 +01:00
fd7b0a341d
Add retrying around inserting channel instance derivations
2024-08-29 09:38:44 +01:00
4ab59f46b4
Stop using profiles for channel instances
...
Just use the store item instead. This should avoid issues where the profile
gets GC'd.
2024-08-27 16:19:03 +01:00
a204bda36d
Handle concatenated data in package-derivation-data->names-and-versions
...
As this broke when constructing the package-data from separate queries per
system.
2024-08-22 12:07:35 +01:00
c486db61f0
Further reduce the chunk size
2024-08-20 12:00:14 +01:00
122b63e131
Increase the WAL threshold
2024-08-20 10:31:20 +01:00
f6eadb0b16
Make the free space requirement configurable
2024-08-20 09:49:41 +01:00
6de908d0f4
Increase the inferior idle seconds
2024-08-18 12:53:12 +01:00
c2ccd81006
Improve the implementation of checking for idle resources
2024-08-18 12:48:29 +01:00
3887435a7d
Improve logging around use of inferiors
2024-08-17 18:31:32 +01:00
210e9a4775
Log the number of inferiors when reporting WAL issues
...
And reduce the threshold.
2024-08-16 21:18:23 +01:00
0592fba35b
Check the WAL size more frequently when using inferiors
...
Since getting an inferior from the pool can take some time, it's not
sufficient to just check prior to attempting to fetch an inferior from the
pool. Instead set a timeout and check periodically.
2024-08-16 11:56:17 +01:00
c335775ed4
Check the WAL size before computing lint warnings
...
As even though this might not use the store, the inferior still holds a store
connection open.
2024-08-16 11:18:40 +01:00
05c3ae5b87
Use more threads for the server
...
Just in case this avoids writing responses blocking new requests coming in.
2024-08-14 21:42:11 +01:00
22eb8d44ca
Move the resource pools to a different thread
...
As the server thread can block for a while when writing big responses.
2024-08-14 21:13:55 +01:00
7f1aaa5e99
Allow setting the scheduler for a resource pool
2024-08-14 21:13:42 +01:00
0c08ce069d
Tweak handling web server errors
2024-08-14 20:40:34 +01:00
b2dcccb264
Name the resource pools
2024-08-14 20:12:19 +01:00
bfb4ff3101
Include the pool name in the resource pool timeout exception
2024-08-14 19:59:36 +01:00
e2e158e33b
Make resource pools a record
...
So that the name is known when requesting a resource from the pool.
2024-08-14 19:56:37 +01:00
eeda1bf33b
Use system-ids for inserting distribution counts
2024-08-12 09:14:36 +01:00
4d0813c331
Add more logging to backfilling guix_revision_package_derivation_distribution_counts
2024-08-12 09:14:36 +01:00
b79c7e83be
Reduce chunk size
...
To try and respond quicker when the WAL is growing.
2024-08-12 09:14:36 +01:00
97b76000ec
Check the WAL size for each chunk
...
Rather than for each system and target, as this should mean the
response (pausing and allowing inferiors and store connections to be closed)
is quicker.
2024-08-12 09:14:36 +01:00
371d76456f
Start trying to handle GC happening while processing revisions
2024-08-12 09:14:36 +01:00
7869082816
Parallelise inserting package derivation distribution counts
2024-08-10 18:28:53 +01:00
885a4758d3
Get the test suite working again
2024-08-07 17:21:49 +01:00
77962f7c2c
Move inserting derivations in to the load-new-guix-revision module
...
And start to more closely integrate it. This makes it possible to start making
it faster by doing more in parallel.
2024-08-07 17:21:28 +01:00
7f746b358b
Add the fiberize utility
2024-08-07 16:50:30 +01:00
3d2335cebe
Stream the render-revision-package-derivations JSON response
...
As it can be large.
2024-08-05 13:45:46 +01:00
d8ade0ea11
Fix the pool metrics
2024-07-29 11:24:24 +01:00
f0f1329461
Tweak timeouts in resource-pool-stats
2024-07-29 11:24:08 +01:00
fbbdc41f5c
Update tests
2024-07-26 22:21:59 +01:00
e8bd27fdb8
Try using 2 threads for the server
2024-07-26 21:42:21 +01:00
2a80304e0c
Add worker thread utils
2024-07-20 00:20:20 +01:00
7df7fd3e52
Compute package derivations in chunks
...
This allows for keeping the inferiors and store connections around for a more
constant period, and allows closing the store connections and allowing the
guix-daemon to clear the WAL file if needed.
2024-07-19 19:45:07 +01:00
ed4ba8eb95
Reduce the WAL size threshold
2024-07-19 19:44:53 +01:00
