58af8e2dc7
Fix git-commit-exists?
2024-11-08 12:52:48 +00:00
3f07df510e
Fix sorting license sets
2024-11-08 12:52:29 +00:00
c58ee6726b
Fix starting with an empty database
2024-11-08 12:52:18 +00:00
717b861d31
Have the background resource pool users be very patient
2024-11-05 09:41:46 +00:00
6bf1747f55
Always assume that resource pool waiters will stick around
...
As I think this is a more efficient design.
2024-11-05 09:41:30 +00:00
d310632f26
Move the inferior cleanup to the end of each system+target
2024-11-04 08:40:45 +00:00
cee8868bfd
Refactor
2024-11-03 21:24:24 +00:00
ed6a4b1577
Use a separate db connection in channel-derivations-by-system-promise
...
As this can run in parallel with something else using conn.
2024-11-02 09:35:23 +00:00
6265c37e89
Increase the sleep-operation time for telling waiters to retry
...
As this being too short was leading call-with-resource-pool to get stuck.
2024-10-31 20:04:16 +00:00
e67edf54bc
Finish chasing the call-with-resource-pool bug
...
This took a while to find as process-job would just get stuck, and this wasn't
directly related to any particular change, just that more fibers increased the
chance of hitting it.
This commit includes lots of the things I changed while debugging.
2024-10-31 20:04:16 +00:00
af93bdcf5e
Tweak the resource pool return behaviour
...
If there's lots of contention for the resource pool, there will be lots of
waiters, so telling all of them to retry whenever a resource is returned seems
wasteful. This commit adds a new option (assume-reliable-waiters?) which will
have the resource pool try to give a returned resource to the oldest waiter,
if this fails, it'll go back to the old behaviour of telling all waiters to
retry.
2024-10-31 16:46:43 +00:00
de5e036ab1
Ensure that call-with-resource-from-pool doesn't get stuck
...
As I think this was happening when it missed the resource-pool-retry-checkout
reply from the resource pool. Handle this case by periodically retrying with a
configurable timeout.
2024-10-31 16:45:09 +00:00
2d7100e75d
Add call-with-default-io-waiters
...
As this is useful when starting threads from a fibers context.
2024-10-31 16:44:15 +00:00
55af7c82e8
Add new fibers utilities
...
The new fibers-map uses the same batching approach that fibers-for-each uses,
and fibers-map-with-progress allows tracking on the results while the map is
happening.
2024-10-31 16:43:13 +00:00
f8ac6e3dd9
Add make-queueing-channel
...
From the build coordinator.
2024-10-31 16:42:42 +00:00
1782a33a18
Add a comment
2024-10-27 14:39:52 +00:00
c650fc6e7a
Rework inserting derivations
...
To add more parallelism.
2024-10-27 14:39:52 +00:00
1e0407e9b6
Use a common derivation-ids-hash-table
...
To allow for calling derivation-file-names->derivation-ids in parallel across
multiple fibers, using the PostgreSQL connection fiber to perform atomic
operations.
2024-10-27 14:39:52 +00:00
38d5501233
Add placeholder derivation source file nar procedures
2024-10-27 14:39:52 +00:00
ff32aa899e
Include the package version in the "error when computing" messages
...
As this helps when there are lots of packages with the same name.
2024-10-27 14:39:52 +00:00
052cf7580c
Ensure the guix derivations are stored prior to attempting to substitute
2024-10-27 14:39:52 +00:00
38e5282546
Add fibers-promise-reset
2024-10-27 14:39:52 +00:00
d53b91dd91
Fix extract-information-from message
2024-10-27 14:39:52 +00:00
b6551842d1
Increase parallelism when loading revisions
2024-10-27 14:39:50 +00:00
f1071cbd4d
Use a dedicated fiber for heap size monitoring
2024-10-27 14:38:50 +00:00
46b86761f5
Fix free-disk-space call
...
As the WAL file might not exist.
2024-10-13 19:12:09 +01:00
c04ea067c2
Make the WAL threshold more flexible
...
So that the WAL can grow more when there's sufficient space. When the
inferiors are closed it takes time to restart them, so doing this less should
speed up processing revisions.
2024-10-10 18:26:35 +01:00
bb5b3731c8
Vacuum some tables when deleting derivations
2024-10-04 08:14:38 +01:00
53ddb26300
Don't use the reserved pool for HTML/JSON responses
...
Just use it for providing substitutes.
2024-09-08 12:58:39 +01:00
395d0b2bd0
Include background pool metrics
2024-09-08 12:58:24 +01:00
8253a21196
Increase the resource pool default timeout a little
2024-09-08 11:54:22 +01:00
eac37b9e8a
Speed up package-description-and-synopsis-locale-options-guix-revision
2024-09-08 11:19:44 +01:00
ea03ca7230
Don't timeout waiting for background pool connections
2024-09-08 10:23:16 +01:00
aca5f8ca05
Add a new background connection pool
...
Mostly to constrain the build server event handlers.
2024-09-08 10:20:21 +01:00
b7ce9fe008
Speed up select-builds-with-context-by-derivation-output
2024-09-07 21:03:57 +01:00
4ecd20b973
Add an index on builds.build_server_build_id
...
As this is used when rendering builds.
2024-09-07 18:45:26 +01:00
d73d20c985
Timeout when trying to process build events
...
As if they relate to derivations that are currently being inserted, they'll
have to wait until the job is processed.
2024-09-03 13:38:56 +01:00
fd7b0a341d
Add retrying around inserting channel instance derivations
2024-08-29 09:38:44 +01:00
4ab59f46b4
Stop using profiles for channel instances
...
Just use the store item instead. This should avoid issues where the profile
gets GC'd.
2024-08-27 16:19:03 +01:00
a204bda36d
Handle concatenated data in package-derivation-data->names-and-versions
...
As this broke when constructing the package-data from separate queries per
system.
2024-08-22 12:07:35 +01:00
c486db61f0
Further reduce the chunk size
2024-08-20 12:00:14 +01:00
122b63e131
Increase the WAL threshold
2024-08-20 10:31:20 +01:00
f6eadb0b16
Make the free space requirement configurable
2024-08-20 09:49:41 +01:00
6de908d0f4
Increase the inferior idle seconds
2024-08-18 12:53:12 +01:00
c2ccd81006
Improve the implementation of checking for idle resources
2024-08-18 12:48:29 +01:00
3887435a7d
Improve logging around use of inferiors
2024-08-17 18:31:32 +01:00
210e9a4775
Log the number of inferiors when reporting WAL issues
...
And reduce the threshold.
2024-08-16 21:18:23 +01:00
0592fba35b
Check the WAL size more frequently when using inferiors
...
Since getting an inferior from the pool can take some time, it's not
sufficient to just check prior to attempting to fetch an inferior from the
pool. Instead set a timeout and check periodically.
2024-08-16 11:56:17 +01:00
c335775ed4
Check the WAL size before computing lint warnings
...
As even though this might not use the store, the inferior still holds a store
connection open.
2024-08-16 11:18:40 +01:00
05c3ae5b87
Use more threads for the server
...
Just in case this avoids writing responses blocking new requests coming in.
2024-08-14 21:42:11 +01:00
