f4be647610
Use a separate fiber to send pool stats
...
As the requester might timeout.
2024-05-02 20:41:04 +01:00
957727c51a
Fix package replacement handling on the revision packages page
2024-04-28 22:04:08 +01:00
6201099003
Instrument requests
2024-04-26 09:46:10 +01:00
824e731a5c
Attempt to avoid the WAL growing excessively
2024-04-25 23:33:12 +01:00
4a09be753c
Use non-blocking inferior-eval-with-store in for glibc-locales
2024-04-25 23:29:50 +01:00
993887fe0c
Further change job store connection handling
...
The guix-dameon WAL is still growing excessively, so avoid doing anything with
the long running store connection except registering temporary roots.
2024-04-25 22:01:22 +01:00
c18589249f
Instrument the resource pool checkout timeouts
2024-04-15 13:54:25 +01:00
f1add88867
Add support for tracking resource pool checkout timeouts
2024-04-15 13:53:35 +01:00
88b9d34fb1
Close store connections earlier
...
As I think the temporary roots on the long running store connection should be
sufficient.
2024-04-15 13:43:37 +01:00
7f5f11048b
Add error handling for startup failures
2024-04-02 12:16:27 +01:00
65f92ca701
Guard against logging failing
...
As I've seen this happening where there are errors writing to FD 1.
2024-04-02 12:16:15 +01:00
15d436032a
Move sleep to start of spawn-port-monitoring-fiber
2024-04-02 12:16:15 +01:00
1dc293c570
Switch to spawn for running sqitch
...
As system* seems to break sigaction picking up SIGINT.
2024-04-02 12:16:15 +01:00
6808532338
Close unused PostgreSQL connections
...
As they take up memory, so free some by closing them.
2024-04-01 22:20:46 +01:00
9fcd615c1f
Remove the per-thread PostgreSQL connection code
...
As this has been replaced by a fibers resource pool.
2024-04-01 22:20:00 +01:00
b5f59189e1
Move backfilling in to the server module and use the connection pool
...
To avoid using the old PostgreSQL connection per thread code.
2024-04-01 21:51:29 +01:00
5f5b8fc91d
Stop using a inferior-package-derivations lock
...
As this isn't for correctness reasons, but resource usage. I'm hoping to
manage this differently.
2024-03-31 19:55:46 +01:00
5d0e84bac4
Make changes for the guile-prometheus update
2024-03-30 21:37:16 +00:00
b4583bb177
Fallback when building channel instances
2024-03-13 15:34:48 +00:00
a667db2f5d
Don't start new jobs when there's low disk space
2024-03-11 19:34:20 +00:00
7d1cc4d325
Attempt to avoid issues with the guix-daemon WAL
2024-03-08 11:29:00 +00:00
5f80856ad1
Allow prioritising jobs from specific repositories
...
To help the QA data service prioritise branches over patches.
2024-03-07 10:02:18 +00:00
184d341671
Report the size of the guix database and WAL file
...
Since the Guix daemon doesn't export these metrics, instrument it here.
2024-03-05 11:15:04 +00:00
ca69d3329d
Add exception handling to the process-jobs script
...
As I'm seeing this exit on beid, but I'm not sure why.
2024-03-05 10:57:41 +00:00
20ad81f0d4
Further fix handling exceptions when computing channel instances
2024-02-13 10:35:04 +00:00
26a751570c
Copy the port monitoring fiber from the build coordinator
...
As the data service has the same issue where it stops listening on the port.
2024-02-13 09:59:39 +00:00
76712e2b00
Instrument resource pool checkout failures
...
As I've got no idea why the resource pools sometimes stop working.
2024-02-10 10:11:29 +00:00
26f517d9c2
Improve exception handling for channel instance derivations
2024-02-03 11:08:04 +01:00
ac1a4cb1e2
Cleanup some with-time-logging
2024-02-02 16:58:06 +01:00
7ecb13e403
Compute lint warnings in parallel
2024-02-01 13:16:08 +01:00
0468bc4321
Speed up loading package metadata
...
By batching the SQL queries.
2024-02-01 13:15:30 +01:00
215a67238a
Ignore inferior-protocol-error when computing channel instances
...
Since this seems to happen for i586-gnu for core-updates currently and I can't
seem to reproduce the issue locally or work out what's wrong.
2024-02-01 10:47:54 +01:00
7acb38377e
Fix exception handling in call-with-temporary-thread
2024-02-01 10:39:11 +01:00
d5e663cc1e
Split up handling of package description data
...
To hopefully see which part is slow.
2024-01-31 20:20:48 +01:00
9779ecf3cd
Speed up fetching package replacements
2024-01-31 12:30:51 +00:00
7d9704192d
Add more logging in when computing channel instance derivations
...
As there's an issue with current core-updates that I'm struggling to track
down.
2024-01-31 08:59:47 +00:00
ba97ade781
Add some time logging in to inferior-packages-plus-replacements
2024-01-28 09:10:01 +00:00
0f7b31c605
Make sure to keep roots for channel instance derivations
2024-01-28 08:18:44 +00:00
39f626aa45
Remove even more time logging
2024-01-28 08:18:13 +00:00
75f27264a6
Improve logging when computing a channel instance derivation fails
2024-01-27 18:41:02 +00:00
062ee1aed2
Log the time to call inferior-packages-plus-replacements
...
As I think this is currently quite slow.
2024-01-27 18:40:01 +00:00
e51d2f8932
Remove some time logging
...
As this is a bit noisy.
2024-01-27 18:39:41 +00:00
e0a6c845e5
Try to fix issues with derivations being GC'ed
2024-01-19 09:57:19 +00:00
a900a1c2ec
Remove drain? #t from process job
...
As it now uses more fibers.
2024-01-18 22:41:02 +00:00
b81e7ea950
Fixup tests
2024-01-18 15:34:40 +00:00
c1d2f3a1b7
Add meaningful parallelism to processing jobs
...
Make parallel use of inferiors when computing channel instance derivations,
and when extracting information about a revision. This should allow for some
horizontal scalability, reducing the impact of additional systems for which
derivations need computing.
This commit also fixes an apparent issue with package replacements, as
previously the wrong id was used, and this hid some issues around
deduplication.
2024-01-18 15:34:40 +00:00
6842a432d6
Split and instrument parts of inferior-packages->package-metadata-ids
...
As parts of it are slow.
2024-01-18 14:42:48 +00:00
e51f87cc7e
Rewrite part of insert-missing-data-and-return-all-ids to avoid filter
...
As filter can use part of the input list, which then prevents modifying the
filtered list.
2024-01-18 14:41:54 +00:00
b8bd56a8f5
Fix par-map&
...
To actually work in parallel.
2024-01-18 14:41:32 +00:00
46b4c87e92
Show backtraces when using parallel fibers and resource pools
2024-01-18 14:41:32 +00:00
