7ecb13e403
Compute lint warnings in parallel
2024-02-01 13:16:08 +01:00
0468bc4321
Speed up loading package metadata
...
By batching the SQL queries.
2024-02-01 13:15:30 +01:00
215a67238a
Ignore inferior-protocol-error when computing channel instances
...
Since this seems to happen for i586-gnu for core-updates currently and I can't
seem to reproduce the issue locally or work out what's wrong.
2024-02-01 10:47:54 +01:00
7acb38377e
Fix exception handling in call-with-temporary-thread
2024-02-01 10:39:11 +01:00
d5e663cc1e
Split up handling of package description data
...
To hopefully see which part is slow.
2024-01-31 20:20:48 +01:00
9779ecf3cd
Speed up fetching package replacements
2024-01-31 12:30:51 +00:00
7d9704192d
Add more logging in when computing channel instance derivations
...
As there's an issue with current core-updates that I'm struggling to track
down.
2024-01-31 08:59:47 +00:00
ba97ade781
Add some time logging in to inferior-packages-plus-replacements
2024-01-28 09:10:01 +00:00
0f7b31c605
Make sure to keep roots for channel instance derivations
2024-01-28 08:18:44 +00:00
39f626aa45
Remove even more time logging
2024-01-28 08:18:13 +00:00
75f27264a6
Improve logging when computing a channel instance derivation fails
2024-01-27 18:41:02 +00:00
062ee1aed2
Log the time to call inferior-packages-plus-replacements
...
As I think this is currently quite slow.
2024-01-27 18:40:01 +00:00
e51d2f8932
Remove some time logging
...
As this is a bit noisy.
2024-01-27 18:39:41 +00:00
e0a6c845e5
Try to fix issues with derivations being GC'ed
2024-01-19 09:57:19 +00:00
a900a1c2ec
Remove drain? #t from process job
...
As it now uses more fibers.
2024-01-18 22:41:02 +00:00
b81e7ea950
Fixup tests
2024-01-18 15:34:40 +00:00
c1d2f3a1b7
Add meaningful parallelism to processing jobs
...
Make parallel use of inferiors when computing channel instance derivations,
and when extracting information about a revision. This should allow for some
horizontal scalability, reducing the impact of additional systems for which
derivations need computing.
This commit also fixes an apparent issue with package replacements, as
previously the wrong id was used, and this hid some issues around
deduplication.
2024-01-18 15:34:40 +00:00
6842a432d6
Split and instrument parts of inferior-packages->package-metadata-ids
...
As parts of it are slow.
2024-01-18 14:42:48 +00:00
e51f87cc7e
Rewrite part of insert-missing-data-and-return-all-ids to avoid filter
...
As filter can use part of the input list, which then prevents modifying the
filtered list.
2024-01-18 14:41:54 +00:00
b8bd56a8f5
Fix par-map&
...
To actually work in parallel.
2024-01-18 14:41:32 +00:00
46b4c87e92
Show backtraces when using parallel fibers and resource pools
2024-01-18 14:41:32 +00:00
5af6233e5b
Make it possible to destroy a resource pool
...
And implement removing idle resources.
2024-01-18 14:41:32 +00:00
15b6dad5a5
Have delete-duplicates/sort! take a equality procedure
...
And change the default, as eq? doesn't always work.
2024-01-18 14:41:32 +00:00
45d9b1ad7c
Add back inferior heap size reporting
...
I'm sure this was present before, but maybe lost during some refactoring.
2024-01-18 14:41:32 +00:00
60980e2668
Use delete-duplicates/sort! in inferior-packages->license-set-ids
...
As it should offer a speedup over delete-duplicates.
2024-01-18 14:41:32 +00:00
49b4841c4e
Use delete-duplicates/sort! in insert-missing-data-and-return-all-ids
...
As it's faster than delete-duplicates for large amounts of data.
2024-01-18 14:41:32 +00:00
241d7e4889
Use libc-locales-for-target
...
To help find the right glibc-locales to use.
2023-12-04 11:11:39 +00:00
8a8dccf2d8
Fix some select-build-servers calls
2023-12-04 11:09:46 +00:00
a3ec1f326d
Set %file-port-name-canonicalization when processing jobs
...
Just in case this helps with performance.
2023-12-04 11:06:27 +00:00
b9026488ed
Memoize computing tokens
...
As I'm not sure how expensive this is, but it doesn't need doing for every
request.
2023-11-24 16:11:57 +00:00
241a704db1
Instrument handling build events
2023-11-24 16:11:42 +00:00
e13febc817
Stop replacements cluttering comparisons
...
This stops the replaced package getting compared with it's replacement.
2023-11-18 10:36:10 +00:00
b2bf948a00
Add more exception handling to make-resource-pool
...
As I'm not sure it's working reliably.
2023-11-17 12:32:14 +00:00
da2a405e8b
Remove hardcoding of builtin:download
...
As this was causing problems with builtin:git-download.
2023-11-16 12:20:55 +00:00
de211ef5ea
Set the names of more threads
...
To help with debugging.
2023-11-16 11:30:23 +00:00
37a07c2d6e
Try to avoid the metrics timing out
...
As that makes debugging problems harder.
2023-11-09 16:44:28 +00:00
7891d1327b
Add missing empty job log entry
2023-11-05 20:01:37 +00:00
4f1ae74d2f
Handle derivations with no sources
2023-11-05 18:49:23 +00:00
341eb4a283
Handle eof when reading logs
2023-11-05 18:49:11 +00:00
b026b8795b
Remove peek
2023-11-05 18:49:04 +00:00
b2707b4828
Restore #:error-port when calling open-inferior
...
As the default is a void port.
2023-11-05 16:12:55 +00:00
1746256ef5
Remove #:use-module (ice-9 custom-ports)
...
As it's not actually used.
2023-11-05 15:59:53 +00:00
2135bc5e49
Update guix-dev.scm and .envrc
2023-11-05 13:46:20 +00:00
03327c0cc3
Include output information in the package page response
...
As this will be useful for QA to say whether the package builds reproducibly
or not.
2023-11-05 13:46:20 +00:00
c3cb04cb80
Use fibers when processing new revisions
...
Just have one fiber at the moment, but this will enable using fibers for
parallelism in the future.
Fibers seemed to cause problems with the logging setup, which was a bit odd in
the first place. So move logging to the parent process which is better anyway.
2023-11-05 13:46:20 +00:00
f5acc60288
Make some sweeping changes to loading new revisions
...
Move in the direction of being able to run multiple inferior REPLs, and use
some vectors rather than lists in places (maybe this is more efficient).
2023-11-02 12:16:17 +00:00
89782b3449
Close the repository when pulling
...
And remove the monitor, as I'm not sure it's necessary.
2023-10-16 21:18:44 +01:00
851a6c17b5
Remove redundant joins from the select build query
2023-10-16 17:19:44 +01:00
82bb174700
Attempt to fetch this Git polling mess
...
Listing remote branches through libgit2 will list branches that don't exist on
the remote. I think branch-list is more listing branch references, and you can
have references to a remote branch where the remote branch doesn't exist. This
isn't very useful here though, as I'm trying to work out what remote branches
exist.
There's remote-ls which might help, but I can't figure out how to get the
commits for branches from that.
Therefore, just bodge the two things together in to a big mess. I seem to be
able to get commits from branch-list that hopefully match what's on the
remote (although I'm not confident about this), and I think remote-ls does
allow checking what branches exist.
2023-10-11 17:53:38 +01:00
70f1824e46
Copy some of the initialisation from (guix git)
...
To go along with just-update-cached-checkout.
2023-10-11 16:33:53 +01:00
