131967dc39
Tweak streaming json
...
Don't set the encoding, this should be handled by knots, and drop the block
size as the previous value seemed excessive.
2025-02-06 17:15:33 +00:00
9e3cfabe77
Fix some nulls
2025-02-06 17:14:47 +00:00
258ee0468b
Update use of resource pools
2025-02-06 17:14:47 +00:00
9bf4eea2fd
Name the temporary threads
2025-02-04 10:45:52 +00:00
6deeccc22f
Remove some knots workarounds
2025-02-03 22:59:34 +00:00
3646de6097
Update exception handling in load-new-guix-revision
2025-02-03 22:59:34 +00:00
931b7bc593
Add a slightly crude method to ignore systems and targets
...
While processing a revision. It would be good to also record what systems and
targets are in the platforms so it's clear what data is missing, but that can
be added later.
2025-02-03 22:59:34 +00:00
acdedb075d
Ensure COLUMNS is set
2025-02-03 13:06:45 +01:00
f9e6daad99
Use the default knots web server exception handler
2025-02-03 12:18:54 +01:00
49f9aa4822
Force add resources parallelism to 1 for some resource pools
...
This'll be the default in coming revisions of knots, and is neccessary for
pools being used to share a single object.
2025-02-02 01:23:49 +01:00
8bce47dc69
Remove use of call-with-utility-thread
...
As I think it's no longer necessary, and the PostgreSQL connection is being
handled here incorrectly.
2025-01-31 13:01:37 +01:00
5cca88ca2a
Work around the resource pool needing a destructor
2025-01-31 08:56:04 +01:00
8f3968af1e
Configure the knots web server connection buffer size
...
Rather than changing the buffer size when processing each request.
2025-01-29 18:34:20 +01:00
98c4fae76f
Move the exception handling to the knots web server
...
Rather than adding another layer. This commit also removes the extra fibers
layer when processing requests. I forget exactly why this was added, but if
the reason resurfaces then it should be addressed in knots.
2025-01-29 18:34:20 +01:00
7dd6be1c3b
Update Knots
2025-01-29 18:34:20 +01:00
83f2981c64
Fix branch package version page if no data is found
2025-01-27 12:25:11 +00:00
9a1c1b30cb
Fix matching NULL
2025-01-27 12:25:04 +00:00
ad30746fab
Fix null handling in result-serializer-simple-list-with-null-handling
2025-01-26 10:10:50 +00:00
25bf45fe77
Handle NULL values for included_branches
2025-01-17 12:23:46 +00:00
c886685e92
Update Guile Knots
2025-01-14 13:49:28 +00:00
d6cfd780e8
Log resource pool stats on errors
2025-01-06 19:31:54 +00:00
52a307612f
Fix log message
2025-01-06 18:53:19 +00:00
d3c87fb1dc
Attempt to reduce memory churn when processing package metadata
2024-12-29 12:23:56 +00:00
cdffef2397
Report on the GC when processing revisions
2024-12-29 12:22:38 +00:00
50127de12a
Parallelise inserting derivation counts
2024-12-29 12:22:38 +00:00
6ce51e8880
Remove random x
2024-12-29 12:22:38 +00:00
ed5419866f
Avoid null characters in package descriptions
2024-12-29 12:12:28 +00:00
93eac06fd6
Update Guile Knots
2024-12-29 11:05:07 +00:00
62d6b59013
Make build_servers.id just generated by default
...
As to make it easier to set explicitly.
2024-12-15 20:54:11 +00:00
73b53f46a3
Add utility functions for configuring the database
2024-12-15 20:17:10 +00:00
8b49884816
Use knots
...
A library of extracted Guile Fibers patterns and utilities.
2024-12-15 20:17:10 +00:00
a33e32275e
Add a comment
2024-12-09 10:53:06 +00:00
d7c7f74ff1
Log differently when using the load-new-guix-revision-inserts lock
2024-12-09 10:53:06 +00:00
859b0093f8
Avoid getting the load-new-guix-revision-inserts lock twice
2024-12-09 10:53:06 +00:00
62fc9b7225
Fix hash display for formatted derivations
2024-12-09 10:53:06 +00:00
5664cde17c
Improve some query formatting
2024-12-09 10:53:06 +00:00
ba97dfc1fe
Fix wrong number of values from channel-derivations-by-system->guix-store-item
2024-12-09 10:53:06 +00:00
5ed98343d7
Rework loading revision data
...
These changes were motivated by switching to a mechanism of loading data that
isn't dependent on the big advisory lock that prevents more than one revision
from being processed at a time.
Since INSERT ... RETURNING id; is used, this can block if another transaction
inserts the same data, and then cause an error when that transaction
commits. The solution is to use ON CONFLICT DO NOTHING, but you have to handle
the case when the INSERT doesn't return an id since the other transaction has
inserted it.
This commit rewrites insert-missing-data-and-return-all-ids to do as described
above, as well as being more efficient in how existing data is detected and to
use more vectors. Other utilities for inserting data are added as well.
2024-12-09 10:53:06 +00:00
b128e9bd7a
Improve null handling
2024-12-01 22:56:12 +00:00
bb84e45c42
Handle conflicts in insert-missing-data-and-return-all-ids
2024-12-01 22:56:12 +00:00
c949deb9d4
Memoize field-can-be-null?
2024-11-29 14:06:08 +00:00
930169751c
Handle conflicts when inserting lint warning message sets
2024-11-29 14:06:08 +00:00
9a38374e7c
Use insert-missing-data-and-return-all-ids for locations
2024-11-29 14:06:08 +00:00
221394eafb
Add in call-with-worker-thread to try to avoid sort problems
2024-11-29 13:49:38 +00:00
cb1bc060c8
Move log message
2024-11-29 10:37:56 +00:00
e14fd88d8f
Don't lookup #f in the hash table
2024-11-29 10:09:41 +00:00
64aeeffd8e
Insert guix_revision_package_derivation_distribution_counts later
...
As this should speed up processing revisions.
2024-11-26 14:12:17 +00:00
620ac16e26
Don't crash when linters crash
...
This is currently happening because some linters try to evaluate parts of
packages for cross-building to aarch64-linux-gnu, but not all packages support
that and some crash in this case.
I'm not quite sure what the correct behaviour should be, but maybe the data
service needs to try and handle these crashes rather than not processing the
entire revision.
2024-11-26 09:27:50 +00:00
112bc95a13
Restore calling update-derivation-ids-hash-table!
...
In derivation-file-names->derivation-ids. This is important for performance
since it avoids reading derivations with an entry already in the database.
2024-11-25 10:56:01 +00:00
bda4269b56
Change to logging the filtered derivations
...
As that's what's relevant here.
2024-11-25 10:55:45 +00:00
