7f49756bac
Track some pg_stat metrics
...
Hopefully this'll help track database things better.
2020-10-01 21:43:41 +01:00
404f39a9ee
Drop default thread count for make-postgresql-connection-channel
...
At least for data deletion, 4 seems unnecessary.
2020-10-01 19:41:13 +01:00
54654417a3
Delete derivations in parallel
...
In an attempt to make this faster.
2020-10-01 19:15:32 +01:00
16600b1a43
Remove the deleting derivations progress output
...
As this is harder to do when deleting derivations in parallel.
2020-10-01 19:14:56 +01:00
fb4c7ecd4c
Delete derivations through a channel
...
Not much different from before, but this will allow parallelising things.
2020-10-01 19:14:11 +01:00
614f9888a5
Add some utilities to use PostgreSQL/Squee through a channel
...
To allow for some concurrency.
2020-10-01 19:13:30 +01:00
3330f034a4
Remove a now redundant part of the maybe-delete-derivation query
...
As this is covered by the big query selecting the derivation ids.
2020-09-30 20:34:33 +01:00
d844b325e2
Stop recursing now that derivation deletion selection is smarter
...
As this probably won't help with performance.
2020-09-30 20:07:41 +01:00
47af6c9661
Attempt to speed up derivation deletion
...
Stop querying for the file-name, as it's unused. Rather than fetching all ids,
then looking at each to see if it can be deleted, do some imperfect but not
too slow checks in the initial query.
2020-09-30 19:38:56 +01:00
39b5df04eb
Remove development code from the process job script
2020-09-28 08:29:20 +01:00
033858410b
Add a JSON page for repository branches
2020-09-27 16:32:56 +01:00
f7933807ac
Add a JSON representation for repositories
2020-09-27 16:26:45 +01:00
02681d7e7a
Fix delete builds for derivation output details set
2020-09-27 16:21:51 +01:00
84907fe040
Implement the JSON representation for system tests
2020-09-27 12:06:18 +01:00
5b13ee2251
Delete builds for unreferenced derivations
2020-09-27 11:11:02 +01:00
52a23a5333
Further data deletion improvements
2020-09-27 11:10:47 +01:00
65e8bf3f8d
Add delete-revisions-from-branch-except-most-recent-n
2020-09-26 19:38:56 +01:00
992a0af63e
Split off delete-revisions-from-branch from delete-data-for-branch
...
To support not deleting all of the revisions.
2020-09-26 18:23:21 +01:00
fb180e1ebd
Replace debug-set! with setenv COLUMNS
...
As that actually seems to work.
2020-09-26 16:42:18 +01:00
6bc1da014f
Better handle loading the (guix i18n) module in the inferior
...
Previously it would only be loaded if the (guix lint) module exists.
2020-09-26 16:07:53 +01:00
faf46565ce
Fix some package search issues
...
Previously, the name wasn't taken in to account when filtering results, so a
search like "git-annex" wouldn't find the git-annex package, since it's
synopsis or description doesn't include the name.
Filtering on the name made the queries much slower, so to address that, the
filtering by revision is moved to a separate part of the CTE, which means
PostgreSQL filters down the rows by quite a lot before it begins filtering by
name.
Also, add in a variant of the query without dashes (-) because that helps with
searches like ruby-engine.
2020-09-26 16:05:06 +01:00
53341c70fc
Change the locale codeset representation
...
From the normalized one, to the one actually contained within glibc. Recent
versions of glibc also contain symlinks linking the normalized codeset to the
locales with the .UTF-8 ending, but older ones do not.
Maybe handling codeset normalisation for queries would be good, but the locale
values ending in .UTF-8 are more compatible and allow the code to be
simplified. For querying, maybe there should be a locales table which handles
different representations.
2020-09-26 11:45:57 +01:00
af2e12a9ef
Add some new metrics about load new revision jobs
2020-09-20 19:13:23 +01:00
fd3ba489d9
Add a metric for the number of revisions
2020-09-20 18:39:46 +01:00
857ac36711
Return a number from count-guix-revisions
2020-09-20 18:38:39 +01:00
e38db9eed9
Set the locale at the start of the process jobs script
...
This might help with the odd [1] errors regarding PostgreSQL queries.
1: invalid byte sequence for encoding "UTF8":
2020-09-20 11:11:03 +01:00
a0e098a6ce
Increase the stack trace width when processing jobs
...
As this might result in more useful error messages.
2020-09-20 10:59:22 +01:00
c596a1c6a9
Add a Prometheus metrics page, with some database metrics
...
The database size is growing, but it's hard to know what parts are growing the
fastest. These metrics will hopefully help with understanding that.
2020-09-06 13:14:31 +01:00
a0cd1097f9
Add guile-prometheus to guix-dev.scm
2020-09-06 13:13:30 +01:00
ce646af308
Tweak inputs in guix-dev.scm
...
Using guile-3.0-latest rather than guile-3.0 to avoid bytecode issues with
guix.
2020-08-26 21:26:30 +01:00
Danjela Lura
50d2e4e158
Add JSON representation for the store item page
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-26 21:26:08 +01:00
Danjela Lura
d5c101dee7
Add JSON representation for the derivation page
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-26 21:20:00 +01:00
Ludovic Courtès
ab68b0fdb3
Use 'guile-3.0' instead of 'guile-next' in 'guix-dev.scm'.
2020-08-26 11:29:11 +02:00
Ludovic Courtès
773fe2a19f
Add dependency on Guile-lzlib.
...
Guix no longer provides the (guix lzlib) module. This commit arranges
to use Guile-lzlib instead.
2020-08-26 11:27:54 +02:00
c7048abe31
Change a package_descriptions index to use MD5 of the description
...
To avoid the index data being too large.
This was first seen in guix commit 1bb4fd64b7bbe5a17eda6f0ca8105283c038f7c8:
psql-query-error (fatal-error PGRES_FATAL_ERROR ERROR: index row size 2808
exceeds maximum 2712 for index "package_descriptions_locale_description_key"
HINT: Values larger than 1/3 of a buffer page cannot be indexed.
2020-08-23 14:42:32 +01:00
Danjela Lura
599c621b88
Postpone the execution of the query that displays locale options
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-14 21:55:59 +01:00
Danjela Lura
7984179f5e
Guard against null package metadata ids
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-13 08:58:09 +01:00
Danjela Lura
086cb9c9d0
Modify the search-packages-in-revision query to make it faster
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:59 +01:00
Danjela Lura
bef826cf2e
Make call-with-time-logging handle multiple return values
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:54 +01:00
Danjela Lura
5b3e5d9441
Update the package_metadata_tsvectors with new entries
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:49 +01:00
Danjela Lura
ee613cdb30
Add the package_metadata_tsvectors table
...
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:35 +01:00
a6a4a84b20
Avoid problems with lint warnings for duplicate packages
...
Use the deduplicated list of packages when fetching lint warnings, to avoid
duplicates. This was first seen in the following Guix commit
843344273c6a587b8e6c84d8406500fd64d8908a.
2020-08-09 16:53:26 +01:00
1e0332136e
Fix rendering the required failed builds for builds by id
...
Use the derivation-file-name from the database, rather than the query
parameter.
2020-07-13 19:12:09 +01:00
a1af644310
Change select-required-builds-that-failed to exclude succeeded builds
...
Only include a failed build if that build server hasn't had any success
building that output. The Guix Build Coordinator can build one output with
many different builds, so this helps avoid showing lots of spurious failures.
2020-07-13 18:55:50 +01:00
745a8bdbf1
Also show related failed builds for scheduled builds
...
Because the Guix Build Coordinator doesn't mark builds blocked by failures as
failed-dependency, they're just left as scheduled.
2020-07-13 18:55:07 +01:00
e9d4e74e56
Add builds to the store item page
2020-07-04 09:31:12 +01:00
5dc9ed65ff
Add a index to derivation_output_details_sets
...
This helps with queries like:
SELECT *
FROM derivation_output_details_sets
WHERE ARRAY[5563227] <@ derivation_output_details_ids;
2020-07-04 08:59:29 +01:00
4b1dc76f18
Improve the revision builds page
...
Allow specifying how many results to show, and say how many results are being
shown.
2020-07-03 20:33:16 +01:00
69d3c83986
Improve select-builds-with-context
...
Support specifying the limit, and improve the ordering.
2020-07-03 20:32:45 +01:00
092f01b6f2
Fix the builds table headers
2020-07-02 20:36:08 +01:00
