Commit graph

932 commits

Author SHA1 Message Date
02681d7e7a Fix delete builds for derivation output details set 2020-09-27 16:21:51 +01:00
84907fe040 Implement the JSON representation for system tests 2020-09-27 12:06:18 +01:00
5b13ee2251 Delete builds for unreferenced derivations 2020-09-27 11:11:02 +01:00
52a23a5333 Further data deletion improvements 2020-09-27 11:10:47 +01:00
65e8bf3f8d Add delete-revisions-from-branch-except-most-recent-n 2020-09-26 19:38:56 +01:00
992a0af63e Split off delete-revisions-from-branch from delete-data-for-branch
To support not deleting all of the revisions.
2020-09-26 18:23:21 +01:00
fb180e1ebd Replace debug-set! with setenv COLUMNS
As that actually seems to work.
2020-09-26 16:42:18 +01:00
6bc1da014f Better handle loading the (guix i18n) module in the inferior
Previously it would only be loaded if the (guix lint) module exists.
2020-09-26 16:07:53 +01:00
faf46565ce Fix some package search issues
Previously, the name wasn't taken in to account when filtering results, so a
search like "git-annex" wouldn't find the git-annex package, since it's
synopsis or description doesn't include the name.

Filtering on the name made the queries much slower, so to address that, the
filtering by revision is moved to a separate part of the CTE, which means
PostgreSQL filters down the rows by quite a lot before it begins filtering by
name.

Also, add in a variant of the query without dashes (-) because that helps with
searches like ruby-engine.
2020-09-26 16:05:06 +01:00
53341c70fc Change the locale codeset representation
From the normalized one, to the one actually contained within glibc. Recent
versions of glibc also contain symlinks linking the normalized codeset to the
locales with the .UTF-8 ending, but older ones do not.

Maybe handling codeset normalisation for queries would be good, but the locale
values ending in .UTF-8 are more compatible and allow the code to be
simplified. For querying, maybe there should be a locales table which handles
different representations.
2020-09-26 11:45:57 +01:00
af2e12a9ef Add some new metrics about load new revision jobs 2020-09-20 19:13:23 +01:00
fd3ba489d9 Add a metric for the number of revisions 2020-09-20 18:39:46 +01:00
857ac36711 Return a number from count-guix-revisions 2020-09-20 18:38:39 +01:00
e38db9eed9 Set the locale at the start of the process jobs script
This might help with the odd [1] errors regarding PostgreSQL queries.

1: invalid byte sequence for encoding "UTF8":
2020-09-20 11:11:03 +01:00
a0e098a6ce Increase the stack trace width when processing jobs
As this might result in more useful error messages.
2020-09-20 10:59:22 +01:00
c596a1c6a9 Add a Prometheus metrics page, with some database metrics
The database size is growing, but it's hard to know what parts are growing the
fastest. These metrics will hopefully help with understanding that.
2020-09-06 13:14:31 +01:00
a0cd1097f9 Add guile-prometheus to guix-dev.scm 2020-09-06 13:13:30 +01:00
ce646af308 Tweak inputs in guix-dev.scm
Using guile-3.0-latest rather than guile-3.0 to avoid bytecode issues with
guix.
2020-08-26 21:26:30 +01:00
Danjela Lura
50d2e4e158 Add JSON representation for the store item page
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-26 21:26:08 +01:00
Danjela Lura
d5c101dee7 Add JSON representation for the derivation page
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-26 21:20:00 +01:00
Ludovic Courtès
ab68b0fdb3 Use 'guile-3.0' instead of 'guile-next' in 'guix-dev.scm'. 2020-08-26 11:29:11 +02:00
Ludovic Courtès
773fe2a19f Add dependency on Guile-lzlib.
Guix no longer provides the (guix lzlib) module.  This commit arranges
to use Guile-lzlib instead.
2020-08-26 11:27:54 +02:00
c7048abe31 Change a package_descriptions index to use MD5 of the description
To avoid the index data being too large.

This was first seen in guix commit 1bb4fd64b7bbe5a17eda6f0ca8105283c038f7c8:

  psql-query-error (fatal-error PGRES_FATAL_ERROR ERROR: index row size 2808
  exceeds maximum 2712 for index "package_descriptions_locale_description_key"
  HINT: Values larger than 1/3 of a buffer page cannot be indexed.
2020-08-23 14:42:32 +01:00
Danjela Lura
599c621b88 Postpone the execution of the query that displays locale options
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-14 21:55:59 +01:00
Danjela Lura
7984179f5e Guard against null package metadata ids
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-13 08:58:09 +01:00
Danjela Lura
086cb9c9d0 Modify the search-packages-in-revision query to make it faster
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:59 +01:00
Danjela Lura
bef826cf2e Make call-with-time-logging handle multiple return values
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:54 +01:00
Danjela Lura
5b3e5d9441 Update the package_metadata_tsvectors with new entries
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:49 +01:00
Danjela Lura
ee613cdb30 Add the package_metadata_tsvectors table
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-08-12 08:53:35 +01:00
a6a4a84b20 Avoid problems with lint warnings for duplicate packages
Use the deduplicated list of packages when fetching lint warnings, to avoid
duplicates. This was first seen in the following Guix commit
843344273c6a587b8e6c84d8406500fd64d8908a.
2020-08-09 16:53:26 +01:00
1e0332136e Fix rendering the required failed builds for builds by id
Use the derivation-file-name from the database, rather than the query
parameter.
2020-07-13 19:12:09 +01:00
a1af644310 Change select-required-builds-that-failed to exclude succeeded builds
Only include a failed build if that build server hasn't had any success
building that output. The Guix Build Coordinator can build one output with
many different builds, so this helps avoid showing lots of spurious failures.
2020-07-13 18:55:50 +01:00
745a8bdbf1 Also show related failed builds for scheduled builds
Because the Guix Build Coordinator doesn't mark builds blocked by failures as
failed-dependency, they're just left as scheduled.
2020-07-13 18:55:07 +01:00
e9d4e74e56 Add builds to the store item page 2020-07-04 09:31:12 +01:00
5dc9ed65ff Add a index to derivation_output_details_sets
This helps with queries like:

  SELECT *
  FROM derivation_output_details_sets
  WHERE ARRAY[5563227] <@ derivation_output_details_ids;
2020-07-04 08:59:29 +01:00
4b1dc76f18 Improve the revision builds page
Allow specifying how many results to show, and say how many results are being
shown.
2020-07-03 20:33:16 +01:00
69d3c83986 Improve select-builds-with-context
Support specifying the limit, and improve the ordering.
2020-07-03 20:32:45 +01:00
092f01b6f2 Fix the builds table headers 2020-07-02 20:36:08 +01:00
bbf8011c95 Fix a bug with the derivation and output history JSON representation
Thanks to zimoun for reporting.
2020-07-01 19:58:36 +01:00
dc8b442e12 Improve the linking to build servers
Add a build-url function that returns the URL to use, and use this where
appropriate.
2020-07-01 19:51:21 +01:00
1e2fefa7cf Change select-build procedures to avoid join issues
The number of build_status rows coming back was being amplified by the INNER
JOIN, using a subquery guards against this.
2020-07-01 19:42:42 +01:00
0242bc6a2e Remove the constraint on having one derivation build per server
As the Guix Build Coordinator can build one derivation many times.
2020-07-01 09:35:40 +01:00
5ee8207565 Avoid duplicate build status entries
This effectively makes adding build status entries idempotent to some degree.
2020-07-01 09:35:29 +01:00
9192550331 Implement more support for builds from the Guix Build Coordinator
Builds from the Guix Build Coordinator might not have timestamps, and the id
from the build server is more important, as one build server can build the
same derivation many times.
2020-07-01 09:35:29 +01:00
879021f21f Fix match in render-builds 2020-06-28 21:44:51 +01:00
911d0779ed Fix compute-token
For some reason the string-filter arguments were the wrong way around.
2020-06-28 21:42:05 +01:00
ddbe4c2192 Support storing the id's representing builds in other services
Like Cuirass, or the Guix Data Service. This isn't so necessary with Cuirass,
as the derivation/output are unique identifiers, but for the Guix Build
Coordinator, one derivation/output can be built many times, with potentially
differing results.
2020-06-27 14:05:51 +01:00
1d48885e6a Make build_status.timestamp nullable
The Guix Build Coordinator doesn't record timestamps for events
currently. This is something to fix, but allow for build statuses without
timestamps as they're not necessary.
2020-06-27 14:05:51 +01:00
d687928a04 Implement insert-build-status using insert-build-statues
To reduce the number of code paths.
2020-06-27 14:05:51 +01:00
Danjela Lura
f60c2eabce Add the packages-translation-availability page
Signed-off-by: Christopher Baines <mail@cbaines.net>
2020-06-26 17:54:19 +01:00