Commit graph

64 commits

Author SHA1 Message Date
Andreas Enge
c63ba7f09c Add guix-data-service-request.
* guix-data-service/client.scm: New file.
* Makefile.am (SOURCES): Register file.
2025-06-27 17:52:12 +02:00
Ludovic Courtès
30b2d91bfb Add support for Forgejo web hooks.
Populate the ‘git_repositories’ and ‘load_new_guix_revision_jobs’ table
when receiving a POST request corresponding to a Forgejo pull-request
webhook.
2025-05-27 11:51:50 +02:00
961441cab0 Reliably perform the background processing for build events
Insert the build ids to a table and remove the ids once processed. If the data
service is restarted and the in memory queue is lost, process all the items
from the database queue.
2025-05-25 13:08:27 +01:00
04f427bcab Update the Makefile for the README name change 2025-05-25 13:08:27 +01:00
ca5f0036f3 Add a heap-profiler module
Taken from a Guile mailing list post.
2024-11-08 22:34:28 +00:00
10bad53ad5 Support polling git repositories for new branches/revisions
This is mostly a workaround for the occasional problems with the guix-commits
mailing list, as it can break and then the data service doesn't learn about
new revisions until the problem is fixed.

I think it's still a generally good feature though, and allows deploying the
data service without it consuming emails to learn about new revisions, and is
a step towards integrating some kind of way of notifying the data service to
poll.
2023-10-09 22:19:02 +01:00
1fb291be40 Add support for incrementally tracking blocked builds
This will hopefully provide a less expensive way of finding out if a scheduled
build is probably blocked by other builds failing or being canceled.

By working this out when the build events are recieved, it should be more
feasible to include information about whether builds are likely blocked or not
in various places (e.g. revision comparisons).
2022-11-11 10:35:09 +00:00
64be52844e Partition the package_derivations_by_guix_revision_range table
And create a proper git_branches table in the process.

I'm hoping this will help with slow deletions from the
package_derivations_by_guix_revision_range table in the case where there are
lots of branches, since it'll separate the data for one branch from another.

These migrations will remove the existing data, so
rebuild-package-derivations-table will currently need manually running to
regenerate it.
2022-05-23 19:10:25 +01:00
869d2522e5 Add the system module to Makefile.am 2021-04-23 18:07:55 +01:00
663bd1411a Add a /package/NAME page
This might be useful for working out when a non-master branch contains a newer
version of a package, or someone has sent in a patch for a newer version
already.
2021-03-14 21:48:43 +00:00
c596a1c6a9 Add a Prometheus metrics page, with some database metrics
The database size is growing, but it's hard to know what parts are growing the
fastest. These metrics will hopefully help with understanding that.
2020-09-06 13:14:31 +01:00
6baef6ae25 Split out querying of build servers and substitute servers
These are related things, but somewhat separate. This change should make it
easier to deal with changes regarding querying build servers, and querying
substitute servers.
2020-05-03 13:23:43 +01:00
d12909d572 Fix some tab issues in the Makefile.am file 2020-03-20 20:43:02 +00:00
77caafb019 Add scripts for generating database dumps 2020-03-02 21:44:29 +00:00
cb353d2c82 Make the test output more verbose 2020-02-27 21:11:27 +00:00
49d10cfe14 Add a new utils module, containing a with-time-logging syntax rule 2020-02-24 21:31:01 +00:00
510e13f175 Add a page listing the available database dumps 2020-02-22 23:24:24 +00:00
27904e8fd7 Fix some comments
Copied over from the Mumi code.
2020-02-16 22:29:11 +00:00
773e5a9c38 Add a module to handle deleting data
This, along with the way of specifying which branches are processed is a way
to manage the data stored within the Guix Data Service.

This only goes so far, it doesn't delete derivations, but it does delete some
of the information related to a revision.
2020-02-15 11:36:31 +00:00
9be7dbac0b Start storing channel instance derivations
These are the ones that relate to Guix pull.
2020-02-11 08:56:24 +00:00
8f0d5b51b1 Store information about system tests 2020-02-03 09:57:02 +01:00
485b79a6de Add a page to show the public key used to sign substitutes 2019-12-27 13:03:17 +00:00
a8d58e0541 Sort the web modules in the SOURCES list in Makefile.am 2019-12-26 09:28:12 +00:00
f78e3126e5 Render nar files for derivations
In the same manor that Guix publish does. This is working towards being able
to serve substitutes for derivations.
2019-12-26 09:25:27 +00:00
639f24bb22 Extract management of package_derivations_by_guix_revision_range
... entries in to a separate module, to split the code up a little further.
2019-12-22 16:09:32 +00:00
2d87bd6340 Add a page for each signing key 2019-12-12 20:07:22 +00:00
8ff27eaa14 Add a new html-utils module
To store code used for rendering HTML across multiple controllers.
2019-12-12 20:07:22 +00:00
b6194e7b3d Begin to add support for importing narinfo files
This commit adds the tables, as well as code to support extracting data from
narinfo files.
2019-11-30 10:51:58 +00:00
32e1fee7ea Add the build-server-token-seed.scm file to Makefile.am 2019-11-26 07:17:09 +00:00
2a3ee20a99 Extract the builds page in to it's own set of modules
To allow for expanding it, without cluttering the root controller.
2019-11-24 20:42:37 +00:00
5663235048 Rework the builds and build_status tables as well as related code
Allow for build status information to be submitted by POST request. This
required some changes to the builds and build_status tables, as for example,
the Cuirass build id may not be available, and the derivation may not be know
yet, so just record the derivation file name.
2019-11-24 20:18:08 +00:00
0ffd8caeeb Add a script to help manage build servers
This computes and displays the tokens needed to send build events to the Guix
Data Service.
2019-11-23 11:50:51 +00:00
3ef99acc79 Add a new channel-news module, along with tables the relevant data 2019-11-21 19:56:19 +00:00
aee903bc8c Add missing files to the SOURCES list in Makefile.am 2019-10-18 18:44:18 +01:00
1ea0c9ba26 Sort the SOURCES in the Makefile.am 2019-10-18 18:13:34 +01:00
c9c7666b49 Don't automatically shutdown the test database
As this leaves a process running, it's better just to stop the database
manually.
2019-10-11 08:43:21 +01:00
06af9ce6de Fix the environment variables in the check-with-tmp-database target
So that they're actually set for sqitch.
2019-10-11 08:36:31 +01:00
eb1709c8e4 Improve the check-with-tmp-database target
To better manage the environment, and stop the database upon completion of the
tests.
2019-10-11 08:32:48 +01:00
c40747a5ee Add a check-with-tmp-database target
This is aimed at testing using pg_tmp, to create a temporary test
database (and instance of PostgreSQL). This is both useful for testing all the
migrations, but should also be useful for running the tests within the guix
package.
2019-10-08 19:45:59 +01:00
44c0c9d0dd Remove non-existent test from Makefile.am 2019-10-01 19:03:17 +01:00
ea253c4515 Add a new script to process emails in an mbox file
This can be used with the mbox files for the guix-commits mailing list to add
older emails in to the database.
2019-09-28 23:55:11 +01:00
f5ff0f8638 Hack building a HTML version of the README in to Makefile.am
This is to be able to serve a readable version of the README on the site,
allowing those without Emacs to more easily read it.
2019-09-21 12:40:46 +01:00
34ed73cdc1 Fix the godir in Makefile.am
I think the .go files should be in lib, rather than share.
2019-09-21 11:30:25 +01:00
1a55022524 Improve the package and package-metadata modules
Add tests around the package module, extract out the use of the
inferior-package record assessors so that they aren't part of the tests, and
switch across the package-metadata module to use
insert-missing-data-and-return-all-ids.
2019-09-05 16:07:23 +02:00
6b9977f62e Store lint warnings in the database
This commit adds the relevant tables and code to store lint warnings in the
database.

Currently, only lint checkers which don't require access to the network will
be run, as this allows the processing to happen without network access. Also,
this functionality won't work in older versions of Guix which don't expose the
lint warnings in a compatible way.
2019-09-01 13:12:10 +01:00
09d927cb99 Add a process-job command
This allows easily processing an individual job by id. This may be useful to
use manually, but also when processing jobs in parallel, as forking doesn't
work well with the libpq library used by squee.
2019-07-12 19:57:24 +01:00
88922d9f11 Tweak the Makefile.am to install the right things
Install the assets and sqitch files, as they are needed. Remove the test
related sources.
2019-06-08 10:27:07 +01:00
544dc1558f Add a new test to cover some of the loading new revisions code 2019-06-02 22:02:12 +01:00
f03a750705 Add some missing test files to Makefile.am 2019-06-02 16:19:06 +01:00
16799a34a9 Store license information for packages
And display this on the package page.

This uses a couple of new tables, and an additional field in the
package_metadata table.

Currently, the order of the licenses in the package definition isn't stored,
as I'm not sure the order in the list is significant.
2019-05-15 08:05:14 +01:00