Commit graph

18 commits

Author SHA1 Message Date
3ba8418656 Allow skipping processing system tests
Generating system test derivations are difficult, since you generally need to
do potentially expensive builds for the system you're generating the system
tests for. You might not want to disable grafts for instance because you might
be trying to test whatever the test is testing in the context of grafts being
enabled.

I'm looking at skipping the system tests on data.guix.gnu.org, because they're
not used and quite expensive to compute.
2023-02-08 14:56:48 +00:00
fcf1177d2c Don't count exiting with a status of 0 as a failure 2022-01-14 15:24:12 +00:00
6102553d94 Record job failure on non-0 exit status
This is helpful when the jobs fail through Guile running out of memory for
example.
2022-01-12 18:17:52 +00:00
f1d8d76c4d Improve the process jobs script signal handling
The return value of sleep is unreliable (see guile bug #53139), so use a
signal handler instead.
2022-01-09 10:30:03 +00:00
7436283989 Increase the default job processing timeout 2022-01-09 10:12:06 +00:00
3368a64c04 Increase the job timeout
As for some jobs that take lots of time to compute and build the channel
instance derivations, it can take more than 24 hours.
2021-02-10 20:31:44 +00:00
65f2f21d3a Support customising the latest branch revision max processes
This makes it possible to set a higher or lower value depending on what you
want.
2020-02-28 20:58:21 +00:00
71f483c816 Add a few imports in preparation for Guile 3 2020-02-27 21:12:16 +00:00
0cea51cc79 Add copyright and license headers to a bunch of source files
That were missing them.
2019-12-26 10:16:55 +00:00
c229ace8a5 Allow customising the maximum number of processes 2019-12-16 19:57:56 +00:00
413ef9ea6e Prioritise processing the latest revisions for branches
Reserve some capacity to process revisions which are the tip of a branch. This
should reduce the time between new revisions appearing, and then being
processed.
2019-07-16 18:20:02 +01:00
6904547bf2 Record jobs as failed when they are killed
So that they aren't retried again and again.
2019-07-14 14:43:47 +01:00
3dfa9212f1 Kill long running load new revision jobs
There are some revisions of Guix which take forever to process (or days at
least). To avoid jobs being processed forever, kill them after they've been
running for a while (default 24 hours).
2019-07-12 23:00:44 +01:00
83ef624b97 Switch to processing jobs in parallel
This should speed up processing new revisions, reduce latency between finding
out about new revisions and processing them, as well as help manage memory
usage, by processing each job in a process that then exits.
2019-07-12 23:00:44 +01:00
14cb5425d4 Improve the job processing
Speed up checking for new jobs, and remove debugging output.
2019-04-10 20:17:44 +01:00
93d8798df5
More easily support stopping guix-data-service-process-jobs 2019-03-29 09:14:23 +00:00
a092db5007
Further improve load-new-guix-revision-jobs
Split the derivations up in to some groups, and run
invalidate-derivation-caches! inbetween to try and reduce the memory
usage.

Also make a couple of other changes to reduce memory usage or protect
against errors.
2019-03-16 17:26:44 +00:00
5a9262b38d
Initial commit
This is a service designed to provide information about Guix. At the
moment, this initial prototype gathers up information about packages,
the associated metadata and derivations.

The initial primary use case is to compare two different revisions of
Guix, detecting which packages are new, no longer present, updated or
otherwise different.

It's based on the Mumi project.

[1]: https://git.elephly.net/software/mumi.git
2019-02-07 22:26:57 +00:00