Switch to using a sequence for the ids in the log parts table, and spawn a
thread to listen for output from the inferior processes, and enter it in to
the database.
Yet again...
This makes the channel-derivations for each system accessible within the
load-new-guix-revision procedure, in preparation for storing them in the
database.
Stop using the system values as targets, and remove package derivation entries
where this is the case.
Switch the non-cross derivation case to have a target of "", rather than
matching the system, as this makes more sense, and is more consistent now that
the target values no longer match the system values.
Hardcode some more correct target values, and use these instead. Hopefully
this can be better integrated with Guix in the future.
This commit also includes a migration attempting to shrink some indexes.
In an attempt to stop the derivations from being garbage collected between the
time they're generated, and when they're potentially read by the Guix Data
Service.
Previously this would just compare on the version if the name was the same,
but there are package definitions that share the name and version (itstool is
one example).
To try and make this more stable, to avoid weird errors, and unstable
comparisons between revisions, use the line number when deduplicating
packages.
I think this might be happening when packages are filtered out as
duplicates (by name and version), but then a reference to a duplicate occurs
somewhere, like in a lint warning.
This seems to work better for both generating the non-cross and cross
derivations. Previously, using the package-transitive-supported-systems
approach didn't generate some cross derivations.
Use the second argument to package-transitive-supported-systems to correctly
identify the different bootstrap path for non x86_64 and i686-linux. The
previous implementation did work, but only up until a merge of core-updates
changed the bootstrap approach.
Previously, they were nix-base32-string encoded, but the representation in the
derivations is base16, so it doesn't make sense to use a different
representation in the database.
Therefore, add some code that runs before the start of each job to convert the
data in the database. It was easier to do this in Guile with the existing
support for working with these bytevector representations. After some
migration period, the code for converting the old hashes can be removed.
Previously, all the entries for the branch were deleted, but not, only the
entries for the branch, that are present in the revision that was loaded will
be deleted. This is more efficient, as it avoids deleting and recreating
entries in the table that shouldn't have changed.
There's already the package_versions_by_guix_revision_range table, but I think
it would be also useful to be able to see how derivations change over time.
Between b13b9384bc43bf93c754c037956c8ef9a99c2b41 and
601171a9bc7ca6e4acb932895a07c0ca9aedfdac, this method failed, so catch the
error to allow loading the affected revisions.
Ideally this would go in to the database somehow as well, but the only idea I
have for that is pass in a pipe, and then spawn a thread to read from the
other end of that pipe in a loop to send the output to the database.
That hasn't been written yet, so just send the output to stderr for now.
Add a new event 'retry', and run jobs where the number of retry events is
greater or equal to the number of failure events.
Also add an index to the git_branches table to make the finding jobs query a
bit faster.
