This commit adds the relevant tables and code to store lint warnings in the
database.
Currently, only lint checkers which don't require access to the network will
be run, as this allows the processing to happen without network access. Also,
this functionality won't work in older versions of Guix which don't expose the
lint warnings in a compatible way.
The licenses table, along with the package_metadata table had duplicate
values. This could happen as the unique constraints on those tables didn't
properly account for the nullable fields.
The duplicates in those tables also affected the license_sets, packages,
package_derivations tables in a similar way. Finally, the
guix_revision_package_derivations table was also affected.
This commit adds a migration to fix the data, as well as the constraints. THe
code to populate the licenses and package_metadata tables is also updated.
Reserve some capacity to process revisions which are the tip of a branch. This
should reduce the time between new revisions appearing, and then being
processed.
There are some revisions of Guix which take forever to process (or days at
least). To avoid jobs being processed forever, kill them after they've been
running for a while (default 24 hours).
This should speed up processing new revisions, reduce latency between finding
out about new revisions and processing them, as well as help manage memory
usage, by processing each job in a process that then exits.
This allows easily processing an individual job by id. This may be useful to
use manually, but also when processing jobs in parallel, as forking doesn't
work well with the libpq library used by squee.
This is working towards running the jobs in parallel. Each job looks at the
records in the database, and adds missing ones. If other jobs, running in
different transactions insert the same missing records at the same time, this
could cause an error.
Therefore, to just avoid this problem, lock before inserting the data. This
will allow the jobs to be processed in parallel, and it shouldn't have too
much of an effect on performance, as the slow bit is outside of the
transaction.
