To better separate the code that needs to happen after a lock has been
acquired to allow concurrently loading revisions without concurrent insertion
issues.
Previously, the query for the jobs page was really slow, as it checked the
load_new_guix_revision_job_log_parts table for each job, doing a sequential
scan through the potentially large table.
Adding an index didn't seem to help, as the query planner would belive the
query could return loads of rows, where actually, all that needed checking is
whether a single row existed with a given job_id.
To avoid adding the index to the load_new_guix_revision_job_log_parts table,
and fighting with the query planner, this commit changes the
load_new_guix_revision_job_logs table to include a blank entry for jobs which
are currently being processed. This is inserted at the start of the job, and
then updated at the end to combine and replace all the parts.
This all means that the jobs page should render quickly now.
Replace the Guile-side HTML escaping with a less complete, but hopefully
faster PostgreSQL side HTML escaping approach.
Also, allow reading part of the log, by default, the last 1,000,000
characters, as this should render quickly.
So that it can easily be shown through the web interface. There's two tables
being used. One which temporarily stores the output as it's output while the
job is running, and other which stores the whole log once the job has
finished.
It isn't very useful, as you have to know the commits. Now that there's Git
branch information, it should be possible to access this in a more useful way.
To the compare/derivations page. Previously, the compare/derivations page was
comparing more than the derivations, notably the package metadata. This change
avoids that, and also reduces the information overload on the compare page.
Conventionally, you'd process the oldest job in a queue, but at the moment, it
would be more useful to have recent information more promptly, and fill in the
historical gaps later. I'm not sure this'll always be the case, but for now,
flip the order in which jobs are processed.
So that they can also be used for the /branch/foo/latest-processed-revision
pages. The content is the same, but the title, link, and some of the links on
the page are different.
This makes is easier to get the latest data for a branch in a single request,
rather than making one request to find the latest revision, then another to
get the data.
And use it for the hosting the inferiors, rather than computing the guix
package at runtime. This simplifies the behaviour when the Guix Data Service
is deployed as a Guix package.
I'm not sure quite how to treat these yet, but for now, just stick with the
previous behaviour of showing revisions which haven't been processed yet,
along with the ones that have been successfully processed.
This'll help the a service manager (like the shepherd) know when the service
is ready, which at the moment, means the database migrations have happened.
This runs Sqitch on startup, which should make managing the database easier,
as you just have to restart the service with this option, and the database
should be updated if necessary.
This indicates currently if the database is accessible and a simple query can
be performed. I'm mostly looking at using this when writing a system test for
this service.
Previously, one of the first things that happened when responding to a request
was a database connection was made, even when serving the CSS. This is
unnecessary, so move the database connection handling in to the controller.
Also, to allow for separating it out from the assets, separate the assets out
from the parts of the controller that require a database connection.
