cbaines/guix-data-service
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Rework loading revision data

Browse source 
These changes were motivated by switching to a mechanism of loading data that
isn't dependent on the big advisory lock that prevents more than one revision
from being processed at a time.

Since INSERT ... RETURNING id; is used, this can block if another transaction
inserts the same data, and then cause an error when that transaction
commits. The solution is to use ON CONFLICT DO NOTHING, but you have to handle
the case when the INSERT doesn't return an id since the other transaction has
inserted it.

This commit rewrites insert-missing-data-and-return-all-ids to do as described
above, as well as being more efficient in how existing data is detected and to
use more vectors. Other utilities for inserting data are added as well.
This commit is contained in:
Christopher Baines 2024-12-05 20:56:23 +00:00
parent b128e9bd7a
commit 5ed98343d7
25 changed files with 874 additions and 1149 deletions
Download patch file Download diff file Expand all files Collapse all files

23
tests/model-package-metadata.scm
View file

@ -34,21 +34,22 @@
mock-inferior-package-foo-2))
(define mock-package-metadata
(map (lambda (mock-inf-pkg)
(list
(mock-inferior-package-home-page mock-inf-pkg)
(mock-inferior-package-location mock-inf-pkg)
`(("en_US.UTF-8" . "Fake synopsis"))
`(("en_US.UTF-8" . "Fake description"))))
mock-inferior-packages))
(list->vector
(map (lambda (mock-inf-pkg)
(list
(mock-inferior-package-home-page mock-inf-pkg)
(mock-inferior-package-location mock-inf-pkg)
`(("en_US.UTF-8" . "Fake synopsis"))
`(("en_US.UTF-8" . "Fake description"))))
mock-inferior-packages)))
(define (test-license-set-ids conn)
(let ((license-id-lists
(inferior-packages->license-id-lists
conn
'((("License 1"
"https://gnu.org/licenses/test-1.html"
"https://example.com/why-license-1"))))))
'#((("License 1"
"https://gnu.org/licenses/test-1.html"
"https://example.com/why-license-1"))))))
(inferior-packages->license-set-ids conn license-id-lists)))
@ -73,7 +74,7 @@
conn
mock-package-metadata
(test-license-set-ids conn))
((x) (number? x))))
(#(x) (number? x))))
#:always-rollback? #t))
(with-postgresql-transaction