cbaines/guix-data-service
1
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
guix-data-service/tests/model-package-metadata.scm

99 lines
3 KiB
Scheme
Raw Normal View History

Remove the sha1_hash from the package_metadata table I'm thinking about adding more fields to this table, and the sha1_hash values will make this tricker. Therefore, remove the value, and adjust the existing code to cope. This commit also adds a new test which coveres some of the changed functionality.
2019-05-12 17:01:51 +01:00
(define-module (test-model-package-metadata)
#:use-module (ice-9 match)
#:use-module (srfi srfi-64)
Store and display the location of packages Store the location a package can be found at, and display this on the package page. If available, link off to the git repository containing the package.
2019-05-13 21:02:53 +01:00
#:use-module (guix utils)
Fix the package-metadata tests
2019-05-16 00:25:45 +01:00
#:use-module (guix tests)
Remove the sha1_hash from the package_metadata table I'm thinking about adding more fields to this table, and the sha1_hash values will make this tricker. Therefore, remove the value, and adjust the existing code to cope. This commit also adds a new test which coveres some of the changed functionality.
2019-05-12 17:01:51 +01:00
#:use-module (tests mock-inferior)
Adapt some license related code to work without mock in the tests With Guile 3, there's a potential for mock to work in even fewer circumstances. So, adapt the code to enable writing the tests without mock.
2020-02-24 21:35:44 +00:00
#:use-module (guix-data-service model license)
Fix the package-metadata tests
2019-05-16 00:25:45 +01:00
#:use-module (guix-data-service model license-set)
Add a missing import
2020-02-27 21:09:39 +00:00
#:use-module (guix-data-service model package-metadata)
Remove the sha1_hash from the package_metadata table I'm thinking about adding more fields to this table, and the sha1_hash values will make this tricker. Therefore, remove the value, and adjust the existing code to cope. This commit also adds a new test which coveres some of the changed functionality.
2019-05-12 17:01:51 +01:00
#:use-module (guix-data-service database))
(test-begin "test-model-package-metadata")
(define mock-inferior-package-foo
(mock-inferior-package
(name "foo")
(version "2")
(synopsis "Foo")
(description "Foo description")
Store and display the location of packages Store the location a package can be found at, and display this on the package page. If available, link off to the git repository containing the package.
2019-05-13 21:02:53 +01:00
(home-page "https://example.com")
(location (location "file.scm" 5 0))))
Remove the sha1_hash from the package_metadata table I'm thinking about adding more fields to this table, and the sha1_hash values will make this tricker. Therefore, remove the value, and adjust the existing code to cope. This commit also adds a new test which coveres some of the changed functionality.
2019-05-12 17:01:51 +01:00
Fix some duplicated values in tables The licenses table, along with the package_metadata table had duplicate values. This could happen as the unique constraints on those tables didn't properly account for the nullable fields. The duplicates in those tables also affected the license_sets, packages, package_derivations tables in a similar way. Finally, the guix_revision_package_derivations table was also affected. This commit adds a migration to fix the data, as well as the constraints. THe code to populate the licenses and package_metadata tables is also updated.
2019-08-04 09:39:40 +01:00
(define mock-inferior-package-foo-2
(mock-inferior-package
(name "foo")
(version "2")
(synopsis "Foo")
(description "Foo description")
(home-page #f)
(location #f)))
Close the load revision inferior prior to inserting data This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
2022-03-11 13:07:34 +00:00
(define mock-inferior-packages
(list mock-inferior-package-foo
mock-inferior-package-foo-2))
(define mock-package-metadata
Rework loading revision data These changes were motivated by switching to a mechanism of loading data that isn't dependent on the big advisory lock that prevents more than one revision from being processed at a time. Since INSERT ... RETURNING id; is used, this can block if another transaction inserts the same data, and then cause an error when that transaction commits. The solution is to use ON CONFLICT DO NOTHING, but you have to handle the case when the INSERT doesn't return an id since the other transaction has inserted it. This commit rewrites insert-missing-data-and-return-all-ids to do as described above, as well as being more efficient in how existing data is detected and to use more vectors. Other utilities for inserting data are added as well.
2024-12-05 20:56:23 +00:00
(list->vector
(map (lambda (mock-inf-pkg)
(list
(mock-inferior-package-home-page mock-inf-pkg)
(mock-inferior-package-location mock-inf-pkg)
`(("en_US.UTF-8" . "Fake synopsis"))
`(("en_US.UTF-8" . "Fake description"))))
mock-inferior-packages)))
Close the load revision inferior prior to inserting data This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
2022-03-11 13:07:34 +00:00
Fix the package-metadata tests
2019-05-16 00:25:45 +01:00
(define (test-license-set-ids conn)
Adapt some license related code to work without mock in the tests With Guile 3, there's a potential for mock to work in even fewer circumstances. So, adapt the code to enable writing the tests without mock.
2020-02-24 21:35:44 +00:00
(let ((license-id-lists
(inferior-packages->license-id-lists
conn
Rework loading revision data These changes were motivated by switching to a mechanism of loading data that isn't dependent on the big advisory lock that prevents more than one revision from being processed at a time. Since INSERT ... RETURNING id; is used, this can block if another transaction inserts the same data, and then cause an error when that transaction commits. The solution is to use ON CONFLICT DO NOTHING, but you have to handle the case when the INSERT doesn't return an id since the other transaction has inserted it. This commit rewrites insert-missing-data-and-return-all-ids to do as described above, as well as being more efficient in how existing data is detected and to use more vectors. Other utilities for inserting data are added as well.
2024-12-05 20:56:23 +00:00
'#((("License 1"
Attempt to reduce memory churn when processing package metadata
2024-12-22 11:05:38 +00:00
"https://gnu.org/licenses/test-1.html"
"https://example.com/why-license-1"))
(("License 1"
Rework loading revision data These changes were motivated by switching to a mechanism of loading data that isn't dependent on the big advisory lock that prevents more than one revision from being processed at a time. Since INSERT ... RETURNING id; is used, this can block if another transaction inserts the same data, and then cause an error when that transaction commits. The solution is to use ON CONFLICT DO NOTHING, but you have to handle the case when the INSERT doesn't return an id since the other transaction has inserted it. This commit rewrites insert-missing-data-and-return-all-ids to do as described above, as well as being more efficient in how existing data is detected and to use more vectors. Other utilities for inserting data are added as well.
2024-12-05 20:56:23 +00:00
"https://gnu.org/licenses/test-1.html"
"https://example.com/why-license-1"))))))
Adapt some license related code to work without mock in the tests With Guile 3, there's a potential for mock to work in even fewer circumstances. So, adapt the code to enable writing the tests without mock.
2020-02-24 21:35:44 +00:00
(inferior-packages->license-set-ids conn license-id-lists)))
Fix the package-metadata tests
2019-05-16 00:25:45 +01:00
Close the load revision inferior prior to inserting data This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
2022-03-11 13:07:34 +00:00
(with-mock-inferior-packages
(lambda ()
(use-modules (guix-data-service model package)
(guix-data-service model git-repository)
(guix-data-service model guix-revision)
(guix-data-service model package-metadata))
Remove the sha1_hash from the package_metadata table I'm thinking about adding more fields to this table, and the sha1_hash values will make this tricker. Therefore, remove the value, and adjust the existing code to cope. This commit also adds a new test which coveres some of the changed functionality.
2019-05-12 17:01:51 +01:00
Close the load revision inferior prior to inserting data This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
2022-03-11 13:07:34 +00:00
(with-postgresql-connection
"test-model-package-metadata"
(lambda (conn)
(check-test-database! conn)
Check that the test database is being used in all the tests
2020-02-24 21:16:41 +00:00
Close the load revision inferior prior to inserting data This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
2022-03-11 13:07:34 +00:00
(test-assert "inferior-packages->package-metadata-ids"
(with-postgresql-transaction
conn
(lambda (conn)
(match
(inferior-packages->package-metadata-ids
conn
mock-package-metadata
(test-license-set-ids conn))
Attempt to reduce memory churn when processing package metadata
2024-12-22 11:05:38 +00:00
(#(x y) (and (number? x)
(number? y)))))
Close the load revision inferior prior to inserting data This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
2022-03-11 13:07:34 +00:00
#:always-rollback? #t))
Fix type issues with the location ids On one code path, they were handled as numbers, whereas elsewhere they were handled as strings. This led to the package-metadata code trying to insert duplicate entries. Instead, just handle them as strings everywhere.
2019-05-14 07:55:17 +01:00
Close the load revision inferior prior to inserting data This means that the lock can be acquired after closing the inferior, freeing the large amount of memory that the inferior process is probably using.
2022-03-11 13:07:34 +00:00
(with-postgresql-transaction
conn
(lambda (conn)
(test-equal "inferior-packages->package-metadata-ids"
(inferior-packages->package-metadata-ids
conn
mock-package-metadata
(test-license-set-ids conn))
(inferior-packages->package-metadata-ids
conn
mock-package-metadata
(test-license-set-ids conn)))
#:always-rollback? #t))))))
Remove the sha1_hash from the package_metadata table I'm thinking about adding more fields to this table, and the sha1_hash values will make this tricker. Therefore, remove the value, and adjust the existing code to cope. This commit also adds a new test which coveres some of the changed functionality.
2019-05-12 17:01:51 +01:00
(test-end)
Reference in a new issue Copy permalink