Introduce data store abstraction layer - #1524
Open
tomjaguarpaw wants to merge 68 commits into
Open
tomjaguarpaw wants to merge 68 commits into
tomjaguarpaw wants to merge 68 commits into
Conversation
Contributor
|
Tick the box to add this pull request to the merge queue (same as
|
Member
Author
|
There's a build error in |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Following a discussion and a draft PR (to my own fork), I am submitting this PR for consideration.
This PR abstracts out the functions that deal with the
acid-statedata store. This PR is a pure refactoring with no change in behaviour. Specifically, for each ofhackage-server's components that useacid-statestorage it does the following:...Componentfunction into the component'sAcidmoduleBackendandStoreBackendfrom the existing acid-state implementation in a function calledacidStoreAfter this refactoring we could, for each such component (perhaps separately), swap out
acid-stateas the data store and replace it, with, for example, Postgres, by defining an appropriate value of typeBackend.The branch is structured as a sequence of trivial refactoring commits.
Now, I don't know for certain that the Hackage maintainers will think this PR is worth merging as merely a pure refactoring. It doesn't actually come with any performance or feature improvements. But maybe it would be considered a beneficial enough architectural improvement to merge, given that it makes it easier for someone to subsequently swap backends. Please do let me know what you think.
Specific improvement example
This replaces several direct uses of the following, where consumers first fetch the entire package index and then operate on a smaller piece of it:
with these new fields of
CoreFeature, which only return the subset of the package index actually required to implement a particular piece of functionality:backed by the following fields of the Core
Store:The current acid-state implementation defines these in terms of its existing
PackageIndex, but consumers no longer need to fetch the wholePackageIndexjust to perform package-name lookup, package-id lookup, or latest-version-per-package selection. A future alternativeBackendcould take advantage of these new methods to load less into memory.Disclaimer: significant use of a coding agent was involved in producing this PR