Skip to content

Introduce data store abstraction layer - #1524

Open
tomjaguarpaw wants to merge 68 commits into
haskell:masterfrom
tomjaguarpaw:abstraction
Open

tomjaguarpaw wants to merge 68 commits into
haskell:masterfrom
tomjaguarpaw:abstraction

Conversation

@tomjaguarpaw

Copy link
Copy Markdown
Member

Following a discussion and a draft PR (to my own fork), I am submitting this PR for consideration.

This PR abstracts out the functions that deal with the acid-state data store. This PR is a pure refactoring with no change in behaviour. Specifically, for each of hackage-server's components that use acid-state storage it does the following:

  • moves the ...Component function into the component's Acid module
  • introduces a component-local abstraction layer with Backend and Store
  • instantiates the Backend from the existing acid-state implementation in a function called acidStore

After this refactoring we could, for each such component (perhaps separately), swap out acid-state as the data store and replace it, with, for example, Postgres, by defining an appropriate value of type Backend.

The branch is structured as a sequence of trivial refactoring commits.


Now, I don't know for certain that the Hackage maintainers will think this PR is worth merging as merely a pure refactoring. It doesn't actually come with any performance or feature improvements. But maybe it would be considered a beneficial enough architectural improvement to merge, given that it makes it easier for someone to subsequently swap backends. Please do let me know what you think.


Specific improvement example

This replaces several direct uses of the following, where consumers first fetch the entire package index and then operate on a smaller piece of it:

queryGetPackageIndex :: m (PackageIndex PkgInfo)
PackageIndex.lookupPackageName :: PackageIndex PkgInfo -> PackageName -> [PkgInfo]
PackageIndex.lookupPackageId :: PackageIndex PkgInfo -> PackageId -> Maybe PkgInfo
PackageIndex.allPackagesByName :: PackageIndex PkgInfo -> [[PkgInfo]]

with these new fields of CoreFeature, which only return the subset of the package index actually required to implement a particular piece of functionality:

queryLookupPackageName :: PackageName -> m [PkgInfo]
queryLookupPackageId :: PackageId -> m (Maybe PkgInfo)
queryLatestPackages :: m [PkgInfo]

backed by the following fields of the Core Store:

lookupPackageName :: PackageName -> m [PkgInfo]
lookupPackageId :: PackageId -> m (Maybe PkgInfo)
latestPackages :: m [PkgInfo]

The current acid-state implementation defines these in terms of its existing PackageIndex, but consumers no longer need to fetch the whole PackageIndex just to perform package-name lookup, package-id lookup, or latest-version-per-package selection. A future alternative Backend could take advantage of these new methods to load less into memory.


Disclaimer: significant use of a coding agent was involved in producing this PR

@mergify

mergify Bot commented Sep 27, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

@tomjaguarpaw

Copy link
Copy Markdown
Member Author

There's a build error in commonmark-extensions-0.2.7.2:

[ 4 of 20] Compiling Commonmark.Extensions.Autolink ( src/Commonmark/Extensions/Autolink.hs, dist/build/Commonmark/Extensions/Autolink.o, dist/build/Commonmark/Extensions/Autolink.dyn_o )
src/Commonmark/Extensions/Autolink.hs:38:11: error: [GHC-88464]
    Variable not in scope:
      getPrecedingTokType
        :: ParsecT
             [Tok]
             (IPState m)
             (Control.Monad.Trans.State.Strict.StateT Commonmark.Tag.Enders m)
             (Maybe TokType)
   |
38 |   mbty <- getPrecedingTokType
   |           ^^^^^^^^^^^^^^^^^^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant