Adding a package database to the GHC API session

The second post in the series.

Intro

It’s hard to get into writing code that uses GHC API. It’s huge there are so many options around and not a lot of introduction-level tutorials.

In this series of blog posts I’ll elaborate on some of the peculiar, interesting problems I’ve encountered during my experience writing code that uses GHC API and also provide various tips I find useful.

I have built for myself a small layer of helper functions that helped me with using GHC API for the interactive-diagrams project. The source can be found on GitHub and I plan on refactoring the code and releasing it separately.

One particular thing I had to do was to add a GHC package database to the GHC API session.

For those familiar with the structure of the interactive-diagrams project: since the workers run in a separate environment, each of them has it’s own chroot jail including each own package database. I had to manually set up a path to package database for each worker so it would pick up the necessary packages.

Package databases

A package database is a directory where the information about your installed packages is stored. For each package registered in the database there is a .conf file with the package details. The .conf file contains the package description (just like in the .cabal file) as well as path to binaries and a list of resolved dependencies:

$ cat aeson-0.6.1.0.1-5a107a6c6642055d7d5f98c65284796a.conf
name: aeson
version: 0.6.1.0.1
id: aeson-0.6.1.0.1-5a107a6c6642055d7d5f98c65284796a

import-dirs: /home/dan/.cabal/lib/aeson-0.6.1.0.1/ghc-7.7.20130722
library-dirs: /home/dan/.cabal/lib/aeson-0.6.1.0.1/ghc-7.7.20130722

depends: attoparsec-0.10.4.0-acffb7126aca47a107cf7722d75f1f5e
         base-4.7.0.0-b67b4d8660168c197a2f385a9347434d
         blaze-builder-0.3.1.1-9fd49ac1608ca25e284a8ac6908d5148
         bytestring-0.10.3.0-66e3f5813c3dc8ef9647156d1743f0ef

You can use ghc-pkg to manage installed packages on your system. For example, to list all the packages you’ve installed run ghc-pkg list. To list all the package databases that are automatically picked up by ghc-pkg do the following:

$ ghc-pkg nonexistentpkg
/home/dan/ghc/lib/ghc-7.7.20130722/package.conf.d
/home/dan/.ghc/i386-linux-7.7.20130722/package.conf.d

See ghc-pkg --help or the online documentation for more details.

Adding a package db

By default GHC knows only about two package databases: the global package database (usually /usr/lib/ghc-something/ on Linux) and the user-specific database (usually ~/.ghc/lib). In order to pick up a package that resides in a different package database you have to employ some tricks.

For some reason GHC API does not export an clear and easy-to-use function that would allow you to do that, although the code we need is present in the GHC sources.

The way this whole thing works is the following:

  1. GHC calls initPackages, which reads the database files and sets up the internal table of package information
  2. The reading of package databases is performed via the readPackageConfigs function. It reads the user package database, the global package database, the “GHC_PACKAGE_PATH” environment variable, and applies the extraPkgConfs function, which is a dynflag and has the following type: extraPkgConfs :: [PkgConfRef] -> [PkgConfRef] (PkgConfRef is a type representing the package database). The extraPkgConf flag is supposed to represent the -package-db command line option.
  3. Once the database is parsed, the loaded packages are stored in the pkgDatabase dynflag which is a list of PackageConfigs

So, in order to add a package database to the current session we have to simply modify the extraPkgConfs dynflag. Actually, there is already a function present in the GHC source that does exactly what we need: addPkgConfRef :: PkgConfRef -> DynP (). Unfortunately it’s not exported so we can’t use it in our own code. I rolled my own functions that I am using in the interactive-diagrams project, feel free to copy them:

-- | Add a package database to the Ghc monad
#if __GLASGOW_HASKELL_ >= 707  
addPkgDb :: GhcMonad m => FilePath -> m ()
#else
addPkgDb :: (MonadIO m, GhcMonad m) => FilePath -> m ()
#endif
addPkgDb fp = do
  dfs <- getSessionDynFlags
  let pkg  = PkgConfFile fp
  let dfs' = dfs { extraPkgConfs = (pkg:) . extraPkgConfs dfs }
  setSessionDynFlags dfs'
#if __GLASGOW_HASKELL_ >= 707    
  _ <- initPackages dfs'
#else
  _ <- liftIO $ initPackages dfs'
#endif
  return ()

-- | Add a list of package databases to the Ghc monad
-- This should be equivalen to  
-- > addPkgDbs ls = mapM_ addPkgDb ls
-- but it is actaully faster, because it does the package
-- reintialization after adding all the databases
#if __GLASGOW_HASKELL_ >= 707      
addPkgDbs :: GhcMonad m => [FilePath] -> m ()
#else
addPkgDbs :: (MonadIO m, GhcMonad m) => [FilePath] -> m ()
#endif             
addPkgDbs fps = do 
  dfs <- getSessionDynFlags
  let pkgs = map PkgConfFile fps
  let dfs' = dfs { extraPkgConfs = (pkgs ++) . extraPkgConfs dfs }
  setSessionDynFlags dfs'
#if __GLASGOW_HASKELL_ >= 707    
  _ <- initPackages dfs'
#else
  _ <- liftIO $ initPackages dfs'
#endif       
  return ()
  • Packages module, contains other functions that modify/make use of extraPkgConfs

Outro

This was the second post in the series and we have seen how to add a package database to the GHC session. Stay tuned for more brief posts and updates.

About these ads

One thought on “Adding a package database to the GHC API session

  1. Pingback: GHC API: Interpreted, compiled and package modules | (parentheses)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s