ANN: hastache 0.6

Announcing: hastache 0.6

Hastache is a Haskell implementation of the mustache templating system.

Quick start

cabal update
cabal install hastache

A simple example:

import Text.Hastache 
import Text.Hastache.Context 
import qualified Data.Text.Lazy.IO as TL

main = hastacheStr defaultConfig (encodeStr template) (mkStrContext context)
    >>= TL.putStrLn

template = "Hello, {{name}}!\n\nYou have {{unread}} unread messages." 

context "name" = MuVariable "Haskell"
context "unread" = MuVariable (100 :: Int)

Read Mustache documentation for template syntax; consult README for more details.

Whats’s new in 0.6?

The interface of the library has been switched from ByteString to (lazy) Text. That means, for example, that the type of hastcheStr function is now the following:

hastacheStr
:: MonadIO m => MuConfig m	-> Text	-> MuContext m -> m Text

The generic context generation (mkGenericContext) now supports functions of the types Text -> Text and Text -> m Text, as well as types with multiple constructors. That is, given a Haskell datastructure

data A = A { str :: String }
       | B { num :: Int }

it is possible to write a template like this:

{{#A}}
A : {{str}}
{{/A}}
{{#B}}
B : {{num}}
{{/B}}

Please take a look at the multiple constructors example if you are interested.

Additionally, a couple of new MuVar instances has been added.

Acknowledgments

Special thanks to Mark Lentczner, Alexander Voikov and others who reported issues, tested the library and provided feedback.

Representable functors in Haskell

I have finally managed to upload a (dry version) of a note about representable functors in Haskell. You can find it here: http://covariant.me/notes/rep-functors.html

Representable functors are functors that are isomorphic to $Hom(A, -)$ or $Hom(-, A)$ for some object $A$. In the note I give examples of some simple representable functors and prove that Maybe is not representable.

Cabal sandbox status in your ZSH prompt

Sometime ago I made a simple script for my zsh setup that allows me to see whether am I in a cabalized sandbox environment or not. I posted it to Haskell-cafe a month ago, but totally forgot to post it to this blog.

The result of checking for the sandbox is cached, which is probably unnecessary; it updates only when the user performs a cabal command or changes a directory.

After you place the contents of the script in your .zshrc file (or in a similar location), you should update your $PROMPT to use $(sandbox_prompt). The prompt I am using, by the way, is

HOST_COLOR=%{$fg_bold[yellow]%}
local ret_status="%(?:%{$fg_bold[green]%}:%{$fg_bold[red]%})%?%{$reset_color%}"
PROMPT=$'\n$(ssh_connection)$HOST_COLOR%n@%m%{$reset_color%}$(my_git_prompt)$(sandbox_prompt) : %~\n[${ret_status}] %# '

and it is based on the oh-my-solarized theme.

ANN: scotty-hastache 0.2

The scotty-hastache library has been updating, following the introduction of the custom exception types in Scotty. Custom exception types (as opposed to plain-text exception mechanism that was previously employed in Scotty) is a more powerful way of handling exceptional situation; an example usage can be found here. Below are the appropriate release notes for scotty-hastache

0.2

Keeping up with the Scotty 0.6 updating. The introduction of the custom exception types is reflected in the scotty-hastache package as well. The main types are now polymorphic over an exception type

type ScottyH e = ScottyT e HState
type ActionH e = ActionT e HState

In addition, the specialized (to Text) types and runners are introduced

type ScottyH' = ScottyH Text
type ActionH' = ActionH Text
scottyH'     :: Port -> ScottyH' () -> IO ()
scottyHOpts' :: Options -> ScottyH' () -> IO ()

Thanks to Kirill Zaborsky for pointing out the incompatibility with 0.6.

As always, the scotty-hastache library is available on Hackage and on GitHub

Interactive-diagrams updates

It has been some time since I’ve written a blog post. Today I would like to present you the latest updates from the interactive-diagrams project.

  • diagrams-ghcjs finally got text support

Big thanks to Joel Burget for the implementation.

The testsuite demo for the package can be found at http://co-dan.github.io/ghcjs/diagrams-ghcjs-tests/Main.jsexe/ as usual.

  • New interactive widgets

We got rid of the old wizard-like widgets in favour of “all-in-one” style widgets. Thanks to Brent and Luite who came up with the type trickery to get this done.

  • New flat design

In order to match your slick & flat iOS7 style I’ve rolled out an update to Bootstrap 3.

  • Ability to paste code with errors. (It’ll ask you to make sure you didn’t make a mistake by accident)
  • Ability to quickly include a number of imports. By checking the “import standard modules” checkbox you’ll automatically bring several useful modules into the scope:
    • Diagrams.Prelude
    • Diagrams.Backend.SVG (or Diagrams.Backend.GHCJS if you are making an interactive widget)
    • Data.Maybe
    • Data.Tuple
    • Data.List
    • Data.Char
  • Other minor UI fixes such as documentation improvement

GSoC 2013, an afterword

The Summer of Code 2013 is over, and here is what I have to say about it.

Introduction

The project is live at http://paste.hskll.org. The source code can be found at http://github.com/co-dan/interactive-diagrams.

I would like to say that I do plan to continue working on the project (and on adjacent projects as well if possible).

Interactive diagrams

Interactive diagrams is a pastebin and a set of libraries used for dynamically compiling, interpreting and rendering the results of user inputted code in a secure environment.

The user inputs some code and the app compiles and renders it. Graphical output alongside with code can be useful for sharing the experiments, teaching beginners an so on. If the users inputs a code that can not be rendered on the server (i.e.: a function), the app produces an HTML5/JS widget that runs the corresponding code.

The produced libraries can be used in 3rd party services/programs.

Screenshot

Screenshot

Technology used

The pastebin is powered by Scotty and scotty-hastache, the access to PosgreSQL db is done via the excellent persistent library. The compilation is done using GHC and GHCJS inside the workers processes powered by the restricted-workers library.

You can read some my previous report on this project which is still pretty relevant.

I plan on updating the documents on the wiki sometime soon.

Progress

The bad news is that I don’t think I was able to 100% complete what I originally envisioned. The good news is that I seem to know, almost exactly, what do I want to improve and how to do that. As I’ve mentioned I plan on continuing with the project and I hope that the project will grow and improve.

One thing that I felt was annoying is the (technical) requirement to use GHC HEAD. Because of that a lot of packages required updates and fixes. Due to changes in GHC and bugfixes in GHCJS I had to perform the tiring and not so productive procedure of updating all the necessary tools, rebuilding everything and so on. But I guess that’s just how computers work and I am sure that in the future (with the release of GHC 7.8 and a new Haskell Platform) the maintenance and installation will be much easier. Another thing that took a lot of my time was configuring the system and setting up the necessary security measures, which was also necessary.

Other stuff that kinda slowed thing down include: the lack of a good build system, in some cases non-American timezone (actually I think that the fact that my mentor, Luite Stegeman, was quite close to me in terms of timezones allowed us to communicated very frequently, as we did), the lack of knowledge of the tools I used (although you can think of it this way: I had an ability to learn exciting new things ;] ).

Among the grand things I plan to do: release a library for manipulating Haskell AST at the GHC level; make an IRC bot using the eval-api and restricted-workers; continue writing my notes/tutorials about GHC API (I have a few drafts laying around).

Some code refactoring should come along and a number of features for the pastebin should be implemented.

Feelings

When the end of the program was approaching I predicted that I would have that sort of conflicted feelings that you usually get when you finish reading a good book – one part of you feels happy because you had an enjoyable experience, yet another part of you doesn’t feel so giddy, because the thing that you enjoyed is over. Well, I didn’t get this with GSoC. I did feel happy, but I didn’t get this touch of sadness. GSoC was a way for me to get into collaborating with people on real-world open source projects, and the end of GSoC for me is a beginning of something else. I can use my experience now to write better code, write more code and write useful code.

Concluding

I had a very exciting summer and I would positively recommend anyone eligible to participate in the Google Summer of Code program. There is, however, a thing to remember. Programmers are known to be the kind of people who set ambitious goals. Reach for something inspiring, ambitious, yet realistic. Make sure to find something in between, that way you’ll have a concrete target that you know that you are able to achieve, but you also have a room for improvement.

PS. Acknowledgments

I would like to thank various people who helped me along the summer: Luite Stegeman, Brent Yorgey, Carter Schonwald, Daniel Bergey, Andrew Farmer; everyone in #diagrams, everyone in the #haskell channel who patiently answered my question; everyone on GitHub who responded to my comments, questions and pull requests. The Haskell community is lucky to have a huge amount of friendly and smart people.

Announcing scotty-hastache

I am happy to announce the first release of the scotty-hastache library. The purpose of this library is to use the light templating language Mustache together with the Scotty web framework. This is done by enriching the ActionM DSL with additional commands.

Example code:

{-# LANGUAGE OverloadedStrings #-}
module Main where

import Text.Hastache
import Web.Scotty.Trans as S
import Web.Scotty.Hastache

main :: IO ()
main = scottyH 3000 $ do
  setTemplatesDir "templates"
  -- ^ Setting up the director with templates
  get "/:word" $ do
    beam <- param "word"
    setH "action" $ MuVariable (beam :: String)
    -- ^ "action" will be binded to the contents of 'beam'
    hastache "greet.html"

This is possible due to the recent changes introduced into the latest scotty-0.5 release, in particular the switch to monad transformers. You can read the scotty-0.5 release notes here.

The scotty-hastache library is available on Hackage and on GitHub

WIP: GHCJS backend for the diagrams library

About

I’ve picked up the development of the diagrams-ghcjs backend for the infamous diagrams library. This backend is what we use for the interactive-diagrams pastebin and it renders diagrams on an HTML5 Canvas using the canvas bindings for ghcjs. The diagrams-ghcjs is a fork of (unmaintained?) diagrams-canvas backend by Andy Gill and Ryan Yates.

The library is still WIP and it requires bleeding edge versions of ghcjs, ghcjs-canvas and ghcjs’ shims to function.

The library is scheduled to be released together with ghcjs.

Demo

The current demo can be found at http://co-dan.github.io/ghcjs/diagrams-ghcjs-tests/Main.jsexe/. It is supposed to work in both Firefox and Chrome.

Problems

  • Text is not implemented. Some work is being done in the text-support branch. Generally, it has been proven hard to implement good text support, even diagrams-svg backend text support is limited;
  • Firefox still does not support line dashing via ‘setLineDash’ and ‘lineDashOffset’. As a result we need to have additional shims.

GHC API: Interpreted, compiled and package modules

The third post in the series.

Intro

It’s hard to get into writing code that uses GHC API. The API itself is and the number of various functions and options significantly outnumber the amount of tutorials around.

In this series of blog posts I’ll elaborate on some of the peculiar, interesting problems I’ve encountered during my experience writing code that uses GHC API and also provide various tips I find useful.

I have built for myself a small layer of helper functions that helped me with using GHC API for the interactive-diagrams project. The source can be found on GitHub and I plan on refactoring the code and releasing it separately.

Today I would like to talk about a different ways of bringing contents of Haskell modules into scope, a process that is necessary for evaluating/interpreting bits of code on-the-fly.

Many of the points I make in this post are actually trivial, but nevertheless I made all of the mistakes I mentioned in this, perhaps post due to my naive approach of quickly diving in and experimenting, instead of reading into the documentation and source code. Now I actually realize that this post should been the first in the series, since it probably deals with more basic (and fundamental) stuff than the previous two posts. But anyway, here it is.

Interpreted modules

Imagine the following situation: we have a Haskell source file with code we want to load dynamically and evaluate. That is a basic task in the GHC API terms but nevertheless there are some caveats. We start with the most basics.

Let us have a file ‘test.hs’ containing the code we want to access:

module Test (test) where
test :: Int
test = 123

The basic way to get the ‘test’ data would be to load ‘Test’ as an interpreted module:

import Control.Applicative
import DynFlags
import GHC
import GHC.Paths
import GhcMonad            (liftIO) -- from ghc7.7 and up you can use the usual
    -- liftIO from Control.Monad.IO.Class
import Unsafe.Coerce

main = defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
    runGhc (Just libdir) $ do
        -- we have to call 'setSessionDynFlags' before doing
        -- everything else
        dflags <- getSessionDynFlags
        -- If we want to make GHC interpret our code on the fly, we
                  -- ought to set those two flags, otherwise we
                  -- wouldn't be able to use 'setContext' below
        setSessionDynFlags $ dflags { hscTarget = HscInterpreted
                                    , ghcLink   = LinkInMemory
                                    }
        setTargets =<< sequence [guessTarget "test.hs" Nothing]
        load LoadAllTargets
        -- Bringing the module into the context
        setContext [IIModule $ mkModuleName "Test"]

        -- evaluating and running an action
        act <- unsafeCoerce <$> compileExpr "print test"           
        liftIO act

The reason that we have to use HscInterpreted and LinkInMemory is that otherwise it would compile test.hs in the current directory and leave test.hi and test.o files, which we would not be able to load in the interpreted mode. setContext, however will try to bring the code in those files first, when looking for the module ‘Test’

dan@aquabox
[0] % ghc --make target.hs -package ghc
[1 of 1] Compiling Main             ( target.hs, target.o )
Linking target ...

dan@aquabox
[0] % ./target
123

Let’s try something fancier like printing a list of integers, one-by-one.

main = defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
    runGhc (Just libdir) $ do
        dflags <- getSessionDynFlags
        setSessionDynFlags $ dflags { hscTarget = HscInterpreted
                                    , ghcLink   = LinkInMemory
                                    }
        setTargets =<< sequence [guessTarget "test.hs" Nothing]
        load LoadAllTargets
        -- Bringing the module into the context
        setContext [IIModule $ mkModuleName "Test"]

        -- evaluating and running an action
        act <- unsafeCoerce <$> compileExpr "forM_ [1,2,test] print"
        liftIO act

But when we try to run it..

dan@aquabox
[0] % ./target
target: panic! (the 'impossible' happened)
  (GHC version 7.6.3 for x86_64-apple-darwin):
        Not in scope: `forM_'

Please report this as a GHC bug:

http://www.haskell.org/ghc/reportabug

Hm, it looks like we need to bring Control.Monad into the scope.

This brings us to the next section.

Package modules

Naively, we might want to load Control.Monad in a similar fashion as we did with loading test.hs

main = defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
    runGhc (Just libdir) $ do
        dflags <- getSessionDynFlags
        setSessionDynFlags $ dflags { hscTarget = HscInterpreted
                                    , ghcLink   = LinkInMemory
                                    }
        setTargets =<< sequence [ guessTarget "test.hs" Nothing
                                , guessTarget "Control.Monad" Nothing]
        load LoadAllTargets
        -- Bringing the module into the context
        setContext [IIModule $ mkModuleName "Test"]

        -- evaluating and running an action
        act <- unsafeCoerce <$> compileExpr "forM_ [1,2,test] print"
        liftIO act

Our attempt fails:

dan@aquabox
[0] % ./target
target: panic! (the 'impossible' happened)
  (GHC version 7.6.3 for x86_64-apple-darwin):
        module `Control.Monad' is a package module

Please report this as a GHC bug:

http://www.haskell.org/ghc/reportabug

Huh, what? I thought guessTarget works on all kinds of modules.

Well, it does. But it doesn’t “load the module”, it merely sets it as the target for compilation, basically it (together with load LoadAllTargets) does what ghc --make does. And surely it doesn’t make much sense to ghc --make Control.Monad when Control.Monad is a module from the base package. What we need to do instead is to bring the compiled Control.Monad module into scope. Luckily it’s not very hard to do with the help of the simpleImportDecl :: ModuleName -> ImportDecl name:

main = defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
    runGhc (Just libdir) $ do
        dflags <- getSessionDynFlags
        setSessionDynFlags $ dflags { hscTarget = HscInterpreted
                                    , ghcLink   = LinkInMemory
                                    }
        setTargets =<< sequence [ guessTarget "test.hs" Nothing ]
        load LoadAllTargets
        -- Bringing the module into the context
        setContext [ IIModule . mkModuleName $ "Test"
                   , IIDecl
                     . simpleImportDecl
                     . mkModuleName $ "Control.Monad" ]

        -- evaluating and running an action
        act <- unsafeCoerce <$> compileExpr "forM_ [1,2,test] print"
        liftIO act

And we can run it

dan@aquabox
[0] % ./target
1
2
123

Compiled modules

What we have implemented so far corresponds to the :load* command in GHCi, which gives us the full access to the source code of the program. To illustrate this let’s modify our test file:

module Test (test) where

test :: Int
test = 123

test2 :: String
test2 = "Hi"

Now, if we try to load that file as an interpreted module and evaluate test2 nothing will stop us from doing so.

dan@aquabo
[0] % ./target-interp
(123,"Hi")

To use the compiled module we have to bring Test into the context the same way we dealt with Control.Monad

main = defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
runGhc (Just libdir) $ do
    dflags <- getSessionDynFlags
    setSessionDynFlags $ dflags { hscTarget = HscInterpreted
                                , ghcLink   = LinkInMemory
                                }
    setTargets =<< sequence [ guessTarget "Test" Nothing ]
    load LoadAllTargets
    -- Bringing the module into the context
    setContext [ IIDecl $ simpleImportDecl (mkModuleName "Test")
               , IIDecl $ simpleImportDecl (mkModuleName "Prelude")
               ]
    printExpr "test"
    printExpr "test2"

printExpr :: String -> Ghc ()
printExpr expr = do
    liftIO $ putStrLn ("-- Going to print " ++ expr)
    act <- unsafeCoerce <$> compileExpr ("print (" ++ expr ++ ")")
    liftIO act

Output:

dan@aquabox : ~/snippets/ghcapi
[0] % ./target
-- Going to print test
123
-- Going to print test2
target: panic! (the 'impossible' happened)
  (GHC version 7.6.3 for x86_64-apple-darwin):
	Not in scope: `test2'
Perhaps you meant `test' (imported from Test)

Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

Note: I had to bring the Prelude into context this time, like a regular module. I tried setting the ideclImplicit option in ImportDecl, but it didn’t work for some reason. Maybe it actually supposed to do not what I think it supposed to do, but something else.

Outro

So, that is it, we have managed to dynamically load Haskell source code and evaluate it. I can only refer you to the GHC haddocs for specific functions that we used in this post, most of them contain way more options that we used and they might prove to be useful to you.