As an example, we will represent a simple user data type:

data User = User

          { name  :: String

          , email :: String

          , admin :: Bool

          }

The generic representation of this type is called a pattern functor. For the user data type, it will look like this:

type instance PF User = C User_User_

  (   S User_User_name_  (K String)

  :*: S User_User_email_ (K String)

  :*: S User_User_admin  (K Bool)

  )

Here we see a few of the functors that are the building blocks of our generic representation: The C type marks constructors, the S type marks record labels, K marks the types that make up the fields of our record, and (:*:) is the product mentioned earlier, and is very similar to the (,) used for constructing tuples. There are three more functors in regular: I, which marks recursive positions in a type; (:+:), which sums together different constructors; and U, for constructors without fields.

To write a generic function, we define a type class which contains this generic function, and give an instance for each of these functors. If we then also define conversion functions from our data type to the generic representation and back, we can apply the function to our data type. Note that for regular, these conversion functions, and the representation given above, can all be generated using Template Haskell.

Since we want to write a generic XML pickler for use with the HXT library, we’ll define a type class containing such a pickler:

class GXmlPickler f where

  gxpicklef :: PU a -> PU (f a)

This says that we can give a generic pickler for one of the functors f, containing a‘s at the recursive positions, if we have a pickler for a‘s. I’ll now show the instances for all the functors, and explain them one by one.

The first is the instance for I, marking recursive positions. Since we get a pickler for the resursive positions, all we have to do is wrap it in the I constructor when unpickling, and remove that constructor when pickling. HXT provides the xpWrap function to transform a pickler like this.

instance GXmlPickler I where

  gxpicklef = xpWrap (I, unI)

Next, we’ll tackle K. Here, we require that the type of the field has its own pickler, and use that. We again use xpWrap to add and remove the constructor of the functor. We provide a special instance for String, since it doesn’t have a standard pickler. We choose to use the pickler that allows empty strings.

instance XmlPickler a => GXmlPickler (K a) where

  gxpicklef _ = (K, unK) `xpWrap` xpickle



instance GXmlPickler (K String) where

  gxpicklef _ = (K, unK) `xpWrap` xpText0

U works similarly. We use the pickler for (), and use xpWrap to go from a () to a U and back.

instance GXmlPickler U where

  gxpicklef _ = (const U, const ()) `xpWrap` xpUnit

The case for (:+:) is interesting. Remember that (:+:) represents a choice between two functors. If we have picklers for both of these, we can define a pickler for the sum as follows: during conversion to XML, we can pattern match on L or R (the constructors of (:+:)) and choose the left or right pickler appropriately. During conversion from XML, we try the first pickler. If it fails (an unpickler returns a Maybe value) we try the second. Since HXT doesn’t seem to have a combinator that follows this logic, let’s define one ourselves:

xpEither :: PU (f r) -> PU (g r) -> PU ((f :+: g) r)

xpEither (PU fl tl sa) (PU fr tr sb) = PU

  (\(x, st) -> case x of

                 L y -> fl (y, st)

                 R y -> fr (y, st))

  (\x -> case tl x of

           (Nothing, _) -> lmap (fmap R) (tr x)

           r            -> lmap (fmap L) r)

  (sa `scAlt` sb)

  where lmap f (a, b) = (f a, b)

Note that we take apart the PU type, and create a pretty printer, a parser and a schema separately. We can use this function in the GXmlPickler instance by supplying it with two picklers, created by recursive calls to gxpicklef.

instance (GXmlPickler f, GXmlPickler g) => GXmlPickler (f :+: g) where

  gxpicklef f = xpEither (gxpicklef f) (gxpicklef f)

The instance for (:*:) is easy again. Since (:*:) combines two functors, we can require that these two have a pickler. We use xpPair to combine these into a pickler for a pair, and then use xpWrap again to convert it into a pickler for (:*:).

instance (GXmlPickler f, GXmlPickler g) => GXmlPickler (f :*: g) where

  gxpicklef f = (uncurry (:*:), \(a :*: b) -> (a, b))

                `xpWrap`

                (gxpicklef f `xpPair` gxpicklef f)

Note that so far, we haven’t created any XML tags ourselves. We’ve only combined picklers. Now we come to the instances for constructors and record selectors. Here, we’ll use the constructor or selector name (converted to lowercase) to generate and parse xml tags. This is done with the xpElem combinator.

instance (Constructor c, GXmlPickler f) => GXmlPickler (C c f) where

  gxpicklef f = xpElem (map toLower $ conName (undefined :: C c f r))

                  ((C, unC) `xpWrap` gxpicklef f)



instance (Selector s, GXmlPickler f) => GXmlPickler (S s f) where

  gxpicklef f = xpElem (map toLower $ selName (undefined :: S s f r))

                  ((S, unS) `xpWrap` gxpicklef f)

We now have picklers for all generic representations built from the standard functors in regular. We still need a top level function that works on our real data types, though. Regular provides the two functions to and from to convert between data types and their generic representation. We can use these together with xpWrap to change our generic pickler into a pickler for real data types. Another matter we have not addressed is the first argument to gpicklef, which is a pickler for the recursive positions. Here we pass the top level function, since the recursive position contains our original data type again.

gxpickle :: (Regular a, GXmlPickler (PF a)) => PU a

gxpickle = (to, from) `xpWrap` gxpicklef gxpickle

And that’s all for the generic XML pickler. Using it is very simple. To use it for our user data type above, we import Generic.Regular, Generic.Regular.TH, Generic.Regular.XmlPickler and Text.XML.HXT.Arrow.Pickle. We can then derive the Regular instance for user (note that we need a little bit of extra code, since Template Haskell cannot generate type family instances yet), and make an XmlPickler instance:

$(deriveAll ''User "PFUser")

type instance PF User = PFUser



instance XmlPickler User where

  xpickle = gxpickle

And that’s it! The package is available as regular-xmlpickler on hackage, and the source can also be found on github. The packages regular and HXT can also be found on hackage.

We figured that using DBXML would give us some important advantages:

• Collections of HTML5 documents will form the basis of data model. Only one trivial conversion from HTML5 to syntactically valid XML is needed to get our documents into the XML database. Once stored we can perform some interesting queries over our data.
• XML databases allow for the storage of complex data layouts without having a strict schema. Without a schema it will be easier to adjust our data model over time without instantly breaking our software.
• XQuery is a very expressive (almost-purely functional) querying language which is at least as powerful as SQL and far more flexible in the structure of the data to target.
• XML can be used to both encode strictly defined datatypes and store free-form documents in the same document collection. This will enable us to put both our meta data and our documents in the same database.
• A quick look on Hackage revealed there is an out-of-the-box easy-to-use Haskell binding available for the Berkeley XML database. No need to create custom bindings ourselves.
• We are in the advantage (or disadvantage) of having Haskell as our language of choice for our server software. Because of the hierarchical nature of both XML and Haskell algebraic datatypes, an XML database feels like a perfect fit.

Once we decided to go for an DBXML back-end we had to figure out how to easily get values form our Haskell program in and out of the database. The rest of this post will be dealing with the last point of our enumeration: how to get a nice mapping from Haskell’s algebraic datatypes to our DBXML back-end.

XML queries

The DBXML binding for Haskell is a shallow wrapper around the existing C++ API. This library allows us to perform the common create, read, update and delete queries for entire XML documents or parts of it. Communication with the XML database happens mainly via XQuery. Queries and query parameters are passed into the API (and results will come out) using Haskell ByteStrings. It is up to the programmer to setup the queries with the right XML structure and encoding. Take for example the (somewhat simplified) type signature of the query function:

query :: Collection -> Query -> Parameters -> IO [ByteString]

This function takes an identification of the XML collection, which is somewhat like a database handle, an XML query, a set of query parameters and returns a possibly empty list of XML snippets as ByteStrings. Too bad that all our domain objects are well-typed Haskell algebraic datatypes and not raw sequences of bytes. We need a simple XML (de)serialization tool for this.

XML picklers

The Haskell XML Toolbox (HXT) is a library containing a (quite extended) collection of XML processing tools. The library has support for XML parsing, pretty printing, XPath queries, XSL stylesheets, DTD, XSD and RelaxNG schemas and a lot more. Interestingly, HXT exposes a type class and an accompanying set of combinators called XML picklers. XML picklers can be used to build conversion functions from Haskell datatypes to XML and vice versa. The type class looks like this:

  class XmlPickler a where

    xpickle :: PU a

So for every type in the XmlPickler type class there is some PU available. The PU datatype is composed of a pair of pickle (serialize) and unpickle (deserialize) functions together with a schema description. Because we won’t be using the schema definitions we will ignore them for now. There is probably no need to ever touch the functions inside the PU type, because the library supplies a vast amount of basic pickler combinators to be used instead.

To illustrate the usage of HXT picklers take this simple Haskell datatype representing a single user in our system:

data User = User

  { name     :: String

  , email    :: String

  , password :: String

  , openID   :: String

  }

Using some of the basic pickler combinators from the library it is very easy to come up with a suitable XmlPickler instance:

class XmlPickler User where

  userPickle =

    xpElem "user"

    $ xpWrap

      ( (\(a, b, c, d, e) -> User a b c d e)

      , (\(User a b c d e) -> (a, b, c, d, e))

      )

    $ xp5Tuple

      (xpElem "username" xpText)

      (xpElem "name"     xpText)

      (xpElem "password" xpText)

      (xpElem "email"    xpText)

      (xpElem "openid"   xpText0)

This instance uses the xp5Tuple function to pickle five sub-picklers into a big tuple. The five fields will be appropriately named elements from which the text value will be used. The tuple will be converted into a value of the User datatype using the xpWrap function. This is all you to need to manually write XML serialization and deserialization code.

A bit off topic but interesting to note is the fact that the xpWrap function can be seen as a pickler specific and bidirectional version of the well known fmap for Functors. The xpWrap is used to define true isomorphisms. When we generalize the type of xpWrap to work arbitrary containers, lets call this function bifmap, and compare the type signatures this similarity becomes obvious:

fmap   :: (a -> b)         -> f a -> f b

bifmap :: (a -> b, b -> a) -> f a -> f b

So, taking the XmlPickler instance for our User datatype we can now easily convert users into XML and read them back in, like the following example:

User "jd" "John Doe" "secret" "john@doe" "none"

<user>

  <username>jd</username>

  <name>John Doe</name>

  <password>secret</password>

  <email>john@doe</email>

  <openid>none</openid>

</user>

Using the xpickle function from the type class and the xunpickleVal from the HXT library we can now write a more suitable query function on top of the raw version. This version does not return ByteStrings, but values of any type we can convert to. Off course, the information in the database should match your datatype, otherwise the unpickler function will just produce parse errors resulting in an empty list.

query :: XmlPickler a => Collection -> Query -> Parameters -> IO [a]

Although this pickler example for our user is a very simple one, even more complicated datatypes, including multi-constructor and possibly mutual recursive datatypes, can quite easily be made an instance of the XmlPickler class. Unfortunately we still have to write them all by hand.

Going generic

After a few of years at the University of Utrecht we learned at least one valuable lesson, never write functions by hand when they can be derived generically. We decided to write a generic XML pickler function using the generic programming library Regular, developed (not entirely coincidentally) at the University of Utrecht. Regular is a relatively simple but powerful tool for writing data type generic functions. The library has support for deriving embedding projection pairs (conversions from and to a generic representation) using Template Haskell and provides enough reflection to inspect constructor names and record labels.

The generic representation is encoded as a type family (the pattern functor, or PF) over the original data type. The generic pickler function we developed has the following signature:

gxpickle :: (Regular a, GXmlPickler (PF a)) => PU a

This means that for every type that we can convert to a generic representation (indicated by the Regular type class) and for every type that has a GXmlPickler instance for its generic representation, we can deliver a PU. The regular-xmlpickler package implements the GXmlPickler type class and the instances for the types of which the representations are composed. So, all we need now for our User datatype is to derive a generic representation and use the generic implementation for the XmlPickler instance.

$(deriveAll ''User "PFUser")

type instance PF User = PFUser



instance XmlPickler User where

  xpickle = gxpickle

Using this automatically derived XML pickler and the query function described above we can now query the DBXML backend for all users that satisfy a certain property:

jd :: IO [User]

jd = query myCollection "/user[username=$name]" [("name", "jd")]

Now we’re able to query a database for pieces of XML and reify these as true Haskell values with almost no boilerplate involved! Because of the bidirectional behavior of the XmlPickler type class it shouldn’t be difficult to imagine that the same trick is applicable to inserting and updating database entries.

We will discuss writing generic functions using the Regular library in a later post.

On the client, there wasn’t much discussion. Javascript is the only viable choice, unless you want to use Flash or similar plugin-based models. But on the server, we had more freedom. Popular choices for web applications are dynamically typed languages like Ruby, Python and PHP, and statically typed languages like Java and C#.

However, there was another option. Two of us (me and Sebas) are enrolled in the Software Technology master program at Utrecht University, where we often use the programming language Haskell. Haskell is a functional programming language, which means that a program is composed of expressions instead of statements, as is the case in an imperative programming language.

As you might have guessed from the title of this post, we decided to use Haskell for our server side programming. What are the features that made us choose Haskell?

First, we like the functional programming model. Modeling programs based on what values are, instead of what bits to move where, is much closer to the way we think. We write down what we mean, and let the compiler figure out how to calculate it. This also leads to shorter code: Haskell is often surprisingly concise.

Another important consideration is the strong type system that Haskell has. It has powerful features that prevent you from running incorrect programs, while not getting in the way, as is often the case in traditional statically typed languages like Java or C#. This is due to the fact that Haskell has type inference: it can deduce the types of variables and functions without programmer specifications.

Something that sets Haskell apart from almost all other programming languages, is its purity: an expression in Haskell is not allowed to do anything on evaluation other than return its value (functions are not allowed to have side-effects). This has many implications. For example, data is immutable: if you modify a list, for example, you create a new copy (although smart compilers share duplicate parts for efficiency). This means that you never have to think about other parts of the code changing your data: you can always reason locally. Another implication is that your language can be lazy (and in fact, Haskell is). This means that expressions are not evaluated unless their results are needed. This can lead to performance improvements, and allows you to use infinite data structures.

If we disallow side effects, how do we perform I/O, for example? Haskell uses an abstraction called a monad to do this. A monad shows up in the type, indicating that we are performing side effects. The I/O monad allows us to force an ordering on actions, allowing us to safely perform the actions in a lazy setting. We can also use monads to keep state, have mutable variables, and many other things.

Purity also makes concurrency a lot easier: if data is immutable, there is no risk in sharing it between different threads. If you need to share mutable data between different threads, Haskell offers a few ways of dealing with this, the most interesting of which is software transactional memory (STM), a lock-free way of building composable atomic actions.

Since Haskell is a functional language, functions are first class: you can create them on the fly, pass them as arguments, return them as results, store them in a data structure, etc. This allows you to easily create powerful abstractions. For example, one often-used Haskell function is map. It takes an argument that turns a value of type a into a value of type b, and uses this to transform a list of a’s into a list of b’s by applying it to each element. This function abstracts away looping over a list. In a similar way we can create our own abstractions. This often leads to the creation of lightweight EDSLs (embedded domain specific languages), especially since Haskell allows the creation of (infix) operators as well.

The combination of conciseness, locality of data, strong types and explicit side-effects has a very important consequence: it makes refactoring easy and risk-free. You can change your code, and if it still type checks, you can often be sure that it still works. Combined with the power of abstraction this means we can quickly develop programs that can be easily understood and modified by others.

Finally, another positive aspect of Haskell is its community. There is an active blog community, a large and helpful IRC channel (#haskell) and an online package repository. There is an industrial strength compiler, a dependency tracking package manager and built-in tools like code coverage.

Of course, there are also a few risks involved in choosing a language like Haskell. Since it doesn’t have a user base as large as some other languages, there are less libraries available. This is alleviated in part by a strong C interface that Haskell provides. Because of laziness, performance problems can be harder to diagnose and fix. And of course, there are less programmers who have experience with Haskell, so finding people to help you can be harder.

But all in all, we feel that the benefits Haskell can bring us outweigh the risks. And most of all, Haskell makes programming fun!