My Diversions » CAL and Open Quark

Functional Programming on Android

Tom Davies — Mon, 27 Jul 2009 13:57:08 +0000

I’ve integrated CAL with a variety of Java based frameworks/environments: Tapestry, GWT and Google AppEngine for instance.

Recently someone asked on the CAL Google Group whether CAL programs could run on Android mobile devices. Android doesn’t use a standard JVM, it uses Dalvik, and it provides a subset of the normal JRE library.

I have put together a trivial proof of concept to show that CAL does indeed work on Android — although it doesn’t cover a great deal of CAL’s runtime behaviour, so there might be problems with larger programs which I haven’t detected.

My sample application just accumulates keypresses and displays the resulting string.

The Android Activity

This class delegates calls to onCreate and onKeyUp to CAL functions.

The state of the application (a String in this case) is initialised to the value returned by the CAL onCreate function, and is then passed to and returned from the onKeyUp function.

public class CalActivity extends Activity {
    private ExecutionContext ec;
    private TextView tv;
    private String state = null;

    public CalActivity() {
        System.setProperty("org.openquark.cal.machine.lecc.non_interruptible", "true");
        ec = StandaloneRuntime.makeExecutionContext(CalActivity.class);
    }

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        tv = new TextView(this);
        try {
            state = Test.onCreate(CAL_Opaque.make(this), CAL_Opaque.make(savedInstanceState), ec);
            tv.setText(state.toString());
        } catch (CALExecutorException e) {
            tv.setText(e.getMessage());
        }
        setContentView(tv);
    }

    @Override
    public boolean onKeyUp(int keyCode, KeyEvent event) {
        char c = (char)event.getUnicodeChar();
        // ignore events which aren't characters
        if (c != 0) {
            try {
                state = Test.onKeyUp(CAL_Opaque.make(this), c, state, ec);
            } catch (CALExecutorException e) {
                tv.setText(e.getMessage());
            }
            tv.setText(state.toString());
            return true;
        }
        return super.onKeyUp(keyCode, event);
    }
}

The CAL Module

module Org.Kablambda.Android.Test;

... imports omitted ...

onCreate :: Activity -> Bundle -> String;
public onCreate activity bundle = "Hello from CAL!\n";

onKeyUp :: Activity -> Char -> String -> String;
public onKeyUp activity ch s = s ++ (fromChar ch);

Activity and Bundle are declared as foreign types, but don’t have any functions defined on them at present.

Modifications to the build process

I added some new targets to the standard build.xml file created by the android SDK.

These properties set the location of our CAL installation, the name of the CAL module we want to call from Java, and the name of the class we want that module’s functions to be exposed in.

The copy-quark-jars target copies the jar files needed by CAL at runtime. They are copied into a temporary directory.

The build-quark-standalone target compiles the CAL workspace workspace.cws into a jar file. This workspace must contain the module specified above and its dependencies. This target assumes that ${quarkdir}/Quark is on the PATH.

Because several of the CAL jars contain duplicate files, which Android doesn’t tolerate, we merge all our jars into a single file, ignoring duplicates.

To deploy your application to the Android Emulator you need to run ant merge-jars reinstall.

Note that during the process of converting the cal.jar file to Dalvik bytecode you’ll see a large number of error messages like this:

[apply] ...while processing com/ibm/icu/util/TimeZoneData.class
[apply] warning: Ignoring InnerClasses attribute for an anonymous inner class that doesn't come with an associated EnclosingMethod attribute. (This class was probably produced by a broken compiler.)

This is something to do with the rather old version of IBM’s International Components for Unicode library — it would be interesting to recompile CAL with a more recent version — or just recompiling that version of icu4j with the Sun compiler.

Further Work

The Android API needs to be exposed to CAL, and a Monad written to sequence Android API operations which have side effects. The CalActivity implementation should store the state as an opaque CalValue, so that implementations can choose what type to use for their state, and of course it needs to delegate all the possible events to CAl functions. There’s plenty of scope for more scaffolding on the CAL side.

As I don’t have an Android phone I doubt that I’ll do any further work on this — unless Dalvik gets an iPhone port

Reconsidering my Combinators

Tom Davies — Mon, 25 May 2009 12:00:14 +0000

Since writing my set of function combinators for matching HTTP requests I’ve realised that there are better approaches.

While the combinators read nicely, once you start extracting attributes from the request things become messy, due to the hAnd combinator composing results as nested pairs.

Here’s an example:

skitchPutHandler =
    let
        storeImage p =
            let (p2, path) = p;
                (maybeContentType,imageData) = p2;
                ...
            in ...
    in
        matchMethod "PUT" `hThen` matchUrl "/skitch" `hThen`
            (getHeader "Content-Type") `hAnd` requireRawData `hAnd` requirePathInfo `hApply` storeImage;

Unpacking the values extracted from the request is cumbersome, and because many are Strings, error prone.

A better approach is to simply use the Maybe monad with the same extraction functions:

simpleSkitchPutHandler req =
    let
        maybeContentType = requireHeader "Content-Type" req;
        ...
    in
        matchMethod "PUT" req `anonymousBind` matchUrl "/skitch" req `anonymousBind`
            requireRawData req `bind` (\rawData ->
            requirePathInfo req `bind` (\path ->
            return $ storeImage path rawData (actualContentType path)));

Optional values are extracted in let expressions, and required values and tests are composed with the monadic bind operators in the main expression. You can make the function a bit more concise and ‘point free’ by writing a monad which composes the Maybe and Reader monads to supply the HTTP request parameter to functions implicitly:

readerMonadSkitchPutHandler =
    let
        ...
    in
        runHttpRequestM (
        method "PUT" `anonymousBind` matchUrlM "/skitch" `anonymousBind`
            rawData `bind` (\rawData ->
            pathInfo `bind` (\path ->
            (optional (header "Content-Type")) `bind` (\maybeContentType ->
            return $ storeImage path rawData (actualContentType path maybeContentType)))));

The function optional has the type HttpRequestM a -> HttpRequestM (Maybe a), that is, it allows a function to succeed by wrapping its result in a ‘Just’.

I’m not convinced that the improvement in readability gained by removing the request parameter is worth the increase in complexity.

So I’ll remove the combinators from my Http module and will rewrite Hangman using just the Maybe monad. I’ll keep the more complicated Monad on the back-burner in case I discover a compelling advantage.

I’d be interested to know what someone with a deeper understanding of functional programming would say.

CAL Hangman on GAE Part 2 — The Datastore

Tom Davies — Tue, 12 May 2009 12:48:48 +0000

Introduction

My implementation of Hangman has only very simple data storage requirements, so my CAL module Datastore doesn’t cover the entire capabilities of Bigtable.

All the application needs is to be able to save a Game instance, put the key of that instance into a cookie in the user’s browser, and then retrieve and update that instance as the user makes each guess, or delete it if the user chooses to restart the game.

Important unexplored areas are:

The creation of entities having another entity as a parent. This is important in Bigtable, because a transaction can only operate on entities in the same ‘entity group’ — that is, entities which share a common parent.
The persistence of references to other entities, and special treatment of these references when performing operations on the entity containing the reference — that is, there is no support for modelling relationships between entities.
No error handling support — any exception will result in a 500 response. In particular, ConcurrentModificationException should be treated differently, as this indicates that a transaction failed due to another transaction modifying the same records. A failure of this type should either be retried, or reported to the user in an application specific manner.
The module is limited to storing types with a single constructor, although there is no difficulty in extending this to support multiple constructor ADTs.

It is implemented using the low level api, not JDO.

Datastore Module

The Datastore module provides a Monad instance for using the AppEngine data store, and a Storeable type class. The source is here.

The Storeable Type Class

Algebraic data types which need to be persisted must be instances of Storeable, which provides metadata needed to persist instances of the type. The metadata includes:

The ‘kind’ of the record — analogous to the name of the table to store instances of the type in, not to kind in the type theory sense.
The data store property names to use for each of the constructor arguments.

When using CAL in statically compiled mode the names of types and the names of constructor parameters are not easily available at runtime. The constructor arguments could simply be stored in properties named arg0...n, but this would make querying and debugging more difficult.

data public Store a = public Store kind :: String fields :: [String];

public class (Outputable a, Inputable a) => Storeable a where
    store :: a -> Store a;
    ;

Note that the type parameter a is not used in the declaration of the constructor arguments — it simply labels the store to ensure that the store used with, for example fromEntity below, is consistent with the type of Storeable we are expecting fromEntity to return. For this reason functions which construct a Store must be explicitly typed, like gameStore below, as otherwise no more specific type than Store a can be inferred.

So the instance for a game of hangman is:

data public Game = 
    private Game word :: String guesses :: String deriving Show, Inputable, Outputable;

gameStore :: Store Game;
public gameStore = Store "Org.Kablambda.AppEngine.Test.Game" ["word","guesses"];

instance Storeable Game where
    store = gameStoreA;
    ;

private gameStoreA a = gameStore;

The public gameStore function is used when we wish to query.

Converting between CAL values and AppEngine Entities

Two conversion functions are used, one in each direction. In both cases the bulk of the work is carried out in Java functions.

From a CAL value to an Entity

toEntity :: Storeable a => a -> Entity;
public toEntity !r =
    jObjectToEntity (store r).Store.kind (outputList $ (store r).Store.fields) (output r);

where:

foreign unsafe import jvm "static method org...jObjectToEntity" 
    public jObjectToEntity :: String -> JList -> JObject -> Entity;

The parameters passed to jObjectToEntity are the kind of entity to create, the list of names to use for the fields of the CAL value, and the CAL value, converted to a JObject by the output function. Storeable instances should all derive Outputable, so output r uses the default implementation of output to convert the algebraic data type instance to an instance of the CAL Java support class AlgebraicValue. The fields are transferred to the AppEngine entity like this:

Entity entity = new Entity(kind);
AlgebraicValue av = ...;
entity.setProperty(DC_NAME, av.getDataConstructorName());
entity.setProperty(DC_ORDINAL, av.getDataConstructorOrdinal());
for (String fieldName : fieldNames) {
    entity.setProperty(fieldName, av.getNthArgument(i++));
}

From an Entity to a CAL value

fromEntity :: Storeable a => Store a -> Entity -> a;
public fromEntity !s !e =
    input $ jEntityToJObject (outputList s.Store.fields) e;

where:

foreign unsafe import jvm "static method org...entityToJObject" 
   public jEntityToJObject :: JList -> Entity -> JObject;

jEntityToJObject reverses the process, creating an AlgebraicValue which is converted to a CAL value by the input function.

The Data Store Monad

Because data store operations operate via side effects we need to create a Monad instance to manage them. This is required or two reasons:

We need to ensure that our data store operations happen in a definite sequence — our puts must happen before we commit the transaction, for example. Lazy evaluation won’t guarantee this without special attention.
Some operation don’t produce results, or produce results which will be ignored, so lazy evaluation won’t cause these operations to happen at all.

We define the following functions for use with our Monad (the type of which is DSM):

put :: Storeable a => a -> DSM Key;

The put function creates a new record, returning the Key it was assigned by the data store.

update :: Storeable a => Entity -> a -> DSM Key;

The update function replaces a previously retrieved entity with a new value, returning the Key (which will always be the same as the old key). Note that the original Entity is used only for the value of its key — all the replacement fields come from the new Storeable value.

delete :: Entity -> DSM ();

The delete function deletes a previously retrieved entity.

find :: (Storeable a) => Store a -> Key -> DSM (Maybe (Entity,a));

The find function looks for a record using a Key and returns it or Nothing if it does not exist. The return value consists of a pair of values: the raw Entity and its value when converted to a Storeable instance. This allows update to be used later. An alternative design would have been to require Storeable instances to be able to store a key, but I think the current design better separates the concerns of data storage from the domain objects.

query :: (Storeable a, Outputable b) => Store a -> [(String,FilterOperator,b)] -> [(String,SortDirection)] -> DSM [(Entity,a)];

The query function returns a sorted list of (Entity,Storeable) pairs, based on a list of filter criteria applied to attributes of the records.

commit :: DSM ();

The runDSM function described below starts and commits the transaction within which the monad is being run, but if you need to operate on entities from more than one entity group you must commit the original transaction and start a new one using the commit function.

These functions are combined with the normal monad operators of bind and anonymousBind, and then the resulting function is ‘run’ with runDSM:

runDSM :: DSM a -> a;

Key Conversion

In the context of a web application we may wish to store keys in cookies or generated URLs. The following functions are provided for extracting Keys from Entity instances, and converting Keys to and from strings.

public keyToString :: Key -> String; 
public stringToKey :: String -> Key;
public getKey :: Entity -> Key;

Example

Suppose we want to retrieve a Game, add a letter to the set of guesses and store it again:

    updateGame :: Key -> Char -> Key;
    updateGame key letter =
        runDSM (find gameStore key `bind` (\p ->
            let (entity, game) = fromJust p;
            in update entity (addGuess letter game)
        ));

where addGuess returns a new game with the given letter added to its set of guesses. This function will terminate with an error if the Game is not found, that is, if find returns Nothing.

CAL Hangman on Google AppEngine, Part 1

Tom Davies — Mon, 27 Apr 2009 08:55:46 +0000

This post describes a simple implementation of the game ‘Hangman’, written in CAL for Google Appengine.

Introduction

Hangman is a simple word guessing game — the computer chooses a word, telling you only how many letters it has. You that it contains a particular letter. Correct guesses fill in those letters in the word, until you have guessed all the letters in the word, or you make too many incorrect guesses and lose.

You can try my Hangman for AppEngine implementation to understand the game. The source is available at launchpad.

Hangman is a very simple game — the state is just the word being guessed, and the letters guessed so far. The point of implementing it in CAL on AppEngine is to develop CAL modules for the AppEngine API classes.

Design

It would be easy to implement hangman as a mostly client-side browser based application, but that wouldn’t have exercised much of the AppEngine API.

To make the task interesting, I decided that the game state would be persisted on the server, and a user’s current game would be identified by a cookie in their browser. So you can make some guesses, close your browser, and come back to the same game later.

Http Module

The Http module provides function for creating a server which handles requests and creates responses. The source is here.

The Server Function

All requests go through a single servlet, which is configured with init-params to reflectively call a static method generated by the CAL stand-alone jar builder.

The function called by the servlet must have the type HttpServletRequest -> JHttpServletResponse. These CAL types are simply imports of the Java HttpServletRequest and HttpServletResponse classes.

data foreign unsafe import jvm "javax.servlet.http.HttpServletRequest"
    public HttpServletRequest deriving Inputable, Outputable;

data foreign unsafe import jvm "javax.servlet.http.HttpServletResponse"
    private JHttpServletResponse deriving Inputable, Outputable;

JHttpServletResponse is private to the module because it is mutable and so shouldn’t be exposed to clients of the module. Instead Http provides a data type HttpServletResponse:

data public HttpServletResponse =
    private HttpServletResponse cookies :: [Cookie] headers :: [Header] body :: HttpServletResponseBody;

data public HttpServletResponseBody =
    private Ok contentType :: String content :: (Array Byte) |
    private Error code :: Int message :: (Maybe String) |
    private Redirect url :: String;

This allows the client to assemble the state of a response, which is then applied to the response in a single operation by the Http module. A monad to sequence operations on the Java response object would have been another alternative, but that wouldn’t enforce correct construction of the response to the same degree — e.g. the client might not set a content type. Note that the actual constructors are private so that the structure of the type is not exposed to clients of this module.

The current data type doesn’t allow the response to be streamed.

As clients can’t see or use JHttpServletResponse they need to use the server function to create a web server:

server :: [HttpServletRequest -> Maybe HttpServletResponse] -> HttpServletRequest -> JHttpServletResponse -> ();

This function takes a list of handlers, each of which is tried in sequence until one returns a response. So the server function for the hangman web application is:

public service =
    server [currentGameHandler, newGameHandler, guessHandler, restartHandler];

Writing Handlers

The Http module provides functions to test and retrieve properties of the request, and combinators to build handlers from these individual functions. The combinators are inspired by Mark Tullsen’s First Class Patterns.

Each combinator combines two functions, each of which extracts a property of the request, and returns a function of the type HttpServletRequest -> Maybe a.

It isn’t clear to me that this is the very best approach — experimenting with a Maybe/Reader monad stack would be interesting.

Combinators

hOr :: (HttpServletRequest -> Maybe a) -> (HttpServletRequest -> Maybe a) -> (HttpServletRequest -> Maybe a);

hOr requires that at least one of two extraction functions succeeds, i.e. returns Just x. The final result is the result of the first extraction function to succeed (if the first extraction function succeeds, the second is not evaluated).

hThen :: (HttpServletRequest -> Maybe a) -> (HttpServletRequest -> Maybe b) -> (HttpServletRequest -> Maybe b);

hThen requires that both the extraction functions succeed, but it discards the result of the first extraction function.

hAnd :: (HttpServletRequest -> Maybe a) -> (HttpServletRequest -> Maybe b) -> (HttpServletRequest -> Maybe (a,b));

hAnd requires that both extraction functions succeed, and returns their results as a pair.

hNot :: (HttpServletRequest -> Maybe a) -> (HttpServletRequest -> Maybe ());

hNot reverses the sense of an extractor, succeeding if it fails and vice-versa.

The following combinators have specialised functions, rather than acting to combine two extraction functions.

hCompose :: (HttpServletRequest -> Maybe a) -> (a -> b) -> (HttpServletRequest -> Maybe b);

hCompose transforms the result of an extraction function.

hApply :: (HttpServletRequest -> Maybe a) -> (a -> HttpServletResponse) -> (HttpServletRequest -> Maybe HttpServletResponse);

hApply combines a sequence of extraction functions with a function that produces a response.

hAttempt :: (HttpServletRequest -> Maybe a) -> (a -> Maybe HttpServletResponse) -> (HttpServletRequest -> Maybe HttpServletResponse);

hAttempt is like hApply, but the response generation function returns Maybe HttpServletResponse, allowing further conditional processing outside the context of combined extraction functions.

Request Attribute Extractors

These functions look at attributes of the response and return their values (which may amount to just signalling their presence or absence).

getParameter :: String -> HttpServletRequest -> Maybe (Maybe String);

getParameter extracts the value of a named parameter from the request. It always succeeds, returning Just Nothing or Just $ Just paramValue.

requireParameter :: String -> HttpServletRequest -> Maybe String;

requireParameter extracts the value of a named parameter, but fails (i.e. returns Nothing) if it is not set.

There are similar get/require Header/Cookie functions.

matchUrl :: String -> HttpServletRequest -> Maybe ();

matchUrl takes a string containing a regular expression, and succeeds if the regular expression matches the path info of the request. It would be useful to have another function which returns the values of capturing groups of the regular expression.

Creating the Response

Utility functions are provided for each common response type.

ok :: HasByteRep a => String -> a -> HttpServletResponse;
public ok contentType content = ...

textHtml :: HasByteRep a => a -> HttpServletResponse;
public textHtml = ok "text/html";

redirect :: String -> HttpServletResponse;
public redirect url = ...

err :: Int -> Maybe String -> HttpServletResponse;
public err code msg = ...

These functions all create requests with default headers. Headers and Cookies are added by applying a function to the request.

addHeader :: Header -> HttpServletResponse -> HttpServletResponse;
addCookie :: Cookie -> HttpServletResponse -> HttpServletResponse;

Example

Rather than an example from the hangman game, here’s a simple example which uses the functions discussed directly.

helloHandler =
    let
        page p = 
            let (firstName, maybeLastName) = p;
            in ok "text/plain" (
                case maybeLastName of
                    Just lastName -> "Greetings, " ++ firstName ++ " " ++ lastName;
                    Nothing -> "Hi there " ++ firstName;
                );
    in
        matchUrl "/hello" `hThen`
        (requireParameter "firstName") `hAnd`
        (getParameter "lastName") `hApply` page;

This handler will return a page for the url /hangman/hello if the firstName parameter is present. To keep the example as simple as possible it uses the text/plain content type to avoid the verbiage of an HTML page.

To be continued…

In the next instalment I’ll describe the CAL module which supports the AppEngine datastore.

Functional Programming on Google AppEngine

Tom Davies — Thu, 09 Apr 2009 08:17:33 +0000

I’ve always hoped that Java would be the next Google App Engine language. Google have provided a solid platform with a ‘real’ JVM and few restrictions.

I deployed the sample application created with the Eclipse plugins a few minutes after my account was upgraded to support Java, and then I went on to deploy an application written in CAL.

There are two ways of integrating CAL and Java. The first is to dynamically load and compile a workspace at run time, and then reflectively call functions in the workspace’s modules. I’ve used this approach in the past, for example for CAL/GWT integration.

There are two reasons not to use this approach for an AppEngine application. Firstly, when CAL loads a workspace it uses a SecurityManager, which GAE doesn’t allow. It’s just used to get the class of the caller of a function so the code could probably be replaced by looking at the stack trace of an un-thrown Exception, but that would involve a custom CAL build. I suspect that this approach would also want to create some local files, which also wouldn’t work on AppEngine.

The other reason not to use this approach is that AppEngine runs your application on (potentially) many different JVM instances, and may stop your application when it is idle and restart it when it receives a request. Nothing I’ve read so far indicates how often this happens, but start up overhead which would be perfectly acceptable in a normal application server is likely to be a bad idea in AppEngine.

CAL modules can also be compiled statically to a ’stand-alone’ jar file.

Each public function in the module becomes a static function on the resulting class, e.g.:

square :: Int -> Int;
public square n = n * n;

becomes:

public static int square(int n, ExecutionContext context)

If we hadn’t explicitly typed square, it would have the inferred type Num a => a -> a and the function would have the signature

public static CalValue square(CalValue n, ExecutionContext context);

Where a CalValue can hold any Java types which CAL knows how to convert to CAL types via the input function, and ExecutionContext really just caches Constant Applicative Forms.

So the components of a minimal CAL application for AppEngine are:

A module which imports HttpServletRequest and HttpServletResponse and provides suitable functions to get their properties and to set attributes of the response.
A function with the type HttpServletRequest -> HttpServletResponse -> ().
A servlet which calls that function.
Extensions to the example build.xml provided by Google which package up the CAL modules into a stand-alone jar, and include the runtime support jars CAL needs.

Additionally all the other AppEngine Services (Data Store, Cache, Mail etc.) need to be made available to CAL (in Monads of course ), but that can be done piecemeal as you need them.

I’ll just give one example from the Http module, as it illustrates the issue of dealing with null being returned by Java methods.

First I import the HttpServletRequest.getParameter method:

foreign unsafe import jvm "method getParameter"
    jGetParameter :: HttpServletRequest -> String -> String;

But it isn’t made public. The ‘real’ getParameter function is defined like this:

public getParameter req s =
    let v = jGetParameter req s;
    in if isNullString v then Nothing else Just v;

Allowing the Maybe type to handle the possibility that a parameter may not be set.

The build.xml changes are:

Now to do something useful with it!

Take Two

Tom Davies — Wed, 09 Jul 2008 03:32:52 +0000

I’m much happier with the second version:

out3 a =
    let
        renderGroup g =
            let 
                char = head g;
                count = length g;
                renderChars :: Int -> String;
                renderChars n =
                    (fromChar char) ++
                    (if n >1 then  " " ++ (renderChars $ n-1) else "");
            in
                if count > 2 then
                    (renderChars 2) ++ 
                    " " ++ (renderChars (count - 2)) ++ ""
                else
                    renderChars count;
        concatWithSep list = (head list) ++ concatMap (\s -> " " ++ s) (tail list);
    in
        concatWithSep $ map renderGroup (group a);

This only uses one ‘exotic’ list function, group:

group :: Eq a => [a] -> [[a]]
    Splits the specified list into a list of lists of equal, adjacent elements.

My program is longer than Matt’s — roughly twice as long — but I think this version is easier to understand.

Matt’s Javascript for loop has some complicated state, contained in some loop variables which are modified in the body of the loop as well as in the for statement itself. I think that’s a recipe for confusion.

Dustin’s Programming Problem Take One

Tom Davies — Tue, 08 Jul 2008 22:07:14 +0000

My colleague Matt Ryall wrote about this simple algorithm for marking up a series of letters — which is complex enough to be interesting.

I wrote a version in CAL:

arr = ['a', 'b', 'c', 'c', 'd', 'e', 'e', 'e', 'e', 'e',
          'f', 'e', 'f', 'e', 'f', 'a', 'a', 'a', 'f', 'f', 'f'];

data State = State string :: String current :: (Maybe Char) count :: Int;

out =
    let
        initial = State "" Nothing 0;
        f :: State -> Char -> State;
        f state char =
            let
                State s current count = state;
                charStr = fromChar char;
                appendSameChar = 
                    if count == 2 then
                        s ++ " " ++ charStr
                    else
                        s ++ " " ++ charStr;
                appendDiffChar =
                    if count > 2 then
                        s ++ " " ++ charStr
                    else
                        s ++ " " ++ charStr;
            in
                case current of
                    Nothing -> State (fromChar char) (Just char) (1 :: Int);
                    Just c ->
                        if c == char then
                            State appendSameChar (Just char) (count+1)
                        else
                            State appendDiffChar (Just char) 1;
            ;
        finalState = foldLeftStrict f initial arr;
    in
        finalState.State.string ++(if finalState.State.count > 2 then
            ""
        else
            "");

Positive proof that you can write verbose, confusing code in any language — but at least this code had no bugs — once it compiled it worked correctly first time.

As a follow up to this I will try to do better!

A CAL webapp with persistent data using GWT, STM and BDB

Tom Davies — Thu, 27 Mar 2008 12:24:26 +0000

aka, attack of the TLAs.

This webapp’s architecture is depicted below:

The browser runs a Javascript thick client compiled from Java by GWT. Some of the classes have CAL annotations. These don’t affect the client, but allow the server side of the GWT RPC protocol to call CAL functions.
The CAL functions store persistent data using Software Transactional Memory. This is a simple Java implementation based on the description in Composable memory transactions. It is given a CAL interface similar to a subset of Haskell’s Control.Concurrent.STM.
Each STM TVar stores its value as a key-value pair in a Berkeley DB Java Edition database.

Any data structure can be built on top of TVars — and each TVar is a mutable reference, these are not functional data structures.

In this application a simple hashmap is used. Skiplists and relaxed balance btrees are other data structures which might allow reasonable concurrency too, while also providing features like in-order traversal.

This illustrates the relationship between the CAL objects in the CAL ExecutionContext and key-value pairs in the BDB:

The root of the persistent data structure is a TVar with a ‘well-known’ id — 1 in the example, which is created by a constant applicative form function. This TVar will retrieve its value from the BDB when it is created, or if no value exists for its id, it will be initialised with a default value, which in the case of a hashmap is an array of TVars, each containing an empty list of key-value pairs. A value stored in a TVar is persisted to the BDB by serialising it using CAL’s default output function and Xstream, which can serialize and deserialize instances which are not serializable and do not have accessible constructors.

TVars themselves have transient values, so only the id is persisted — the value is lazily loaded when required, using the id. So even though a TVar may persist a complicated tree of CAL algebraic values, this stops at the first TVar. (The root TVar is never persisted itself — only its value is stored).

You can get a snapshot here — just unpack it and run ant run, then point your browser at http://localhost:8080/caltest.html. Or browse the source.

The ant build script includes a target run-tests which runs some Selenium tests. Stop the server before running that target.

Note that the source code includes various bits of half-baked rubbish, in addition to that described above!

GWT as a CAL client

Tom Davies — Sat, 15 Mar 2008 09:12:04 +0000

I’ve been interested in GWT as a way of building rich Internet applications since it appeared, and I’m very pleased to see it getting better and better.

So it’s natural that I’d want to try using it with CAL, a functional language quite similar to Haskell which runs on the JVM.

I used a similar approach to marshalling Javabeans to CAL algebraic types as I used before, but this time I haven’t used any bytecode manipulation — as the Java classes are needed at compile time for the GWT client there isn’t any point in generating them at runtime (although generating them as a separate build step might be useful). I’ve also extended the previous work to include mapping a Java 5 enum to a CAL algebraic type which has constructors with zero parameters.

So in our GWT client we can write:

CaltestServiceAsync service = GWT.create(CaltestService.class);
((ServiceDefTarget) service).setServiceEntryPoint(
    GWT.getModuleBaseURL() + "CaltestService");
service.processPerson(
    new Person(Salutation.MR, "Jim", "Earl", "Jones"), new MyAsyncCallback(...));

and call the CAL function:

public processPerson p =
    let Person s f m l = p; 
    in Person s (toUpperCase f) (lift toUpperCase m) (toUpperCase l);

where the types are:

data Salutation = MR | MRS deriving Inputable, Outputable;
data Person =
    Person salutation :: Salutation 
              firstName :: String 
              middleName :: (Maybe String) 
              lastName :: String 
    deriving Inputable, Outputable;

All this is done via three different annotations, and a special subclass of the GWT RemoteServiceServlet.

The first annotation, @Cal is applied to the GWT service interface, and indicates the CAL module to map the functions on the interface to:

@Cal(workspace = "myworkspace.cws", module = "TDavies.GwtTest")
public interface CaltestService extends RemoteService {
    @Cal
    Person processPerson(Person p);
...

The CAL types Person and Salutation need to be mapped to Java classes:

Person is a simple Javabean with getters and setters for each attribute:

@CalBean(workspace = "myworkspace.cws", module = "TDavies.GwtTest",
    constructorName = "Person")
public class Person implements IsSerializable {
    private Salutation salutation;
    private String firstName;
    private String lastName;
    private String middleName;
...

Note that middleName has the type Maybe String in the CAL type. A value of null maps to Nothing while a value of "x" maps to Just "x".

Salutation is an enum:

@CalEnum(workspace = "myworkspace.cws", module = "TDavies.GwtTest",
    type = "Salutation")
public enum Salutation {
    MR, MRS
}

The names of the enum’s values must be identical to the names of the CAL constructors.

A subclass of RemoteServiceServlet checks for the annotations and transforms the values in both directions.

The source code for this experiment is available via anonymous svn from http://tgdavies.beanstalkapp.com/eddy/browse/trunk/cal. Please note that this repository contains various other half-baked and half-finished experiments! Look at the build.xml file to see how to set up an environment — you’ll need to supply OpenQuark, GWT and Jetty.

In my next post I’ll describe how to persist information on the server.

CAL and Tapestry 5, Part 2: Algebraic Types and Forms

Tom Davies — Wed, 03 Oct 2007 11:29:17 +0000

In my previous post I described how to use a CAL function as part of the implementation of a Java class.

This post looks at interfacing CAL to Tapestry 5 using the ‘Java Bean’ conventions of getter and setter methods for the fields in an object.

Tapestry 5 provides a BeanEditForm component which simplifies providing CRUD operations for Beans. This is described in the second part of the Tapestry 5 tutorial.

By creating a Java class which provides a Bean with fields equivalent to the constructor parameters of a CAL algebraic data type we can use CAL to provide the data model for a web UI created with Tapestry. A Tapestry application doesn’t need to have a compile-time dependency on the fields in a Bean which will be edited in a BeanEditForm. This means that we can simply declare an abstract class, associate it with a CAL data constructor and have immediate web-based CRUD for that type.

On the Java side, we make the following changes to the code described in the tutorial.

The Address class becomes:

@CalBean(workspace="myworkspace.cws", module="TDavies.Address", constructorName = "Address")
public abstract class Address implements CalBeanBase {
}

The type referred to by the annotation looks like this:

data Address = Address
    honorific :: String
    firstName :: String
    lastName :: String
    street1 :: String
    street2 :: (Maybe String)
    city :: String
    state :: String
    zip :: (Maybe Int)
    email :: String
    phone :: (Maybe String)
    fax :: (Maybe String)
deriving Inputable, Outputable, Show;

The type must be an instance of Inputable and Outputable.

The CalBeanBase interface provides the information about the Bean needed for copying between an instance of Address (or any other Bean constructed from a CAL type) and an AlgebraicValue:

/**
 * This interface describes the structure of a Bean derived from a CAL type.
 * The information allows values to be transferred between bean instances and
 * CAL AlgebraicValue instances, using the BeanConverter class.
 */
public interface CalBeanBase {
    public static class FieldSpec {
        public String name;
        public BeanValueConverter converter;

        public FieldSpec(String name, BeanValueConverter converter) {
            this.name = name;
            this.converter = converter;
        }
    }
    public List getFieldSpecs();
    String getDataConstructorName();
    int getDataConstructorOrdinal();
}

BeanValueConverter implementations copy between a CAL value and a field. At present the only non-trivial implementation is the MAYBE converter, which maps Nothing to null.

The BeanConverter class has these methods:

/**
 * Copy values between CalBeanBase subclasses and CAL AlgebraicValue instances.
 */
public class BeanConverter {
    /**
     * Create a new AlgebraicValue which has been constructed using
     * the field values from bean.
     */
    public static AlgebraicValue toCal(CalBeanBase bean) {
        ...
    }

    /**
     * Set the field values in bean from the constructor parameters
     * in v.
     */
    public static void fromCal(CalBeanBase bean, AlgebraicValue v) {
        ...
    }
}

At runtime Javassist implements the CalBeanBase interface and adds the fields, getters and setters needed to make a bean. It does this by looking up the CAL type given in the annotation and introspecting it.

In addition, setters for fields which are not of type Maybe a are given a @Validate("required") annotation. @Order annotations are used to keep the fields in the same order as they appear in the CAL type. (The tutorial is out of date in this respect, and doesn’t mention the @Order annotation).

To exercise this technique we can make the CreateAddress class less trivial:

       // make _address persistent, so we can see the results of processing it on submit
  @Persist
  private Address _address;
       ... getter and setter unchanged ...
       // CAL function -- the body will be replaced with a call to TDavies.Address.sendAddress
  @Cal
  public AlgebraicValue sendAddress(Object address) {return null;}

  public void onSubmit()
  {
      BeanConverter.fromCal(_address, sendAddress(BeanConverter.toCal(_address)));
  }

When we submit the form, the bean’s field values are converted to an AlgebraicValue and passed to the CAL function sendAddress. The value returned by the function is used to set the fields of the bean, which is then displayed.

The sendAddress function just converts some of the data to upper case:

sendAddress :: Address -> Address;
public sendAddress o =
    let
        address = o;
    in
        Address
            address.Address.honorific
            (toUpperCase address.Address.firstName)
            (toUpperCase address.Address.lastName)
            (toUpperCase address.Address.street1)
            (case address.Address.street2 of Nothing -> Nothing; Just s -> Just $ toUpperCase s;)
            (toUpperCase address.Address.city)
            (toUpperCase address.Address.state)
             address.Address.zip
             address.Address.email
             address.Address.phone
             address.Address.fax;

There are a number of enhancements which could be made to this scheme:

Unlike the example, Java Enums are not supported as field values.
Only required validation can be specified, not regex or other types.
There is a problem allowing optional field values for non-string values — this may be a Tapestry limitation.
The BeanConverter methods should be on the CalBeanBase interface. Not only would the onSubmit implementation above be simpler, but the existing methods on the CalBeanBase interface could be removed, and implemented as private methods called by the toCal/fromCal implementations.

To really tell whether these techniques are worthwhile, I need to write a non-trivial web app using them!