Building an assessment app in two days -- 6th commit

After a long pause while I fixed that bug, back to getting the app up and running.

The sixth commit has basic sign-in, sign-up, and sign-out functionality working, which means we have our first proper controller on the server and our first proper service on the client.

Earlier I said I wasn't going put in email/password log in. Well, I then realised I would need to.

Later on, I'm going to need to try out creating a course (as staff) and then preenrolling a different user (as a student). That means I need two accounts, but I only have one GitHub log in.

Handling users on the server

UserController contains the various concise actions that we've defined on the server:

• self to return JSON for the currenlty logged in user
• signUp to sign up
• logIn to log in
• logOut to log out.

At the top of the controller, there is the interesting line

implicit val userToJson = UserToJson

This is used to convert the responses to JSON format.

If you then have a look at UserToJson, you'll see it's a slightly different strategy for converting to JSON than most libraries. The key function is:

def toJsonFor(u:User, a:Approval[User]) = { 
  // etc

This is from recognising that very often we want to give different information depending on who's asking. For instance, in this case we only want to give out the JSON data for the user if they are asking about themselves, not someone else.

And, it returns a Ref[JsValue] rather than just a JsValue, in case it has any other work it needs to do before it can return a result.

Handling users on the client

UserService is the corresponding component on the client. Again, it has self, signUp, logIn, and logOut actions which are all neatly concise.

Angular.js has built into it promises. These are very much like the way Promises and Futures work in Scala on the server.

Which is perhaps why I quite like working with Angular.js in the browser!

Building an assessment app in two days -- bug resolved

Ouch that was a painfully long delay...

The pause in committing code was because I hit an issue between DataAction and Play that took a little while to resolve. So there's a commit on the handy project that's just been pushed to resolve it.

The gory details can wait for another time, but the short version is that Play has a distinction between EssentialAction and Action, and these two Play classes don't work quite as nicely together as I naively assumed. This was causing problems in DataAction.one(parse.json). It was a bit of a fiddly workaround that was needed.

So, back to it, but having taken a few hours out to deal with that, I might be having a late night after all.

Building an assessment app in two days -- 5th commit.

So, it's late on Wedndesday morning and I probably should get a wriggle on if this is going to be done by tomorrow.

The commits I've been making so far are mostly basic plumbing -- wiring together the different components we're going to use. At some point, I'll publish these as a template project (using giter8), so that future projects can start from a pre-plumbed base.

But, someone has to plumb the first one.

Fifth commit: The basics of the Angular.js app

The 5th commit sets up the basics of our Angular.js app.

Here's a little 404 pic for evidence:

It might look a little odd that a 404 Not Found is telling me that I've got that part of the app wired up correctly, but it's because that error page is part of our Angular.js app -- it's not just plain HTML from the server.

Angular.js app

The Angular.js app is set up in two parts

• app/assets/javascripts/modules/base.coffee

  This declares the Assessory module, and is the first thing we call

• app/assets/javascripts/modules/app.coffee

  This configures the app and declares the routes, and will usually be the last thing we call (because it's going to be using a few controllers we haven't created yet)

We use require.js to enforce the order in which all the scripts are run, and also to combine and minify the scripts in production so that the browser doesn't have to make so many requests for javascript files.

The rest of this post is gory details about how we're getting Angular.js to render the error page, and how the header at the top is our first Angular directive.

(It's here because I feel I ought to explain what's going on, but the chances are you'll want to skip it for more interesting posts later.)

That 404 page comes from the client

That friendly little fellow with the map (sorry for my poor drawing) is also being rendered by our Angular.js app.

What happens is this: the / route (the home page) is configured to use a template that I haven't written yet. So, Angular.js gets a 404 error when it requests it.

Our app has configured Angular.js that when it has a routing error (such as this), it should set an error variable in $scope.

# Handles route change errors so that the user doesn't just see a blank page
Assessory.angularApp.controller('ErrorController', ['$scope', ($scope) ->
  $scope.$on("$routeChangeError", (event, current, previous, rejection) ->
    $scope.error = rejection
  )
  $scope.$on("$routeChangeSuccess", () ->
    $scope.error = null
  )
])

$scope is an Angular.js concept -- it contains the data for rendering a fragment of the page. There are usually quite a few scopes, corresponding to different parts that are showing.

The error in this scope (the root scope) causes Angular.js to show the error portion of the main template:

  <div ng-show="error">
    <ng-include src="'client-error-template'"></ng-include>
  </div>

The client error template is pre-embedded in the first HTML we sent the browser, and includes some switches to change what's shown depending on what the error was.

Site header directive

At the top of the image, you can see a site header. That's our first directive. The template is just a fragment of HTML in views.components.directive_siteHeader.scala.html.

If you look in includeDirectives.scala.html you can see how we use Play's templating to insert the template into the HTML that we initially send the browser -- this means that Angular.js never has to make a request to the server to fetch that template, because we've already provided it.

<!-- Site UI components --> 
<script type="text/ng-template" 
        id="directive_siteHeader.html"
>
  @views.html.partials.components.directive_siteHeader()
</script>

(includeDirectives is in turn embedded into the main HTML in main.scala.html.)

The other side of the directive is the Javascript code that declares it to Angular.js

That is in assets.javascripts.components.SiteHeader.coffee and looks like this

define(["modules/base"], (l) -> 

  Assessory.angularApp.directive("siteHeader", () -> 
    {
      restrict: 'E'
      templateUrl: "directive_siteHeader.html"
    }
  )

)

Around the outside, we have a require.js call that makes sure modules/base is loaded first. (That's going to create Assessory.angularapp)

And then we create a directive that tells Angular.js to replace <site-header></site-header> with the content of our site header template.

Building an assessment app in two days -- 4th commit.

The fourth commit has been pushed to GitHub.

Starting to set up the client

This sets up the beginnings of the Angular.js / Play app. Though the app doesn't do anything yet.

A quick run-down on its contents:

• Global.scala

  This contains the application start-up code for Play. At the moment, it has three things to set up:

  • Set up the database, looking in application.conf for connection details (which in turn looks up some environment variables)

  • Set the home action for DataAction

    DataAction is particularly designed for "single page apps" such as this one.

    When the app is running in the client, Angular.js is going to request data in JSON format from the server, and render it appropriately in HTML.

    But at any stage, the user might click "Refresh", in which case the browser will make a request to the server on that URL. And in that case, we don't want to return JSON data, but the HTML and Javascript that will bootstrap our Angular.js app.

    DataAction uses the Accepts HTTP header to tell the difference between a request from our Angular.js app, and a request from a browser asking for HTML.

    If a request is made with an Accepts header looking for a JSON response, then the DataAction will run and return (asynchronously) data in JSON format. But if the request is made with an Accepts header asking for HTML, then it returns the home action (delivering our Angular.js app to the browser to get started).

  • Set the look up method for RefById and RefManyById.

    This configures how items are retrieved from the database. The basic handy library, (where Ref, RefById, LazyId, and RefManyById, amongst others, are defined) is database agnostic, so this is how we wire it up to our ReactiveMongo database layer.

    The DAO classes we have defined automatically inherit partial functions for looking up the classes they handle. (The DAO trait in handy-reactivemongo defines an appropriate partial function). So, the lookup method just has to call the partial functions from the DAO classes to see which one applies.

    (If you're new to Scala, a "partial function" is essentially a function that can say "actually, no I can't handle that argument after all". So, we can keep a set of partial functions for looking up different classes of item by their ID, and the partial functions themselves can decide whether or not they apply to the reference we gave them.)

• Application.scala

  This contains a few key "actions" for our app.

  • index

    This action delivers the HTML for our Angular.js app

  • The default action

    Angular.js includes its own routing (its own handling of URLs) in the browser. In a single page app, the routing on the client side is somewhat decoupled from the routing on the server side.

    This means it's perfectly possible that there will be a valid URL in the client that isn't defined as an explicit action on the server.

    But we still need to handle the request if the user hits "refresh" and causes their browser to make an HTTP GET request to the server for that URL.

    The default action causes those requests to return the index action (containing our Angular.js app).

  • The partials action

    Periodically, Angular.js needs to request the HTML for a new template to show. As we define new views, we will be defining new partial templates.

    If we gave each partial template its own entry in the routes file, then in development, every time we add a new partial template, Play would want to recompile all the controllers as the routes file (containing all the controllers' action URLs) would have changed. If we instead put all the partials into this one action, then adding a new partial template only causes the Application controller to recompile.

• main.js

  We're going to use require.js to combine and minify our Javascript (because we're going to have a lot of small Javascript files by the time we're finished).

  At the moment, there are no Javascript files to load, so we're "requiring" an empty array of files.

  We have set it up, though, so that when all the (zero) libraries have loaded, require should tell Angular.js to bootsrap.

• includeDirectives.scala.html

  This is where we'll embed the HTML for Angular.js template directives. But at the moment, there aren't any.

Building an assessment app in two days -- third commit.

The third commit puts in place the first Data Access Object for the database: UserDAO

This uses ReactiveMongo, which is a non-blocking database driver for MongoDB. That means that when the database is processing a query, the thread is not left waiting, but can get on with other tasks.

Each of the methods returns Ref.

You're probably getting a bit tired of seeing Ref everywhere, but if you glance over to the test, you should see how they can be chained together in a fairly easy to read style. (If you're familiar with Scala.)

For instance, in this snippet, you can read those for statements as sequential actions that happen one after another. They happen asynchronously, but in order. First we save the user, then we push a new session, then we fetch the user again using the session, and then we extract the name from the user we got back.

"push sessions correctly" in {      
  val u = UserDAO.unsaved.copy(name=Some("Cecily Cardew"))
  val returnedName = for (
    saved <- UserDAO.saveNew(u);      
    pushed <- UserDAO.pushSession(
      saved.itself, 
      ActiveSession(key="mysession")
    );
    fetched <- UserDAO.bySessionKey("mysession"); 
    name <- fetched.name
  ) yield name      

  returnedName.toFuture must be_==(
    Some("Cecily Cardew")
  ).await      
}

There are a few design decisions taking place in this commit.

• User contains a sequence of active sessions.

  This is going to let us list your active sessions in the browser, and remotely log yourself out if you've left yourself logged in on another computer.

• There's no save, only saveDetails and a lot of push.. methods.

  When we save a user, we don't want to overwrite the sessions -- in case a concurrent request is updating them while we're processing this save. (Whether or not it's concurrent within the database, there might be two requests from the browser in flight.)

  Accordingly, saveDetails does not update the active sessions or any of the other lists.

• Password hashing is already in there

  Although we're not supporting log-in with passwords, the default classes in handy-appbase-core already create a salt and a hash method, so that it's easy to store passwords salted and encrypted. (This uses encryption that comes standard in the JVM)

Building an assessment app in two days -- second commit.

The second commit is where I need to start thinking about the app a bit more. We're going to establish a few basic data model classes. They'll change as the rest of the code is written, but this commit will make a start on it.

Our situation gives us a slightly unusual need:

• The students are all in groups already
• The software hasn't been written yet, so the students are not in the database
• We can't use their UQ logins (institutional policy reasons)
• But we have all the students' GitHub usernames in a spreadsheet

When students log in, they need to be able to get their groups — which means the groups need to be entered before the students have created their accounts in the database.

That means we're going to need a concept of a "pre-enrol", so that when a student has logged in, they can automatically find both the course and their group, in this case using their GitHub username.

Although I'm writing this initially for our course, I'll try to make the data model reasonably general.

Things you'll notice about the data model

• There are lots of Refs. This is part of my handy library. A ref is a reference to a data object. It might be the object itself; it might be a Future that will return the object when a database fetch has completed. It might be a LazyId if all we have so far is the ID of the object. There are quite a few possibilities for what a Ref can be.

  Ref is a monad, which means that it has flatMap and map methods that I'm going to use extensively, and ensures that at the end of an algorithm I'll still have something that meets Ref's contract.

  It's what I call an "ad-hoc" monad, however, because we haven't predetermined the specific kind of Ref we're going to end up with.

  More information will go up on handy's documentation site (And there's a paper I want to write on this style of app development soon.)

• Each of the data classes extends HasStringId.

  This means a little more than just that it has a string id. Ref.getId looks for an "implicit argument" that can produce a canonical id for an object. (For instance, so that if you use Integer ids, Ref(classOf[Foo], 10:Int) and Ref(classOf[Foo], "10") resolve to the same item.)

  Within the handy library, there's an appropriate object GetsStringId for handling this for objects whose canonical IDs are Strings.

  (In the database layer, however, we're going to be converting to and from MongoDB's BSONObjectID class. I just don't want to expose object IDs in the API.)

So, in this commit we have some incomplete data classes:

• User

  A user in the system. This inherits from one in handy-appbase-core, and includes types for Identity and PasswordLogin (though we won't be using the PasswordLogin for now)

• Identity

  A social login identity, such as a GitHub account. (Or, in time, LTI for logging in directly from Blackboard, but university policy prevents us from doing that yet.)

• Task

  Well, if we're asking them to do a peer critique, maybe we'll have other tasks for students in time.

• Course

  A course (we'll try to open this up for others too)

• Preenrol

  A way of pre-registering students for courses if all you have is a social identity

• Group

  A group, for the group critique

• GroupSet

  Sometimes, in courses, students are in more than one group. For instance, they may have a tutorial group as well as a project group. Or their group may change every so often, but you'd like to preserve the historical groups they've been in in the past so you can look back on their previous work.

  GroupSet will support this

• GPreenrol

  Preenrol for groups

• Question

  The critique is going to need a survey form of some sort, which will be made up of questions. We'll establish the different kinds of question in a later commit.

  (Likewise, there's an Answer trait)

• Critique

  A critique

• CritTask

  The task of doing a critique

Building an assessment app in two days -- first commit.

So it's Tuesday lunchtime, and now that the meetings of yesterday evening and this morning are done, I can get started.

(I've got another meeting in an hour.)

The repository is now up on GitHub at http://github.com/impressory/assessory

The first commit just establishes the basic structure of the project. (The link takes you to github's page showing the committed code.)

We have three projects:

• Outermost, the Play app itself
• asserssory-api, which will contain classes for the data in the app.

• assessory-reactivemongo, which will contain our database layer. The project will use ReactiveMongo, which is a non-blocking driver for MongoDB.

  And when I get to writing the database components, it'll also use handy-reactivemongo that contains some useful common code, and wraps the returned data as a Ref.

There are no tests yet, because there's no code to test. But on my machine at least, the Play project starts.

If you check it out, you might find that sbt isn't wholly content about the "SNAPSHOT" dependencies and keeps trying to re-resolve them every time you run a command, even if it only just did so. This might make your build slower than ideal. This doesn't happen on my machine because I've built those dependencies and run sbt publish-local on them, so that sbt only has to look as far as the ~/.ivy2 directory on my laptop.

I'll resolve that in due course.