API services & MVC

You might have heard of this thing called MVC before, maybe you read about it in 1988 in The Journal of Object Oriented Programming, or you were a fanatic user of Smalltalk-76 in the 1970s? Or maybe you are (or work with) an aspiring software developer, in the business of modern web and mobile applications.

MVC (or Model-View-Controller) is an architectural pattern on which to build software. The basic idea in it is to separate internal data models from the user interface via the controller and view. It is dominating web and mobile development, and although some alternatives exist, almost all relevant server-side software is developed with an MVC-compliant (or an MVC-variant-compliant) framework.

A typical representation of the MVC pattern

You should care because

The thing is, MVC is made for full-stack development, where the software consists of the database, user interface and business logic. But, consider for a second, the use case of most mobile applications. Yes, in most cases the MVC-stack is present, but it is no longer the same as it once was. The client application is the view, the backend is the controller and the database is the model. Now as a self-proclaimed backend developer I’m going to focus on the backend part of said use case.

What MVC actually looks like now

Of course the above diagram still follows the principles of MVC, the problem comes when you expand one of those pieces, namely the backend. In that piece of the MVC you can most certainly find another implementation of the MVC pattern, with abstracted database implementations and endpoints. This would be okay if all parts of the backend are fairly complex, but most of the time it’s made up of a few steps:

1. Get request for some data
2. Find said data in database
3. Convert said data to correct format
4. Return said data

These sorts of cases seem more like a pipeline than a complex multi-layer application to me, and makes me wonder why would you build a server-side API application following the MVC pattern in these cases. If you could significantly reduce the complexity and size of the application with a simple pattern shift, shouldn’t you do it?

Example case

Let’s say you have a very standard setup where you have some form of SQL database, a mobile client of operating system X and then a Scala Play 2.X backend. Now you notice that Play is a framework intended for use with the MVC architecture (as most backend frameworks are at the moment) and can be used either with mobile clients as an API or as a stand-alone web application. Our example client calls the backend with the call:

HTTP GET https://our.mobileclient.com/users

Now what happens in the backend is that first the Play application checks the routes-file to see where that URL is mapped to (the first part of the view-layer in Play):

GET   /users          controllers.HelloClient.users

From there we can see that the actual view is in the controller HelloClient. When building a stand-alone web application, the Controller is equivalent to MVC's controller, but in the case of an API it's more of a hybrid between a view and a controller:

object HelloClient extends Controller {
  def users = Action {
      Ok(HelloService.getAll)
  }
}

And inside the Service we find that it's just routing the call to the model and transforming it to JSON. A level such as Services isn't necessary in a Play application, but it is a place where you can put the business logic in, to keep concerns nicely separated. HelloService could look something like this:

object HelloService {
   def getAll: JSON = {
     UserRepository.getAll.toJson
   }
 }

And now, finally, we get to the model part, which would seem to be a two-part layer, where UserRepository defines the actual storing (in the database), and User is the class that defines the actual model. UserRepository here is missing it's two helper functions "userParser" and "rowToUser" which are used to translate stuff the SQL library gives us into a User-object. But overall they would look something like this:

case class User(id: UUID, name: String)

object UserRepository {
  def getAll: List\[User\] =
    DB.withConnection {
      implicit c =>
        SQL”select * from user_account”
          .as(userParser.*)
          .flatMap(rowToUser)
    }
}

To someone who has some experience with Scala this may be a fairly normal approach, but as I first laid my eyes upon this architecture, it got me thinking that surely this isn’t the best way to do such stuff. Thinking about it a mobile backend API only has to provide the API, and the payload (and of course some logic, but that’s not in all the cases).

What if it looked something like this

What if we had a framework that allowed us to flatten the architecture almost completely? After noticing that most of our calls (or in this case the single call) is just a direct pipeline from database to return, we remove all the copypasta and let the framework handle the repetitive stuff. Working our way from the bottom up, we abstract the database connection altogether (it will be defined in application.conf or a similar file). The model only needs to know what table the data is located in, and the framework handles the rest.

@table(”user_account”)
case class User(id: UUID, name: String) extends JsonModel with DBConnection

object User extends EndPoint {
  @GET("/users")
  def users = Action {
    Ok(User.findAll)
  }
}

Now the companion object is where the endpoint magic happens. The annotation tells what route this function responds to, Ok is the response type and findAll is a function that returns a list of users, which the framework translates to JSON. Of course all of this expects the variables to be named the same way in all levels (JSON, database and model).

Now what if we want to include some more functionality to our program? Say we want to find all users with book loans and return a list of users with a list of their loaned books, for example. This is the case where a model is not the best solution, but extra logic is called for, so just add the correct class and work there.

object BookLoansService extends EndPoint {
  @GET("/users/withloans")
  def usersWithLoans = Action {
    val loans = BookLoan.findAll
    val users = loans.map(User.find(_.userid))
    Ok(users.map(user => user.addAttribute("loans",
        loans.filter(_.userid == user.id))))
  }
}

So the example pattern proposed here would boil down to Models & Services where most calls go directly to the models, taxing the server application as little as possible, and then the calls that require some extra functionality are the ones we work in.

Example architecture pattern for M&S

So I have this library called [insert library name]

Yes, there are libraries (mostly for JavaScript) that implement some or all of the requirements expressed here and you can use those to implement the Models & Services-pattern in your application. I personally don't like combining applications together from multiple little libraries, preferring instead a semi-ready framework, to avoid compatibility problems.

I did spend a few days scouring the internet for frameworks that would implement just about anything other than MVC, but my search only turned up JavaScript libraries. Also, I stumbled on the fact that most popular and used frameworks on the internet tend to be MVC. So if you do find something give me a holler on Twitter.

What I’m talking about here is more about changing the entire mindset and pattern when starting work on an API backend. Yes, you could do a flattish architecture in for example Scala, but you would still need to make the abstractions of databases and routes yourself, or endure extremely bloated models. This would very much undermine the usefulness of the flat architecture, trading maintainability for flatness.

In conclusion

Although this is just a napkin-scribble-level proposition, I find the idea of a lightweight optimized (for API applications) pattern very alluring. And even though memory or CPU isn’t as valuable nowadays I believe this sort of solution would save a lot of grey hairs in revisions and would significantly ease the pain of creating a new application.

Pros of Models & Services:

• Higher abstraction level => comprehensibility
• Flat architecture => maintainablity
• Overall simplicity => models allow for simplistic end-to-end use of the API with minimal implementation

Pros of MVC:

• Lower abstraction level => controllability
• Widely used => ease of handover
• Many implementations => tested and understood

* Of course not all endpoints are as simple as that, but I propose that those are the exceptions, and the needs of the few should not incur in to a new layer in the whole architecture.