Adventures in PWA Land Part II: The Back-End

This is the second and final part in a 2-post series about how I built a Progressive Web App. You can find part 1 here.

During my adventure with PWAs, I reached the point where the front-end was ready. What I needed now was to connect everything to a back-end server, so I started to build one. Instead of working with a technology I already knew, I decided to push a bit further and learn a new programming language. The candidate was obvious. Golang is very popular nowadays.

If you are curious to see the final product, head over to https://dude-expenses.com. It is nothing more than a simple expense tracker. The front-end is a Progressive Web App built with Babel/Webpack/React/Redux. The back-end is a server application written in Go. Feel free to create an account. I am using it all the time, but hey, there are no terms of use, privacy policy or guarantees.

The whole application is open-source and currently lives in Github. Feel free to browse the front-end and back-end code. The app is deployed on a Digital Ocean droplet and Nginx proxies requests in front of the API (back-end) and the app (front-end).

Let us take a closer look at the back-end.

Getting started

Before starting the project, my experience with Golang was limited to having read small blog posts about creating a web app in Golang. Unfortunately, most resources out there seem to limit themselves to a basic "Hello World" server. Most blog posts and tutorials can be considered duplicates.

I started investigating more and decided to try an online course as well. I watched the Go: Getting Started course on Pluralsigh. It was a nice introduction, although a little slow paced. I began searching for experienced Go programmers and went through all their blog posts, something I still continue to do. The ones that stood out were Matt Ryer, Dave Cheney, Matt Silverlock, Brad Fitzpatrick, and the Golang mailing list.

I also read the official Effective Go document, but, I must admit, I did not quite grasp much at the time.

Writing the code

Not long after, I decided to put what little Go knowledge I had to use. I already knew what I wanted to build. A JSON-only API with proper headers. Every response should be JSON and follow a certain structure. The API should support authentication via JWTs. All requests should be logged in a machine readable format, one line per request. All these are pretty straightforward and kind of basic, if you have been building APIs for some time.

However, I have to admit, working with Go took me waaay too long to get started. I cannot remember any other language that has had this effect. With Ruby, you get Rails. In Node.js, I would probably go with Express.js, when starting out. Java has the SpringMVC framework. I do not know anybody straying from ASP.Net, and Python guys will use either Django, Flask, or Pyramid. Go has some web app frameworks, but the general idea is that the standard library has all that you need to get started.
Another huge struggle for me was setting up my Go environment. Installing Golang is a breeze, but, if you want to actually start developing, you have to set up your GOPATH. If you follow the officially suggested method everything is covered in roses. If you have a different opinion on how your workspace should be organized, you are in trouble. Guess what—I had a different opinion.

I am used to working in a lot of different projects, written in various languages. Back when I was doing Java, I hated that I had to organize my workspace according to my IDE's preferred way of working. I decided to set a different GOPATH per project. I had a lot of trouble deciding on project structure to make this work at first, but, after a while, it worked.

Here is what I am currently doing

- bin
- src
  - github.com
    - gkats
      - dude-expenses-backend
        - app
        - auth
        - db
        - expenses
        - log
        - passwords
        - users
        main.go
        Makefile
- pkg
.gopath
README.md

Another problem for me was debugging. From what I understood, the officially suggested method to debug Go programs used to be GDB. However, it no longer supports recent versions of Go well. There are other alternatives like delve. A lot of people use development environments with built-in debugger support, like Visual Studio Code, which, I think, uses delve under the hood. I limited myself to print debugging for this project, and it thankfully ended up being just enough.

Last problem I faced was the strong type checking. This was a good thing though. Working mostly with dynamic languages has made me a bit "lazy" in certain ways. With Go, you have to pay close attention to how you design your programs. The end benefit, apart from catching errors at compile time, is that all your functions explicitly declare the data types they work upon. No magic, no implicit type conversions.

What went well

I would like to touch upon some things that I believe worked out smoothly in the end. Truth is that if I were to write the app again, I would do some things differently. Still, I think it is worth mentioning these things here.

Database migrations

I decided to include the SQL code for setting up the database in the project. I followed the Rails paradigm of ActiveRecord migrations. It basically treats the migrations as replayable, incremental operations for the database schema. I created a db/migrate folder with SQL scripts to set up the database schema. I then created a small Go program to connect to the specified database and execute the scripts. It is bundled with the source code. I know there are ready-made solutions for this out there, but Go makes this so easy to do, and I could not resist the extra learning opportunity.

Each migration has a timestamp. The program creates a schema_migrations table and stores the last timestamp run. That way, it knows which scripts have been already executed on subsequent runs.

Handler wrappers

One way to setup a server in Go is to declare some routes and use the net/http package to listen for requests. When setting up routes, you configure certain functions to execute when an endpoint is hit. These functions are called HTTP handlers. You can pass in any function you want, as long as it implements the Handler interface. The simplicity of the interface is refreshingly liberating.

Based on ideas by Matt Ryer and Matt Silverlock, I put handler wrappers (or handler middleware) to good use. Let us take a look at an example. The Handler interface is satisfied, as long as the type implements a function called ServeHTTP, which takes a pointer to an http.Writer and an http.Response as arguments. For example, we can have a type that retrieves a list of expenses from the database and returns them in a slice (array). Simple, you just have to implement a ServeHTTP function in such a way that it does exactly that. Now what if you want to add logging to this? And what if you want to add logging to more handlers?

Leveraging the fact that functions are first class citizens in Go you can come up with something like this

type indexHandler struct {
  // You probably pass a reference to the database or your store here...
}

func Index() *indexHandler {
  return &indexHandler
}

func (h *indexHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  // Get the list of expenses from the database
  // Use the response writer to render them back to the client making the request
}

type loggingHandler struct {
  // You probably pass a reference to a configured logger here
  logger *Logger
}

func WithLogging(logger *Logger, next http.Handler) *loggingHandler {
  return &loggingHandler{next: next, logger: logger}
}

func (h *loggingHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  h.logger.Log(r)
  h.next.ServeHTTP(w, r)
}

// Wrap the index handler with the logging middleware.
http.Handle("/expenses", withLogging(&Logger{}, Index()))

You can achieve the above by making the middleware return a handler function (HandlerFunc) instead of a Handler. I was excited when I first unlocked the power of this pattern.

Custom handlers

I took the above approach one step further. I figured, what does my API do? It takes a request with JSON body, performs some checks, fetches some data and writes a response setting the status, some headers and the response body in JSON. Hmm... this smells like an interface.

I ended up creating my own app.Handler interface that follows the http.Handler interface but further augments it to return an app.Response. The app.Handle function is a good example of a function that returns an http.HandleFunc but handles setting headers and responding with JSON all the time.

type Response interface {
  Status() int
  Body() interface{}
}

func Handle(next Handler) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    encoder := json.NewEncoder(w)

    // CommonHeaders is middleware to set headers like Content-Type
    // The response is an app.Response
    response := CommonHeaders(next).ServeHTTP(w, r)
    w.WriteHeader(response.Status())
    encoder.Encode(response.Body())
  })
}

I was able to get a good abstraction for responses with the status code and the body. All that the handler function had to do is encode the body to JSON. More information can be found in app/response.go here. There are structs for most common responses, like 200 OK, 422 Unprocessable Entity, 500 Internal Server Error, and so on...

The whole application is flexible in the sense that it can use simple http.Handler wrappers and middleware, but the final wrapper always returns an app.Handler.

Global/Context variables

I am not a fan of global variables. They tend to create confusion by causing side-effects. They also make code hard to test. What I did instead is that I set up an Env object to hold "global" variables like the database object, the logger, the routes, and so on...

type Env struct {
  db         *sql.DB
  router     *mux.Router
  userId     string
  logStream  io.Writer
  authSecret string
}

func New(authSecret string, dbUrl string) *Env {
  return &Env{
    router:     mux.NewRouter(),
    logStream:  os.Stdout,
    authSecret: authSecret,
  }
}

func main() {
  env := app.New(*authSecret)
  // wrapper over sql.DB
  database := db.Configure(dbUrl)
  defer database.Close()
  env.SetDB(database)
}

All handlers subsequently receive a pointer to the "global" *Env object as an argument. Truth is, this design provides good encapsulation but still forces you to pass on an object with more data than you may need. If I were to do it again, I would probably have handlers accept arguments that implement an interface and be more explicit about what they need.

// Current implementation
type indexHandler struct {
  env *app.Env
}

func Index(env *Env) *indexHandler {
  return &indexHandler{env: env}
}

// Better
type indexHandler struct {
  logger *Logger
  reader *ExpensesReader
}

func Index(logger *Logger, reader *ExpensesReader) *indexHandler {
  return &indexHandler{logger: logger, reader: reader}
}

// caller
http.Handle("/expenses", Index(env.GetLogger(), env.GetDB()))

Deployments

Deploying Go code is by far the easiest deployment process I have ever experienced. You go install your program and you get a binary which you can copy to your server and run. I used systemd to be able to run the program as a service and have it restart on failure.

You can even cross-compile your binary for other target platforms and operating systems.

Post-mortem

Golang is relatively small in comparison to other languages, but its standard library is really powerful. Development time is relatively slower in contrast to other languages, but the big win is that all Go code is pretty much the same. Once you establish and explain a set of patterns, any developer can quickly pick up your project. And the code will look like it was written by the same person.

I have still got a lot to learn about writing idiomatic Go. As is true with all languages, it takes a while to master it, and you have to crash and burn until you find your voice. I have found that the "Go way" is not hard. It might look limiting at first, but the sooner you embrace it, the more liberating it will become.

This was my first "real" project with Go. Despite all my struggles with the language, I would definitely use it again. In fact, I am already using it for another personal project. But that is a story for another time.