Bridging Scala and the Front-End

In this blog post we will talk about Bridges, a simple library that generates front-end code from your Scala ADTs and reduces the friction of changing your data model. We will present the project, show how to use it, and list some common traps to avoid when creating your ADTs. We will also showcase some advanced features for projects using Scala and Elm.

When developing a web-based application (or any application with separate back- and front-end codebases), changes to the back-end model often cause a cascade of manual changes on the forward layers. We need to modify the corresponding data types in the front-end, and also utilities such as encoders and decoders for JSON and URL parameters. The industry has developed solutions as Protocol Buffers to tackle this problem, and one of the main features people like about Scala.js is being able to reuse your back-end model in the front-end.

But we can’t always use Scala.js, and sometimes we may just want something simpler than Protocol Buffers. Ideally we want a lightweight tool that keeps our front-end in sync with our back-end data model, without having to use a 3rd intermediate language to define them.

We propose Bridges as a solution for this issue. Bridges is in the early stages of development, but is already being used in production at Underscore in codebases front-ends written in Elm and Flow.

Introducing Bridges

Bridges is, at its core, a simple library: it uses shapeless to translate our ADTs into an intermediate language. Then we can use that language to generate valid representations of those ADTs in the front-end.

The advantage of using an intermediate representation is that Bridges can generate output for multiple front-end languages. Currently there is support for Typescript, Flow, and Elm, but that can be easily extended for other languages.

The main difference with other tools like Protocol Buffers is that the intermediate model is automatically derived at compile time. We are writing code in Scala and obtaining the front-end representation automatically, so any change to our Scala ADT is automatically picked up by the type-checker for our front-end codebase.

Bridges intermediate language

Bridges intermediate language tries to directly represent a Scala ADT using another ADT. At the core we have two types:

The type DeclF[A] represents a named declaration, that is the representation of a type along its name. This named declaration can be used as a top level element, or as a field when defining a Product or Sum for the representation of an ADT.

The type Type maps elements found in a Scala ADT. We have a definition for Prod (Product) and Sum, as well as definitions for basic types in Scala. We also use the special type Ref to represent types defined by the user.

The definitions for both types follow:

final case class DeclF[+A](name: String, tpe: A)

sealed abstract class Type extends Product with Serializable
final case class Prod(fields: List[Decl])      extends Type
final case class Sum(products: List[ProdDecl]) extends Type
final case class Ref(id: String)               extends Type
final case object Str                          extends Type
// and others...

With these definitions we can transform a given case class into a corresponding Type structure:

case class Value(value: String) extends AnyVal

DeclF("Value", Str)

// second example
case class Pair(a: String, b: Int)

DeclF("Pair", Prod(List(
          DeclF("a", Str),
          DeclF("b", Intr)
        )))

DeclF and multiple target languages

Note that DeclF (defined in the previous section) is parameterised by A. By default A is Type, as in

type Decl = DeclF[Type]

but this representation allows us to swap the Type representation used by our intermediate language, if needed. This is a requirement that has been added as Bridges started targeting several languages with different capabilities.

Currently Bridges is targeting 3 languages: Elm, Typescript and Flow. Elm has a Haskell-like syntax that matches perfectly the intermediate language we described in the previous section. But Flow, for example, differentiates between union types, intersection types, and structs. This doesn’t map in a straightforward way to our ADT Products and Sums.

Solving this mismatch between the target languages was creating friction in the intermediate language, thus the solution to add A to DeclF: if your language may need to cover special cases, you can create your own intermediate language and use it, so your resulting code matches expectations.

From intermediate to final representation

Once we have our DeclF representing the Scala code to translate, we need a way to obtain a String with the syntax for the target language. This is achieved via the Renderer[A] trait and its various implementations. Renderer is defined as:

trait Renderer[A] {
  def render(decl: DeclF[A]): String
}

and we implement one instance for each target language. For example, for Elm and Flow we have:

class ElmRenderer extends Renderer[Type] { ... }
class FlowRenderer extends Renderer[FlowType] { ... }

Note that while Elm used the default Type describe before, the FlowRenderer class uses its own FlowType intermediate representation to be able to define intersection types.

Using Bridges with Typescript

With the above, we have all the pieces we need to generate our final representations. To use Bridges in our application, we simply need to import the library using the latest version (currently 0.11.0):

libraryDependencies += "com.davegurnell" %% "bridges" % "0.11.0"

Let’s see it in action with some sample ADTs:

final case class Color(red: Int, green: Int, blue: Int)

sealed abstract class Shape extends Product with Serializable
final case class Circle(radius: Double, color: Color) extends Shape
final case class Rectangle(width: Double, height: Double, color: Color) extends Shape

We’ll start by generating the intermediate representation of Color for Typescript:

import bridges.typescript.syntax._

val declaration = decl[Color]
// declaration: bridges.typescript.TsDecl = 
//	  DeclF(Color,
//			Struct(List(DeclF(red,Intr), DeclF(green,Intr), DeclF(blue,Intr))))

As you can see, Bridges tells us the Color is a structure tagged as Color that forms a Struct with three numeric fields, one per each field in our class. Note that declaration is of type TsDecl which is defined as DeclF[TsType] as Typescript requires its own intermediate language.

Now let’s request the Typescript representation of this declaration:

import bridges.typescript._

TsTypeRenderer.render(declaration)
// res0: String = export type Color = { red: number, green: number, blue: number };

Or we can request the implementation for all our ADTs, as follows:

import bridges.typescript._
import bridges.typescript.syntax._

TsTypeRenderer.render(List(decl[Color],
  decl[Circle],
  decl[Rectangle],
  decl[Shape]
))
// res1: String =
//		export type Color = { red: number, green: number, blue: number };
//		export type Circle = { radius: number, color: Color };
//		export type Rectangle = { width: number, height: number, color: Color };
//		export type Shape = { type: "Circle", radius: number, color: Color } | { type: "Rectangle", width: number, height: number, color: Color };

The output of render() is a typical representation of an ADT in Typescript: a set of structural types and a tagged union based on a discriminator field called type.

Typescript and guards

Having an intermediate language means that we can do more than just render our types in the syntax of the target language. For example, once we are running our Typescript code we will need to load Json and convert it to one of the types defined above. This is a repetitive task, as we can use Bridges to generate that code for us.

By creating a new trait TsGuardRenderer that uses DeclF[TsType] as inout we can generate the following:

TsGuardRenderer.render(decl[Color])
// res0: String =
// export function isColor(v: any): boolean {
//   return typeof v.red === "number" && typeof v.green === "number" && typeof v.blue === "number";
// }
//
// export function asColor(v: any): ?Color {
//   return isColor(v)
//     ? v as Color
//     : throw new Error("Expected Color, received " + JSON.stringify(v, null, 2));
// }

The code we generated allows us to verify some Json is a valid Color by using isColor and, if it is, we can obtain a Color using asColor.

Bridging Scala and Elm

Instead of Typescript we may want the Elm output, which we can obtain by replacing the language type parameter:

import bridges.core.syntax._
import bridges.elm._

Elm.render(List(declaration[Color],
  declaration[Circle],
  declaration[Rectangle],
  declaration[Shape]
))

// res2: String =
//   type alias Color = { red: Int, green: Int, blue: Int }
//
//   type alias Circle = { radius: Float, color: Color }
//
//   type alias Rectangle = { width: Float, height: Float, color: Color }
//
//   type Shape = Circle Float Color
//     | Rectangle Float Float Color

Elm has direct language support for ADTs, which means we can use the default Type instead of having to create our own intermediate representation.

Generating JSON encoders and decoders

For projects that use Elm we can also generate JSON encoders and decoders (support for Typescript and Flow codebases is coming). This means we can use these encoders and decoders in our Elm code, ensuring we are working with a version that matches the Json sent by the back-end.

For this functionality to work, we make a few assumptions about our data model:

  • All Elm types generated by Bridges will belong to the same module;

  • Our Elm project must include the NoRedInk/elm-decode-pipeline and danyx23/elm-uuid dependencies;

  • When we encode the ADTs into Json, any CoProduct we generate will use a field type to discriminate between values. For Circe users, see this example on how to achieve this.

These assumptions mean that we can predict the shape of the decoder and generate valid Elm syntax.

Bridges provides two methods to generate Json encoders and decoders:

  • decoder(dec: DeclF[Type]) will generate the decoder for our ADT;

  • encoder(dec: DeclF[Type]) will generate the encoder for our ADT.

Let’s see some examples, using the ADT we defined in the previous section. First let’s generate a decoder for Color:

import bridges.core.syntax._
import bridges.elm._

Elm.decoder(decl[Color])

// res1: String =
//   decoderColor : Decode.Decoder Color
//   decoderColor = decode Color
//     |> required "red" Decode.int
//     |> required "green" Decode.int
//     |> required "blue" Decode.int

And let’s build the encoder for the same type:

import bridges.core.syntax._
import bridges.elm._

Elm.encoder(decl[Color])

// res1: String =
//   encoderColor : Color -> Encode.Value
//   encoderColor obj = Encode.object [
//      ("red", Encode.int obj.red),
//      ("green", Encode.int obj.green),
//      ("blue", Encode.int obj.blue) ]

For a more complex ADT like Shape, we get a more complex decoder that expects the type field as a discriminator:

import bridges.core.syntax._
import bridges.elm._

Elm.encoder(decl[Shape])

// res1: String =
//   decoderShape : Decode.Decoder Shape
//   decoderShape = Decode.field "type" Decode.string
//      |> Decode.andThen decoderShapeTpe
//
//   decoderShapeTpe : String -> Decode.Decoder Shape
//   decoderShapeTpe tpe =
//      case tpe of
//         "Circle" -> decode Circle
//            |> required "radius" Decode.float
//            |> required "color" (Decode.lazy (\\_ -> decoderColor))
//         "Rectangle" -> decode Rectangle
//            |> required "width" Decode.float
//            |> required "height" Decode.float
//            |> required "color" (Decode.lazy (\\_ -> decoderColor))
//         "ShapeGroup" -> decode ShapeGroup
//            |> required "leftShape" (Decode.lazy (\\_ -> decoderShape))
//            |> required "rightShape" (Decode.lazy (\\_ -> decoderShape))
//         _ -> Decode.fail ("Unexpected type for Shape: " ++ tpe)

As well as the matching encoder:

import bridges.core.syntax._
import bridges.elm._

Elm.encoder(decl[Shape])

// res1: String =
// encoderShape : Shape -> Encode.Value
// encoderShape tpe =
//    case tpe of
//       Circle radius color -> Encode.object [
//          ("radius", Encode.float radius),
//          ("color", encoderColor color),
//          ("type", Encode.string "Circle") ]
//       Rectangle width height color -> Encode.object [
//          ("width", Encode.float width),
//          ("height", Encode.float height),
//          ("color", encoderColor color),
//          ("type", Encode.string "Rectangle") ]
//       ShapeGroup leftShape rightShape -> Encode.object [
//          ("leftShape", encoderShape leftShape),
//          ("rightShape", encoderShape rightShape),
//          ("type", Encode.string "ShapeGroup") ]

The Bridges codebase includes several examples of different ADTs along with expected output for each. Check the tests, specifically ElmJsonDecoderSpec and ElmJsonEncoderSpec, for more information.

Creating complete Elm modules

To compile the Elm code we’ve produced so far, we need we to join the fragments and add the required imports. Bridges’ Elm module provides another method to help with this:

  • buildFile[L](module: String, decls: List[Declaration]) returns a pair of Strings: a file name and the contents of the file.

buildFile uses the other methods discussed above to provide a convenient, batteries-included way of generating Elm code. Let’s see an example based on an ADT we have used on this post:

import bridges.core.syntax._
import bridges.elm._

Elm.buildFile("CustomModule", decl[Color])

// res1: (String, String)
// res1._1: String =
//   Color.elm
//
// res1._2: String =
//   module CustomModule.Color exposing (..)
//
//   import Json.Decode as Decode
//   import Json.Decode.Pipeline exposing (..)
//   import Json.Encode as Encode
//
//   type alias Color = { red: Int, green: Int, blue: Int }
//
//   decoderColor : Decode.Decoder Color
//   decoderColor = decode Color
//      |> required "red" Decode.int
//      |> required "green" Decode.int
//      |> required "blue" Decode.int
//
//   encoderColor : Color -> Encode.Value
//   encoderColor obj = Encode.object [
//      ("red", Encode.int obj.red),
//      ("green", Encode.int obj.green),
//      ("blue", Encode.int obj.blue) ]

You can use the filename and content from buildFile to create an Elm source file at a relevant location as part of your build pipeline.

Gotchas

Here are a few problems and workarounds you may encounter using Bridges:

Overriding definitions for specific types

Sometimes, a case class in the back-end may include a type that we don’t want to derive in the front-end. For example, we may have a custom data type Foo on the back-end that we want to represent with a simple type like Int in the UI.

We can modify the way values are translated to the intermediate language using implicits. For this example, we would add the following to our code generator:

implicit val fooEncoder: BasicEncoder[Foo] =
  Encoder.pure(Num)

This will convert any reference to Foo to a numeric value in our generated code.

Refined types

Refined is a Scala library for refining types with type-level predicates that constrain the set of values described by some underlying type (for example, the set of positive integers, or the set of strings of a certain length). When using Bridges on Refined values, it is very important that the following import from refined-shapeless is in scope, to avoid compilation errors:

import eu.timepit.refined.shapeless.typeable._

For example, given the following refined type RefinedString we can generate our declaration as follows:

import eu.timepit.refined._
import eu.timepit.refined.api.Refined
import eu.timepit.refined.collection.Size
import eu.timepit.refined.numeric.Interval.ClosedOpen
import eu.timepit.refined.shapeless.typeable._

type RefinedString = String Refined Size[ClosedOpen[W.`1`.T, W.`100`.T]]

final case class ClassWithRefinedType(name: RefinedString)

decl[ClassWithRefinedType] 
// res0: bridges.typescript.TsDecl = 
// 		DeclF(ClassWithRefinedType,
			Struct(List(DeclF(name,Str))))

Note that the type for RefinedString is Str when using the intermediate language, we have removed the refinement and preserved the base type.

Conclusions

We have presented Bridges, a library to generate front-end code for our apps based on our back-end ADTs. We showed how we use this in to generate valid code for TypeScript and Elm, simplifying development and reducing the effort needed to keep both front-end and back-end in sync.

Acknowledgements

Bridges is a collaboration between Dave Gurnell and myself inspired by work by Stephen Kennedy. Thanks to Miles Sabin for his help solving shapeless-related queries.


Like what you're reading?

Join our newsletter


Comments

We encourage discussion of our blog posts on our Gitter channel. Please review our Community Guidelines before posting there. We encourage discussion in good faith, but do not allow combative, exclusionary, or harassing behaviour. If you have any questions, contact us!