This evening I went to a YOW Night where Ken Scambler (@KenScambler) spoke about the introduction and evolution of using Scala at REA Group. Here’s my notes…
Functional Scala Benefits
The benefits of going functional are to get to code that is: Modular, Abstract, Composable.
Modularity is about being able to fit entire sections of code in your head without having to consider things going on outside that code, and also about being able to replace small parts without affecting the whole.
To write a total function (a function that returns a result for all possible input values), you need to elevate all possibilities into the type system. For example, you can’t throw an exception, you have to encode that possibility of an error into the return value somehow.
Abstraction should reduce changes to code, because unnecessary detail is not all across the code.
Whole systems can be composed from functional components.
Functional programming is not about picking up a hipster language. It’s about producing better software.
Functional Scala at REA
The team at REA picked Scala partly because they wanted strong types, and having object-oriented capabilities made the learning curve smaller for the team of Java programmers. Also, someone converted part of a service to Clojure as an example, which made everyone want to try Scala.
They used Play2 as the web framework to start with, but found it too heavy, very opinionated, and they were fighting against it.
With the second Scala project, they forgot to invest in learning and that resulted in an okay technical result and a poor sentiment.
The second team to start on Scala (third project) picked a different tech stack (unfinagled, argonaut and slick). In went well, and the first team stole lots of their good ideas and learning for their next project.
They then tried to use monad transformers for dependency injection. This proved to be a massive learning curve, and even the senior devs didn’t get it. The technical result was okay, but only two people really understood the code.
Ken had two great comments after this experience:
“Ultimately, we have to bring people along, not leave them behind.”
and
“If you can’t make a new concept learnable, you shouldn’t use it.”
A team with no Scala mentors tried to employ Scala while learning through trial and error and code katas. It didn’t work out so well.
Emerging functional design trends at REA
Emerging design trends they’e converged towards:
- Replaced inheritance and mixins with static functions.
- Abandoned partial functions (eg throws exception) in favour of total functions.
- Replaced exceptions with sum types (Either)
- Says Ken: “exceptions are essentially a goto”
- Replaced strings and primitives with wrapper types. String has too many possible values. Your total functions don’t want to deal with all of those.
Learning Functional Scala
He said the “Functional Programming in Scala” book is really good.
It’s important that learning is done during work time. REA uses the “guild” model made famous at Spotify where groups of people from across the organisation but interested in the same technology meet together to learn from each other.
On their latest project, #17 with Scala, they’re using: unfiltered, argonaut, slick, and free monads for dependency injection. They put a huge investment into people’s learning for this project and consequently had a great result. People were able to understand the code and the service has a large pure functional core with thin layers of interpreter and web access at the front.
“The only purpose of mocks is to point out design flaws in your code, so you can fix them and remove the mocks.”
Ken finished off by observing:
“FP is a tall tree, but there is so much low hanging fruit.”
Want to learn more?
Hi Graham,
This is a very good summary. Thanks.
I am learning Haskell now. Though I have not got to anything as advanced as “Free Monads”. Having heard the talk and read your link, I really feel I get the concept and could explain to someone why they are useful.
REA uses Free Monads to abstract out real world actions such as DB queries, returning HTTP responses and suchlike. I wonder if they would be useful also to create ‘business abstractions’ like this:
p5 = do s <- nextShipment
markAsSent s
end
Where in this case I don't even care if I am on a web server, or what kind of data storage is being used.
Then you are almost at the point where lot of business-specific behavior can be quickly developed with confidence that it will work as intended.
Hey Martin,
Precisely – your use case is spot on! You are essentially defining a “little language” and expressing your program’s high-level intent with it.
There’s a design challenge: what do you put in your algebra of instructions, what goes in the code, and what goes in the interpreter?
You want the instructions to not be too low level, because you want the programs to express the things you care about, and not get bogged down in details. On the other hand, you can easily compose bigger scripts out of little ones, so you want them low level enough that you can use them as building blocks.
Generally you want to keep the interpreter as thin as possible, since it is by design a kind of “naughty corner” for messy things that are basically untestable — and you swap it out in its entirety when testing the rest of the application. It should never make decisions — it should be just pulling the trigger.
So there’s still some scope for good and bad decisions within a Free Monad design, but I think they are decidedly first world problems. 🙂