Functional Programming in the Real World - Handling Effects

During those workshops we are going to learn how to handle situation when need to handle side effects and create functions which may return unpredictable results.


  1. Understand what kind of problems may rise when effects are hidden and not explicitly sygnalized in the type system..
  2. Learn how make an Effects part of Type system
  3. How to compose mulitple effects of the same type
  4. How to represent effects of Optionality , Failure , Time and Non Determinism.
  5. How to use pure functions in the context of side effects

Part 1 - functions in the real world

Lab File :

First we are going to provide couple implementations of a function which return first element of a list

def firstElement1Exception(l: List[Int]): Int = l.head
def firstElement1Null(l: List[Int]): Int = if (l.size > 0) l.head else null.asInstanceOf[Int]
def firstElement1OrDefault(l: List[Int], default: Int): Int = if (l.size > 0) l.head else default
def firstOptionalElement(l: List[Int]): Option[Int] = if (l.size > 0) Some(l.head) else None

The problem is that an empty list does not have the first element - how we can handle this situation?

In the demonstration part we are going to check how to safely use each variant of these function and as a consequence - what construction will appear in our code.

  • In try example we need to add try-catch block. We "need" but we are not "forced"to add it. So there is always danger that we will not handle it properly. Later we will discuss why checked exception did not solved this problem
  • We could return null but does caller know that we could return null?
  • We could provide default value. But do we always have logical default value. What if we need to distinguis between natural and default result?
  • Finally we can make this edge case a aprt of Type system


In the exercises section we are going to execute one of the two following scenarios :

  1. call UsersDAO and find given User
  2. convert user into HTML
  3. display his info on a sucess page

or in case of missing user in database

  1. call UsersDAO but there is no user with given Id
  2. display error page

Exercise : handle both scenario with possible missing value

In dao you can find 4 variants of find emthods :

def findOrNull(id: Int): User = if (database.contains(id)) database(id) else null
def findOrException(id: Int): User = if (database.contains(id)) database(id) else throw new RuntimeException("break computation!")
def findOrDefault(id: Int, defaultUser: User): User = if (database.contains(id)) database(id) else defaultUser
    //what is default here?
def findOption(id: Int): Option[User] = database.get(id)

Try to handle two mentioned business scenarios with each variant and check what consequences each solution triggers.

There is provided skeleton for each part in the code

//    val optionalUser = UsersDAO.findOption(1)
//      displayPage(userToHTML(optionalUser.get))
//      displayError("There is no user with this id")

Part 2 - Understand Optionality Effect

Lab File :

In the last exercise we used Option just to sygnalize that value is potentially missing and then we really treated it like object representation of if condition which is actually a bad practice

Having Option as a type allow us to execute pure functions which are referentially transparent in context of missing value

val some: Option[Int] = Some(7)

val multiplyByTwo: Int => Int = i => i * 2 //is this function pure ?
val display: Int => String = i => s"  -- RESULT : $i" // does this function have any knowledge about context? => println(s"  -- multiplied by two : $i"))

But what if the value is really missing - what will be executed and how to handle this situation?

Because Option type has available set of very useful high order functions so it is very easy to use pure functions to handle happy path and also to build second path in case of empty value

val default = some.filter(_ > 10).getOrElse(-1)
some.filter(_ > 10).orElse(Some(-1)).map(i => s" -- filter(remove) & orElse -1 : i=$i").foreach(println)
some.filter(_ < 10).orElse(Some(-1)).map(i => s" -- filter & orElse : i=$i").foreach(println)

We can also use Pattern Matching to handle each Option type accordingly :

some match {
        case Some(value) => println(s"  -- pattern matching value : $value")
        case None => println(s"  --  pattern matching is empty")

finally we can just pass to option set of recipies in case of success and failure.

val onSuccess: Int => String = i => s"business value $i"
val onError: String = "raise an error"
val result = some.fold(onError)(onSuccess)

This is a lot of new knowledge - let's train it now


Exercise : handle both scenario with high order functions

In following lines use combinators map and fold and also pure functions from Conversions and FrontEnd to implement both scenarios

 val html1 = UsersDAO.find(1) //use map & fold
 val html2 = UsersDAO.find(2) //use map & fold

In the following example try to use combinators map and orElse and getOrElse

UsersDAO.find(2) //use map & orElse & foreach to display result
val html3 = UsersDAO.find(2) //use map & getOrElse

Additional Exercise : implement lift function

As a warmup implement lift function which lifts pure functions into effects level

def lift[A, B](f: A => B): Option[A] => Option[B] = ???

Additional Exercise : implementing optionality with custom types

Now we have a custom type which symbolizes optionality

sealed trait Maybe[+A]
case class Just[A](value: A) extends Maybe[A]
case object Empty extends Maybe[Nothing]

implement lift and map fot this new type

def map[A, B](m: Maybe[A])(f: A => B): Maybe[B] = ???
def liftMaybe[A, B](f: A => B): Maybe[A] => Maybe[B] = ???

Part 3 - Multiple Optional Effects

Till now we analyzed only couple simple examples when we have a single Optionality Effect but what to do when we have multiple such types in our calculations?

For example we have a User who may or not have Page which may or not have Picture

Take a look at the Exercise source code :

In Demonstration you will find an example how to use the new operator flatMap . Each time when you have an effect and you want to perform transformation which also triggers the Effect - think flatMap


Exercise : display User last meetup

find user last meetup and implement pure function which maps user to HTML

def userLastMeetup(id: Int): HTML = ???
def meetupHistoryToHTML(mh: MeetupHistory) = ???

In Our domain you have a meetup history record with optional picture

val meetupHistory1 = MeetupHistory("FP Scala", Some(Picture("SCALA_LOGO")))
val meetupHistory2 = MeetupHistory("FP Java", None)

The problem is that potenatial user maybe has just registered and he has no meetup history

val user1 = User(1, "FirstUser", "", List(meetupHistory1, meetupHistory2))
val user2 = User(2, "SecondUser", "", List())

Additional Exercise : implement flatMap for our custom type

def flatMap[A, B](m: Maybe[A])(f: A => Maybe[B]): Maybe[B] = ???

Additional Exercise : implement new combinator 'map2'

What if you have two Options and a two arguments pure function? We need to introduce a new operator which lifts multi-argument functions to effects level.

def map2[A, B, C](m1: Maybe[A], m2: Maybe[B])(f: (A, B) => C): Maybe[C] = ???

This is a prelude to a new very powerful Functional Programming Abstraction - stay tuned!

Part 4 - For Comprehension


When we have a lot of effect types using map and flatMap can make our code less readable.

      val some1 = Some(1)
      val some2 = Some(2)
      val some3 = Some(3)
      val some4 = Some(4)

      val result = some1.flatMap { a =>
        some2.flatMap { b =>
          some3.flatMap { c =>
   { d => a + b + c + d }

Fortunatelly Scala introduces a very powerful language construct - for-comprehension

val resultComprehension = for {
        s1 <- some1
        s2 <- some2
        s3 <- some3
        s4 <- some4
} yield s1 + s2 + s3 + s4

The result of both code samples is the same but the second one is a lot cleaner. During exercises we will gain some experience with usage of this construct.


Additional Exercise : join optional values

Join couple optional values with for comprehension

def join(o1: Option[String], o2: Option[String], o3: Option[String]): Option[String] = ???

Additional Exercise : implement map2 for Maybe type

 def map2[A, B, C](m1: Maybe[A], m2: Maybe[B])(f: (A, B) => C): Maybe[C] = ???

Additional Exercise : sequence of effects

//use foldRight & map2   || flatMap & map & recursion
def sequence[A](l: List[Maybe[A]]): Maybe[List[A]] = ???

Part 5 - Error Effect

Lab File :

While Option tell us that something Is present/Is missing Try is prepared to handle multiple error path for different exceptions

Try has two subtypes Success and Failure

println(s"\n  -- pattern matching")
goodResult match {
     case Success(lines) => println(s"file size PM : "+lines.size)
     case Failure(exception) => throw exception

You can recover to happy path if you know how to handle specific exception

 println(" \n -- Recover Bad")
  case e:Exception =>  List(s"something went wrong  : [${e.getMessage}]")

or you can apply independent handler procedures

println(s"\n  -- transform Good")
val businessFunction:List[String] => Try[Int] = l=> Try(l.length)
val errorHandler : Throwable => Try[Int] = _ => Try(0)
goodResult.transform(businessFunction,errorHandler).map(i=>s"file length $i").foreach(println)


Exercise : add error effects

For comprehension may be useful here :

def tryToAdd(s1: String, s2: String, s3: String, s4: String) :Try[String] = ???

Additional Exercise : implement map2 and sequence for Try

def map2[A,B,C](t1:Try[A],t2:Try[B])(f:(A,B)=>C):Try[C] = ???
def sequence[A](l: List[Try[A]]): Try[List[A]]

Additional Exercise : implement map3 for Maybe type

A pattern starts to emerge

def map3[A, B, C,D](m1: Maybe[A], m2: Maybe[B],m3:Maybe[C])(f: (A, B,C) => D): Maybe[D] = ???

Part 6 - Some Theory - Effects and composition


But why exactly effects removed from type system cause so much problem? Let's imagine couple simple functions which we want to compose :

val toInt:String=>Int = _.toInt
val addOne:Int=>Int=_+1
val multiplyByTwo:Int => Int=_*2
val simpleComposition=toInt andThen addOne andThen multiplyByTwo
println("  -- pure result : "+simpleComposition("5"))

Everything works fine but there is something very dangerous lurking inside :

simpleComposition("notANumber") // this will throw an exception

Now look at this. Because Our function has hidden side effect it can return different result in different places of a program so Referential Transparency is broken as a start and we need to be prepared for all exceptions thrown by each function which composed the final one.

 val resultWithSideEffect=try{

    simpleComposition("notANumber")  //here returns -1
  }catch {
    case e:Exception => -1
//      simpleComposition("notANumber") // here returns 0
  case e:Exception=> 0
//    simpleComposition("notANumber") //here dies

Now what if we want to handle this situation and still keep simple types? Well we can not compose function which throws exceptions because it just leave computations so we must somehow signalize that actually our function has two possible paths :

val toIntWithException:String=>(Int,Exception)

But in case of happy path we don't have Exception and in case of exception we may not have value - we can use null to signalize this optionality - we already saw what kind of problems null generates.

 val toIntWithException:String=>(Int,Exception)=input=>try{
} catch{
  case e:Exception => (null.asInstanceOf[Int],e)

But what If second function has just simple type Int => Int ? We need need somehow to wrap this function into (Int,Exception) => Int ... ant then again we may encounter the same pronblem which solve set of embedded ifs

The truth is that function val toInt:String=>Int = _.toInt is a partial function To make it total we need to introduce Effects into Type system

val toIntTotal:String=>Try[Int] =input =>  Try(input.toInt)

now we can transform/map this value without looking for any hidden execution paths. In case of pure functions we have map combinator and we saw that we can easily mixIn other functions with effects with flatMap

Part 7 - Other Effects



To symbolize Time Effect - calculations may be finished soon or never - Scala sues Future

For example we can have two external services and we need to wait untill information will move through wires.

object Service1{
  def call(input:Int):Future[Int]=Future{

object Service2{
  def call(input:Int):Future[Int]=Future{

Than we can build result with already known for comp[rehension

val comprehensionResult=for{
  } yield r2

    case v => println(s"  -- comprehensionResult : $v")

Non Determinism

This may be surprise but effect of non determinism is a ... List of multiple values. Why?

Theoretically function can return only one value


What if we have


How to specify what exactly is a result? For example we want to calculate root of a positive value

 val allRoots:Double=>List[Double]=input => {
        val result = Math.sqrt(input)
        List(-result, result)

Now we can use high order functions to specify one final result

println("-- handling non determinism1 : "+allRoots(4.0).map(Math.abs).head )
println("-- handling non determinism2 : "+allRoots(4.0).filter(_>0).head )

We can also use special reduction oeprators

Let say we want to count profit made on a given day. We call for a purchase from that day and we receive multiple purchases.

val purchasesOnDate: String => List[Purchase]= _ match {
      case ("01-01-2016") => List(Purchase("tv","01-01-2016",300),Purchase("console","01-01-2016",200))

Now we need to reduce our purchases to single value

println("  -- counting profit - moving from set of possible values into a single value")
val singleProfit=purchasesOnDate("01-01-2016").map(_.profit).reduce(_+_)
println("  -- singleProfit : "+singleProfit)


Exercise : all possible combinations

You have two multi value results

val drivers=List(Driver("John"),Driver("Jane"),Driver("Kubica"))
val cars = List(Car("Polonez"),Car("Porsche"))

calculate possible combinations

def combinations():List[(String,String)] = ???

Additional Exercise : applying function with effects

map2 allow us to specify new very powerful combinator. Because map2 apply two-arg function to effects than we can close a Function (which is also a value in FP) inside an Effect and then we can apply this function multiple times in context of this effect.

This may be more clear in code :

take look at function type: (Option[Int=>Int]) => (Option[Int] => Option[Int])

val applicative1ArgInt: (Option[Int=>Int]) => (Option[Int] => Option[Int]) = 
function => firstOption => map2(function,firstOption)((f,argument)=>f(argument))

and the second one , again notice

(Option[Int=>Int=>Int]) => (Option[Int] => Option[Int=>Int])

val applicative2ArgIntInt: (Option[Int=>Int=>Int]) => (Option[Int] => Option[Int=>Int]) = function => firstOption =>

To see how universal this pattern may be try to implement map & map2 with it

def mapApp(o1:Option[Int])(f:Int=>Int):Option[Int] = ???
def map2App(o1:Option[Int],o2:Option[Int])(f:(Int,Int)=>Int):Option[Int] = ???


If we want to generalize function from previous exercise it would look like this :

 def applicative[A,B](fab:Option[A=>B])(oa:Option[A]):Option[B] = map2(fab,oa)((f,arg)=>f(arg))

Still it only handles Option type. We already had similar issue when we generalized map function during workshop 2.

We also saw that with map2 we can apply function in context of given effect. If we combine this with currying we receive a powerful abstract concept of applaying multiple arguments in context of a given effect

  • Functor : (A=>B) => (F[A] => F[B])
  • Applicative Functor : F[A=>B] => (F[A]=>F[B])
  • Applicative Functor : F[A=>B=>C] => (F[A]=>F[B=>C])
  • Applicative Functor : F[A=>B=>C=>D] => (F[A]=>F[B=>C=>D])

Cats library already provides Applicative Functors for most Effects. In cats they are called Apply

So Implement universal map2 and map3 with this pattern

def universalMap2[A,B,C,M[_]](m1:M[A],m2:M[B])(fab:(A,B)=>C)(implicit applicative:Apply[M]): M[C] = ???
def universalMap3[A,B,C,D,M[_]](m1:M[A],m2:M[B],m3:M[C])(fabc:(A,B,C)=>D)(implicit applicative:Apply[M]): M[D] = ???

Part 8 - Combine different effects


This is quite complex topic but I decided to put it here just for completness of Effects topic

What if for example we have two effects at the same time? Foe example when we are calling external webservice we have time - Future[T] and posibility that given value will be missing - Option

In this situation we can use predefined MonadTransformer . In Graphical explanation it will look like this :

Let's imagine that in the code we have two web services :

 def service1(name:String) : Future[Option[Int]] ...
 def service2(id:Int) : Future[Option[Company]]

We can not write simple for comprehension because argument of second web service Int is not the same as result of the first one Option[Int].

Solution with MonadTransformer

    import cats.std.future._

    val result=for{
    } yield

    val finalValue: Future[Option[String]] = result.value


Read :

results matching ""

    No results matching ""