Building services with Ktor is a lot of fun, but because it is so flexible, it can sometimes be hard to decide on a structure or which libraries to use. This talk aims to provide a set of “opinions” that have worked well for me and the teams I’ve worked with.

Resources

• Ktor Quickstart - Ktor Quickstart template
• Ktor Documentation - The official Ktor documentation.
• KotlinConf 2026 - The conference website.

Slides

The slides for the talk can be found here: Opinionated Ktor Services - Slides (Coming soon)

Video

The recording of the talk will be available on the Kotlin YouTube channel later this year.

Thank you to everyone who attended the talk! It was great to meet so many of you in Munich. If you have any questions, feel free to reach out to me on the Kotlin Slack, Twitter, BlueSky or LinkedIn.

Good morning everyone,

I hope you all had a great trip to Amsterdam and are well-rested for this exciting conference. I am thrilled to be here today to discuss Kotlin, Open-Source, and Functional Programming.

Thank you all for coming to my talk about Arrow’s trajectory towards 2.0.

Before we dive into the main topic, let me briefly introduce myself. My name is Simon.

I have been working with Kotlin for the past eight years, since 2015, and I am a fanatic open-source developer.

Mostly active on GitHub and over the past five years, my primary focus has been on Arrow, but I also contribute to other projects and organisations.

Currently, I work as a Principal Engineer at Xebia Functional, previously known as 47 Degrees.

My favorite place to engage with the community is the Kotlin Slack, but you can also find me on Mastodon and Twitter. I exclusively discuss Kotlin, with a focus on Arrow and back-end related topics.

During this talk, I plan to cover five topics. Firstly, I will shed light on the history of Arrow and the reasoning behind some decisions made in the past. As we dive into this topic, I will also discuss the complexity of functional programming from a Kotlin perspective. While these opinions are my own, I believe they are shared by many in the Kotlin and Arrow community.

Next, I will share my thoughts on the importance of embracing the language and the ecosystem. A uniform and cohesive ecosystem of libraries that work well together is crucial not only for productivity and collaboration within the community, but also for the success of the language.

In addition, I will discuss one of the hardest aspects of open-source software - evolving APIs and libraries. While it can be frustrating and expensive to break code, sometimes it’s necessary in the path towards success.

Lastly, I will mention other projects in the ecosystem and highlight the community in and around Arrow.

To understand the history of Arrow, we need to take a big jump back to a time before Kotlin was on my radar. In 2013, Mario Arias created funKtionale. Developers working in Scala with Spring wanted to use Kotlin’s seamless interoperability with Java. However, they quickly realized that Kotlin lacked a set of features they had grown used to in Scala. This led to the creation of funKtionale, which aimed to solve this problem.

Fast-forward to 2017, a Spanish community named FineCinnamon created a learning project called Katz, heavily inspired by the Typelevel ecosystem in Scala. However, it quickly evolved and changed its name to Kategory to become its own entity as it grew larger over time.

Around the same time, I was also on a learning adventure and created Optikal, an Optics library in Kotlin inspired by Monocle from Scala. After I made the project public, the Kategory maintainers approached me and invited me to the Lambda World conference, where they asked me to join the Kategory efforts. Excited by the opportunity to join forces, I agreed.

Soon after, Mario from funKtionale joined us, and we all merged our efforts to prevent conflicting libraries in the community. To avoid imposing a name, we created a new open-source organization called Arrow.

Arrow’s goal is to facilitate uniform and idiomatic functional programming for everyone in Kotlin, preventing fragmentation and competing libraries.

All of this happened during what I refer to as the pre-0.12.x era, which heavily relied on Typeclasses and Higher Kinded Types - two features that don’t exist in Kotlin but are available in Scala.

While I don’t want to delve too deeply into these technical concepts, I would like to briefly show how they were used in Arrow pre-0.12.x to give you an idea of how far we’ve come.

It’s clear that in the early days, Arrow code looked quite foreign to many Kotlin developers, especially those coming from a Java background where these features didn’t exist.

It’s worth noting that Kotlin’s Structured Concurrency didn’t exist at the time either.

The heavy use of generics and the Kind type to emulate Higher Kinded Types made the code look and feel non-idiomatic. The concept of Higher Kinded Types and the F generic leaked everywhere, as shown in the example of the ListK wrapper used to wrap a Kotlin List.

All of this complexity made it challenging for developers to work with these abstractions, particularly since Kotlin didn’t natively support this functionality and relied on tricks and hacks to make it work.

In this code snippet, we can observe how the GistsApi is used with Mono type from Project Reactor.

However, due to the F abstraction and Higher Kinded Types, we have to use the fix method to “fix” the Higher Kinded representation to access the resulting mono value, and finally we can call the code…

So, what’s the point of all this F and Higher Kinded Type complexity?

F is often used to write code that is independent of the runtime or framework being used. This allows for higher-level, more abstract code that doesn’t make any assumptions about the effect being used.

However, achieving this level of abstraction required a lot of complexity and obscurity. We had to use tricks and hacks like Kind and ListK to work around Kotlin’s lack of support for Higher Kinded Types. Fortunately, this is now a thing of the past.

With the introduction of KotlinX Structured Concurrency in September 2018, we no longer need to rely on these abstractions to the same extent. It’s a big step forward that has made our code simpler and easier to understand. But it’s important to remember where we came from and appreciate the progress we’ve made.

Around 2018-2019, with the announcement and release of Structured Concurrency, we decided to move away from our previous approach and look for new solutions. This led to an overhaul of Arrow, and marked a major milestone with the release of versions 0.12.x and 0.13.x.

In version 0.12.x, we deprecated the use of Higher Kinded Types and Typeclasses, which had powered the machinery in our earlier code examples.

With the release of version 0.13.x, we completely removed these features, and as a result, we were able to support Kotlin Multiplatform in Arrow. This was a significant step forward and demonstrated the benefits of simplifying our code and aligning more closely with the idioms of the language.

In summary, using Arrow for functional programming in Kotlin originally presented a steep learning curve. It required familiarity with the programming style used in Scala or Haskell, resulting in non-idiomatic code that was difficult to comprehend and explain. This also caused friction with IDEA and other tooling at the time.

Moreover, Arrow’s approach attempted to emulate concepts that were not available in Kotlin, leading to a foreign and complex development experience. This went against one of Arrow’s primary goals: to provide functional programming that is idiomatic and accessible to everyone using Kotlin.

Fortunately, with the popularity of KotlinX Structured Concurrency and Kotlin Coroutines, we sought a solution that seamlessly integrates these technologies with Arrow without compromising any of its features. In the following sections, we will explore how we have achieved this, eliminating complexity and delivering a more elegant and intuitive experience for developers.

In this updated definition of our GistsApi, you will notice that all references to F have disappeared, and we have eliminated our custom ListK wrapper as well. Now, everything looks like common Kotlin code, and there is no trace of Arrow anymore.

This is because we can leverage the suspend keyword, which takes care of everything that we need, and with the power of Coroutines, we can seamlessly compose or mix-and-match different functional concepts, as we will see shortly.

As you can see, we have achieved the same goal as before, but in a much simpler way. With a single GistsApi, we can use it in combination with many runtimes such as Project Reactor, ReactiveX or JDK8 Futures.

Furthermore, building an integration layer with any framework or runtime environment that you need to work with is quite easy and straightforward.

In the 1.x series of Arrow, we continued to prioritize making the library more Kotlin idiomatic. Building on the success of Structured Concurrency and coroutines, we went all-in on using suspend and domain-specific languages (DSLs).

One challenge with functional APIs is that they often have complex names and signatures, and FP libraries tend to have a large API surface to allow composing all different shapes of functions. This can be overwhelming for developers, as we experienced with Arrow in Kotlin.

To address this, we gathered feedback from the community and through frequently asked question we were able to truly pinpoint the pain points of Arrow. We focused on adopting a DSL-style approach to functional programming, using more idiomatic names and addressing these common pain points.

This Kotlin code using Either might look familiar if you’ve worked with functional programming languages before. We use Either to handle processing a GithubUser, which can result in either a ProcessingFailed error or a ProcessedUser. To wrap these values in the Either container, we use the left and right constructors.

While this approach is perfectly valid, we believe we can make the code even more idiomatic and readable by using DSLs and other Kotlin language features.

In its DSL form, the same function can be simplified using the eager function to open the DSL block. In the 1.x.x series, we still need to distinguish between suspending and non-suspending DSLs using eager. And we use the ensure DSL to check a predicate. If the predicate is not matched, it raises the typed error of ProcessingFailed. The ensure function works similarly to the Kotlin Std require method, but for typed errors instead of exceptions.

With the DSL, we no longer need to explicitly wrap our results in left or right. This is automatically taken care of by the DSL. This same syntax is available for all types in Arrow and can be easily leveraged for your own types.

Interestingly, we can also leverage this same DSL to remove all wrappers altogether, as we’ll see later. But first, let’s take a look at another example where this pattern is used to simplify APIs.

In functional programming, transforming every element of a List using Either, or a different data type, is typically done using traverse. Although it is an easy concept to learn, many people still find it challenging to use. This is why the joke “the answer is always traverse” is often mentioned, implying that developers forget to use traverse when it’s needed.

The reason behind this is that we usually associate transforming every element of a List with the map function. While every Kotlin developer is familiar with map, we can’t use it to transform every element of a List to an Either since we need an Either containing a List. Hence, we need to use traverse to flip the order of the containers.

In a DSL style, we no longer need to use traverse to transform every element of a List using Either or another data type. Instead, we can simply use map and other standard collection operators. In the DSL style, we “unwrap” the Either by using bind. This approach works for any data type in Arrow or any custom data type you’ve built a DSL for using Arrow.

By using the APIs that we’re all already familiar with from the Kotlin Std and reducing the API surface of Arrow, we’ve significantly reduced the learning curve of doing FP in Kotlin with Arrow. This makes it much more user-friendly and less confusing to figure out which API to use.

We are excited to announce the release of version 1.2.0-RC, which includes the complete API for Arrow 2.0. This release marks a significant milestone as Arrow 2.0 has been able to shed some of its legacy, obscure, and complex APIs inherited from pre-0.12 era. The new version is expected to be released later this year, possibly alongside Kotlin 2.0.

The Arrow 2.0 series boasts a small and uniform API that covers all use-cases from the very beginning, with a greatly reduced learning curve. This makes it much easier for developers to learn and use Arrow. Let’s take a look at some final examples!

This snippet is very similar to the previous one, but with one key difference: it no longer requires the eager function. We no longer have to distinguish between suspending and non-suspending DSLs.

Thanks to Kotlin’s support for DSLs, Arrow DSL functions now have a special color, making them easier to spot and read.

We’re also using Kotlin Contracts in more places to take advantage of the Kotlin compiler’s specialized support. For instance, we’re using ensureNotNull, which is similar to requireNotNull from the Kotlin Standard Library, to check a value and smart-cast it to non-null once it’s been verified.

All these features allow Arrow to provide the same semantics you’d expect from functions in the Kotlin Standard Library, making it feel more idiomatic and natural.

API naming can be a challenging task, and while languages like Haskell have valid reasons to use more obscure API names, this is not the case in Kotlin. This has been especially true since post-0.12.0.

Therefore, we have chosen to follow the conventions of the Kotlin Standard Library for our API naming. In fact, all the names you see here were inspired by Kotlin’s Result type. This approach should make the names feel right at home and familiar to anyone who is already accustomed to the Kotlin Std library.

One of the most exciting features of Arrow’s functional programming in Kotlin is the use of context receivers, which I believe will be a major innovation and a significant quality-of-life improvement for Kotlin developers in general.

With context receivers, we can forget about wrapper types and instead define co-effects. For example, instead of returning an Either of ProcessingFailed or a ProcessedUser when processing a GithubUser, we can add a context that can raise a ProcessingFailed and simply return a ProcessedUser, without the need for an Either wrapper.

Furthermore, we can now access our DSL inside the method body without having to open a DSL block like we did before, as all DSLs in Arrow are built on top of the Raise type.

For example, when we need to traverse a collection and transform every element, we can simply use map without even requiring the use of bind. This is because processing our user now simply returns ProcessedUser.

This same DSL approach is applicable to many patterns.

Here are two small examples:

• One is for Resource safety, where we define an acquire and a finalizer action in a direct style.
• The other is the Saga pattern, where we define an action and a compensating action also in a direct style.

What is so interesting about these DSLs is that they all compose together without ever requiring nesting of wrappers or any special techniques to combine functionalities such as Monad Transformers.

I mentioned earlier that by reducing the API surface, we were able to decrease the binary size without compromising on features. This had a drastic effect on the overall size of Arrow!

Before version 0.12.x, the binary size was slightly over 4,1MB. With the 1.0.0 release, we were able to reduce that to just over 3MB. But with the latest 2.0 version, the binary size is now under 500KB!

This is a really significant improvement because it means that Arrow has a much smaller impact on the size of your projects or libraries. With such a small API surface and binary size, it’s now even easier to include Arrow in your projects without worrying about the size impact.

The evolution of Arrow over the past 10 years has demonstrated the importance of iterative API development. Community feedback and interaction have been critical to this process, allowing us to build a stronger library together.

Learning a new language can be challenging, as you might have experienced. While you can be productive in Kotlin within days, as we become more familiar with a language, new patterns and techniques emerge, sometimes requiring us to let go of old patterns learned in other languages. For example, the rise of Structured Concurrency led to the removal of Arrow’s IO, Higher Kinded Types, and Typeclasses.

While these changes are positive, they also come with a downside: breaking existing code. This a burden I find very heavy to bear, and Arrow worked hard to mitigate the impact of these changes on our users.

The first and perhaps most obvious way to deal with API changes is through Kotlin’s Deprecated annotation, which supports the ReplaceWith parameter. This parameter enables a library to suggest a new alternative API. For example, we can use it to recommend the use of the DSL-based recover method instead of the old handleErrorWith method.

When opening the action window in IntelliJ, the deprecated method will suggest automatically refactoring to the suggested alternative API specified in the ReplaceWith annotation, which can be applied to the entire project. However, we have found that this approach doesn’t cover all cases and may not always work as intended. Additionally, it is not well documented and has some edge cases that need to be considered.

Another promising option for dealing with breaking changes is OpenRewrite, an open-source tool for large-scale automated refactorings developed by Moderne, a revolutionary automated code remediation platform.

Although Kotlin support in OpenRewrite is still experimental, I found it to be very promising and was among the first to use it extensively. The community and maintainers were extremely friendly and helpful.

Let’s take a closer look at how we can use OpenRewrite to refactor our codebase.

To set up OpenRewrite, we only need to configure a few things in Gradle.

Firstly, we need to add the OpenRewrite Gradle plugin to our project’s plugin block.

Next, we add a dependency for the rewrite project, which contains the recipes that describe the various refactorings available.

Finally, we can activate specific recipes by including them in our Gradle configuration. For example, we could activate the recipe for refactoring the Raise DSL.

We can then simply run the rewriteRun task from Gradle.

OpenRewrite will process our project and apply the recipes that we have activated. After the task completes, OpenRewrite will list the processed files and which recipes have been applied to each of them.

OpenRewrite is a truly powerful tool that allows us to automate large-scale refactorings, making code maintenance and modernization a breeze.

While Kotlin support is still experimental, it’s evolving rapidly, and any issues or bug reports are typically fixed within a day or two. This gives me great hope for the future of this tool. It’s currently only supported for the JVM and commonMain sourceSets, so the refactorings cannot be applied to JS and Native sourceSets at the moment.

At the moment, only one recipe has been finished, but more refactorings and improvements are being made. The aim is to ensure that no one will be left behind on older Arrow versions as the community moves towards 2.0.

Equally important to all these changes is having good documentation. With this new version of Arrow, we have a brand new website that emphasizes use-cases and industry patterns instead of just explaining how things work.

The previous website and documentation were quite technical, but we have now re-oriented the documentation to better focus on how Arrow can be used and applied in your projects and applications.

The new Arrow website also provides an extensive migration guide that explains the reasoning behind all changes in detail, as well as what the expected new code should be and why.

This guide can be helpful if you need to do a manual migration or if OpenRewrite or IntelliJ’s ReplaceWith is not working for your project.

At Arrow, we believe that functional programming should be accessible to everyone in Kotlin. To make it easier for users to discover and learn about all the libraries that facilitate this, we’ve added a dedicated ecosystem page to the Arrow website.

We also want to encourage developers who are building functional libraries in Kotlin but aren’t yet listed on the page to reach out to us. By working together, we can create a vibrant community where users can easily find and use the best functional libraries available.

Before I conclude, I want to express my gratitude to the entire Arrow community. To all the contributors who have dedicated their time and effort to make Arrow what it is today, to all the users who trust and utilize Arrow in their projects, and to everyone who provided feedback and supported us along the way. Thank you all for your invaluable contributions and support.

We’re very active on both Slack and GitHub, and you can find a large portion of the Arrow and functional programming community there. I invite all of you to join us if you haven’t already and come chat with us.

Before I wrap up, I’d like to extend my gratitude to Xebia Functional for their continuous support and sponsorship of Arrow throughout the years, and for building the new Arrow website. If you visit their booth, you can get a discount coupon for Arrow and Kotlin courses provided by Xebia Functional.

Also, you can find me at the Arrow booth, next to the Xebia Functional booth. I’ll be there for the entire conference and would be more than happy to answer any questions you may have about Arrow, functional programming, or Kotlin. I hope to see you there!

Thank you all for attending my talk today. I hope you found it informative and useful. I also want to thank the organizers for putting together this fantastic conference and for giving me the opportunity to speak to all of you.

Before you leave, please remember to vote for your favorite talks and speakers. Your feedback helps us improve and deliver better content in the future.

And most importantly, I hope you have a great time at KotlinConf! Enjoy the rest of the sessions and have fun networking with other Kotlin enthusiasts. Thank you!

The code can be found in the Ktor K8s Zero Downtime repository.

Modern programming often requires us to make strong guarantees about how our applications terminate. One way to do this is through the use of structured concurrency, which allows us to reason about parallel processes and how they relate to each other.

In this session, we will learn how to use Arrow’s Resource Domain-Specific Language (DSL) to reason about resource safety in the same way we use structured concurrency to avoid leaking resources. We will also see how this can be combined with KotlinX Coroutines to build complex use cases in a simple and elegant way using Kotlin DSLs.

This talk will cover some of the issues you might encounter when not respecting graceful shutdown, and how we can leverage Structured Concurrency to reason about these issues.

The code can be found in the Ktor Arrow Example repository.

In this session, we are going to build an end-to-end example that uses the patterns we have discussed in the previous session Functional programming in Kotlin with Arrow.kt about functional programming with Kotlin and Arrow.kt.

This time, we will look at the complete workflow from beginning to end, from handling the request, interacting with the database, and sending a response. All in a functional style, of course.

Currently, the only receiver we can define in Kotlin is through extension functions. In the following function, fun List<Int>.sum(): Int, we call List<Int> the receiver of the function.

We can call the above function as a member function on the type List<Int>. Note that for a custom defined extension function it requires an import, but not for most functions from the Kotlin Std.

listOf(1, 2, 3).sum()

What are Context Receivers

With Context Receivers we can introduce additional receivers. Let’s take another function from the Kotlin Std List<A>.sortedWith(comparator: Comparator<A>): List<A> and refactor it to use context receivers.

With context receivers we can redefine this function without the Comparator<A> argument in the parameter list. Where we call Comparator<A> the Context receiver, and List<A> the Extension Receiver

context(Comparator<A>)
fun <A> List<A>.sort(): List<A> =
  sortedWith(this@Comparator)

We can now call this function only from within the context of Comparator<A>, so we can use Kotlin’s scope functions to bring Comparator<A> into context.

with(Comparator.naturalOrder<Int>()) {
  listOf(3, 5, 1).sort() // [1, 3, 5]
}

Comparator.naturalOrder<Int>().run {
  listOf(4, 6, 2, 1).sort() // [1, 2, 4, 6]
}

What happens if we mark a function with context(Comparator<A>), is that it gets an implicit argument that the compiler automatically fills in for us when we’re in its scope. If we look at the JVM signature we will see public <A> List<A> sort(List<A> extension, Comparator<A> comporator), where both receivers are actually regular arguments on the JVM and Kotlin takes care of passing them around for us.

Context Receivers in action

So now that we have an understanding of how Context Receivers work, and what terminology is involved, lets take a look at some cool patterns.

In a previous blogposts we’ve discussed Effect & inline & suspend. Currently, in Kotlin with Arrow we’d write a function like this fun readFile(path: String?): Effect<FileError, Content> = effect { ... }. We’re forced to have a wrapper in the return type, Effect<FileError, Content>, to signal to the type system that the function results in Content but can fail with FileError.

With Context Receivers we can write code inside the effect context, so we can rewrite the function as:

@JvmInline value class Content(val body: List<String>)

sealed interface FileError
@JvmInline value class SecurityError(val msg: String?) : FileError
@JvmInline value class FileNotFound(val path: String) : FileError
object EmptyPath : FileError {
  override fun toString() = "EmptyPath"
}

context(EffectScope<FileError>)
suspend fun readFile(path: String?): Content {
  TODO("All functionality from effect { } available here")
}

So we can now write functions like we love in Kotlin, suspend fun with the happy-path in the return type. There are two ways that we can call this function:

val res: Effect<FileError, Content> = effect {
  readFile("")
}

context(EffectScope<FileError>)
suspend fun allFiles(vararg path: String): List<Content> =
  path.map { readFile(it) }

However, we can also introduce new context receivers as we go along in our program, for example logging.

fun interface Logger {
  suspend fun info(msg: String): Unit
  suspend fun warn(msg: String): Unit
  suspend fun error(msg: String): Unit
}

context(EffectScope<FileError>, Logger)
suspend fun allFiles(vararg path: String): List<Content> {
  info("Processing files")
  return path.map { readFile(it) }
}

val res2: Effect<FileError, Content> = with(Logger(::println)) {
  effect {
    allFiles("path1", "path2")
  }
}

Kotlin Context Receivers enable a popular pattern in FP called “Effect Handlers”, which refers to the technique of constraining functionality to an interface and enabling its functionality by bringing it into scope. In the example above the Logger handler enables the info, warn and error functionality. We can resolve the _EffectScope_ part of our _context_ but leave the `Logger` handler as _context_.

context(Logger)
suspend fun allFilesOrEmpty(vararg path: String): List<Content> =
  effect<FileError, List<Content>> {
    allFiles(*path)
  }.orElse { emptyList() }

Here we were able to make resolve the EffectScope<FileError> context, by wrapping in effect and resolving the error by defaulting to an emptyList. This makes it very easy to solve errors across certain layers in your application, in case you’d want to resolve DbError before consuming them in the service layer.

Let’s look at a quick example of what that might look like:

data class User(val uuid: UUID)
data class DbError(val error: Throwable)

interface Users {
  context(EffectScope<DbError>)
  suspend fun fetchUser(uuid: UUID): User = User(uuid)
  
  companion object Default : Users
}

data class Profile(val uuid: UUID)
data class NetworkError(val error: Throwable)

interface Profiles {
  context(EffectScope<NetworkError>)
  suspend fun fetchProfile(user: User): Profile = Profile(user.uuid)

  companion object Default : Profiles
}

context(EffectScope<DbError>, EffectScope<NetworkError>, Profiles, Users)
suspend fun service(): Profile {
  val user = fetchUser(UUID.randomUUID())
  return fetchProfile(user)
}

Here we have defined a service function, that talks to the Persistence layer, and the Network layer. Both layers have their own error hierarchy, and we would like to keep them separate, so we don’t have to create a single error hierarchy for all layers. This also makes it more explicit from which layers the error came from.

suspend fun program(): Profile? = with(Profiles.Default) {
  with(Users.Default) {
    effect<DbError, Profile?> {
      effect<NetworkError, Profile?> {
        service(Persistence, Network)
      }.tapLeft { networkError -> println(networkError) }
        .orNull()
    }.tapLeft { dbError -> println(dbError) }
      .orNull()
  }
}

Conclusion

Context receivers give us an elegant way of constraining functions to which context they can be called from. It allows for composing context without requiring inheritance, and bringing them into scope/context by using Kotlin’s scope functions.

The biggest improvement I’m still hoping for is with that allows multiple parameters, that would allow us to flatten a lot of code.

with(Users.Default, Profiles.Default) {
  // Users & Profiles available here
}

After having tried context receivers I cannot wait for them to become stable, and will probably start using them in all my toy projects already and perhaps even in some production code.

Thanks for reading my blog, and thoughts on Kotlin’s context receivers. Hope you enjoyed it!

A Gradle project with the code from this blogpost can be found on GitHub here. If you want to try it out do not forget to set up your IDEA as well, instructions can be found here.

What I love most about Kotlin are some small features, that make the language very elegant. Such as smart-casting (contracts), suspend, receiver/extension functions & delegation.

But at this point in time my favorite feature is the combination of inline and suspend.

Inline recap

The inline feature is a feature everyone in Kotlin is using regularly, knowingly or not. A function can be marked inline when it has lambdas as parameters, also known as Higher Order Functions. A Higher Order Function is a function that takes other functions as parameters, or returns a new function. We call map a HOF since it takes another function transform as a parameter.

inline fun <A, B> Iterable<A>.map(transform: (A) -> B): List<B>

Iterable.map is probably the best known one. It maps all elements in a List of type A to a List of type B using the transform function (A) -> B. When these patterns were first introduced, there was some concern about performance due to for-loops being more efficient at byte-code level.

inline is a Kotlin compiler feature that can completely get rid of that concern. If we look at the function body of Iterable.map we see a for-loop.

So let’s see what the bytecode of listOf(1, 2).map(Int::toString) looks like:

val destination = ArrayList<String>()
for (item in listOf(1, 2)) {
  destination.add(item.toString())
}  

It looks the same as if we’d have written the for-loop by hand. Awesome, we can write high-level code without losing any of the benefits of writing low-level code!

However, inlining is not unique to Kotlin. Many other languages also support inlining.

suspend recap

suspend in Kotlin is extremely powerful, and it has many interesting use-cases. Suspension allows us to wrap callbacks, and write imperative code over callbacks.

suspend fun example(): Int = suspendCoroutine { continuation ->
    callbackCode { int, error ->
      if(error == null) continuation.resume(int)
      else continuation.resumeWithException(error)
    }
}

suspend fun twice() = example() + example()

In concurrency that means we can work with JVM Future or JS Promise without having to write callback based code. This is very powerful, as we can now write “regular imperative” to describe powerful async workflows, but it also allows for very interesting use-cases outside of concurrency. A great example is discussed at length in my previous blogpost.

The compiler also verifies that suspending code can only be called from other suspending code, so that you can never accidentally call suspending code from places that do not support it.

suspend fun example(): Int = suspendCoroutine { continuation ->
  callbackCode { int, error ->
    if(error == null) continuation.resume(int)
    else continuation.resumeWithException(error)
  }
}

fun once() = example()
// Cannot call suspending fun example from non-suspending fun once

The suspend functionality is also known as Continuation Passing Style, which means that the compiler can automatically pass Continuation between function for us. It does so in an extremely efficient manner, and that allows us to write callback based code in an immediate manner.

suspend + inline

Now that we’ve discussed how suspend and inline works, we can take a look at my favourite Kotlin feature which is support between suspend + inline!

Let’s start from a non-inline example.

fun <A, B> Iterable<A>.map(transform: (A) -> B): List<A> = TODO()

suspend fun fetchUser(id: Int): Result<Int> = Result.success(id)

suspend fun example5() = listOf(1, 2, 3).map { fetchUser(it) }

This fails to compile with Suspension functions can be called only within coroutine body even though our example function is marked as suspend. This is because we’re creating a non-suspending lambda transform to pass to the map function, and from within that non-suspending lambda we cannot call the suspend function fetchUser.

But when we mark this function inline it suddenly works. That’s because it’s not creating a non-suspending lambda anymore, but instead in our example we now have again our optimised for-loop and the resolve function is being called from within.

Since the compiler knows that the non-suspending lambda will not exist due to inline, it allows suspend to be called from inside. This is similar to smart-casting, the compiler has certain knowledge about the program that it allows us to do something that is regularly not allowed.

When designing libraries, or higher order functions it’s important to keep inline in mind. Not only for performance reasons, but also for suspend.

It’s one of the features of the Kotlin Compiler that is easily overseen, since it works so transparently.

More examples

Let’s see some other interesting examples, and some caveats.

fun process(i: Int): Either<String, Int> = i.right()

val res: List<Either<String, Int>> = (0..9).map { process(it) }

In the above example we have a list of 10 items, and want to process every element. When we use Iterable.map we get List<Either<String, Int>>, which is not that interesting. Actually we’d want Either<String, List<Int>>, either the first error, or all processed results.

If you’re familiar with Arrow, or the Cont blogpost you’ve seen a DSL where we can use suspend to work over Either.Right in a nicer way. Since we can combine inline with suspend, we can extract the value out of Either inside Iterable.map.

fun process(i: Int): Either<String, Int> = i.right()

val res: Either<String, List<Int>> = either.eager<String, Int> {
  (0..9).map { process(it).bind() }
}

Now we get the result we’re interested in Either<String, List<Int>>, either the first error, or all processed results. If you’re familiar with functional programming, you might see that this replaces the Traverse and Monad hierarchy. So when practicing FP in Kotlin, we can forget about a lot of complexity since it’s possible to provide powerful semantic sugar through suspension. Note: here we use either.eager which uses suspend but doesn’t allow any foreign suspending code.

Another interesting example is Sequence from Kotlin Std, where suspend is used to offer a powerful DSL for building lazy sequences. sequence offers suspend fun yield(a: A): Unit which can be used to emit/yield a value to the Sequence. Below we can use it to implement flatMap through suspend and inline.

fun <A, B> Sequence<A>.flatMap(transform: (A) -> Sequence<B>): Sequence<B> =
  sequence {
    forEach { a: A ->
      f(a).forEach { b: B -> yield(b) }
    }
  }

1. Open a sequence DSL block
2. Loop over all elements in Sequence<A>
3. transform A into Sequence<B>
4. Loop over all elements of Sequence<B> and yield them

The reason we can call yield from the double nested forEach is because both are marked as inline, and thus we can call yield safely from inside.

Caveat

This pattern is sadly not foolproof, there are some gotchas in terms of resource-safety and cancellation.

coroutineScope {
  listOf(1, 2, 3)
    .map { async { ... } }
    .map { it.await() }
}

This code is not cancellation safe, if a task fails it can never cancel the other tasks ahead of it in the list. Due to awaiting the Deferred in the order of the list instead of all at the same time. To await all the Deferred at the same time you should use awaitAll.

coroutineScope {
  listOf(1, 2, 3)
    .map { async { ... } }
    .awaitAll()
}

Conclusion

The way these two features work so seamlessly together is in my opinion an underappreciated feature of the Kotlin compiler and one of my favourites.

An upcoming feature in Kotlin, Context receivers, is going to make this pattern more powerful. More on that in a next post.

Thanks for reading my blog on my favorite Kotlin feature(s). I hope you enjoyed it, and a Happy New Year!

Arrow strives to be a great functional companion library for Kotlin. This is not just a matter of style: with this functional style we can describe complex patterns in an easy way, and we aim to showcase a few of them in the context of a Ktor project. We’ll discuss how to handle retries using circuit breakers and schedulers, and how to do validation in a powerful fashion. Along the way, we’ll show a bit of other Arrow projects, like optics and analysis.

Kotlin has support for CPS, or Continuation Passing Style, which means that the compiler has a style in which it can automatically pass Continuation between functions that are marked with suspend.

This means that in every suspend fun we have access to an implicit Continuation parameter through suspendCoroutine { effect: Continuation -> ... }, or any of it’s lower level variants.

This Continuation only has a single type parameter Continuation<A>, and the result of running this suspend/Continuation based program is either A, or Throwable. This is represented by the Result<A> type from the Kotlin Standard Library.

In functional programming we often prefer to interrupt the program with a typed value R instead of Throwable. So we’d like a Continuation that instead of only being able to result in A | Throwable can result in A | R | Throwable, you can never escape that a program can also result in Throwable and suspend explicitly takes Throwable into account to operate correctly.

To achieve this we define a new type interface Effect<R, A>, which represents a function of suspend () -> A that can fail with R (and Throwable). It’s defined by suspend fun <B> fold(f: suspend (R) -> B, g: suspend (A) -> B): B.

What is interesting about the Effect<R, A> type is it’s a suspend function fold, so it doesn’t rely on any wrappers such as Either, Ior or Validated. Only when fold is called it will create a Continuation and run the computation, this means Effect can encode side-effects as pure values. This makes Effect<R, A> a very efficient generic runtime.

To construct a Effect<R, A> we simply call the effect<R, A> { } constructor, which exposes a rich syntax through the lambda receiver suspend EffectScope<R>.() -> A.

Working with Effect<R, A>

Let’s write a small program to read a file from disk, and instead of having the program work exception based we want to turn it into a polymorphic type-safe program.

We start by defining a small function that accepts a String, and does some simply validation to check that the path is not empty. If the path is empty, we want to program to result in EmptyPath. So we’re immediately going to see how we can raise an error of any arbitrary type R by using the function shift. The name shift comes shifting (or changing, especially unexpectedly), away from the computation and finishing the Continuation with R.

object EmptyPath

fun readFile(path: String): Effect<EmptyPath, Unit> = effect {
  if (path.isEmpty()) shift(EmptyPath) else Unit
}

Here we see how we can define a Effect<R, A> which has EmptyPath for the shifted type R, and Unit for the success type A.

Patterns like validating a Boolean is very common, and the Effect DSL offers utility functions like kotlin.require and kotlin.requireNotNull. They’re named ensure and ensureNotNull to avoid conflicts with the kotlin namespace. So let’s rewrite the function from above to use the DSL instead.

fun readFile2(path: String?): Effect<EmptyPath, Unit> = effect {
  ensureNotNull(path) { EmptyPath }
  ensure(path.isNotEmpty()) { EmptyPath }
}

You can get the full code here.

Now that we have the path, we can read from the File and return it as a domain model Content. We also want to take a look at what exceptions reading from a file might occur FileNotFoundException & SecurityError, so lets make some domain errors for those too. Grouping them as a sealed interface is useful since that way we can resolve all errors in a type safe manner.

@JvmInline
value class Content(val body: List<String>)

sealed interface FileError
@JvmInline value class SecurityError(val msg: String?) : FileError
@JvmInline value class FileNotFound(val path: String) : FileError
object EmptyPath : FileError {
  override fun toString() = "EmptyPath"
}

We can finish our function, but we need to refactor our return value from Unit to Content and our error type from EmptyPath to FileError.

fun readFile(path: String?): Effect<FileError, Content> = effect {
  ensureNotNull(path) { EmptyPath }
  ensure(path.isNotEmpty()) { EmptyPath }
  try {
    val lines = File(path).readLines()
    Content(lines)
  } catch (e: FileNotFoundException) {
    shift(FileNotFound(path))
  } catch (e: SecurityException) {
    shift(SecurityError(e.message))
  }
}

The Effect returned by readFile function represents a suspend fun that will return:

• the Content of a given path
• a FileError
• An unexpected fatal Throwable (OutOfMemoryException)

The returned Effect<FileError, Content> can executed into a value.

suspend fun test() {
  readFile("").toEither() shouldBe Either.Left(EmptyPath)
  readFile("knit.properties").toValidated() shouldBe  Validated.Invalid(FileNotFound("knit.properties"))
  readFile("gradle.properties").toIor() shouldBe Ior.Left(FileNotFound("gradle.properties"))
  readFile("README.MD").toOption { None } shouldBe None

  readFile("build.gradle.kts")
    .fold({ _: FileError -> null }, { it })
    .shouldBeInstanceOf<Content>()
    .body.shouldNotBeEmpty()
}

You can get the full code here.

Handling errors

Handling errors of type R is the same as handling errors for any other data type in Arrow. Effect<R, A> offers handleError, handleErrorWith, redeem, redeemWith and attempt.

As you can see in the examples below it is possible to resolve errors of R or Throwable in Effect<R, A>.

val failed: Effect<String, Int> =
  effect { shift("failed") }

val resolved: Effect<Nothing, Int> =
  failed.handleError { it.length }

val newError: Effect<List<Char>, Int> =
  failed.handleErrorWith { str ->
    effect { shift(str.reversed().toList()) }
  }

val redeemed: Effect<Nothing, Int> =
  failed.redeem({ str -> str.length }, ::identity)

val captured: Effect<String, Result<Int>> = effect<String, Int> {
  1
}.attempt()

suspend fun test() {
  failed.toEither() shouldBe Either.Left("failed")
  resolved.toEither() shouldBe Either.Right(6)
  newError.toEither() shouldBe Either.Left(listOf('d', 'e', 'l', 'i', 'a', 'f'))
  redeemed.toEither() shouldBe Either.Right(6)
  captured.toEither() shouldBe Either.Right(Result.success(1))
}

You can get the full code here.

Effect<E, A> and Either<E, A>

We’ve briefly mentioned the difference between Either<E, A> and Effect<E, A> but let’s compare them in some more detail.

Either<String, Int> represents a value of either a String or an Int, while Effect<String, Int> represents a function that will result in an Int, interrupt with a String, or throw a Throwable.

So Effect<String, Int> can represent a function that results in either a String, an Int or throw a Throwable. That is exactly what the either DSL of Arrow Core does, and let’s see what the implementation might look like with Effect.

Since Effect<E, A> can interrupt with a String we can implement bind for Either<String, Int>. Here we implement bind how it’s implemented in the library, it’s defined in EffectScope<E> which is the DSL available inside effect { }.

interface EffectScope<E> {
  suspend fun <A> shift(e: e): A
  
  suspend fun <A> Either<E, A>.bind(): A =
    when(this) {
      is Either.Right -> value
      is Either.Left -> shift(value)
    }
}

When we encounter Either.Right we can simply return the value, and when we encounter Either.Left we short-circuit (interrupt) the function. This allows us to use the effect { } DSL similarly to how we use either { } from Arrow Core.

suspend fun test() {
  effect<String, Int> {
    val x: Int = 1.right().bind()
    val y: Int = shift("Failed")
  }.toEither() shouldBe Either.Left("failed")
}

You can get the full code here.

At the end of our DSL we call toEither to finally turn the Effect result into Either. So we can re-define the either DSL from Arrow with following function:

suspend fun <E, A> either(block: suspend EffectScope<E>.() -> A): Either<E, A> = effect(block).toEither()

So we can consider Effect a higher abstraction to write programs that can result in Either, Validated, Option, Ior, Result, etc

Structured Concurrency

Effect<R, A> can automatically operate with KotlinX’s Structured Concurrency by leveraging kotlin.cancellation.CancellationException introduced in Kotlin 1.4. It’s used by shift to raise error values of type R inside the Continuation since it effectively cancels/short-circuits it. For this reason shift adheres to the same rules as Structured Concurrency

For more details see the Effect<R, A> documentation

Conclusion

The Effect type will soon become available in Arrow! There is also RestrictedCont which exposes the same functionality without requiring suspend.

Thank you for reading the first informal (test) blog on my Github Pages, I hope you enjoyed it! Feel free to reach out on KotlinLang Slack for feedback or questions.

Stay tuned for more informal blogs!