Welcome to the Kotlin Notebook!
Basics
Let's start with something simple: execute a cell with a simple arithmetics and immediately see the result.
2 + 2
Execution results could be saved in variables and reused
val result = 3 * 14
result
(Out[3] as Int) / 2
Variables defined in the notebook can lose their nullability if they are not actually nulls
val a1: Int = 1
val a2: Int? = 2
val a3: Int? = null
a1 + a2 // OK, a2 was converted to Int
a1 + a3 // compile-time error
Rich outputs
Outputs might be not only plain text. They could also be images and HTML. HTML can contain CSS and JavaScript.
HTML("""
<p>Counter: <span id="ctr">0</span> <button onclick="inc()">Increment</button></p>
<script>
function inc() {
let counter = document.getElementById("ctr")
counter.innerHTML = parseInt(counter.innerHTML) + 1;
}
</script>
""")
NB! If your outputs contain JS, notebook should be marked as trusted.
Images could be loaded by link. In this case, it won't show if the link breaks or if you lose Internet connection
%use lib-ext(0.11.0-398)
Image("https://kotlinlang.org/docs/images/kotlin-logo.png", embed = false).withWidth(300)
You can also embed images. In this case they will stay in the notebook forever
val kotlinMascot = Image("https://blog.jetbrains.com/wp-content/uploads/2023/04/DSGN-16174-Blog-post-banner-and-promo-materials-for-post-about-Kotlin-mascot_3.png", embed = true).withWidth(400)
kotlinMascot
The cell can also have several outputs, to achieve it use DISPLAY()
function
DISPLAY(HTML("<h2>Kodee is back!</h2>"))
DISPLAY(kotlinMascot)
With Kotlin Notebook, you can also render LaTeX formulae
LATEX("c^2 = a^2 + b^2 - 2 a b \\cos\\alpha")
You can also output BufferedImage's. They are embedded into notebook
import java.awt.Color
import java.awt.image.BufferedImage
val width = 300
val height = width
val image = BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB)
val graphics = image.createGraphics()
graphics.background = Color.BLACK
graphics.clearRect(0, 0, width, height)
graphics.setRenderingHint(
java.awt.RenderingHints.KEY_ANTIALIASING,
java.awt.RenderingHints.VALUE_ANTIALIAS_ON
)
graphics.color = Color.WHITE
graphics.fillRect(width / 10, height * 8 / 10, width * 10 / 20, height / 10)
graphics.dispose()
image
Generally, you can display any output using mimeResult
function.
We're using Jupyter approach for outputs.
mimeResult(
MimeTypes.PLAIN_TEXT to "JetBrains logo",
MimeTypes.HTML to "<b>JetBrains</b> logo"
)
Using libraries and dependencies
You can always turn on source and binary dependencies of a current project. To do it, use the corresponding button in the toolbar.
import java.io.File
import javax.imageio.ImageIO
fun showScreenshot(id: Any) {
DISPLAY(ImageIO.read(File("screenshots/screenshot$id.png")))
}
showScreenshot(1)
It is also possible to set these options for the newly created notebooks
showScreenshot(2)
Of course, you can depend on various JVM libraries even if you don't have a project in the current scope.
The simpliest option we offer is to use predefined library descriptors, you can find which are available using :help
command or here.
:help
Kernel version: 0.11.0.381
Kotlin version: 1.8.20
JVM version: 11
Commands:
:help - display help
:classpath - show current classpath
:vars - get visible variables values
Magics:
%use - injects code for supported libraries: artifact resolution, default imports, initialization code, type renderers
Usage: %use klaxon(5.5), lets-plot
%trackClasspath - logs any changes of current classpath. Useful for debugging artifact resolution failures
Usage: %trackClasspath [on|off]
%trackExecution - logs pieces of code that are going to be executed. Useful for debugging of libraries support
Usage: %trackExecution [all|generated|off]
%useLatestDescriptors - use latest versions of library descriptors available. By default, bundled descriptors are used. Note that default behavior is preferred: latest descriptors versions might be not supported by current version of kernel. So if you care about stability of the notebook, avoid using this line magic
Usage: %useLatestDescriptors [on|off]
%output - output capturing settings
Usage: %output --max-cell-size=1000 --no-stdout --max-time=100 --max-buffer=400
%logLevel - set logging level
Usage: %logLevel [off|error|warn|info|debug]
Supported libraries:
kravis (https://github.com/holgerbrandl/kravis) - Kotlin grammar for data visualization
plotly (https://github.com/mipt-npm/plotly.kt) - [beta] Plotly.kt jupyter integration for static plots.
exposed (https://github.com/JetBrains/Exposed) - Kotlin SQL framework
lets-plot-gt (https://github.com/JetBrains/lets-plot-kotlin) - Lets-Plot visualisation for GeoTools toolkit
deeplearning4j (https://github.com/eclipse/deeplearning4j) - Deep learning library for the JVM
kraphviz (https://github.com/nidi3/graphviz-java) - Graphviz wrapper for JVM
serialization (https://github.com/Kotlin/kotlinx.serialization) - Kotlin multi-format reflection-less serialization
krangl (https://github.com/holgerbrandl/krangl) - Kotlin DSL for data wrangling
roboquant (https://roboquant.org) - Algorithmic trading platform written in Kotlin
kmath (https://github.com/mipt-npm/kmath) - Experimental Kotlin algebra-based mathematical library
gral (https://github.com/eseifert/gral) - Java library for displaying plots
webtau (https://github.com/testingisdocumenting/webtau) - WebTau end-to-end testing across layers
deeplearning4j-cuda (https://github.com/eclipse/deeplearning4j) - Deep learning library for the JVM (CUDA support)
gradle-enterprise-api-kotlin (https://github.com/gabrielfeo/gradle-enterprise-api-kotlin) - A library to use the Gradle Enterprise API in Kotlin scripts or projects
coroutines (https://github.com/Kotlin/kotlinx.coroutines) - Asynchronous programming and reactive streams support
mysql (https://github.com/mysql/mysql-connector-j) - MySql JDBC Connector
lets-plot-dataframe (https://github.com/JetBrains/lets-plot-kotlin) - A bridge between Lets-Plot and dataframe libraries
multik (https://github.com/Kotlin/multik) - Multidimensional array library for Kotlin
reflection (https://kotlinlang.org/docs/reflection.html) - Imports for Kotlin Reflection
smile (https://github.com/haifengl/smile) - Statistical Machine Intelligence and Learning Engine
kandy-echarts (https://github.com/Kotlin/kandy) - Kotlin plotting DSL for Apache ECharts
spark-streaming (https://github.com/JetBrains/kotlin-spark-api) - Kotlin API for Apache Spark Streaming: scalable, high-throughput, fault-tolerant stream processing of live data streams
rdkit (https://www.rdkit.org/) - Open-Source Cheminformatics Software
combinatoricskt (https://github.com/shiguruikai/combinatoricskt) - A combinatorics library for Kotlin
lib-ext (https://github.com/Kotlin/kotlin-jupyter) - Extended functionality for Jupyter kernel
khttp (https://github.com/jkcclemens/khttp) - HTTP networking library
plotly-server (https://github.com/mipt-npm/plotly.kt) - [beta] Plotly.kt jupyter integration for dynamic plots.
londogard-nlp-toolkit (https://github.com/londogard/londogard-nlp-toolkit) - A Natural Language Processing (NLP) toolkit for Kotlin on the JVM
lets-plot (https://github.com/JetBrains/lets-plot-kotlin) - ggplot-like interactive visualization for Kotlin
openai (https://openai.com/blog/chatgpt) - OpenAI API for Jupyter Notebooks
fuel (https://github.com/kittinunf/fuel) - HTTP networking library
kalasim (https://www.kalasim.org) - Discrete event simulator
kaliningraph (https://github.com/breandan/kaliningraph) - Graph library with a DSL for constructing graphs and visualizing the behavior of graph algorithms
kandy (https://github.com/Kotlin/kandy) - Kotlin plotting DSL for Lets-Plot
kotlin-dl (https://github.com/Kotlin/kotlindl) - KotlinDL library which provides Keras-like API for deep learning
kotlin-statistics (https://github.com/thomasnield/kotlin-statistics) - Idiomatic statistical operators for Kotlin
jdsp (https://github.com/psambit9791/jDSP) - Java library for signal processing
default - Default imports: dataframe and Lets-Plot libraries
dataframe (https://github.com/Kotlin/dataframe) - Kotlin framework for structured data processing
biokotlin (https://bitbucket.org/bucklerlab/biokotlin) - BioKotlin aims to be a high-performance bioinformatics library that brings the power and speed of compiled programming languages to scripting and big data environments.
klaxon (https://github.com/cbeust/klaxon) - JSON parser for Kotlin
datetime (https://github.com/Kotlin/kotlinx-datetime) - Kotlin date/time library
spark (https://github.com/JetBrains/kotlin-spark-api) - Kotlin API for Apache Spark: unified analytics engine for large-scale data processing
Let's try kotlinx.serialization
library
%use serialization
It allows us to serialize and deserialize classes.
import kotlinx.serialization.Serializable
@Serializable
class User(val firstName: String, val lastName: String)
val bob = User("Alex", "Green")
Json { prettyPrint = true }.encodeToString(bob)
"firstName": "Alex",
"lastName": "Green"
}
It is possible to specify descriptors' and underlying libraries' versions, write and contribute your own descriptors and much more. You can read about it here
Also, you can add dependencies for any Maven libraries you want
USE {
repositories {
// Any additional repositories. Maven central is already included
// maven("<url>")
}
dependencies {
// Here we add kandy plotting library
implementation("org.jetbrains.kotlinx:kandy-lets-plot:0.4.3")
}
// Sometimes library integration are loaded transitively and you don't want them to do it.
discardIntegrationTypeNameIf {
it.startsWith("org.jetbrains.kotlinx.dataframe.")
}
}
Renderers and other integration features
Let's try to conduct an experiment: we'll throw 50 dices and count the sum of points on them. Then, we'll repeat this experiment some reasonable number of times and plot the distribution using kandy library we've just loaded.
import kotlin.random.Random
fun diceNTimesSum(n: Int): Int {
return (1..n).sumOf { Random.nextInt(1, 7) }
}
val experimentData = (1..100000).map { diceNTimesSum(50) }.groupBy { it }.mapValues { it.value.size }.entries.sortedBy { it.key }
val experimentX = experimentData.map { it.key }
val experimentY = experimentData.map { it.value }
val gaussPlot = plot {
bars {
x(experimentX)
y(experimentY)
}
}
gaussPlot
gaussPlot::class
As you can see kandy's Plot
object was rendered to some image. That's because kandy
library defines a renderer for this type of objects. We also can define renderers ourselves.
%use dataframe
val bob = User("Bob", "Brown")
bob
User
isn't a data class, that's why it was rendered this way.
USE {
// Match is based on runtime type here, beware of type erasure
render<User> { listOf(it).toDataFrame() }
}
bob
We can also use full syntax for defining renderers. It's verbose but let you do many things
class User2(val name: String)
USE {
addRenderer(object : RendererHandler {
override fun replaceVariables(mapping: Map<String, String>): RendererHandler {
return this
}
override val execution: ResultHandlerExecution
get() = ResultHandlerExecution { host, res ->
FieldValue("Value of ${res.name} is a user with name ${(res.value as User2).name}", null)
}
override fun accepts(value: Any?): Boolean {
return value != null && value::class == User2::class
}
})
}
User2("Felix")
What do the libraries bring except for dependencies and renderers? First of all, default imports. These imports are implicitly added to all subsequent cells.
fun loadFileFromGitHub(repositoryUrl: String, filePath: String): String {
val rawUrl = "$repositoryUrl/raw/master/$filePath"
val url = URL(rawUrl)
val connection = url.openConnection()
connection.setRequestProperty("Accept", "application/vnd.github.v3.raw")
val inputStream = connection.getInputStream()
val content = inputStream.bufferedReader().use { it.readText() }
return content
}
fun loadDescriptor(name: String) {
val text = loadFileFromGitHub("https://github.com/Kotlin/kotlin-jupyter-libraries", "$name.json")
DISPLAY(MIME(
"text/markdown" to "```json\n$text```"
))
}
Notice imports
section in the following descriptor:
loadDescriptor("kaliningraph")
{
"link": "https://github.com/breandan/kaliningraph",
"description": "Graph library with a DSL for constructing graphs and visualizing the behavior of graph algorithms",
"dependencies": [
"com.github.breandan:kaliningraph:0.1.4"
],
"imports": [
"edu.mcgill.kaliningraph.*",
"edu.mcgill.kaliningraph.matrix.*",
"edu.mcgill.kaliningraph.circuits.*",
"org.ejml.data.*",
"org.ejml.kotlin.*"
],
"renderers": {
"edu.mcgill.kaliningraph.LabeledGraph": "HTML(($it as edu.mcgill.kaliningraph.Graph<*, *, *>).html())",
"edu.mcgill.kaliningraph.circuits.Gate": "HTML(($it as edu.mcgill.kaliningraph.circuits.Gate).graph.html())",
"edu.mcgill.kaliningraph.circuits.NFunction": "HTML(($it as edu.mcgill.kaliningraph.circuits.NFunction).graph.html())",
"edu.mcgill.kaliningraph.circuits.ComputationGraph": "HTML(($it as edu.mcgill.kaliningraph.Graph<*, *, *>).html())",
"edu.mcgill.kaliningraph.matrix.BMat": "HTML(\"<img src=\\\"${($it as edu.mcgill.kaliningraph.matrix.BMat).matToImg()}\\\"/>\")",
"edu.mcgill.kaliningraph.matrix.BSqMat": "HTML(\"<img src=\\\"${($it as edu.mcgill.kaliningraph.matrix.BSqMat).matToImg()}\\\"/>\")"
}
}
You might have noticed that almost every descriptor in Kotlin/kotlin-jupyter-libraries
contains link and description.
They are used to build :help
command output and our README
Creating an integration
So, we learned the concept of the integration. It is a wrapper on top of the usual Kotlin library that simplifies and elevates the experience of using this library in the notebook.
Comprehensive guide to writing library integrations can be found here. To inspire you, there is an example of the dataframe
integration below.
%use dataframe
val df = DataFrame.read("https://raw.githubusercontent.com/mwaskom/seaborn-data/master/iris.csv")
Let's try to investigate the data that we have just read.
df.petal_length
df.filter { petal_width >= 1.5 && petal_length < 4.5 }
There is also a special variable in the integration to manage display options. For example, we can limit number of displayed rows.
dataFrameConfig.display.rowsLimit = 5
df
Let's now see what's happening there under the hood. First, we enable debugging logging that will print the code that is executed under the hood.
%trackExecution
Then, we define some other dataframe.
val df1 = dataFrameOf("A", "B", "C")(1, "Str1", null, 2, "Str2", 3)
val df1 = dataFrameOf("A", "B", "C")(1, "Str1", null, 2, "Str2", 3)
Executing:
@DataSchema
interface _DataFrameType1
val ColumnsContainer<_DataFrameType1>.A: DataColumn<Int> @JvmName("_DataFrameType1_A") get() = this["A"] as DataColumn<Int>
val DataRow<_DataFrameType1>.A: Int @JvmName("_DataFrameType1_A") get() = this["A"] as Int
val ColumnsContainer<_DataFrameType1?>.A: DataColumn<Int?> @JvmName("Nullable_DataFrameType1_A") get() = this["A"] as DataColumn<Int?>
val DataRow<_DataFrameType1?>.A: Int? @JvmName("Nullable_DataFrameType1_A") get() = this["A"] as Int?
val ColumnsContainer<_DataFrameType1>.B: DataColumn<String> @JvmName("_DataFrameType1_B") get() = this["B"] as DataColumn<String>
val DataRow<_DataFrameType1>.B: String @JvmName("_DataFrameType1_B") get() = this["B"] as String
val ColumnsContainer<_DataFrameType1?>.B: DataColumn<String?> @JvmName("Nullable_DataFrameType1_B") get() = this["B"] as DataColumn<String?>
val DataRow<_DataFrameType1?>.B: String? @JvmName("Nullable_DataFrameType1_B") get() = this["B"] as String?
val ColumnsContainer<_DataFrameType1>.C: DataColumn<Int?> @JvmName("_DataFrameType1_C") get() = this["C"] as DataColumn<Int?>
val DataRow<_DataFrameType1>.C: Int? @JvmName("_DataFrameType1_C") get() = this["C"] as Int?
val ColumnsContainer<_DataFrameType1?>.C: DataColumn<Int?> @JvmName("Nullable_DataFrameType1_C") get() = this["C"] as DataColumn<Int?>
val DataRow<_DataFrameType1?>.C: Int? @JvmName("Nullable_DataFrameType1_C") get() = this["C"] as Int?
df1.cast<_DataFrameType1>()
Executing:
val df1 = res54
As you can see, a marker interface _DataFrameType1
is created and property accessors are generated for it. Let's check what's the type of df1
now:
::df1
::df1
So, df1
now is not simply a DataFrame<*>
, it's DataFrame<_DataFrameType1>
. That's exactly what allows us to statically resolve defined property accessors on it.
%trackExecution off
API that DataFrame uses to achieve all of these is open, and you can try it yourself! The code of the integration is available here
Inter-cell API
To use Kotlin Notebook API you don't necessarily need to create a separate file with integration. As it was shown above, you can place in USE { }
call the same integration that you would
place in the standalone library or JSON descriptor. However, notebook offers API to get some information about the current notebook session and to set it up.
The main entry point is notebook
: it allows you to investigate what cells were already executed, what libraries were loaded, what renderers and variable processors are loaded and gives the ability to unload them.
Read the documentation of the Notebook
interface for reliable and actual information about this API
notebook.cellsList.take(5).map { "> " + it.code }.joinToString("\n")
> val result = 3 * 14
> result
> (Out[3] as Int) / 2
> val a1: Int = 1
val a2: Int? = 2
val a3: Int? = null
notebook.kernelVersion
notebook.cellVariables
Also, a couple of options is available with SessionOptions
object.
SessionOptions.resolveSources // could be switched off to speed up dependencies loading process
SessionOptions.resolveMpp // could be switched on to resolve MPP libraries such as kotlinx-serialization from "universal" Maven coordinates
Sharing the notebooks
Essentially, Kotlin notebooks are Jupyter notebooks. It means they could be opened, edited and run with any Jupyter client such as Jupyter Notebook, Jupyter Lab and Datalore. We plan to integrate with Datalore better in the future. The only limitation is that project dependencies will be not included into the notebook in other clients. However, you can build the logic based on Jupyter client type:
if (notebook.jupyterClientType != JupyterClientType.KOTLIN_NOTEBOOK) {
// load substitutive dependencies
}
Notebooks could be also loaded and viewed on GitHub (including gists). Limitation there is that JS isn't executed in the outputs.
To overcome it in kandy
, we add extra SVG output that is hidden from JS.
Kotlin Notebooks could be also converted to some other format using nbconvert tool.
What's next?
Kotlin Notebook project is experimental and actively developed. Use the latest version of plugin to be in sync. We eagerly need your stories and feedback: share them in the dedicated #notebooks Slack channel or in our issue tracker.
See you in the next cell!