JVM Weekly #3 - Microstream, State of Clojure and Kotlin's Roadmap
JVM Weekly #3 - Microstream, State of Clojure and Kotlin's Roadmap
1. What is MicroStream?
We wrapped up our previous edition on the new version of Micronaut, and this week we have a follow-up to that. While we reported that the new version of the framework will include support for MicroStream in the form of a Micronaut module, there is another announcement: MicroStream had announced that it had joined the Micronaut Foundation. The tool has become a silver sponsor of the framework and both plan to strengthen the collaboration further. So let's revisit the topic, and in a slightly longer form, talk about what MicroStream is and what problems it answers.
Do you know such a term as object-relational impedance mismatch? Behind this wonderful term, you will find a problem as old as ORM frameworks (which should be expanded as Object-Relational Mapping). It stems from the fact that the object model and the relational model are strongly incompatible with each other, and even at the level of the concepts themselves.
After Wikipedia, freely translated:
Objects relate to each other, forming what in mathematical terms is referred to as a directed graph (a network containing loops and cycles). Relational schemas, on the other hand, are tabular and based on relational algebra, which defines related heterogeneous tuples (groups of data fields in a "row" with different types for each field), where relationships are always reversible (foreign keys can be traced backwards because INNER JOIN is symmetric), a feature more similar to undirected graphs.
ORMs have always had to deal with this mismatch, for better or worse, by modeling data in a very skewed way. Often to the point of forcing a specific way of writing code to suit the framework, or creating intermediate layers. All that ended up in a recent fashion for using frameworks as close to SQL as possible.
MicroStream approaches the problem differently. Instead of using a relational model, it serializes objects into a so-called ObjectGraph. This way, no redundant conversions are required, the CPU doesn't get consumed and we save the planet altogether.
They also boast that their DataGraph can be persisted both in-memory, in an external cache, and in any database that accepts binary data. The use of indexes for data retrieval has been replaced by the creation of intelligent subgraphs, allowing to efficiently reach a specific subset of the entire memory dump. The tool is supposed to be small and very "native" to Java, integrating well with Java's native APIs, such as the Stream API, and allowing easy serialisation also of objects over which we have no control.
And where is the budget for Micronaut sponsorship coming from? In the Community version, the developers of MicroStream only allow PostgreSQL, MongoDB, SQLite, and file systems. More advanced solutions, such as S3 and other blob storage, are available in the commercial release. Unfortunately, I have not been able to find a list of paying customers of the solution anywhere.
If you haven't heard enough about MicroStream yet, foojay.io has just started a series on the tool. Its first part can be found here, along with a much broader introduction to the topic, along with some broad potential use cases. I've been hitting up MicroStream over and over again in recent weeks, and while it's hard to tell how much of this is a grassroots community effort and how much a marketing stunt... well, it worked. Maybe they've just decided it's time to start monetizing heavily.
Sources
The State of Clojure 2022
We haven't had Clojure for a long time, huh? Well, it's a good thing that there's an opportunity to catch up with what's been happening in the ecosystem of this king of modern lisp-like languages.
Let's start with this - is Clojure being used in real-world projects? As a matter of fact, we are seeing a growing number of people using Clojure commercially. Interestingly, this is associated with losses in all other groups. I wonder if this means some kind of inbreeding and just some shuffling within a fairly constant group? I happen to be missing data on the growth of the entire ecosystem in the survey.
Let's fly further. Where are people coming from in the Clojure ecosystem? Java is a fairly natural transit direction, JavaScript too (ClojureScript is one of the driving forces behind the whole project), but... Python? I have to admit that this is a rather surprising transit direction for me. It would be funny as if it was the result of some single major transit from Python to Clojure 😉 But the effect is pretty constant, so maybe some Clojure-reader can enlighten me?
Not all the graphs can be found in the original post, for access to some of the data you have to work a bit harder. I was very curious how the usage is distributed in different parts of the industry, but I didn't find it in the post accompanying the results, so I looked into the "pure" data. It turns out that the most common place to use Clojure is... Web Development and Open Source projects, and only third are commercial projects (I wonder how these compare to Web Development). I suspect that a big part of this is due to ClojureScript.
Similarly, you also have to bend over for domain-specific usage data. Here, however, there is no surprise - Clojure has always been associated with FinTechs, and this is quite well reflected in the table above.
Finally, I've left myself my favorite graph, the one about involvement in the language community. I wonder to what extent the group filling out the survey are people with some inclination to "give back" (or active declaratively), but it's surprising that almost a quarter of them are involved in open-source tool development, and almost half are involved in evangelizing their own organization. In my experience - those invested in Clojure have it that way. Believe me, I know something about it from my own example.
A bit like lawyers. How will you know a Clojure programmer? Because he will tell you about it.
The survey data probably shouldn't be taken as the ultimate source of truth about the ecosystem, but it still provides a window into the Clojure community. You can find the full results here, including answers to the open questions and more details. In case anyone was wondering - 2352 people took part in the survey.
Sources:
3. Update of the Kotlin Roadmap!
After a period of drought, when it comes to new information about Kotlin, the situation slowly begins to unravel - we finally got a new version of the language's Roadmap. Within it, JetBrains decided to include all their plans for the second half of 2022, but also reveal a bit of the secret about what the beginning of 2023 will bring.
There's no denying that Kotlin developers have taken to heart the comments of users complaining about sub-optimal developer experience of the language, manifesting itself in long compilation times. Therefore, the key change that the language creators put special emphasis on is a new compiler.
K2, as it is called, has two goals. Firstly, it is intended to be significantly faster in itself than existing solutions. At the same time, its implementation will allow developers to innovate more easily by getting rid of technical debt. This is because K2 differs from existing solutions in that it is intended to provide a common abstraction for the entire Kotlin Multiplatform, all thanks to a new intermediate format. At the moment the JVM variant is leading the way, but there is already talk of a version for Kotlin JS, Kotlin Native, and for mobile releases. The Roadmap also includes themes to stabilize the API to allow plugin writing, as well as an official beta release... unfortunately no dates have been given for now. The full implementation of K2 will still take some time, which is why Kotlin developers are planning to include new features related to the existing compiler, such as streamlining Kotlin scripts.
As we have already mentioned the Multiplatform version, this one will also soon receive some improvements, especially in the context of the Android code. Multiplatform is finally going to be released as a Beta version, and a separate, detailed note was recently devoted to it, describing what the "promotion" process will look like.
We can also see further investment in tooling. The stabilization of code analysis and the avoidance of redundant compilation (which is, on the face of it, potentially even more effective than even the fastest new compiler) are intended to complement the K2 effort.
As far as the JVM and JS versions of Kotlin are concerned, no new items related to it have appeared on the roadmap. In general, it can be seen that the stabilization (in the case of JS) and standardization (in the case of JVM) of the new intermediate format (internal representation - IR) for both languages takes a lot of effort and drags on a bit. So we are still waiting e.g. for the support of kapt (Kotlin Annotation Processor) in the new JVM IR, and also for better compilation times using it.
To sum up the Roadmap update - it is clear that the times of big changes in the language are behind us. Only one new (very cosmetic change) in its syntax - special syntax for until operator - shows that Kotlin as a language is already feature complete. Now, the focus is mainly on supporting new platforms, but also on making the whole ecosystem as comfortable as possible to work with - both by external programmers and the language developers themselves. It is not a bad decision - after all, it is with great integration with tools and ease of use that Kotlin won the hearts of users.
Bonus: KotlinDL 0.4
And while we're on the subject of Kotlin, at the beginning of the week a new version of the KotlinDL library appeared, numbered 0.4, which introduces a number of new features, of which the most eye-catching one (and thus also used in advertising this edition) is the Pose Detector.
In addition to it, the new KotlinDL brings support for two new models: EfficientDet and EfficientNet, and new layers beyond what the standard TensorFlow library brings. If you want more information, including about the other new features in this version, the official Release Notes are really well written, so I encourage you to read them.
Sources
PS: The edition was created while listening NINA - Sleepwalking (Full Album).