March & April 2024 Long-Term Project Updates

By Kathy Davis

Our 2024 long-term developers report on an incredible body of work for March and April. A heartfelt thanks to all!

Bozhidar Batsov: CIDER
Michiel Borkent: squint, neil, clj-kondo,nbb, CLI, and more
Toby Crawley: clojars-web
Thomas Heller: shadow-cljs
Kira McLean: Scicloj Libraries and more
Nikita Prokopov: Datascript, Clj-reload, Clojure Sublimed, and more
Tommi Reiman: Reitit 7.0. Malli
Peter Taoussanis: http-kit, Nippy, Telemere, and more

Bozhidar Batsov

The biggest news in the past couple of months is that CIDER now supports clj-reload, as an alternative of clojure.tools.namespace! You can find the relevant PR here.

Some other noteworthy changes:

Interested people should check out the recent changes in cider-nrepl and orchard, as that’s where the bulk of the work has happened.

We’ve yet to release a new version of CIDER, but I’m expecting this will happen before the end of May.

P.S. We might be close to a solution for the broken interrupt functionality in nREPL on newer JDKs (20+). See this for details.

Michiel Borkent

Updates In this post I’ll give updates about open source I worked on during March and April 2024. To see previous OSS updates, go here.

Sponsors

I’d like to thank all the sponsors and contributors that make this work possible. Without you, the below projects would not be as mature or wouldn’t exist or be maintained at all.

Current top tier sponsors:

If you want to ensure that the projects I work on are sustainably maintained, you can sponsor this work in the following ways. Thank you!

If you’re used to sponsoring through some other means which isn’t listed above, please get in touch.

On to the projects that I’ve been working on!

Updates

Here are updates about the projects/libraries I’ve worked on last month.

Other projects

There are many other projects I’m involved with but that had little to no activity in the past month. Check out the Other Projects section (more details) of my blog here to see a full list.

Published: 2024-04-30 Tagged: clojure oss updates

Toby Crawley

April 2024

Commit Logs: clojars-web

Mostly maintenance work this month related to error handling and tighening error boundaries.

March 2024

Commit Logs: clojars-web, infrastructure

Thomas Heller

Time was mostly spent on doing maintenance work and some bugfixes. As well as helping people out via the typical channels (e.g. Clojurians Slack).

Current shadow-cljs version: 2.28.4 Changelog

Notable Updates

Kira McLean

This is a summary of the open source work I’ve spent my time on throughout March and April, 2024. Overall it was a really insightful couple of months for me, with lots of productive discussions and meetings happening among key contributors to Clojure’s data science ecosystem and great progress toward some of our most ambitious goals.

Sponsors

This work is made possible by the generous ongoing support of my sponsors. I appreciate all of the support the community has given to my work and would like to give a special thanks to Clojurists Together and Nubank for providing me with lucrative enough grants that I can reduce my client work significantly and afford to spend more time on these projects.

If you find my work valuable, please share it with others and consider supporting it financially. There are details about how to do that on my GitHub sponsors page. On to the updates!

Grammar of graphics in Clojure

With help from Daniel Slutsky and others in the community, I started some concrete work on implementing a grammar of graphics in Clojure. I’m convinced this is the correct long-term solution for dataviz in Clojure, but it is a big project that will take time, including a lot of hammock time. It’s still useful to play around with proofs of concept whilst thinking through problems, though, and in the interest of transparency I’m making all of those experiments public.

The discussions around this development are all also happening in public. There were two visual tools meetups focused on this over the last two months (link 1, link 2). And at the London Clojurians talk I just gave today I demonstrated an example of one proposed implementation of a grammar-of-graphics-like API on top of hanami implemented by Daniel.

There are more meetups planned for the coming months and work in this area for the foreseeable future will look like researching and understanding the fundamentals of the grammar of graphics in order to design a simple implementation in Clojure.

Clojure’s ML and statistics tools

I spent a lot of time these last couple of months documenting and testing out Clojure’s current ML tools, leading to many great conversations and one blog post that generated many more interesting discussions. The takeaway is that the tools themselves in this area are all quite mature and stable, but there are still ongoing discussions around how to best accommodate the different ways that people want to work with them. The overall goal in this area of my work is to stabilize the solutions so we can start advocating for specific ways of using them.

Below are some key takeaways from my research into all this stuff. Note none of these are my decisions to make alone, but represent my current opinions and what I will be advocating for within the community:

Foundations of Clojure’s data science stack

I continue to work on guides and tutorials for the parts of Clojure’s data science stack that I feel are ready for prime time, mainly tablecloth and all of the amazing underlying libraries it leverages. Every once in a while this turns up surprises, for example this month I was surprised at how column header processing is handled for nippy files specifically. I also fixed one bug in tablecloth itself, which I discovered in the process of writing a tutorial earlier in March. I have a pile of in-progress guides focusing on some more in-depth topics from developing the London Clojurians talk that I’m going to tidy up and publish in the coming months.

The overarching goal in this area is to create a unified data science stack with libraries for processing, modelling, and visualization that all interoperate seamlessly and work with tablecloth datasets, like the tidyverse in R. Part of achieving that is making sure that tablecloth is rock solid, which just takes a lot of poking and prodding.

London Clojurians talk

This talk was a big inspiration for diving deep into Clojure’s data science ecosystem. I experimented with a ton of different datasets for the workshop and discovered tons of potential areas for future development. Trying to put together a polished data workflow really exposed many of the key areas I think we should be focusing on and gave me a lot of inspiration for future work. I spent a ton of time exploring all of the possible ways to demonstrate a broad sample of data science tools and learned a lot along the way.

The resources from the talk are all available in this repo and the video will be posted soon.

Summary of future work

I mentioned a few areas of focus above, below is a summary of the ongoing work as I see it. A framework for organizing this work is starting to emerge, and I’ve been thinking about in terms of four key areas:

Visualisation

Machine learning

Statistics

Foundations

Going forward

My overarching goal (personally) is still to write a canonical resource for working with Clojure’s data science stack (the Clojure Data Cookbook), and I’m still working on finding the right balance of documenting “work-in-progress” tools and libraries vs. delaying progress until I feel they are more “ready”. Until now I’ve let the absence of stable or ideal APIs in certain areas hinder development of this book, but I’m starting to feel very confident in my understanding of the current direction of the ecosystem, enough so that I would feel good about releasing something a little bit more formal than a tutorial or guide and recommending usages with the caveat that development is ongoing in some areas. And while it will take a while to get where we want to go, I feel like I can finally see the path to getting there. It just takes a lot of work and lot of collaboration, but with your support we’ll make it happen! Thanks for reading.

Nikita Prokopov

Hello hello hello, I hope you like some open-source, because I have some for ya. This past two months have been busy!

DataScript:

Humble UI:

Skija:

clj-reload, a smarter way to reload Clojure code:

Clojure Sublimed, Clojure development environment for Sublime Text:

New library: sane-math

Blogging:

Best, Niki

Tommi Reiman

Finally! Reitit 0.7.0 is out. It has been over a year in the making, spanning multiple libraries: reitit, malli, schema-tools, [spec-tools](https://github.com/metosin/spec-tools and ring-swagger-ui. Big thanks to everyone involved. There is a lot of draft work that has been queued and can now be worked on. New releases should come more frequently in the future.

Reitit 0.7.0 (all 8 alphas flattened)

0.7.0 (2024-04-30)

The OpenAPI3 release, Year in the making - the changes span over multiple repositories!

[metosin/malli "0.16.1"] is available but we use "0.10.1"
[metosin/muuntaja "0.6.10"] is available but we use "0.6.8"
[metosin/spec-tools "0.10.6"] is available but we use "0.10.5"
[metosin/schema-tools "0.13.1"] is available but we use "0.13.0"
[metosin/jsonista "0.3.8"] is available but we use "0.3.7"
[com.fasterxml.jackson.core/jackson-core "2.17.0"] is available but we use "2.14.2"
[com.fasterxml.jackson.core/jackson-databind "2.17.0"] is available but we use "2.14.2"
[ring/ring-core "1.12.1"] is available but we use "1.9.6"
[metosin/ring-swagger-ui "5.9.0"] is available but we use "4.15.5"

Malli

0.16.1 (2024-04-30)

0.16.0 (2024-04-20)

0.15.0 (2024-03-23)

;; function of arg:int -> ret:int, where arg < ret
[:=> 
 [:cat :int] 
 :int 
 [:fn (fn [[[arg] ret]] (< arg ret))]]

borkdude/edamame 1.3.23 -> 1.4.25

Something else

Back at Greece, to relax and to Open Source.

os-greece

Peter Taoussanis

A big thanks to Clojurists Together, Nubank, lambdaschmiede, and other sponsors of my open source work!

Hi folks! 🦎

It’s been a very productive couple months! Have been able to continue full-time on open source. Output included:

http-kit v2.8.0

http-kit v2.8.0 final is now on Clojars 👍

http-kit is a simple, high-performance event-driven HTTP client+server for Clojure.

This is the first major stable http-kit release since 2023-06-30, and includes work from 10 contributors. Big thanks to everyone involved! 🙏

Some highlights include:

Nippy v3.4.0

Nippy v3.4.0 final is now on Clojars 👍

Nippy is a fast and mature binary serialization library for Clojure.

This is the first major stable Nippy release since 2023-10-11, and includes:

Telemere v1.0.0 first public pre-releases

My main focus during this period has been has been Telemere.

Telemere is a major new library (and along with Tempel, my first all-new library in 7+ years). It’s a structured telemetry library for Clojure/Script, and a highly-polished modern rewrite of Timbre without any of the historical constraints.

It offers a superset of the functionality found in traditional and structured logging, and out-the-box support for SLF4J, OpenTelemetry, clojure.tools.logging, etc.

The latest release can be found here (currently beta5).

Folks happy with Timbre have zero pressure to update, I’ll continue to support Timbre as usual. But Telemere offers a lot of new features and improvements (see README), and migration is often pretty easy.

Will note that based on Clojure survey feedback, I’ve been putting a lot more emphasis lately on beginner-oriented support.  For Telemere this includes the most comprehensive wiki and API docs I’ve yet included with a library.

Please do let me know if this stuff is helpful, since it adds a lot to the development effort! 🙏

There’s also a new Telemere Slack channel and short demo video.

Telemere’s a small library, but it’s been a lot of work getting the details just right. I’m happy with the results, and excited for folks to try it out.

Telemere is in many ways represents the refinement and culmination of ideas brewing over 12+ years in Timbre, Tufte, Truss, etc.

Ultimately the hope is for it to help enable Clojure/Script systems that are observable, robust, and debuggable. The wiki intro is probably a good place to start if you’re interested in hearing more.

London Clojurians talk

This was actually recorded back in February, but I have folks still occasionally mentioning that they’d missed it earlier - so I’ll share a reminder here.

The talk is available on YouTube, and is titled Some Controversial Truths: _challenging some commonly-held ideas about Clojure, software engineering, and startups; and sharing the 1 secret that will solve all your problems.

Big thanks to Bruno Bonacci for hosting!

Interview with Daniel Compton

Likewise for folks that missed it earlier - Daniel Compton recently posted a chat we had about my open source work. Big thanks to Daniel for hosting!

Upcoming work

My current roadmap can always be found here.

Current objectives for May-June include:

- Peter Taoussanis