October 2019 Monthly Update

By Nola Stowe

October was our third and final month with this round of projects, check out their monthly update!

Shadow CLJS

Released version 2.8.69

Inspect

I spent the last few weeks implementing the first version of a new “Inspect” feature. It was inspired by REBL, fulcro-inspect and re-frame-10x. My first goal with this is to bring the console.log experience from the Browser to all other JS platforms and Clojure itself. console.log in the Browser is great because it is basically a prn on steroids. Instead of getting one large blob of text you get a structured output you can easily inspect visually. It is however limited in the Browser, basically abusing the built-in Browser devtools. “Inspect” runs in an independent UI (currently the shadow-cljs Web UI) and can be used to inspect any supported runtime. These include react-native, node and Clojure which previously had a rather basic prn experience.

This combines a generic remote Data Viewer along with support for tap> in the runtime to “expose” new values to the UI. Later versions of “Inspect” will also include direct REPL integration and some other features found in similar tools.

I’m very excited about this feature and it is just getting started. I wrote a bit more about it here. This is all still undergoing a lot of changes but has become part of my normal workflow already. I hope others will find it useful as well.

Bugfixes

Meander

UX

Some users had intermittently reported problems with defsyntax and its use of clojure.spec.alpha/conform. After some digging around I discovered the problem was due to a patch to the core specs library which changed the names of some of the keys thus breaking the use of conform in defsyntax. To fix the problem, defsyntax now has its own specs and no longer relies on clojure.core.specs.alpha.

In other areas corrections to or movement of specs were made to improve the quality of life for users using spec instrumentation.

One of the most significant improvements in October came in the form of ClojureScript build size. We had some reports of Meander resulting in very large ClojureScript builds (even with advanced compilation) causing people to be unable to use the library seriously. Fortunately, with the help of Thomas Heller and Jimmy Miller, we were able to completely eliminate the build size problem altogether.

Performance/Code generation improvements

The performance of n or more patterns when using the search macro by define runtime was given a boost by writing functions specially suited to the task.

Continuing with improvements to reference (the symbols that look like %foo) compilation last month, this month I fixed a subtle bug which could cause code size to increase substantially with by combining cyclic references and or patterns. While patterns would still match as expected, the resulting code size and compiler performance in these cases was unacceptable. Additionally, unused functions are no longer produced by reference code generation.

To reduce code size, the internal process which attempts to merge two nodes in the intermediate representation made some progress. Merging is attempted when two nodes have similar properties. If their properties are the same, then an attempt is made to merge their children. In the past, if any of the submerges failed, the whole merge would fail. This is not the desired outcome. Since the two nodes are similar enough to combine the merge process now combines the two nodes and produces a branch node (a choice between patterns) with the children. This reduces the generated code size significantly and avoids doing extra work.

Substitution got better this month with with pattern enhancements/fixes. Recursive references to map patterns which used memory variables i.e. %m {!k !v & %m} used to stack overflow, now they don’t. The substitution compiler now uses the same acyclic reference inlining as the match compiler avoiding the cost of function calls at runtime.

Enhancements

Toward the end of the month I wanted to shift my attention to making the cata pattern matching operator available on the substitution side of rewrite and rewrites clauses. The semantics of the substitution version of the cata pattern is to recursively rewrite with a constructed value via substitution. So (cata [?x ?y]) on the substitution side would first construct [?x ?y] then recurse. This provides an elegant way to write non-trivial recursive rewrite systems which are very useful when work with recursively structured data. While my goal was to have it complete by the end of the month, I just couldn’t quite pull it off in time. However, this work is near completion and I expect to make it available soon.

Calva

October 1-15

Theme: Changes Spree

Two weeks, seven new Calva versions. We got into a flow, changing, testing, releasing. Christian Fehse kept a speed where I had a hard tine keeping up with testing and giving feedback. But I managed 😄, and even got a few changes in myself, plus some documentation.

Changes

Documentation and promotion

I wrote a small blog post about the streamlined, and friendly, Calva contribution process. This had me start thinking about the mess that was supposed to pass for user documentation. It got me to:

October 16-31

Theme: Quality through issue hunting

As the Calva user base grows (which is currently happening) the influx of issues is also growing. During the funding period we have been able shrink the total issues count from near the 100s down to, at the time of this writing, 62. This even though close to 150 issues where opened during the same period! The last two weeks of October continued with the issue-squashing.

Also, with help from Michiel Borkent, aka @borkdude, creator of clj-kondo, we managed to bring the Clojure linting story for VS Code users down to:

– ”Install the Calva Extension.”

🎉 Even for Windows users. 🥂

Changes

Documentation

In an effort to help potential contributors to Calva figure out the project, I wrote a sort of a manifest: The Tao of Calva.

We have also worked on closing some gaps in the developer documentation, adding these articles to the wiki:

Other

This report concludes the Clojurists Together 2019 Q3 sponsorship of Calva. We will probably apply again, sometime soon, Calva is far from finished!

Meanwhile, a few days ago I got my GitHub Sponsors profile approved, so now you can sponsor me there. Please do!

CIDER