April 2018 Monthly Update

By Daniel Compton

Clojurists Together news

This is the third and final update from clj-http and Figwheel. Both projects finished their work at the end of April. Both projects have done great work, and we are really impressed with what they’ve been able to achieve. We recently announced that we are funding CIDER and ClojureScript for our Q2 funding round. Those two projects have just started, we’ll have more details on what they’re working on next month. If you’d like to follow along with Mike Fike’s work on ClojureScript, he has a public work log up.

We continue to work on automating the sign-ups on the website, to make a faster and smoother flow for people who’d like to join. If you’d like to help out on that, or other issues, our site is now up on GitHub.

clj-http updates

April 1 - 15

Hello again Clojurists!

These past two weeks have been focused on two particular tasks for clj-http:

A user opened up an issue about clj-http’s cookie policy configuration, talking about the documentation being incorrect with regard to plugging your own cookie policy in. I looked into this and discovered he was correct, the wrong part of cookie policies selection was configurable. I spent a while reading up on how Apache implements their cookie spec and cookie spec providers, and added a couple of new ways to configure the cookie parsing in clj-http. These new options are the :cookie-spec and :cookie-policy-registry settings in the body. These settings will let someone either provide a spec provider (there are several that the Apache HTTP client makes available) or to complete write their own cookie spec (I’ve also added documentation for how to do this). From there, a user has complete control over how cookies are parsed from headers, how the cookie values are validated, and how headers get transformed back into cookies.

The other thing I’ve been focusing on has been the Apache 5.0 upgrade. I’m happy to report that the apache5-upgrade branch now compiles, and that you can use it in a REPL to perform requests! There are still some features that are commented out, as I have not yet discovered the equivalent 5.0 versions (there are many changes in the 5.0 upgrade and all the documentation has not been released). I’ve been going through clj-http’s tests to get them passing, but so far this is a much smoother transition than the 3.x -> 4.0 upgrade in the past. While that upgrade took many, many months to transition to the new and un-deprecated APIs, I’m hoping to be much more on top of this major release. Having this branch ready means that it will be a much shorter transition once the new Apache client version is out of beta and 5.0 is released.

That’s it for these past two weeks!

April 15 - 30

Alright on to what I’ve been up to in the last couple of weeks:

Thank you Clojurists Together members again for your support of the project and the honor of being picked and being able to work on clj-http!

Figwheel updates

April 1 - 15

Complete Figwheel REPL and start Figwheel Main

During the last 20 days, I have put most of my energy into completing Figwheel REPL.

The most difficult part of working on it was the design and trade-offs that surrounded the server that is used for the REPL connection. In the end, I settled on using the Jetty Server because it is the most commonly used Ring server in the ecosystem.

Getting a nice design together so that the server can be used for both a REPL connection and an initial development server was challenging. I settled on using ring-defaults along with some middleware that makes the server smarter about caching compiled ClojureScript assets.

I’m happy with the end product because it exposes all the ring.jetty.adapters/run-jetty configuration options which in turn should allow things like SSL support. It also exposes all of the ring/ring-defaults configuration options as well, to allow configuration of common ring server middleware.

The final REPL also allows you to completely customize/replace the what I am calling the :ring-stack (the ring-stack is what wraps the :ring-handler) which is currently ring-defaults composed with some extra middleware for Figwheel. Also, for the brave you can replace the :ring-server itself, which would be a much more involved task.

Of course this setup will still also allow you to supply a :ring-handler configuration option. You will still be able to use your own server for development and have the Figwheel server operate as simple websocket server for REPL only communication.

The Figwheel REPL also now serves a default index.html so that establishing an initial REPL connection is a simple matter. Unfortunately the REPL is not complete yet, but the rest of its development will be better informed by starting work on figwheel-main.

figwheel-main

Work on figwheel-main has started and I’m still getting my bearings on getting the best behavior from it. Much like the work on the REPL the really hard part is getting a good design.

The main idea driving figwheel.main is that it should provide all the functionality of cljs.main in addition to figwheel functionality. I want ClojureScript users to have less cognitive overhead when it comes to tooling.

Figwheel main defaults to the figwheel-repl and adds a --build or -b flag so that you can supply the name of a build. So the typical invocation would be:

clj -m figwheel.main -b dev -r

This command will start the familiar figwheel experience.

In this case dev refers to a file named dev.cljs.edn.

The contents of which are:

{:main example.core}

This is all the figwheel needs now. It will supply smart defaults for :output-to, :output-dir, :asset-path. It will also supply a smart default for what directory to watch.

So in other words you can have a development server, repl, figwheel reloading, by just supplying a build edn file like above and a src/example/core.cljs, and a deps.edn.

This much is working already, but there is still much more work to do. I’m really hoping to get initial releases of these libraries out by the end of the month.

Thanks again!

A big thanks to everyone for supporting this work!!

April 15-30 Figwheel Main - Feature Complete not Friendly Complete

Summary

Figwheel Main now provides all the features of Figwheel Sidecar and more. However, it is not yet as friendly as I am wanting it to be.

I’m hoping to publish an alpha snapshot of Figwheel Main soon with a more hardened one out by the end of May.

A Rewrite

Figwheel initially evolved in an environment that was very different from today. The ClojureScript compiler has changed a great deal, along with the tooling landscape, and my experience writing Clojure.

Early code design decisions were made, with the objective of exploring a relatively unknown problem space. The priority being was to get these tools out to programmers sooner than later, not to write an exemplary codebase.

The figwheel-main rewrite sheds oodles of accidental complexity introduced by an earlier me. This earlier Bruce was attempting to provide a design structure to give himself the illusion of control over the problem. In reality, structure was only there to comfort, and help me convince myself that I was doing things the “right” way. In the end, this structure just made it difficult to evolve the codebase.

Side-note: Rewriting figwheel was a fantastic experience that allowed my growing Clojure experience to guide me to much better solutions.

A rewrite has been an opportunity for me to address so many holes and long running issues that would have been very hard to fix in Figwheel Sidecar.

The first of these issues was to minimize the configuration that’s needed to get an initial ClojureScript development session up an running. figwheel-main delivers this:

In your deps.edn

;; deps.edn
{:deps {com.bhauman/figwheel-main {:mvn/version "0.1.0-SNAPSHOT"}
 :paths ["src" "target"]]}

And on the command line:

clojure -m figwheel.main

The above will launch a browser and connect a REPL to it, from here you can do REPL driven development.

And if you want to in work on a particular build with figwheel style reloading just add a build config file.

For example in dev.cljs.edn

{:main example.core}

And in src/example/core.cljs

(ns example.core)
(enable-console-print!)
(prn "hello world!")

and run the command:

clojure -m figwheel.main -b dev

You’ll notice there is no need for an index.html, :output-to, :output-dir, :asset-path, :source-paths, and oodles of other configuration that is needed to get up an running with ClojureScript.

So little in fact, I’m considering adding a --gen command line option to add namespaces, index.html, and other initial config. I’m thinking these will be backed up by github gists.

This is a much needed clean up of the initial configuration to get started with a hot loading ClojureScript environment. It is easier to understandable and friendlier. I can’t imagine what newcomers think when they see the project.clj generated by lein new figwheel but it certainly not a feeling of comfort.

Another thing I wanted to fix was the overall complexity of the configuration options. The new configuration is also much simpler and flatter.

There is a main configuration file: figwheel-main.edn

For example:

{:watch-dirs ["src" "dev"]
 :ring-server-options {:port 9500}}

And if you want to modify that configuration for an individual build add meta-data to the build config file:

For example: in your dev.cljs.edn

^{:watch-dirs ["src"]
  :ring-server-options {:port 9501}}
{:main example.core}

This makes configuration much more understandable as there are no longer two levels of config with different options available. Only one level that can be overridden by a the meta-data on a build.

The use of meta-data here also enables you to effortlessly re-use the configuration with cljs.main commands.

Another, major issue was the promiscuous REPL connections or “where the heck is this being evaluated?”

As mentioned before the new REPL allow names the individual connections and allows you to choose the connection so send evaluations to. When you evaluate code in the REPL it only goes to one connection unless you specify otherwise in your config.

Add to the above changes a long list of other improvements that I was able to make with the introduction of rebel-readline-cljs, figwheel-repl and figwheel-core.

The biggest feature is that the figwheel-main codebase is much much smaller and much easier to grok. The new figwheel is now in a much better position to move into the future and integrate with other tooling.

I already have a long list of improvements that I want to make now that the codebase much more approachable.

Conclusion

I consider the Clojurists Together funding of Figwheel to be a success! The funding gave me both an excuse and an opportunity to really focus on Figwheel, and do the things that I knew needed to happen for it to continue to provide value to the ClojureScript community.

Much thanks goes to Daniel Compton and everyone who participated in Clojurists Together.