Reveal:
Read Eval Visualize Loop for Clojure

Demo

Clojars Project Github page Slack Channel

Rationale

Repl is a great window into a running program, but the textual nature of its output limits developer’s ability to inspect the program: a text is not an object, and we are dealing with objects in the VM.

Reveal aims to solve this problem by creating an in-process repl output pane that makes inspecting values as easy as selecting an interesting datum. It recognizes the value of text as a universal interface, that’s why its output looks like a text: you can select it, copy it, save it into a file. Unlike text, reveal output holds references to printed values, making inspecting selected value a matter of opening a context menu.

Unlike datafy/nav based tools, Reveal does not enforce a particular data representation for any given object, making it an open set — that includes datafy/nav as one of the available options. It does not use datafy/nav by default because in the absence of inter-process communication to datafy is to lose.

Not being limited to text, Reveal uses judicious syntax highlighting to aid in differentiating various objects: text java.lang.Integer looks differently depending on whether it was produced from a symbol or a class.

Give it a try

The easiest way to try it is to run a Reveal repl:

clj \
-Sdeps '{:deps {vlaaad/reveal {:mvn/version "1.0.130"}}}' \
-m vlaaad.reveal repl

Executing this command will start a repl and open Reveal output window that will mirror the evaluations in the shell.

Features

tap> support

Clojure 1.10 added tap> function with the purpose similar to printing the value for debugging, but instead of characters you get the object. Reveal repls show tapped values in their output windows — you won’t need println anymore!

Tap demo

Eval on selection

You can evaluate code on any selected value by using text input in the context menu. You can write either a single symbol of a function to call or a form where *v will be replaced with selected value.

Eval on selection demo

Inspect object fields and properties

Any object in the JVM has class and fields, making them easily accessible for inspection is extremely important. With java-bean contextual action you get a debugger-like view of objects in the VM. Access to this information greatly improves the visibility of the VM and allows to explore it. For example, for any class you have on the classpath you can get the place where it’s coming from:

Java bean demo I learned about it after implementing this feature :)

Look and feel customization

Reveal can be configured with vlaaad.reveal.prefs java property to use different font or theme:

Light theme

URL and file browser

You can open URL-like things and files: both internally in Reveal and externally using other applications in your OS e.g. file explorer, browser or text editor.

Browse demo

Doc and source

Reveal can show you documentation and sources for various runtime values — namespaces, vars, symbols, functions, keywords (if they define a spec). Like cljdoc, it supports [[wikilink]] syntax in docstrings to refer to other vars, making them accessible for further exploration.

Doc and source demo

Ref watchers

Reveal can watch any object implementing clojure.lang.IRef (things like atoms, agents, vars, refs) and display it either automatically updated or as a log of successors.

Ref watchers demo

Charts

Reveal can show data of particular shapes as charts that are usually explorable: when you find an interesting data point on the chart, you can then further inspect the data in that data point.

The simplest shape is labeled numbers. Labeled means that those numbers exist in some collection that has a unique label for every number. For maps, keys are labels, for sequential collections, indices are labels and for sets, numbers themselves are labels.

A pie chart shows labeled numbers:

Pie chart demo

Other charts support more flexible data shapes — both because they can show more than one data series, and because they can be explored, where it might be useful to attach some metadata with the number. Since JVM numbers don’t allow metadata, you can instead use tuples where the first item is a number and second is the metadata. Bar charts can display labeled numbers (single data series) or labeled numbers that are themselves labeled (multiple data series):

Bar chart demo

Line charts are useful to display progressions, so Reveal suggests them to display sequential numbers (and labeled sequential numbers):

Line chart demo

Finally, Reveal has scatter charts to display coordinates on a 2D plane. A coordinate is represented as a tuple of 2 numbers and as with numbers, you can use a tuple of coordinate and arbitrary value in the place of coordinate. Reveal will suggest scatter charts for collections of coordinates and labeled collections of coordinates.

Scatter chart demo

Table view

There are cases where it is better to make sense of the value when it is represented by a table: collections of homogeneous items where columns make it easier to compare corresponding parts of those items, and big deeply nested data structures where it’s easier to look at them layer by layer.

Table view demo

…and more

Reveal was designed to be performant: it can stream syntax-highlighted output very fast. In addition to that, there are various helpers for data inspection:

Using Reveal

UI concepts and navigation

Concepts

Reveal UI is made of 3 components:

Navigation:

Customization

Reveal can be customized using vlaaad.reveal.prefs java property that contains an edn map of UI preferences. Supported keys (all optional):

Example:

$ clj -A:reveal \
-J-Dvlaaad.reveal.prefs='{:font-family "Consolas" :font-size 15}' \
-m vlaaad.reveal repl

User API

The main entry point to Reveal is vlaaad.reveal ns that has various repls and lower-level functionality for data inspection.

repl

It is a repl wraps clojure.main/repl with additional support for :repl/quit and tap>. It is as simple as it gets. I use it all the time. Example:

$ clj -A:reveal -m vlaaad.reveal repl
# Reveal window appears
Clojure 1.10.1
user=>

io-prepl

This prepl works like clojure.core.server/io-prepl. Its purpose is to be run in a process on your machine that you want to connect to using another prepl-aware tool. Example:

$ clj -A:reveal \
-J-Dclojure.server.reveal='{:port 5555 :accept vlaaad.reveal/io-prepl}'

Now you can connect to this process using any socket repl and it will show a Reveal window for every connection:

$ nc localhost 5555
# reveal window appears
(+ 1 2 3) # input
{:tag :ret, :val "6", :ns "user", :ms 1, :form "(+ 1 2 3)"} # output

remote-prepl

Reveal is the most useful when it runs in the process where the development happens. This prepl, unlike the previous two, is not like that: it connects to a remote process and shows a window for values that arrived from the network. It can’t benefit from easy access to printed references because these references are pointing to values deserialized from bytes, not values in the target VM. It’s still nice and performant repl, and it’s useful when you want to use Reveal to talk to another process that does not have Reveal on the classpath (e.g. production or ClojureScript prepl).

Example:

  1. Start a prepl without Reveal on the classpath:
    $  clj \
    -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.764"}}}' \
    -J-Dclojure.server.cljs-prepl='{:port 50505 :accept cljs.server.browser/prepl}'
    
  2. Connect to that prepl using Reveal:
    $ clj -A:reveal -m vlaaad.reveal remote-prepl :port 50505
    # at this point, 2 things happen:
    # 1. Browser window with cljs prepl appears
    # 2. Reveal window opens
    
    # input
    js/window
    
    # output
    {:tag :ret, 
     :val #object [Window [object Window]], 
     :ns "cljs.user", 
     :ms 25, 
     :form "js/window"}
    

ui

Calling this function will create and show a Reveal window. It returns a function that you can submit values to — they will appear in the output panel. All built-in visual repls are thin wrappers of other repls that submit values to a window created by this generic function. You can use it to create custom Reveal-flavored repls, or, instead of using it as a repl, you can configure Reveal to only show tapped values.

Example:

(require '[vlaaad.reveal :as reveal])

;; open a window that will show tapped values:
(add-tap (reveal/ui))

 ;; submit value to the window:
(tap> {:will-i-see-this-in-reveal-window? true})

Editor integration

With user API you should be able to configure your editor to use Reveal, but there are still some points worthy of discussion.

Cursive

For Cursive, you should create a “local repl” run configuration with “clojure.main” repl type. For prefs, use “JVM Args” input, but note that it splits args on spaces, so you should use commas, e.g. -Dvlaaad.reveal.prefs={:theme,:light}. This is the most simple setup that allows IDEA to start your application and establish a repl connection for sending forms.

Sometimes this setup is not ideal: you might want to start an application using some other means and then connect to it using IDEA. In that case, you should not use “remote repl” run configuration, since it will rewrite your forms and results to something unreadable. Instead, you should still use the “local repl” run configuration that uses a remote repl client that connects to your process. Example:

  1. Make your target process a reveal server:

    clj -A:reveal -J-Dclojure.server.repl='{:port 5555 :accept vlaaad.reveal/repl}'
    
  2. Add a dependency on remote-repl to your deps.edn:

    {:aliases 
     {:remote-repl {:extra-deps {vlaaad/remote-repl {:mvn/version "1.1"}}}}}
    
  3. Create a “local repl” run configuration with “clojure.main” repl type, make it “Run with Deps” with remote-repl alias, and in Parameters specify -m vlaaad.remote-repl :port 5555.

Nrepl-based editors

For development workflows that require nrepl Reveal has a middleware that will show evaluation results produced by nrepl: vlaaad.reveal.nrepl/middleware, you will need to add it to your middleware list. The minimum required version of nrepl is 0.6.0.

Example of using this middleware with command line nrepl entry point:

$ clj -A:reveal -m nrepl.cmdline --middleware '[vlaaad.reveal.nrepl/middleware]'

Alternatively, you can create .nrepl.edn file in your project directory that will be picked up by nrepl. Example .nrepl.edn file:

{:middleware [vlaaad.reveal.nrepl/middleware]}

Windows

It’s probably a good idea to add -Dfile.encoding=UTF-8 to JVM options.

If you want to use reveal from WSL, you will need X server (e.g. X410). Make sure you have libgtk-3-0 installed (e.g. with sudo apt install libgtk-3-0). If you have HiDPI screen, you should set GDK_SCALE env variable to appropriate scale factor (e.g. export GDK_SCALE=2).

Extending reveal

There are 3 ways to extend Reveal to your needs: custom formatters, actions, and views. All three are available in vlaaad.reveal.ext namespace (aliased as rx in following examples).

One feature that they all share is annotations — non-intrusive metadata that exists alongside your objects in the Reveal state. Unlike datafy/nav based tooling, it does not obstruct your objects, leaving Clojure’s metadata exactly as it is in your program, and, since the annotation is alongside the object, Reveal allows any object to be annotated — not just IMetas.

Formatters

Formatters define how values are shown in the output panel. Formatter dispatch is a multimethod that looks at :vlaaad.reveal.stream/type meta key or, if it’s absent, at object’s class. The recommended way to extend this multi-method is using (defstream dispatch-val [x ann?] sf) macro that automatically marks the formatted region with the value that is being formatted. There is a small set of functions that perform the streaming of the formatting in an efficient way called streaming functions (sfs for short).

Low-level text emitting sfs

These are usually used in the defstream body to configure how something looks as text. Such sfs don’t mark the text they emit with values that will be available for inspection, instead they rely on their context (e.g. defstream) to mark what they emit. There is only 5 of them:

Delegating sfs

These sfs allow you to stream other values using their default streaming. This is also a place to annotate the streamed values.

Annotations are only useful if they are used, and they are used from actions. There is an example that configures formatting with annotations and uses these annotations for powerful data inspections here.

Overriding sfs

These sfs allow modifying some aspect of a streaming:

Actions

If selected text in Reveal UI has associated value, requesting a context menu on it will show a popup that checks all registered actions and suggest applicable ones. Use (defaction id [x ann?] body*) macro to register new actions.

Action body should return a 0-arg function to indicate that this action is available: this function will be executed when the action is selected in the context menu popup. Any other results, including thrown exceptions are ignored. The action body should be reasonobly fast (e.g. not performing disk IO) since all actions are always checked when the user asks for a popup. Returned function, on the other hand, may block for as long as needed: Reveal will show a loading indicator while it’s executed.

Minimal action example that shows shows how strings look unescaped (e.g. display "hello\nworld" as hello and world on separate lines):

(rx/defaction ::unescape [x]
  (when (string? x)
    #(rx/stream-as-is 
       (rx/as x (rx/raw-string x {:fill style/string-color})))))

As mentioned earlier, there is a bigger example that shows how actions and formatting can build on each other to aid with data exploration:

Actions demo

Views

A major difference between Output panel and Results panel is that the latter can show any graphical node allowed by Reveal’s UI framework (JavaFX). Reveal is built on cljfx — declarative, functional and extensible wrapper of JavaFX inspired by react. Reveal converts all action results to cljfx component descriptions with vlaaad.reveal.view/Viewable protocol that shows them with output panel view by default. You can reify an instance of the protocol with (view-as-is desc) function.

Short cljfx intro

To learn cljfx/JavaFX, you should go through cljfx readme and examples to get familiar with semantics and explore JavaFX javadoc to find available views. This might be a big task, so to get a feel for it here is this short introduction.

To describe a node, cljfx uses maps with a special key — :fx/type — that defines a type of node, while other keys define properties of that node. Value on :fx/type key can be a keyword (kebab-cased JavaFX class name) or a function (that receives a map of props and returns another description). Some examples of most commonly used descriptions:

;; showing a text
{:fx/type :label
 :text (str (range 10))}

;; showing a button with a callback:
{:fx/type :button
 :text "Deploy"
 :on-action (fn [event] (deploy-to-production!))}

;; combining views together
{:fx/type :v-box ;; vertically
 :children [{:fx/type rx/value-view ;; built-in component
             :value msft-stock}
            {:fx/type :h-box ;; horizontally
             :children [{:fx/type :button
                         :text "Sell"
                         :on-action (fn [_] (sell! :msft))}
                        {:fx/type :button
                         :text "Buy"
                         :on-action (fn [_] (buy! :msft))}]}]}

While cljfx supports using maps to define callbacks, you should only use functions — behavior of map event handling is an implementation detail that is subject to change.

Built-in components

Reveal provides an access to various built-in components:

Pluggable context menu

Fancy visualizations don’t have to be leaf nodes that you can only look at — wouldn’t it be nice to select a data point on a plot and explore it as a value? Reveal supports this continued data exploration for built-in views like charts and tables out of the box. In addition to that it provides a way to install the action popup on any JavaFX node with a special component — popup-view:

{:fx/type rx/popup-view
 :value (the-ns 'clojure.core)
 :desc {:fx/type :label
        :text "The Clojure language library"}}

This description shows label that you can request a context menu on, and its popup will suggest acions on clojure.core ns. There is a bigger example showing how to create a custom view for a chess server that displays active games as chess boards and allows inspecting any piece:

Custom views demo

Closing thoughts

If repl is a window to a running program, then Reveal is an open door — and you are welcome to come in. I get a lot of leverage from the ability to inspect any object I see, and I hope you will find Reveal useful too.