Value Scrubbing / Graphical Value Pickers

[doug-moen]

"Value Scrubbing" is the ability to modify a value in a Curv program using direct manipulation. For example, tweaking a numerical value using a graphical slider. While you drag the indicator on the slider, the shape in the Viewer window updates in real time. This is an important "live coding" feature that makes it easy to explore the effect that different parameter values have on a parametric shape.

Here are two approaches that we could take in Curv. They aren't mutually exclusive, and they serve different types of users.

Value Pickers in the Viewer Window

In one approach, you modify the source code and declare which parameters can be "scrubbed" using an interactive value picker. The declaration includes the type of value picker, and additional information like the start and end values of a numeric slider. When the program is evaluated and the shape is displayed, a collection of value pickers is displayed in the Viewer window along with the shape. This is similar to the Thingiverse customizer, the OpenSCAD customizer, and Fragmentarium.

In this approach, a developer can design a parametric shape, and create a "user interface" for tweaking the parameters. Then another user, who may not understand the code, can use the value pickers in the Viewer window to customize the shape.

Value Pickers in the Editor Window

In the other approach, you select a literal constant in the source code, and a value picker appears above the constant for scrubbing the value. Examples:

• Bret Victor video: https://www.youtube.com/watch?v=PUv66718DII&t=11m45s
• The Book Of Shaders, which implements this UI within a web page: https://thebookofshaders.com/05/

Used by developers, for writing Curv programs.

[doug-moen]

How to implement value pickers in the editor window

If you are using a conventional text editor to edit Curv programs, then it isn't possible to implement value pickers in the editor window. The default gedit isn't powerful enough, and vim running in a terminal window can't do this either. Maybe an existing IDE could be used (with what plugins/extensions?), and/or maybe we implement a text editor using Qt, as part of an integrated Curv GUI.

@doug-moen said:

I've thought about embedding Curv in a conventional IDE. But the features that will turn Curv into a reactive visual programming environment, like the glslEditor slider mechanic and other Brett Victor-inspired features, are not found in IDEs.

That kind of points to writing a GUI in C++, probably using Qt, just like OpenSCAD and libfive Studio.

@s-ol replied:

as for the IDE debate - as a vim user I really enjoy curv's current editor-independent live-reloading and general application paradigm. Building 'yet another IDE' sounds a bit "silly" to me - LSP sounds like a better choice for "standard level support". For scrubbing values etc. maybe the LSP protocol can be extended - after a quick look at the specification it seems the protocol would easily support something like that:

the rename/prepareRename mechanism seems similar for example; the client sends a prepareRename to find out whether a location in the text is rename-able, if so, it can preset renaming-UI, and when the user enters a name the rename request can go through. In this second phase the language server generates the project-wide change-sets and the editor only commits them.

Mapping this to scrubbing, there could be a prepareScrub request that yields information about the UI to be displayed (value type, value range etc.). When this request succeeds a corresponding UI is activated that sends previewScrub requests that allow the preview (doubling as the language server) to preview the value changes in real time and finally a finishScrub request to finalize and write back the changes. (previewScrub and finishScrub might also be the same).

This extension sounds general enough to propose to LSPs at large and simple enough to fork and maintain in one 'officially-curv-supported' IDEs LSP implementation even if the proposal doesn't become part of the specification.

[doug-moen]

Implementing value pickers in the viewer window

This is my current project. I'm building an experimental proof-of-concept. The idea is to first create working code, then improve the design.

A parametric shape is a shape value containing additional metadata which describe its parameters. This allows the Viewer window to display graphical sliders for modifying numeric shape parameters. The shape animates as the sliders are moved. (A primary design goal is to represent parametric shapes as first class values, rather than by placing a special interpretation on source files that contain specially formatted comments.)

Right now, you can construct parametric shapes using code like this:

make_parametric << {
    diam :: slider(0.5,4) = 1;
    len :: slider(2,8) = 4;
} -> let
    candy = sphere diam >> colour red;
    stick = cylinder {h: len, d: diam/8} >> move(0, 0, -len/2);
in
    union(candy, stick)

diam and len are the two parameters that you control using sliders. This code runs, but the sliders do not appear in the Viewer window yet. And we might want to replace make_parametric with a better syntax.

Parametric shapes are supported by the following language features:

• parametric records, currently constructed using make_parametric.
• value picker predicates, like slider(lo,hi), which specify the type and range of a graphical value picker.
• predicate patterns (name :: predicate) which are used here to associate shape parameters with value picker types.

Parametric Records

A parametric record remembers how it was constructed. It remembers its construction parameters, and it allows you to selectively change those parameters and make a modified copy of the record. This metadata is stored in two record fields: parameter and call.

• A parametric record has a set of named parameters. The names and values of these parameters are stored as a record value in the parameter field.
• A parametric record has a function which constructs a new instance of the record, when given a parameter record as an argument. This is stored in the call field. Whenever a record value R has a call field, that means it can be called like a function, so R(arg) is the same as R.call(arg).

make_parametric is the current API for constructing a parametric record. (Although you can also create one "by hand", I wanted a more convenient, higher level syntax.) make_parametric is a built-in function, which takes a function F as an argument, and returns a parametric record as a result. The function argument F looks like this:

{ d :: slider(1,10) = 2 } -> cube d

On the left of the -> is a record pattern, with one field pattern for each named parameter in the parametric record. Each field pattern contains the parameter name (eg, d), then :: and a value picker predicate like slider(1,10), then = value specifying the initial value of the parameter.

On the right side of the -> is an expression that use the named parameters to construct a record value, which in this case is a shape, cube d.

Predicate Patterns

A predicate is a function that returns true or false. Predicates are used to classify values. For example, there are some built in predicates like is_bool and is_num that classify values according to their type.

Within a function definition like this:

incr f (n :: is_num) = n + 1;

the phrase n :: is_num is a predicate pattern that restricts the parameter n to being a number. This is similar to a type declaration in other languages.

Value Picker Predicates

To support parametric shapes, we are defining a set of value picker predicates. These are predicate functions that carry extra metadata that describe the type of a value picker, and additional parameters specifying the range of values supported by the value picker.

The initial prototype will just support sliders: slider(lo,hi) specifies a slider that ranges between the numbers lo and hi. In the future, I would also like to support boolean checkboxes, colour pickers, drop down lists giving a fixed set of alternatives, and more.

[s-ol]

I would like to see logarithmic sliders on the Todo list here!

If you don't want to deal with all the edge cases of UI inputs and get a lot of different input means "for free" you might want to look into dear imgui, it's a well known, robust and variable UI framework that stays out of your way and doesn't need to own or invade your data structures, which makes integrating it very easy.

[doug-moen]

@s-ol Thanks for the recommendations. Logarithmic sliders are a good idea.

I am looking into GUI toolkits. I haven't tried any yet.

• Dear IMGUI:
  • Stores state in global variables. Not thread safe. If I use it, then I can't have more than one Viewer window open at the same time, running in different threads. Conflicts with the design of class Viewer, which contains all of the state for a Viewer window. But there is a goal to fix this in the future.
  • Separation of GUI library from backends. Backends for OpenGL 2, OpenGL 3, Vulkan, DX9-12, Metal.
• NanoGUI:
  • Supports multiple Viewer windows: all state is properly encapsulated in classes.
  • No GUI/backend separation. Supports OpenGL 3 or later only.
• Nuklear:
  • No global variables. Separates GUI from backends, but fewer backends than IMGUI. Fewer widgets. No radial colour picker.
• QT: a huge dependency, requiring a major investment in time to switch to.

[doug-moen]

There is now an experimental implementation of slider controls in the Viewer window.
See: examples/lollipop.curv.

• It is a work in progress.
• The syntax is experimental and subject to change.
• It is implemented using Dear IMGUI. As part of integrating IMGUI, Curv now requires OpenGL 3.2 (up from OpenGL 2.1).

[doug-moen]

The syntax is now more convenient: the old make_parametric function has been replaced by a parametric keyword for declaring picker parameters. There is now a checkbox value picker, and more in the pipeline. Look in examples/picker for working example code.

[doug-moen]

First public preview, plus documentation: https://github.com/doug-moen/curv/blob/master/examples/picker/README.md