acb's technical journal

Custom Max for Live user interface elements using jsui

If you're writing a Max for Live instrument or effect, at some point you may find the standard user interface components that come with Max—the dials, sliders buttons and menus, and the built-in grid and step sequencer—to be limiting. Fortunately, however, it is possible to create custom interface elements in JavaScript, using the jsui object, albeit with some gotchas.

jsui allows you to create a responsive custom UI in a box in your Max for Live control's interface, by writing code for drawing the interface and responding to user events in JavaScript. If you're familiar with writing web apps using JavaScript, HTML and CSS, however, you're in for a surprise, as the only familiar thing will be the JavaScript syntax. Max does not make use of any other part of the web stack, instead replacing it with a programming model drawn from OpenGL, an API used for 2D and 3D graphics.

In short, programming a jsui widget is similar to graphical user interface programming in other systems. You have to define a drawing function which draws the interface and event handlers to respond to mouse clicks. You will also want to interact with the rest of the Max patch around your widget, handling incoming data and messages and sending data out.

Which all sounds easy in principle, though there are a few gotchas. One of them is Max's use of OpenGL, though this is mitigated by a fairly straightforward basic interface. One thing you will have to deal with is jsui's somewhat peculiar way with screen coordinates. Events such as mouse clicks are reported in screen coordinates (i.e., pixel coordinates), with the origin (0,0) being in the top left corner; however, drawing is done in OpenGL viewport coordinates; i.e., the control's view rectangle is defined as having the origin at the centre, with the vertical axis going from -1 at the bottom to 1 at the top, and the horizontal axis to whatever the aspect ratio is. This makes sense for the traditional OpenGL use-case, rendering scenes seen from a camera, but for UI elements, not so much. However, it is possible to reconfigure this, as we will see.

An example component

For an example, we will be building a widget for editing a list of 1s and 2s, representing single- and double-length events. For simplicity's sake, we will assume that the sum of the list (the total duration) is constant, and the only thing that changes is the mixture of 1s and 2s. The user can click on any cell and toggle the event there between a 1 and a 2. The widget looks something like this:

To create this widget, we create a jsui element. Max immediately gives us a box with a rendered dial in it, and some JavaScript code which implements the dial. We need to edit this code, to replace it with our own. The only way to see and edit the JavaScript code in a jsui object is to send an open message to it, so we attach one like so, and press it:

This opens a window with about 200 lines of code. Much of the API is documented in the Max documentation and online, so rather than focussing on it, we will look at the steps required to create our control.

Keeping state

Our object will keep several pieces of state, in a number of variables: sequence, an array of integers, each of which is either 1 or 2, and maxlen, which is the (constant) total duration, like so:

var sequence = [ 1, 1, 1, 1, 1, 1, 1, 1];
var maxlen = 8;

For the time being, we are keeping these as global variables, as in the default dial control, but for anything more complex, encapsulating them in a class would be more manageable.

Setting up coordinates

As mentioned above, the default jsui configuration sets up display coordinates quite differently from device coordinates. This causes a few complications, from needing to convert between two coordinate systems to making drawing lines on pixel boundaries (useful if we're drawing a screen user interface) tricky. However, it is possible to configure jsui to use screen coordinates. To do so, we find the line at the top which sets up the default 2D coordinates, which looks like:

sketch.default2d();

and replace it with the following:

with(sketch) {
    default2d();
    glmatrixmode("projection");
    glloadidentity();
    glortho(0., size[0], size[1], 0., -1,100.);
}

And now our drawing coordinates are identical to pixel coordinates. Be advised, though, that coordinates correspond to pixel boundaries not centres, and thus lines should always be drawn from the middle of pixels, i.e., 0.5.

Drawing the control

Drawing is done by a method named draw(), and operates mostly on the sketch object (which represents the control's screen area). As this method is called before the control interacts with the user in any way, it is also a good place to calculate any geometry we may need to, and store it in global variables for later use.

/* Global variables we fill in draw() */
var cellwidth = 0;

function draw() {
    var i, x ,e;
    cellwidth = Math.floor(sketch.size[0]/maxlen);
    var ymid = Math.ceil(sketch.size[1]/2.0);
    var radius = (cellwidth/2.0)-2.0;

    // clear background to white
    sketch.glclearcolor(1.0, 1.0, 1.0, 1.0);
    sketch.glclear();			

	// set foreground to black
    sketch.glcolor(0.0, 0.0, 0.0, 1.0);
    x = 0;
    for (i=0; i < sequence.length; i++) {
        e = sequence[i];
        if(e==1) {
            sketch.moveto(x+(cellwidth/2.0), ymid);
            sketch.circle(radius);
        } else if (e==2) {
	    	sketch.moveto(x+cellwidth, ymid);
            sketch.roundedplane(cellwidth/2.0, cellwidth-2.0, radius);
        }
        x += cellwidth * e;
    }
};

This loops through the list of items, and for each one, draws a single- or double-width cell. When run, it will produce an image like this:

Resizing and aspect ratio

Of course, the aspect ratio is wrong for the data we wish to display (a row of eight square cells). However, if you try to resize it, you'll notice that the element resists your attempts to change its aspect ratio, and insists on remaining square. This is because of the onresize function in the script, which is called every time the element's size changes; the implementation in the sample file we started with calls another defined function named forcesize, which in turn makes it square again. If we comment out the onresize function, we can resize the widget freely. Or, if we want it to have our cells to have a square aspect ratio, we can set it thus:

function forcesize(w,h)
{
    if (w!=h*8) {
        h = Math.floor(w/maxlen);
        w = h*maxlen;
        box.size(w,h);
    }
};

Now we can resize the control, and get something looking like this:

(If the control appears blank, save the JavaScript file; when it changes on the disk, Max will reload and rerun it, including the code setting up the window's geometry.)

User interaction

We now have an array of built-in values, and a function which draws them. However, we want the user to be able to change the values. In particular:

  • If the user clicks on a single-length cell, we want to turn it and the following cell into a double-length cell. (If the following cell is a double-length cell, the second half of it becomes a single-length cell. If the cell clicked is the last cell, we ignore it.)
  • If the user clicks on a double-length cell, we want to convert it into two single-length cells.

These operations are fairly easy to perform on JavaScript arrays. First, however, we'll need a function to determine which array index a cell in the display falls in:

function listIndexForCell(c) {
    var i, a;
    for(i=0, a=0; i < this.sequence.length; i++) {
        if(c >= a && c < a+this.sequence[i]) 
            return i;
        a += this.sequence[i];
    }
    return -1;
};

Then we can write a function for toggling the array element at a cell:

function toggleCell(c) {
    var li = listIndexForCell(c), 
        cur = sequence[li];
    if(li > -1) {
        if(cur == 1) {
            if (li < sequence.length-1) {
                if(sequence[li+1] == 1) {
                    // replace '1 1' with 2
                    sequence.splice(li, 2, 2);
                } else { 
                    // replace '1 2' with '2 1'
                    sequence.splice(li, 2, 2, 1);
		}
	    }
        } else if (cur == 2) {
	    // replace with 1 1
	    sequence.splice(li, 1, 1, 1);
        }
    }
    draw();
    refresh();
};

We can then hook this up to the click handler:

function onclick(x,y,but,cmd,shift,capslock,option,ctrl) {
    var cell = Math.floor(x/cellwidth);
    toggleCell(cell);
};
onclick.local = 1;

We can now click on cells and watch them toggle between single- and double-length events, like so.

Talking to Max

For our control to be useful, we need to be able to communicate with the Max patch it is embedded in; we want to:

  • Allow the patch to set the sequence displayed, by sending a list of length values to its inlet.
  • Send its current sequence to its outlet as a list of length values.

Setting the internal state from a list is fairly straightforward; we define a list method which will be called by Max when a list is received:

function list() 
{
    sequence = arrayfromargs(arguments);
    draw();
    refresh();
}

Sending the contents of the sequence array through the first outlet is even easier; it just involves calling outlet(0, sequence). We define a bang method to do this when a bang is received:

function bang()
{
    outlet(0, sequence);
}

It is also a good idea to add this line to the end of the onclick method, if we want the value to be sent out whenever the user changes the pattern.

Now we can embed the control in a patch, like so:

Saving state

Because your custom widget is not one of the built-in live.* controls, it has no way of automatically saving its state with presets or the Live set it's embedded in. As such, left to its own devices, it will revert to a default state every time it loads, which is somewhat less than ideal.

Luckily, it is possible to manually save state in Live, by sending it to a pattr object. This is described in the Max for Live documentation, in the section on how to allow non-Live-specific controls to be saved as presets. It works with anything that can send Max data, including custom controls. The gist of it is that you need to:

  1. Create a pattr object, with a name.
  2. In the inspector, set Parameter Mode Enable. You will need to set its type; if it sends anything other than a number, this will be Blob (i.e., its data is stored as an opaque binary object). If you wish to make the parameter saveable, Parameter Visibility should be “Stored Only” and not “Hidden”. (For single-number parameters, you can enable automation, so that the user can draw curves for them in Live, but this is not an option for lists stored in blobs.)
  3. Link the output of your control to the pattr's input, and the pattr's output to your control's input. Then when the patch loads, the pattr will send its saved data to the control, and when the control sends its state out, the pattr will save it and store it with the patch state when the Live document is saved.

The wiring should look like so:

Summary

The final JavaScript code looks like this:

/*
Basic jsui-based rhythm editor control.
By Andrew Bulhak http://dev.null.org/acb/
*/
 
with(sketch) {
    default2d();
    glmatrixmode("projection");
    glloadidentity();
    glortho(0., size[0], size[1], 0., -1,100.);
}

// state
var sequence = [ 1, 1, 1, 1, 1, 1, 1, 1];
var maxlen = 8;
var cellwidth = 0;

function listIndexForCell(c) {
    var i, a;
    for(i=0, a=0; i < sequence.length; i++) {
        if(c > =a && c < a+sequence[i]) return i;
        a += sequence[i];
    }
    return -1;
};

function toggleCell(c) {
    var li = listIndexForCell(c), 
        cur = sequence[li];
    if(li > -1) {
        if(cur == 1) {
            if (li < sequence.length-1) {
                if(sequence[li+1] == 1) {
                    // replace '1 1' with 2
                    sequence.splice(li, 2, 2);
                } else { 
                    // replace '1 2' with '2 1'
                    sequence.splice(li, 2, 2, 1);
		        }
	        }
        } else if (cur == 2) {
	    // replace with 1 1
	    sequence.splice(li, 1, 1, 1);
        }
    }
    draw();
    refresh();
};

function onclick(x,y,but,cmd,shift,capslock,option,ctrl) {
	var cell = Math.floor(x/cellwidth);
	toggleCell(cell);
};
onclick.local = 1;

function draw() {
    var i, x ,e;
    cellwidth = Math.floor(sketch.size[0]/maxlen);
    var ymid = Math.ceil(sketch.size[1]/2.0);
    var radius = (cellwidth/2.0)-2.0;

    // clear background to white
    sketch.glclearcolor(1.0, 1.0, 1.0, 1.0);
    sketch.glclear();			

	// set foreground to black
    sketch.glcolor(0.0, 0.0, 0.0, 1.0);
    x = 0;
    for (i=0; i < sequence.length; i++) {
        e = sequence[i];
        if(e==1) {
            sketch.moveto(x+(cellwidth/2.0), ymid);
            sketch.circle(radius);
        } else if (e==2) {
	    	sketch.moveto(x+cellwidth, ymid);
            sketch.roundedplane(cellwidth/2.0, cellwidth-2.0, radius);
        }
        x += cellwidth * e;
    }
};

function list() 
{
    sequence = arrayfromargs(arguments);
    draw();
    refresh();
}

function bang()
{
	outlet(0,sequence);
}

function forcesize(w,h)
{
	if (w!=h*8) {
		h = Math.floor(w/maxlen);
		w = h*maxlen;
		box.size(w,h);
	}
}
forcesize.local = 1; //private

function onresize(w,h)
{
	forcesize(w,h);
	draw();
	refresh();
}
onresize.local = 1; //private

draw();

Further work

I deliberately kept this code simple; for production code, it could (and, in practice, should) be refactored somewhat:

  • it would be good practice to encapsulate instance variables (like sequence and cellwidth) and methods (like toggleCell() and draw()) in an object class.
  • the code's handling of inputs is not as robust as it should be for production use; it should guard against illegal values on its input.
  • This code has no facility for configuring the colours of interface elements, drawing in black on white. For an example of how to improve this, see the jsui documentation in Max and the default JavaScript that jsui provides.

These improvements are beyond the scope of this article and left as an exercise for the reader.

There are no comments yet on "Custom Max for Live user interface elements using jsui"

Want to say something? Do so here.

Post pseudonymously

Display name:
URL:(optional)
To prove that you are not a bot,
please enter the text in the image on the right
in the field below it.

Your Comment:

Please keep comments on topic and to the point. Inappropriate comments may be deleted.

Note that markup is stripped from comments; URLs will be automatically converted into links.