Bitwig Studio Tutorial 4:
Observers

Bidirectional Communication with your Controller.

On KVR, many users wanted to simply map CCs freely in Bitwig, but have bidirectional communication, so that motor-faders, LEDs or virtual controls in apps like the Lemur react and reflect the current state of things in Bitwig Studio and vice versa.

For this to work, we will look into the concept of “Observers”, which is a core concept throughout the whole BWS Controller API and therefore very important to grasp.

If like me you come from less event-driven coding in C and other such languages, this may feel strange at first. You don’t simply request the current value of a control, since that could interfere with more important tasks. Instead, you set up observers that then provide and update the data whenever there is a change, and only then, so you can react to such changes however you want.

Makes sense, but makes things also work a bit different from classical linear programming.

 

EDIT: Please be aware that this tutorial was created for Bitwig Studio 1.0. In version 2.x the API has changed a lot and there are now other ways to get at the values of parameters although this article is still valid AFAIK.
Please refer to the latest API docs to make sure you get the latest information!

 

In this tutorial I will create an extension to my “TomsGenericMultiControl.control.js” which you may be familiar with already – find it here if not: Toms Generic Bitwig Scripts on Github
The end result resembles my script “TomsMultiBiController.control.js” if you want to see things in a bigger context.

I want to be able to not only map all CCs to GUI controls, but also update my controller when something changes in BWS or if I load a new Song etc. For this we need basically three ingredients:

  • First a place to store all the values.
  • Second a definition of the connection between those values and the CCs – these are the already mentioned “Observers”.
  • Third a place to send the values back to the controller.

Let’s start with the most simple part, the variables:

loadAPI(1);

host.defineController("TomsScripts", "TomsMultiBiController", "1.0", "f03747e0-d9f0-11e3-9c1a-0800200c9a66");
host.defineMidiPorts(1, 1);
host.addDeviceNameBasedDiscoveryPair(["YourMidiInPortNameHere"], ["YourMidiOutPortNameHere"]);

var LOWEST_CC = 1;
var HIGHEST_CC = 119;

var ccValue = initArray(0, ((HIGHEST_CC - LOWEST_CC + 1)*16));
var ccValueOld = initArray(0, ((HIGHEST_CC - LOWEST_CC + 1)*16));

The first couple of lines you may know already from my previous scripts.

We define a controller with Category, Name, Version and a unique UUID (for instance from here: http://www.famkruithof.net/uuid/uuidgen), the UUID being unique is extremely important since this is how the controller is recognised internally, not by the name! So be sure if you create your own scripts to get a unique UUID as the first step.

Next we define a Midi Port with one Input and one Output.

Then there is the optional Discovery Pair for auto-detection. If you want, you can replace the two placeholder names with the exact names of the Midi In and Midi Out of your controller.

Next in line are two constant definitions of the lowest and the highest mappable CCs (Constants like this usually use all caps in their names to make clear that they are not meant to be changed). CC 0 is Bank Select which we leave alone. The CCs above 119 are also reserved for things like Panic and Midi Modes and are left out since they may interfere with the normal working of your gear.

Now to the first new element: We create a “var” (-iable) called “ccValue” and we do so using a small helper function that comes with Bitwig but isn’t part of the Java-API. Instead it is part of a collection of very helpful (Java-) scripts that reside in the installation directory in a folder called “resources\controllers\api\”. At the time of this writing there are four scripts, “midi.js” with helpful midi tools, “sysex.js” making life easier with sysex data, “api_v1.js” which contains the “println()” function for easy output to the Bitwig Script Console and finally “utils.js” which holds several utility functions, one of them being “initArray” – it looks like this:

function initArray(value, length) {
   var arr = [], i = 0;
   arr.length = length;
   while (i < length) { 
      arr[i++] = value; 
   }
   return arr;
}

You provide it with an initialisation value and the length of the array you want to create and it builds it setting all array members to that given value in a “while” loop. You could create this array in other ways, but this is a rather convenient method that works no matter how large the array.

 


Basic Programming Knowledge:

An Array is basically a list of values. Since we have a lot of CCs to handle, we don’t want to create (in this case) 1904 separate variables, but instead one variable that contains 1904 values that can be accessed by an “index”, which is written into square brackets [ ].

If I want to access the 55th value in above array “ccValue”, I would write that as ccValue[54]. Since the index is zero-based, 1904 values are located from index 0 to index 1903 (and yes, this often leads to confusion and is the reason why you will often see the usage of -1 or +1 when arrays are accessed to provide for this offset).

After creation an array at first contains “undefined” values, so it is good practice to initialise it with some basic neutral values, in this case zeroes.


 

So back to our script, we tell the function to set all array members to zero and make it a length of the highest CC minus the lowest CC + 1 and multiplied by 16 for the 16 midi channels. In this case we could just write 119*16 but we want to keep it flexible in case somebody only wants to use the first 20 CCs for instance, so it makes sense to use the variables we set up earlier so they can be changed in one central place.

We need a second identical array ccValueOld later on, so it’s created in exactly the same way.

First step completed ;-)

Now for the meat:

In the scripts “Init” function, which is executed once when the script is loaded, we now want to create the relation or connection between the parameter values in the array and the mapped functions in Bitwig Studio:

userControls = host.createUserControls((HIGHEST_CC - LOWEST_CC + 1)*16);
for(var i=LOWEST_CC; i<=HIGHEST_CC; i++) {
   for (var j=0; j<16; j++) {
      var c = i - LOWEST_CC + j * (HIGHEST_CC-LOWEST_CC+1);
      userControls.getControl(c).setLabel("CC " + i + " - Channel " + j);
      userControls.getControl(c).addValueObserver(127, getValueObserverFunc(c, ccValue));
   }
}

The first line you already know from earlier scripts, we create a bank with as many User Controls as needed for all the CCs and save the “access code” to the resulting collection (the “object”)  in a variable called “userControls” which we then can use to work on it’s “members” (the actual controls in this case).

User Controls are basically unspecific controls that can be mapped freely and aren’t connected to a specific function in BWS – exactly what we want here.

In other areas of the script I wrote out the midi channels etc. verbose, since it can be easier for beginners to understand and test code when everything is written down and clearly visible, but of course it would be crazy to do so for our 1904 Controls. Therefore we use a loop with two layers to step through all of them.

 


Basic Programming Knowledge:

If you should be unfamilar with “for”-loops (otherwise you can skip this part):
It is a way to repeat a certain portion of code a predefined number of times. For this the for-loop needs three things, first a variable is created that is used to count the repetitions:

for (var i = LOWEST_CC;

That part is only executed the first time the loop runs, so here the variable “i” is created and set to the value of our constant “LOWEST_CC”, the value we want to start counting from. The semicolon ends that first part. Next is:

i < = HIGHEST_CC;

This part is executed every time the loop loops and means “(loop as long as) i is smaller or equal HIGHEST_CC”. Since we set i to LOWEST_CC in the first step, which in our example is 1, the loop starts with i having a value of 1 and then loops as long as it is not exceeding HIGHEST_CC.
More generally this part says “loop as long as the expression is true”.

The last part of the for-loop increases the variable i:

i++)

This is a shorthand notation for “i = i + 1” which is used so often that even the alternative notation of “i += 1” seemed too much for the programmers concerned… ;-)

This part is executed after each loop, so the comparison in step 2 has something fresh to do…
But even if i++ is the most common, you can just as well count down with i– (two minus signs) or in greater steps like i = i + 5, that is totally up to you.


 

So this first loop counts from 1 to 119 and for each loop it executes the code in the curly brackets…

…which in our case contains another for loop…   :-)

This is also something very common, since sometimes it is easier or more convenient to have multiple layers of repetition. In this case we basically separate the 119 CC numbers from the 16 midi channels this way.

The inner for-loop does exactly the same as the outer technically, but loops only 16 times for the 16 midi channels and uses the variable j for it.

Now finally inside the loop we first compute our index, that is, the number of the current control we want to map in the array we created at the beginning.

This formula became a bit involved but I hope you can follow:

The new variable “c” is our index. We take the variable i from the outer loop and subtract LOWEST_CC (since I starts with LOWEST_CC as a value, this gives us zero as a start). We then add

(j - 1) * (HIGHEST_CC - LOWEST_CC + 1)

j is 1-16 but we need 0 to 15 (so that in the first loop, the result is zero). The second part provides the “jumps” in steps of 119 per Midi Channel. The resulting sequence of c is going through the first CC for each Midi Channel, then the second etc. If you want to dissect it further, let the values print to the console (with “println(c);” ), but be warned, printing many values can bring BWS to almost a halt ;-)

Ok, now we have our index, let’s do something with it:

First, we give the controls a name:

userControls.getControl(c).setLabel("CC " + i + " - Channel " + j);

As you can see, we now go through each and every userControl we created, using c as the index. The setLabel command needs a string, so we construct one from the information we have inside our loop (and this is why the two layered loops were chosen).

i always contains the current CC number and j contains the current Midi Channel, so we can concatenate the name of the current userControl very easy. The result would be something like “CC 5 – Channel 16” for I = 5 and j = 16.

This name shows up when you map the control with a right mouse click to any control in BWS. You can also see assigned controls in the browser panel if you select the Studio I/O view. All assigned controls show up in the list below the used controller:

BitwigControllerAssignment

Now, we get to the new stuff – Be prepared for some JavaScript trickery ahead:

userControls.getControl(c).addValueObserver(127, getValueObserverFunc(c, ccValue));

Again we use our index value in “c” to select a specific control and use “addValueObserver” to add a Value Observer to it. ;-)

This function expects first a number representing the “resolution” of our control – which for plain Midi data is 128 (7 Bit, 128 steps from 0 to 127) and second a so called “callback-function” that can be used to update the value whenever it changes.

And this is the meat of the observer-business: Since everything in a controller script is driven by events (changes coming from the controller or changes coming from the application), you need a “live” connection between parameters in BWS and your controller. You only want to update the value when it changes, so you basically need to tell BWS how to update the value itself when needed and therefore you provide it with a function that does that.

Simple enough in theory, but now it is getting slippery:

The callback function you provide can only have one parameter itself (the value), but we need two, since we are dealing with many parameters: First the current index “c” into our array and second, the array itself to save the value to.

Disclaimer: if you don’t understand what comes next, don’t feel bad, it’s rather weird indeed for a non-programmer-brain. ;-)

What we do is, we provide addValueObserver with a function that creates a function, and that first function can actually have two parameters since it returns a function that has only one.

Confused? Good! You are one healthy human I would say ;-)

function getValueObserverFunc(index, varToStore) {
   return function(value) {
      varToStore[index] = value;
   }
}

This is the function that we call. It has two parameters, the first is the current index and the second is the variable to store the value in. This function in turn contains a function that is returned to the caller with

return function(value)

And as you can see, this function only has one parameter as required. But since it is created in the scope of the surrounding function, we can also access “index” and use it to select the correct place in the array to save the value to.

 


Basic Programming Info:

“Scope” is a term that describes the validity and lifetime of variables in code.
There are “global variables” which are declared outside of any functions. They can be accessed from everywhere in your code, which sometimes is good but often is problematic, since if you write into the same variable from different places, it can be hard to keep track what is written where when. So usually it’s recommended to avoid global variables. They exist for the whole runtime of a script or programm. They have “global scope”.

And then there are “local variables” which are only accessible from inside certain code constructs. In almost every other language than JavaScript there is the concept of “block scope” which means, that a variable I create inside a code block – for instance an if block or a for-loop – is local to that block and it’s sub-blocks and destroyed after that code block is finished. 
But JavaScript is different. It doesn’t have block scope, but only “function scope”. This means that only variables I declare with “var” inside of a function are “hidden” from the outside world and other functions,  they are local to that function.

But wait, there’s more:
There is another concept in JavaScript called “Closure”:
If I have one function inside another function, the inner function can access the variables of the outer function even after that outer function has returned!

This is totally alien to any user of C-like languages where local variables cease to exist as soon as a function has returned (= has finished what it was doing and the program returned to where the function was originally called from).
 But exactly this principle is at work here: The inner function with only one parameter can still access the index variable of the outer function even after that outer function has done it’s work and returned.


 

That was the weird part. The rest is peachy. ;-)

Now that we have our arrayed variables wired with an observer each to update them whenever something changes, we only need to in turn update our controller if the value changes – and only then.

Bitwig Studio has internal parameter thinning, that is, it reduces the amount of controller data that has to be sent through Midi connections, since Midi isn’t really made to transport thousands of values per millisecond. The data you are getting from the observers is already reduced and combined to a sensible data rate that should be manageable.

To show another part of the API, we don’t send the midi data directly from the observers but use the flush() function instead, which provides a central place to send controller feedback. Like init() or exit() it doesn’t have to be called explicitly, but is automatically called each time observed parameters have changed:

function flush() {
   for(var i=LOWEST_CC; i<=HIGHEST_CC; i++)
   {
      for (var j=1; j<=16; j++) {
         var c = i - LOWEST_CC + (j-1) * (HIGHEST_CC-LOWEST_CC+1);
         if (ccValue[c] != ccValueOld[c]) {
            sendChannelController(j-1, i, ccValue[c]);
            ccValueOld[c] = ccValue[c];
         }
      }
   }
}

First you will recognise the same loop construct we used before. Once again we go through all our controls, this time to see if something has changed.

For that we finally use our second array, ccValueOld. We compare if the value at the current index is the same as in our ccValue array (which is updated automatically by BWS via the observer) and only do something if it’s different.

 


Basic Programming Knowledge:

!= stands for “is not equal”, so the test is “true” if the two values are different.
“Execute the code in the curly brackets if the two values are NOT equal”.


 

If that is the case, we send out midi to the controller.

We use a helper function from the API helpers in midi.js to do so: “sendChannelController”. It first needs the Midi Channel (we need to subtract 1 because it wants the Channels numbered 0 to 15), then the CC number and finally the actual value, which is stored in ccValue.

Finally we update the variable ccValueOld for the current index to the same value as ccValue so that we can compare it again in the next flush() iteration to see if something has changed then compared to now.

And this concludes this rather epic tutorial…

Let me know in the comments if it was helpful, if you find any errors or if something is missing or you have questions left not answered here.

Happy Scripting!!!

Cheers,

Tom

Posted in Audio, English, Tutorials | Tagged , , , , | 9 Comments

9 Responses to Bitwig Studio Tutorial 4:
Observers

  1. Ludow says:

    hello Toms

    thank you for your script

    I have a problem with the pitch bend on my Roland Juno DI.
    When I record, pitch bend is captured.

    But for playback, pitch bend information is not transcribed on Juno DI.

    According to the Midi Monitor software, MIDI message is “Pitch Wheel”

    You have an idea to solve the problem

    I also have full MIDI/SYSEX implementation for Roland Juno DI if needed.
    http://www.media.rolandus.com/manuals/JUNO-Di_MI.pdf

    English is not my first language (I’m french), so please excuse any mistakes

    thank you

  2. Thomas says:

    Can you check if on the track you record to there is a checkmark in the Inspector on “Convert Pitch-bend”? That should only be active for factory synths (it converts Pitch-bend to per-Note Expressions). Simply uncheck it and try again. If that doesn’t solve it, maybe open a thread on KVR – more convenient place for discussions ;-)

    Cheers,

    Tom

  3. Ludow says:

    It works , you are awesome
    thank you for your help

  4. Thomas says:

    Great!

    Cheers,

    Tom

  5. kirkwood says:

    Very cool script! I’m getting it working on my BCR2000. Values are going into bitwig but not coming out to adjust the led rings.

    I’ve been able to make this work in my other DAWs. So i’m not sure whats up.

    I put some debug prints in your script to analize flush and the onMidi. the same data is being passed in and out of those. So it appears to me there is an issue with
    this function: sendChannelController()…

    But i’m really not sure or what do do next… any ideas?

    • Thomas says:

      Well, did you check out the BCR2000 and BCF2000 scripts that come with Bitwig? They may contain a clue. I’m not aware of bugs with sendChannelController so I don’t know what’s up. Are you sure you are sending the right CCs on the correct channel?

      Cheers,

      Tom

    • Ch00rD says:

      Fwiw, I’ve tested my script with a BCR-2000, and everything seems to work as expected here, including feedback to the encoder rings. The sendChannelController() function seems to work as expected, too (although I’m a JavaScript noob, too, so don’t take my word for that last one). Perhaps try again, and let me know in the comments if there’s anything else I can do to help. Good luck!

  6. Ch00rD says:

    Hi Tom, and thank you very much for sharing your scripts and tips! :)

    I have forked your “TomsMultiBiController” script, and heavily edited it to implement support for 14-bit MIDI CC pairs; which seems to work as expected, at least so far… :) I’ll probably extend it with more functionality as I go along (I’m a complete Bitwig noob), but I wanted to share it here as a ‘thank you’ in its current (still relatively basic and straightforward) state, as that may be more useful for other users for further learning and hacking.

    Btw, will you be at the Musikmesse next week? I remember us briefly meeting over a lovely spiked coffee at the stand of u-he’s crew, I hope to make it next week, so perhaps we can meet up and grab ourselves another one. Or two. Or three… ;)

    In any case, cheers!

    while (!drunk) do (
    coffee++;
    coffee += rum;
    );

    • Thomas says:

      Hey Ch00rD,
      I’m currently working on completely different stuff (procedural 3D with Rhino/Grasshopper :-) ) but it’s good to see that the scripts still provide some insight.
      Yeah, Musikmesse was lovely, but sadly I won’t be there this year. I hope you’ll have a good time! ;-)

      Cheers and thanks for your helpful replies!

      Tom

Leave a Reply

Your email address will not be published. Required fields are marked *

*