CSP channels in javascript/python

The Code

• https://github.com/ubolonton/js-csp
• https://github.com/ubolonton/twisted-csp

Async IO

Reactor pattern

• Single-threaded event loop
• IO happens elsewhere
• Register callbacks after signaling the world
• Use callbacks to handle external signals

Where?

• Node.JS
• Twisted, Tornado, Tulip
• Vert.x
• Browser!

Callback Hell

fs.readdir(source, function(err, files) {
  if (err) {
    console.log('Error finding files: ' + err)
  } else {
    files.forEach(function(filename, fileIndex) {
      console.log(filename)
      gm(source + filename).size(function(err, values) {
        if (err) {
          console.log('Error identifying file size: ' + err)
        } else {
          console.log(filename + ' : ' + values)
          aspect = (values.width / values.height)
          widths.forEach(function(width, widthIndex) {
            height = Math.round(width / aspect)
            console.log('resizing ' + filename + 'to ' + height + 'x' + height)
            this.resize(width, height).write(destination + 'w' + width + '_' + filename, function(err) {
              if (err) console.log('Error writing file: ' + err)
            })
          }.bind(this))
        }
      })
    })
  }
})

Problems

• Loss of context
• No local reasoning
• Forced creation of non-reusable functions
• Hard to combine with normal programming constructs

Promises?

Nice try, but not enough

CSP

Communicating Sequential Processes

Processes

Independent logical threads of execution

go(function*() {
  yield sleep(1000);
  return 1;
});

Lightweight processes

Communicating

Mechanism: channels

Unbufferred

Synchronization

Buffered

Queue

Taking

var value = yield take(ch);

Putting

var stillOpen = yield put(ch, value);

Selecting

var r = yield alts([ch1, ch2, [ch3, value]]);
if (r.channel === ch1) {
  ...
}

Basic patterns

Fastest chosen

// Fastest bidder wins
yield alts([fromClient1, fromClient2, fromClient3]);

Timeout channel

for (;;) {
  yield alts([keydowns, timeout(50)]);
  console.log("Type faster!");
}

for (;;) {
  yield alts([pings, timeout(5000)]);
  console.log("Client hasn't pinged us for 5 seconds!");
}

Backpressure

Slow down fast producer by using a fixed-size buffer to avoid saturating consumer

Other patterns

Roll-up with timeout

Avoid sending too many emails on repeated failures

go(function*() {
  for (var t = timeout(60000);;) {
    for (var count = 0;;count++) {
      var r = yield alts([errors, t]);
      if (r.channel === t) {
        if (count > 0) {
          yield put(mails, count + " errors in the last 60 seconds");
        }
        break;
      }
    }
  }
});

Control channels

go(function*() {
  for (;;) {
    var value = yield take(control);
    if (value === "off") {
       continue;
    }
    for (;;) {
      var r = yield alts([control, works])
      if (r.channel === works) {
         doWork(r.value);
      } else {
        if (value === "off") {
          break;
        }
      }
    }
  }
});
// In other places
yield put(control, "on");
yield put(control, "off");

Comparison with Promises/Deferreds

Local reasoning

Encapsulated in the process

Sequential reasoning

Combined with the previous => serialized access to local state

Separation of concerns

Each process handles a single task

Behaviors vs. data: processes vs. channels

Composability

With normal programming construct: conditionals, loops, exceptions

Series of events

Long-running processes handle repeated signals

Testability

Wire channels to fake generated data and logging instead of real IO

Again this comes to channels being a sequence of things, not individual things

Faster Maybe?

The cost is paid per-process, not per-signal

Implementation

Underneath

• Callback (what else can it be?)
• Low-level API: putAsync, takeAsync

On Top

• Killer idea: generators
• Single best feature of ES6
• yield is 2-way communication

CSP in other languages

Go

• n-to-n
• Built-in
• Typed
• Syntax checking
• Read-only, write-only (type checking)
• Lightweight processes (goroutines)

Clojure/Clojurescript

• n-to-n
• Library, macro-ed
• Runtime checking
• Communications must be top-level
• Lightweight processes (go blocks) for IO-bound operations
• Normal threads for CPU-bound operations

Rust

• 1-to-1, n-to-1
• Built-in
• Typed
• Syntax checking
• Read/write pair
• Lightweight processes (tasks)

Future directions

For js/py CSP

Better interface for real-time client-server communication

Channels on both side

Better interface for cross-browser-tab communication

Channels on all tabs

Deadlock detection

• Process state
• Waitables that are not channels

Settle on a common error handling strategy

• Exceptions?
• Error on result channel?
• Keeping stacks?

Contact

ubolonton on Github, LinkedIn, Skype, Facebook, Twitter, Google+…

Any question, comment, feedback, or correction is welcome.