CSP channels in javascript/python

Nguyễn Tuấn Anh

ubolonton@gmail.com

Table of Contents

The Code

Async IO

Reactor pattern

  • Single-threaded event loop
  • IO happens elsewhere
  • Register callbacks after signaling the world
  • Use callbacks to handle external signals

Where?

  • Node.JS
  • Twisted, Tornado, Tulip
  • Vert.x
  • Browser!

Callback Hell

fs.readdir(source, function(err, files) {
  if (err) {
    console.log('Error finding files: ' + err)
  } else {
    files.forEach(function(filename, fileIndex) {
      console.log(filename)
      gm(source + filename).size(function(err, values) {
        if (err) {
          console.log('Error identifying file size: ' + err)
        } else {
          console.log(filename + ' : ' + values)
          aspect = (values.width / values.height)
          widths.forEach(function(width, widthIndex) {
            height = Math.round(width / aspect)
            console.log('resizing ' + filename + 'to ' + height + 'x' + height)
            this.resize(width, height).write(destination + 'w' + width + '_' + filename, function(err) {
              if (err) console.log('Error writing file: ' + err)
            })
          }.bind(this))
        }
      })
    })
  }
})

Problems

  • Loss of context
  • No local reasoning
  • Forced creation of non-reusable functions
  • Hard to combine with normal programming constructs

Promises?

Nice try, but not enough

CSP

Communicating Sequential Processes

Processes

Independent logical threads of execution

go(function*() {
  yield sleep(1000);
  return 1;
});

Lightweight processes

Communicating

Mechanism: channels

Unbufferred

Synchronization

Buffered

Queue

Taking

var value = yield take(ch);

Putting

var stillOpen = yield put(ch, value);

Selecting

var r = yield alts([ch1, ch2, [ch3, value]]);
if (r.channel === ch1) {
  ...
}

Basic patterns

Fastest chosen

// Fastest bidder wins
yield alts([fromClient1, fromClient2, fromClient3]);

Timeout channel

for (;;) {
  yield alts([keydowns, timeout(50)]);
  console.log("Type faster!");
}
for (;;) {
  yield alts([pings, timeout(5000)]);
  console.log("Client hasn't pinged us for 5 seconds!");
}

Backpressure

Slow down fast producer by using a fixed-size buffer to avoid saturating consumer

Other patterns

Roll-up with timeout

Avoid sending too many emails on repeated failures

go(function*() {
  for (var t = timeout(60000);;) {
    for (var count = 0;;count++) {
      var r = yield alts([errors, t]);
      if (r.channel === t) {
        if (count > 0) {
          yield put(mails, count + " errors in the last 60 seconds");
        }
        break;
      }
    }
  }
});

Control channels

go(function*() {
  for (;;) {
    var value = yield take(control);
    if (value === "off") {
       continue;
    }
    for (;;) {
      var r = yield alts([control, works])
      if (r.channel === works) {
         doWork(r.value);
      } else {
        if (value === "off") {
          break;
        }
      }
    }
  }
});
// In other places
yield put(control, "on");
yield put(control, "off");

Comparison with Promises/Deferreds

Local reasoning

Encapsulated in the process

Sequential reasoning

Combined with the previous => serialized access to local state

Separation of concerns

Each process handles a single task

Behaviors vs. data: processes vs. channels

Composability

With normal programming construct: conditionals, loops, exceptions

Series of events

Long-running processes handle repeated signals

Testability

Wire channels to fake generated data and logging instead of real IO

Again this comes to channels being a sequence of things, not individual things

Faster Maybe?

The cost is paid per-process, not per-signal

Implementation

Underneath

  • Callback (what else can it be?)
  • Low-level API: putAsync, takeAsync

On Top

  • Killer idea: generators
  • Single best feature of ES6
  • yield is 2-way communication

CSP in other languages

Go

  • n-to-n
  • Built-in
  • Typed
  • Syntax checking
  • Read-only, write-only (type checking)
  • Lightweight processes (goroutines)

Clojure/Clojurescript

  • n-to-n
  • Library, macro-ed
  • Runtime checking
  • Communications must be top-level
  • Lightweight processes (go blocks) for IO-bound operations
  • Normal threads for CPU-bound operations

Rust

  • 1-to-1, n-to-1
  • Built-in
  • Typed
  • Syntax checking
  • Read/write pair
  • Lightweight processes (tasks)

Future directions

For js/py CSP

Better interface for real-time client-server communication

Channels on both side

Better interface for cross-browser-tab communication

Channels on all tabs

Deadlock detection

  • Process state
  • Waitables that are not channels

Settle on a common error handling strategy

  • Exceptions?
  • Error on result channel?
  • Keeping stacks?

Contact

ubolonton on Github, LinkedIn, Skype, Facebook, Twitter, Google+…

Any question, comment, feedback, or correction is welcome.