Rust Integration with GC Langs

ubolonton

Created: 2018-08-25 Sat 14:30

What is Rust?

  • Fast system language
  • Safe, high-level
  • Good tooling
  • Great community

Brief Feature List

  • Statically managed memory: ownership, borrowing
  • Separation of data and behavior: trait
  • Generics
  • Pattern matching
  • Type inference
  • No nulls
  • Type-checked concurrency
  • Aspects of functional programming
  • Lexical macros

Why Integrate with GC Langs?

  • Easily write fast, parallel (GIL workaround) native extensions
  • Add Rust to existing codebase with low friction
  • Glue together different GC runtimes

Existing Bindings

Idiomatic Rust bindings on top of C/C++ FFI

Javascript: Neon

  • npm integration
  • Largest community
  • Other: Josephine (SpiderMonkey)

Python: PyO3

  • setuptools integration
  • Requires nightly
  • Other: milksnake

Elixir: Rustler

  • Looks cleanest
  • hex integration

Other

  • Ruby: Helix
  • R: rustr
  • Emacs: emacs-module-rs
  • JVM/Go: Maybe you can start one?

Common Patterns

Context

  • Main interaction point
  • Usually created by GC runtime
  • Short-lived (non-static)
Context<'a>                     // Neon
Python<'p>                      // PyO3
NifEnv<'a>                      // Rustler
&Env                            // emacs-module-rs

Handle

  • GC runtime owns data
  • Rust sees opaque handles
  • Context converts handles into borrowed/owned data
  • Borrow rules are checked dynamically
Handle<'a, T: Managed + 'a> // Neon
PyObject                    // PyO3
NifTerm<'a>                 // Rustler
Value<'e>                   // emacs-module-rs

Function Calls

Rust Calling GC-Lang Functions

  • Call a function on context
  • Pass handles
pub fn call(&self, name: &str, args: &[Value]) -> Result<Value>

GC Lang Calling Rust Functions

  • Special functions taking context and handles
  • Macros for declaration
pub fn(env: &Env, args: &[Value]) -> Result<Value>

GC Interaction

Rust-created Data

  • Ownership is transferred to GC runtime
  • "Internal mutability", RefCell, Arc

Rust Data Referencing GC Data

  • GC doesn't know about this
  • Solution 1: Require trace hints on transferred data
  • Solution 2: Disallow referencing GC data without rooting/refcount-wrapping

Removing GC Data from Rust Data

  • GC can no longer reach this
  • Solution 1: Require &mut Context to call into GC runtime
  • Solution 2: Disallow referencing GC data without rooting/refcount-wrapping

Newly-created GC Data on Rust Stack

  • GC doesn't know about this
  • Solution 1: Require explicit rooting/refcount-wrapping new GC data
  • Solution 2: Add implicit tracking to runtime/binding

Error Handling

  • GC: non-local exits (exceptions)
  • Rust: Result<T, E>
  • Conversion at function call borders

Issues

  • Most bindings have little docs (but work well once set up)
  • Writing bindings are hard, with many interleaving concerns (it's fun though)
  • Many runtimes' C interfaces are not well-documented
  • Runtimes with parallel mutability are hard (JVM, Go)