Interop with Other Languages

! This section is a draft. Many important details have yet to be ironed out.

A major goal of Puck is minimal-overhead language interoperability while maintaining type safety.

The problems of interop

There are three issues that complicate language interop: 1. The language of communication, i.e. the C ABI. 2. Conflicting type systems, i.e. Python vs. Rust 3. Conflicting memory management systems, i.e. tracing / reference counting vs. ownership

For the first, Puck is being written at the same time as the crABI ABI spec is in development. crABI promises a C-ABI-compatible, cross-language ABI spec: which would dramatically simplify the task of linking to object files produced by other languages (so long as languages actually conform to the ABI). It is being led by the Rust language team, and both Nim and Swift developers have expressed interest in it, which bodes quite well for its future.

For the second, Puck has a type system of similar capability to that of Rust, Nim, and Swift: and thus interop with those languages should be a straightforward exchange of types. Its type system is strictly more powerful than that of Python or C, and so interop requires additional help. Its type system is equally as powerful as but somewhat orthogonal to Java's, and so interop will be a little more difficult.

For the third: Puck uses what amounts to a combination of ownership and reference counting: and thus it is exchangeable in this regard with Rust. Nim and Swift, by contrast, use reference counting: which is not directly compatible with ownership, as attempting to use an owned type as a GC'd reference will immediately lead to a use-after-free. Puck may have to explore some form of gradual typing at linking-time to accommodate making its functions available for use. Using functions from GC'd languages, however, is perfectly doable with the refc type: though this may necessitate copying object graphs over the call boundary.

There is additional significant work being put into the use of Wasm as a language runtime. Wasm allows for - among other things - the sharing of garbage collectors, which means that any garbage-collected language compiling to it can simply use the primitive refc type to denote a garbage-collected reference. This does not, however, immediately work off the bat with ownership: as ownership necessitates certain invariants that garbage collection does not preserve. There is active research into fixing this: notably RichWasm, which retrofits a structural type system with ownership atop Wasm. Such extensions necessitate the runtime environment to implement them, however, and so Puck may have to explore some form of gradual typing for the broader Wasm ecosystem.

Usability

use std.io
use rust.os.linux
use nim.os.sleep
...

Languages often focus on interop from purely technical details. This is very important: but typically little thought is given to usability (and often none can be, for necessity of compiler support), and so using foreign function interfaces very much feel like using foreign function interfaces. Puck attempts to change that.

@[form(this-function)]
pub func this_function() = ...

A trivial concern is that identifiers are not always the same across languages: for example, in Racket this-function is a valid identifier, while in Puck the - character is disallowed outright. Matters of convention are issues, too: in Puck, snake_case is preferred for functions and PamelCase for types, but this is certainly not always the case. Puck addresses this at an individual level by attributes allowing for rewriting: and at a language level by consistent rewrite rules.

...todo...

Existing systems to learn from: - The Rust ABI - rust-interop - CBindGen - swift-bridge - Kotlin C interop - rust-lang-interop - extern in Rust - NimPy - JNim - Futhark - Haxe's callfunc