🧚 puck - an experimental programming language

A place where I can make some bad decisions.

Puck is an experimental, memory safe, structurally typed, interface-first, imperative programming language. It aims to be consistent and succinct while performant: inspired by the syntax and metaprogramming of Nim, the error handling of Swift, the memory management of Rust and Koka, the async/await and comptime of Zig, and the module system of OCaml.

# Note: These declarations are adapted from the standard prelude.

## The Result type. Represents either success or failure.
pub type Result[T, E] = union
  Okay(T)
  Error(E)

## The Err class. Useful for dynamically dispatching errors.
pub type Err = class
  str(Self): str
  dbg(Self): str

## A Result type that uses dynamically dispatched errors.
## The Error may be any type implementing the Err class.
pub type Result[T] = Result[T, ref Err]

## Implements the `dbg` function for strings.
## As the `str` function is already defined for strings,
## this in turn means strings now implicitly implement Err.
pub func dbg(self: str) = "\"" & self & "\""

# Note: These declarations are adapted from the standard prelude.

## Syntactic sugar for dynamic result type declarations.
pub macro !(T: type) =
  quote Result[`T`]

## Indirect access. Propagates `Error`.
pub macro ?[T, E](self: Result[T, E]) =
  quote
    match `self`
    of Okay(x) then x
    of Error(e) then return Error(e)

## Opens the std.tables module for unqualified use.
use std.tables

pub type Value = str
pub type Ident = str
pub type Expr = ref union # tagged, algebraic unions
  Literal(Value)
  Variable(Ident)
  Abstraction(param: Ident, body: Expr)
  Application(body: Expr, arg: Expr)
  Conditional(condition: Expr,
    then_branch: Expr, else_branch: Expr)

## Evaluate an Expr down to a Value, or return an Error.
pub func eval(context: mut Table[Ident, Value], expr: lent Expr): Value! =
  match expr # structural pattern matching and guards are supported but not shown
  of Literal(value) then
    Okay(value.clone) # ownership necessitates we explicitly clone
  of Variable(ident) then
    context.get(ident) # whitespace is significant but flexible
      .err("Could not find variable {} in context!"
      .fmt(ident)) # uniform function call syntax allows arbitrary piping/chaining
  of Application(body, arg) then
    if body of Abstraction(param, body as inner_body) then # compact matching with if
      context.set(param, context.clone.eval(arg)?)
      context.eval(inner_body) # all values must be handled: returns are implicit
    else
      Error("Expected Abstraction, found body {} and arg {}".fmt(body.clone, arg.clone))
  of Conditional(condition, then_branch, else_branch) then
    if context.clone.eval(condition)? == "true" then
      context.eval(then_case)
    else
      context.eval(else_case)
  of _ then Error("Invalid expression {}".fmt(expr))

# The top-level module declaration can be elided if the file shares the same name.
pub mod tables =
  ## The Table class. Any sort of table - no matter the underlying
  ## representation - must implement these methods.
  pub type Table[K, V] = class
    get(lent Self, lent K): lent V?
    get(mut Self, lent K): mut V?
    set(mut Self, lent K, V): V?
    pop(mut Self, lent K): V?
    clear(mut Self)
    size(lent Self): uint
    init(varargs (K, V)): Self

  ...

  pub mod hashtable =
    use std.hashes

    pub type HashTable[K, V] = struct
      ...

Why Puck?

Puck is primarily a testing ground and should not be used in any important capacity. Don't use it. Everything is unimplemented and it will break underneath your feet.

That said: in the future, once somewhat stabilized, reasons why you would use it would be for:

• The syntax, aiming to be flexible, predictable, and succinct, through the use of uniform function call syntax, significant whitespace, and consistent scoping rules
• The type system, being modern and powerful with a strong emphasis on safety, algebraic data types, optional and result types, first-class functions, generics, interfaces, and modules
• The memory management system, implementing a model of strict ownership with an optimized reference counting escape hatch
• The metaprogramming, providing integrated macros capable of rewriting the abstract syntax tree before or after typechecking
• The interop system, allowing foreign functions to be usable with native syntax/semantics from a bevy of other languages

This is the language I keep in my head. It sprung from a series of unstructured notes I kept on language design, that finally became something more comprehensive in early 2023. The overarching goal is to provide a language capable of elegantly expressing any problem, and explore ownership and interop along the way.

How do I learn more?

• The basic usage document lays out the fundamental semantics of Puck.
• The syntax document provides a deeper and formal look into the grammar of Puck.
• The type system document gives an in-depth analysis of Puck's extensive type system.
• The modules document provides a more detailed look at the first-class module system.
• The error handling document gives a look at the various kinds of error handling available.
• The memory management document gives an overview of Puck's memory model.
• The metaprogramming document explains how using metaprogramming to extend the language works.
• The asynchronous document gives an overview of Puck's colourless asynchronous support.
• The interop document gives an overview of how the first-class language interop system works.
• The standard library document provides an overview and examples of usage of the standard library.
• The roadmap provides a clear view of the current state and future plans of the language's development.

These are best read in order.

Note that all of these documents (and parts of this README) are written as if everything already exists. Nothing already exists! You can see the roadmap for an actual sense as to the state of the language. I simply found writing in the present tense to be an easier way to collect my thoughts.

This language does not currently integrate ideas from the following areas of active research: effects systems, refinement types, and dependent types. It plans to base (un)safety tracking, exception handling, and async/await on a future effects system. It plans to integrate refinement types in the future as a basis for range[] types, and to explore safety and optimizations surrounding integer overflow.

Primary References

• The Rust I wanted had no future
• Notes on a smaller Rust
• Notes on the next Rust compiler

An Overview of Puck

Puck is an experimental, high-level, memory-safe, statically-typed, whitespace-sensitive, interface-oriented, imperative programming language with functional underpinnings.

It attempts to explore designs in making functional programming paradigms comfortable to those familiar with imperative and object-oriented languages, as well as deal with some more technical problems along the way, such as integrated refinement types and typesafe interop.

This is the language I keep in my head. It reflects the way I think and reason about code.

I do hope others enjoy it.

Variables and Comments

let ident: int = 413
# type annotations are optional
var phrase = "Hello, world!"
const compile_time = std.os.file_name

Variables may be mutable (var), immutable (let), or compile-time evaluated and immutable (const). Type annotations on variables and other bindings follow the name of the binding (with : Type), and are typically optional. Variables are conventionally written in snake_case. Types are conventionally written in PascalCase. The type system is comprehensive, and complex enough to warrant delaying full coverage of until the end. Some basic types are of note, however:

• int, uint: signed and unsigned integers
  • i[\d]+, u[\d]+: arbitrary fixed-size counterparts
• float, decimal: floating-point numbers
  • f32/f64/f128: their fixed-size counterparts
  • dec64/dec128: their fixed-size counterparts
• byte: an alias to u8, representing one byte
• char: an alias to u32, representing one Unicode character
• bool: defined as union[false, true]
• array[T, size]: primitive fixed-size arrays
• list[T]: dynamic lists
• str: mutable strings. internally a list[byte], externally a list[char]
• slice[T]: borrowed "views" into the three types above

Comments are declared with # and run until the end of the line. Documentation comments are declared with ## and may be parsed by language servers and other tooling. Multi-line comments are declared with #[ ]# and may be nested. Taking cues from the Lisp family of languages, any expression may be commented out with a preceding #;.

Functions and Indentation

Functions are declared with the func keyword. They take an (optional) list of generic parameters (in brackets), an (optional) list of parameters (in parentheses), and must be annotated with a return type if they return a type. Every function parameter must be annotated with a type. Their type may optionally be prefixed with either lent, mut or const: denoting an immutable or mutable borrow (more on these later), or a constant type (known to the compiler at compile time, and usable in const exprs). Generic parameters may each be optionally annotated with a type functioning as a constraint.

Whitespace is significant but flexible: functions may be declared entirely on one line if so desired. A new level of indentation after certain tokens (=, do, then) denotes a new level of scope. There are some places where arbitrary indentation and line breaks are allowed - as a general rule of thumb, after operators, commas, and opening parentheses. The particular rules governing indentation may be found in the syntax guide.

Uniform Function Call Syntax

func inc(self: list[int], by: int): list[int] =
  self.map(x => x + by)

print inc([1, 2, 3], len("four")) # 5, 6, 7
print [1, 2, 3].inc(1)  # 2, 3, 4
print [1].len # 1

Puck supports uniform function call syntax: and so any function may be called using the typical syntax for method calls, that is, the first parameter of any function may be appended with a . and moved to precede it, in the style of a typical method. (There are no methods in Puck. All functions are statically dispatched. This may change in the future.)

This allows for a number of syntactic cleanups. Arbitrary functions with compatible types may be chained with no need for a special pipe operator. Object field access, module member access, and function calls are unified, reducing the need for getters and setters. Given a first type, IDEs using dot-autocomplete can fill in all the functions defined for that type. Programmers from object-oriented languages may find the lack of object-oriented classes more bearable. UFCS is implemented in shockingly few languages, and so Puck joins the tiny club that previously consisted of just D, Nim, Koka, and Effekt.

Basic Types

Boolean logic and integer operations are standard and as one would expect out of a typed language: and, or, xor, not, shl, shr, +, -, *, /, <, >, <=, >=, div, mod, rem. Notably:

• the words and/or/not/shl/shr are used instead of the symbolic &&/||/!/<</>>
• integer division is expressed with the keyword div while floating point division uses /
• % is absent and replaced with distinct modulus and remainder operators
• boolean operators are bitwise and also apply to integers and floats
• more operators are available via the standard library (std.math.exp and std.math.log)

The above operations are performed with operators, special functions that take a prefixed first argument and (often) a suffixed second argument. Custom operators may be implemented, but they must consist of only a combination of the symbols = + - * / < > @ $ ~ & % | ! ? ^ \ for the purpose of keeping the grammar context-free. They are are declared identically to functions.

Term (in)equality is expressed with the == and != operators. Type equality is expressed with is. Subtyping relations may be queried with of, which has the additional property of introducing new bindings to the current scope in certain contexts (more on this in the types document).

let phrase: str = "I am a string! Wheeee! ✨"
for c in phrase do
  stdout.write(c) # I am a string! Wheeee! ✨
for b in phrase.bytes() do
  stdout.write(b.char) # Error: cannot convert from byte to char
print phrase.last() # ✨

String concatenation uses a distinct & operator rather than overloading the + operator (as the complement - has no natural meaning for strings). Strings are unified, mutable, internally a byte array, externally a char array, and are stored as a pointer to heap data after their length and capacity (fat pointer). Chars are four bytes and represent a Unicode character in UTF-8 encoding. Slices of strings are stored as a length followed by a pointer to string data, and have non-trivial interactions with the memory management system. More details can be found in the type system overview.

Conditionals and Pattern Matching

Basic conditional control flow uses standard if/elif/else statements. The when statement provides a compile-time if. It also takes elif and else branches and is syntactic sugar for an if statement within a const expression (more on those later).

All values in Puck must be handled, or explicitly discarded. This allows for conditional statements and many other control flow constructs to function as expressions: and evaluate to a value when an unbound value is left at the end of each of their branches' scopes. This is particularly relevant for functions, where it is often idiomatic to omit an explicit return statement. There is no attempt made to differentiate without context, and so expressions and statements often look identical in syntax.

Exhaustive structural pattern matching is available with the match/of statement, and is particularly useful for the struct and union types. of branches of a match statement take a pattern, of which the unbound identifiers within will be injected into the branch's scope. Multiple patterns may be used for one branch provided they all bind the same identifiers of the same type. Branches may be guarded with the where keyword, which takes a conditional, and will necessarily remove the branch from exhaustivity checks.

The of statement also stands on its own as an operator for querying subtype equality. Used as a conditional in if statements or while loops, it retains the variable injection properties of its match counterpart. This allows it to be used as a compact and coherent alternative to if let statements in other languages.

Error Handling

type Result[T] = Result[T, ref Err]
func may_fail: Result[T] = ...

Error handling is done via a fusion of functional monadic types and imperative exceptions, with much syntactic sugar. Functions may raise exceptions, but by convention should return Option[T] or Result[T, E] types instead: these may be handled in match or if/of statements. The effect system built into the compiler will track functions that raise errors, and warn on those that are not handled explicitly via try/with statements anywhere on the call stack.

A bevy of helper functions and macros are available for Option/Result types, and are documented and available in the std.options and std.results modules (included in the prelude by default). Two in particular are of note: the ? macro accesses the inner value of a Result[T, E] or propagates (returns in context) the Error(e), and the ! accesses the inner value of an Option[T] / Result[T, E] or raises an error on None / the specific Error(e). Both operators take one parameter and so are postfix. The ? and ! macros are overloaded and additionally function on types as shorthand for Option[T] and Result[T] respectively.

The utility of the ? macro is readily apparent to anyone who has written code in Rust or Swift. The utility of the ! function is perhaps less so obvious. These errors raised by !, however, are known to the compiler: and they may be comprehensively caught by a single or sequence of with statements. This allows for users used to a try/with error handling style to do so with ease, with only the need to add one additional character to a function call.

More details may be found in error handling overview.

Blocks and Loops

loop
  print "This will never normally exit."
  break

for i in 0 .. 3 do # exclusive
  for j in 0 ..= 3 do # inclusive
    print "{} {}".fmt(i, j)

Three types of loops are available: for loops, while loops, and infinite loops (loop loops). For loops take a binding (which may be structural, see pattern matching) and an iterable object and will loop until the iterable object is spent. While loops take a condition that is executed upon the beginning of each iteration to determine whether to keep looping. Infinite loops are infinite are infinite are infinite are infinite are infinite are infinite and must be manually broken out of.

There is no special concept of iterators: iterable objects are any object that implements the Iter[T] class (more on those in the type system document): that is, provides a self.next() function returning an Option[T]. As such, iterators are first-class constructs. For loops can be thought of as while loops that unwrap the result of the next() function and end iteration upon a None value. While loops, in turn, can be thought of as infinite loops with an explicit conditional break.

The break keyword immediately breaks out of the current loop, and the continue keyword immediately jumps to the next iteration of the current loop. Loops may be used in conjunction with blocks for more fine-grained control flow manipulation.

block
  statement

let x = block
  let y = read_input()
  transform_input(y)

block foo
  for i in 0 ..= 100 do
    block bar
      if i == 10 then break foo
      print i

Blocks provide arbitrary scope manipulation. They may be labelled or unlabelled. The break keyword additionally functions inside of blocks and without any parameters will jump out of the current enclosing block (or loop). It may also take a block label as a parameter for fine-grained scope control.

Module System

Code is segmented into modules. Modules may be made explicit with the mod keyword followed by a name, but there is also an implicit module structure in every codebase that follows the structure and naming of the local filesystem. For compatibility with filesystems, and for consistency, module names are exclusively lowercase (following the same rules as Windows).

A module can be imported into another module by use of the use keyword, taking a path to a module or modules. Contrary to the majority of languages ex. Python, unqualified imports are encouraged - in fact, are idiomatic (and the default) - type-based disambiguation and official LSP support are intended to remove any ambiguity.

Within a module, functions, types, constants, and other modules may be exported for use by other modules with the pub keyword. All such identifiers are private by default and only accessible module-locally without. Modules are first-class and may be bound, inspected, modified, and returned. As such, imported modules may be re-exported for use by other modules by binding them to a public constant.

More details may be found in the modules document.

Compile-time Programming

## Arbitrary code may execute at compile-time.
const compile_time =
  match std.os.platform # known at compile-time
    of Windows then "windows"
    of MacOS then "darwin"
    of Linux then "linux"
    of Wasi then "wasm"
    of _ then "unknown platform"

## The propagation operator is a macro so that `return` is injected into the function scope.
pub macro ?[T](self: Option[T]) =
  quote
    match `self`
    of Some(x) then x
    of None then return None

## Type annotations and overloading allow us to define syntactic sugar for `Option[T]`, too.
pub macro ?(T: type) =
  quote Option[`T`]

Compile-time programming may be done via the previously-mentioned const keyword and when statements: or via macros. Macros operate directly on the abstract syntax tree at compile-time: taking in syntax objects, transforming them, and returning them to be injected. They are hygenic and will not capture identifiers not passed as parameters. While parameters are syntax objects, they can be annotated with types to constrain applications of macros and allow for overloading. Macros are written in ordinary Puck: there is thus no need to learn a separate "macro language", as syntax objects are just standard unions. Additionally, support for quoting removes much of the need to operate on raw syntax objects. A full description may be found in the metaprogramming document.

Async System and Threading

The async system is colourblind: the special async macro will turn any function call returning a T into an asynchronous call returning a Future[T]. The special await function will wait for any Future[T] and return a T (or an error). Async support is included in the standard library in std.async in order to allow for competing implementations. More details may be found in the async document.

Threading support is complex and also regulated to external libraries. OS-provided primitives will likely provide a spawn function, and there will be substantial restrictions for memory safety. I really haven't given much thought to this.

Memory Management

# Differences in Puck and Rust types in declarations and at call sights.
# note: this notation is not valid and is for illustrative purposes only
func foo(a:
  lent T → &'a T
  mut T → &'a mut T
  T → T
):
  lent T → &'a T
  mut T → &'a mut T
  T → T

let t: T = ...
foo( # this is usually elided
  lent t → &t
  mut t → &mut t
  t → t
)

Puck copies Rust-style ownership near verbatim. &T corresponds to lent T, &mut T to mut T, and T to T: with T implicitly convertible to lent T and mut T at call sites. A major goal of Puck is for all lifetimes to be inferred: there is no overt support for lifetime annotations, and it is likely code with strange lifetimes will be rejected before it can be inferred. (Total inference, however, is a goal.)

Another major difference is the consolidation of Box, Rc, Arc, Cell, RefCell into just two (magic) types: ref and refc. ref takes the role of Box, and refc both the role of Rc and Arc: while Cell and RefCell are disregarded. The underlying motivation for compiler-izing these types is to make deeper compiler optimizations accessible: particularly with refc, where the existing ownership framework is used to eliminate unnecessary counts. Details on memory safety, references and pointers, and deep optimizations may be found in the memory management overview.

Types System

# The type Foo is defined here as an alias to a list of bytes.
type Foo = list[byte]

# implicit conversion to Foo in declarations
let foo: Foo = [1, 2, 3]

func fancy_dbg(self: Foo) =
  print "Foo:"
  # iteration is defined for list[byte]
  # so it implicitly carries over: and is defined on Foo
  for elem in self do
    dbg(elem)

# NO implicit conversion to Foo on calls
[4, 5, 6].foo_dbg # this fails!

Foo([4, 5, 6]).foo_dbg # prints: Foo: 4 5 6

Finally, a few notes on the type system are in order. Types are declared with the type keyword and are aliases: all functions defined on a type carry over to its alias, though the opposite is not true. Functions defined on the alias must take an object known to be a type of that alias: exceptions are made for type declarations, but at call sites this means that conversion must be explicit.

# We do not want functions defined on list[byte] to carry over,
# as strings function differently (operating on chars).
# So we declare `str` as a struct, rather than a type alias.
pub type str = struct
  data: list[byte]

# However, the underlying `empty` function is still useful.
# So we expose it in a one-liner alias.
# In the future, a `with` macro may be available to ease carryover.
pub func empty(self: str): bool = self.data.empty

# Alternatively, if we want total transparent type aliasing, we can use constants.
pub const MyAlias: type = VeryLongExampleType

If one wishes to define a new type without previous methods accessible, the newtype paradigm is preferred: declaring a single-field struct, and manually implementing functions that carry over. It can also be useful to have transparent type aliases, that is, simply a shorter name to refer to an existing type. These do not require type conversion, implicit or explicit, and can be used freely and interchangeably with their alias. This is done with constants.

Types, like functions, can be generic: declared with "holes" that may be filled in with other types upon usage. A type must have all its holes filled before it can be constructed. The syntax for generics in types much resembles the syntax for generics in functions, and generic constraints and the like also apply.

Structs and Tuples

# standard alternative syntax to inline declarations
type MyStruct = struct
  a: str
  b: str

# syntactic sugar for tuple[str, b: str]
type MyTuple = (str, b: str)

let a: MyTuple = ("hello", "world")
print a.1 # world
print a.b # world

let c: MyStruct = {a = a.0, b = a.1}
print c.b # world

Struct and tuple types are declared with struct[<fields>] and tuple[<fields>], respectively. Their declarations make them look similar at a glance: but they differ fairly fundamentally. Structs are unordered and every field must be named. They may be constructed with brackets. Tuples are ordered and so field names are optional - names are just syntactic sugar for positional access. Tuples are both constructed and optionally declared with parentheses.

It is worth noting that there is no concept of pub at a field level on structs - a type is either fully transparent, or fully opaque. This is because such partial transparency breaks with structural initialization (how could one provide for hidden fields?). The @[opaque] attribute allows for expressing that the internal fields of a struct are not to be accessed or initialized: this, however, is only a compiler warning and can be totally suppressed with @[allow(opaque)].

Unions and Enums

type Expr = union
  Literal(int)
  Variable(str)
  Abstraction(param: str, body: ref Expr)
  Application(body: ref Expr, arg: ref Expr)

Union types are composed of a list of variants. Each variant has a tag and an inner type the union wraps over. Before the inner type can be accessed, the tag must be pattern matched upon, in order to handle all possible values. These are also known as sum types or tagged unions in other languages.

Union types are the bread and butter of structural pattern matching. Composed with structs and tuples, unions provide for a very general programming construct commonly referred to as an algebraic data type. This is often useful as an idiomatic and safer replacement for inheritance.

type Opcode = enum
  BRK INC POP NIP SWP ROT DUP OVR EQU NEQ GTH LTH JMP JCN JSR STH JCI JMI
  LDZ STZ LDR STR LDA STA DEI DEO ADD SUB MUL DIV AND ORA EOR SFT JSI LIT

print Opcode.BRK # 0
...

Enum types are similarly composed of a list of variants. These variants, however, are static values: assigned at compile-time, and represented under the hood by a single integer. They function similarly to unions, and can be passed through to functions and pattern matched upon, however their underlying simplicity and default values mean they are much more useful for collecting constants and acting as flags than anything else.

Classes

pub type Iter[T] = class
  next(mut Self): T?

pub type Peek[T] = class
  next(mut Self): T?
  peek(mut Self): T?
  peek_nth(mut Self, int): T?

Class types function much as type classes in Haskell or traits in Rust do. They are not concrete types, and cannot be constructed - instead, their utility is via indirection, as parameters in functions or as ref types in structures, providing constraints that some concrete type must meet. They consist of a list of function signatures, implementations of which must exist for the given type passed in in order to compile.

Their major difference, however, is that Puck's classes are implicit: there is no impl block that implementations of their associated functions have to go under. If functions for a concrete type exist satisfying some class, the type implements that class. This does run the risk of accidentally implementing a class one does not desire to, but the author believes such situations are few and far between and well worth the decreased syntactic and semantic complexity. As a result, however, classes are entirely unable to guarantee any invariants hold (like PartialOrd or Ord in Rust do).

As the compiler makes no such distinction between fields and single-argument functions on a type when determining identifier conflicts, classes similarly make no such distinction. Structs may be described with their fields written as methods. They do distinguish borrowed/mutable/owned parameters, those being part of the type signature.

Classes are widely used throughout the standard library to provide general implementations of such conveniences like iteration, debug and display printing, generic error handling, and much more.

Syntax: A Casual and Formal Look

Call Syntax

There is little difference between a function, macro, and operator call. There are only a few forms such calls can take, too, though notably more than most other languages (due to, among other things, uniform function call syntax): hence this section.

# The standard, unambiguous call.
routine(1, 2, 3, 4)
# The method call syntax equivalent.
1.routine(2, 3, 4)
# A block-based call. This is only really useful for macros taking in a body.
routine
  1
  2
  3
  4
# A parentheses-less call. This is only really useful for `print` and `dbg`.
# Only valid at the start of a line.
routine 1, 2, 3, 4

Binary operators have some special rules.

# Valid call syntaxes for binary operators. What can constitute a binary
# operator is constrained for parsing's sake. Whitespace is optional.
1 + 2
1+2
+ 1, 2 # Only valid at the start of a line. Also, don't do this.
+(1, 2)

As do unary operators.

# The standard call for unary operators. Postfix.
1?
?(1)

Method call syntax has a number of advantages: notably that it can be chained: acting as a natural pipe operator. Redundant parenthesis can also be omitted.

# The following statements are equivalent:
foo.bar.baz
foo().bar().baz()
baz(bar(foo))
baz
  bar
    foo
baz bar(foo)
baz foo.bar

Indentation Rules

The tokens =, then, do, of, else, block, const, block X, and X (where X is an identifier) are scope tokens. They denote a new scope for their associated expressions (functions/macros/declarations, control flow, loops). The tokens ,, . (notably not ...), and all default binary operators (notably not not) are continuation tokens. An expression beginning or ending in one of them would always be a syntactic error.

Line breaks are treated as the end of a statement, with several exceptions.

pub func foo() =
  print "Hello, world!"
  print "This is from a function."

pub func inline_decl() = print "Hello, world!"

Indented lines following a line ending in a scope token are treated as belonging to a new scope. That is, indented lines following a line ending in a scope token form the body of the expression associated with the scope token.

Indentation is not obligatory after a scope token. However, this necessarily constrains the body of the associated expression to one line: no lines following will be treated as an extension of the body, only the expression associated with the original scope token. (This may change in the future.)

pub func foo(really_long_parameter: ReallyLongType,
another_really_long_parameter: AnotherReallyLongType) = # no indentation! this is ok
  print really_long_parameter # this line is indented relative to the first line
  print really_long_type

Lines following a line ending in a continuation token (and, additionally not and () are treated as a continuation of that line and can have any level of indentation (even negative). If they end in a scope token, however, the following lines must be indented relative to the indentation of the previous line.

let really_long_parameter: ReallyLongType = ...
let another_really_long_parameter: AnotherReallyLongType = ...

really_long_parameter
  .foo(another_really_long_parameter) # some indentation! this is ok

Lines beginning in a continuation token (and, additionally )), too, are treated as a continuation of the previous line and can have any level of indentation. If they end in a scope token, the following lines must be indented relative to the indentation of the previous line.

pub func foo() =
  print "Hello, world!"
pub func bar() = # this line is no longer in the above scope.
  print "Another function declaration."

Dedented lines not beginning or ending with a continuation token are treated as no longer in the previous scope, returning to the scope of the according indentation level.

if cond then this
else that

match cond
of this then ...
of that then ...

A line beginning with a scope token is treated as attached to the previous expression.

# Technically allowed. Please don't do this.
let foo
= ...

if cond then if cond then this
else that

for i
in iterable
do ...

match foo of this then ...
of that then ...

match foo of this
then ...
of that then ...

This can lead to some ugly possibilities for formatting that are best avoided.

# Much preferred.

let foo =
  ...
let foo = ...

if cond then
  if cond then
    this
else that
if cond then
  if cond then this
else that

for i in iterable do
  ...
for i in iterable do ...

match foo
of this then ...
of that then ...

The indentation rules are complex, but the effect is such that long statements can be broken almost anywhere.

Expression Rules

First, a word on the distinction between expressions and statements. Expressions return a value. Statements do not. That is all.

There are some syntactic constructs unambiguously recognizable as statements: all declarations, modules, and use statements. There are no syntactic constructs unambiguously recognizable as expressions. As calls returning void are treated as statements, and expressions that return a type could possibly return void, there is no explicit distinction between expressions and statements made in the parser: or anywhere before type-checking.

Expressions can go almost anywhere. Our indentation rules above allow for it.

# Some different formulations of valid expressions.

if cond then
  this
else
  that

if cond then this
else that

if cond
then this
else that

if cond then this else that

let foo =
  if cond then
    this
  else
    that

# Some different formulations of *invalid* expressions.
# These primarily break the rule that everything following a scope token
# (ex. `=`, `do`, `then`) not at the end of the line must be self-contained.

let foo = if cond then
    this
  else
    that

let foo = if cond then this
  else that

let foo = if cond then this
else that

# todo: how to handle this?
if cond then if cond then that
else that

# shrimple
if cond then
  if cond then that
else that

# this should be ok
if cond then this
else that

match foo of
this then ...
of that then ...

Reserved Keywords

The following keywords are reserved:

• variables: let var const
• control flow: if then elif else
• pattern matching: match of
• error handling: try with finally
• loops: while do for in
• blocks: loop block break continue return
• modules: pub mod use as
• functions: func varargs
• metaprogramming: macro quote when
• ownership: lent mut ref refc
• types: type struct tuple union enum class

The following keywords are not reserved, but liable to become so.

• impl object interface concept auto effect case
• suspend resume spawn pool thread closure static
• cyclic acyclic sink move destroy copy trace deepcopy

The following identifiers are in use by the standard prelude:

• logic: not and or xor shl shr div mod rem
• logic: + - * / < > <= >= == != is
• async: async await
• types: int uint float i[\d]+ u[\d]+
  • f32 f64 f128
  • dec64 dec128
• types: bool byte char str
• types: void never
• strings: & (string append)

The following punctuation is taken:

• = (assignment)
• . (chaining)
• , (parameters)
• ; (statements)
• : (types)
• # (comment)
• @ (attributes)
• _ (unused bindings)
• | (generics)
• \ (string/char escaping)
• () (parameters, tuples)
• [] (generics, lists)
• {} (scope, structs)
• "" (strings)
• '' (chars)
• `` (unquoting)
• unused on qwerty: ~ % ^ $
  • perhaps leave $ unused. but ~, %, and ^ totally could be...

A Formal Grammar

We now shall take a look at a more formal description of Puck's syntax.

Syntax rules are described in extended Backus–Naur form (EBNF): however, most rules surrounding whitespace, and scope, and line breaks, are modified to how they would appear after a lexing step.

Identifiers

Ident  ::= (Letter | '_') (Letter | Digit | '_')*
Letter ::= 'A'..'Z' | 'a'..'z' | '\x80'..'\xff' # todo
Digit  ::= '0'..'9'

Literals

Int ::= '-'? (DecLit | HexLit | OctLit | BinLit)
Float ::= '-'? DecLit '.' DecLit
BinLit ::= '0b' BinDigit ('_'? BinDigit)*
OctLit ::= '0o' OctDigit ('_'? OctDigit)*
HexLit ::= '0x' HexDigit ('_'? HexDigit)*
DecLit ::= Digit ('_'? Digit)*
BinDigit ::= '0'..'1'
OctDigit ::= '0'..'7'
HexDigit ::= Digit | 'A'..'F' | 'a'..'f'

Chars, Strings, and Comments

CHAR    ::= '\'' (PRINT - '\'' | '\\\'')* '\''
STRING  ::= SINGLE_LINE_STRING | MULTI_LINE_STRING
COMMENT ::= SINGLE_LINE_COMMENT | MULTI_LINE_COMMENT | EXPRESSION_COMMENT
SINGLE_LINE_STRING  ::= '"' (PRINT - '"' | '\\"')* '"'
MULTI_LINE_STRING   ::= '"""' (PRINT | '\n' | '\r')* '"""'
SINGLE_LINE_COMMENT ::= '#' PRINT*
MULTI_LINE_COMMENT  ::= '#[' (PRINT | '\n' | '\r' | MULTI_LINE_COMMENT)* ']#'
EXPRESSION_COMMENT  ::= '#;' SINGLE_STMT
PRINT ::= LETTER | DIGIT | OPR |
          '"' | '#' | "'" | '(' | ')' | # notably the dual of OPR
          ',' | ';' | '[' | ']' | '_' |
          '`' | '{' | '}' | ' ' | '\t'

Values

Value ::= Int | Float | String | Char | Array | Tuple | Struct
Array  ::= '[' (Expr (',' Expr)*)? ']'
Tuple  ::= '(' (Ident '=')? Expr (',' (Ident '=')? Expr)* ')'
Struct ::= '{' Ident '=' Expr (',' Ident '=' Expr)* '}'

Variables

Decl  ::= Let | Var | Const | Func | Type
Let   ::= 'let' Pattern (':' Type)? '=' Expr
Var   ::= 'var' Pattern (':' Type)? ('=' Expr)?
Const ::= 'pub'? 'const' Pattern (':' Type)? '=' Expr
Pattern ::= (Ident ('as' Ident)?) | Char | String | Number | Float |
            Ident? '(' Pattern (',' Pattern)* ')'

Declarations

Func  ::= 'pub'? 'func' Ident Generics? Parameters? (':' Type)? '=' Body
Macro ::= 'pub'? 'macro' Ident Generics? Parameters? (':' Type)? '=' Body
Generics   ::= '[' Ident (':' Type)? (',' Ident (':' Type)?)* ']'
Parameters ::= '(' Ident (':' Type)? (',' Ident (':' Type)?)* ')'

All arguments to functions must have a type. This is resolved at the semantic level, however. (Arguments to macros may lack types. This signifies a generic node.)

Types

TypeDecl ::= 'pub'? 'type' Ident Generics? '=' Type
Type     ::= TypeStruct | TypeTuple | TypeEnum | TypeUnion | SugarUnion |
             TypeClass | (Modifier* (Type | ('[' Type ']')))
TypeStruct ::= 'struct' ('[' Ident ':' Type (',' Ident ':' Type)* ']')?
TypeUnion  ::= 'union'  ('[' Ident ':' Type (',' Ident ':' Type)* ']')?
SugarUnion ::= '(' Ident ':' Type (',' Ident ':' Type)* ')'
TypeTuple  ::= 'tuple' ('[' (Ident ':')? Type (',' (Ident ':')? Type)* ']')?
TypeEnum   ::= 'enum'  ('[' Ident ('=' Expr)? (',' Ident ('=' Expr)?)* ']')?
TypeClass  ::= 'class' ('[' Signature (',' Signature)* ']')?
Modifier   ::= 'ref' | 'refc' | 'ptr' | 'lent' | 'mut' | 'const'
Signature  ::= Ident Generics? ('(' Type (',' Type)* ')')? (':' Type)?

Control Flow

If     ::= 'if' Expr 'then' Body ('elif' Expr 'then' Body)* ('else' Body)?
When   ::= 'when' Expr 'then' Body ('elif' Expr 'then' Body)* ('else' Body)?
Try    ::= 'try' Body ('with' Pattern (',' Pattern)* 'then' Body)+ ('finally' Body)?
Match  ::= 'match' Expr ('of' Pattern (',' Pattern)* ('where' Expr)? 'then' Body)+
While  ::= 'while' Expr 'do' Body
For    ::= 'for' Pattern 'in' Expr 'do' Body
Loop   ::= 'loop' Body
Block  ::= 'block' Ident? Body
Const  ::= 'const' Body
Quote  ::= 'quote' QuoteBody

Modules

Mod ::= 'pub'? 'mod' Ident '=' Body
Use ::= 'use' Ident ('.' Ident)* ('.' ('[' Ident (',' Ident)* ']'))?

Operators

Operator ::= 'and' | 'or' | 'not' | 'xor' | 'shl' | 'shr' |
             'div' | 'mod' | 'rem' | 'is' | 'in' | Opr+
Opr ::= '=' | '+' | '-' | '*' | '/' | '<' | '>' |
        '@' | '$' | '~' | '&' | '%' | '|' |
        '!' | '?' | '^' | '.' | ':' | '\\'

Calls and Expressions

This section is (quite) inaccurate due to complexities with respect to significant indentation. Heed caution.

Call ::= Ident ('[' Call (',' Call)* ']')? ('(' (Ident '=')? Call (',' (Ident '=')? Call)* ')')? |
         Ident Call (',' Call)* |
         Call Operator Call? |
         Call Body
Stmt ::= Let | Var | Const | Func | Type | Mod | Use | Expr
Expr ::= Block | Const | For | While | Loop | If | When | Try | Match | Call
Body ::= (Stmt ';')* Expr

References:

• Statements vs. Expressions
• Swift's Lexical Structure
• The Nim Programming Language
• Pietro's Notes on Compilers

Typing in Puck

! This section needs a rewrite. Proceed with low standards.

Puck has a comprehensive static type system, inspired by the likes of Nim, Rust, and Swift.

Basic types

Basic types can be one-of:

• bool: internally an enum.
• int: integer number. x bits of precision by default.
  • uint: same as int, but unsigned for more precision.
  • i[\d]+, u[\d]+: arbitrarily sized integers
• float: floating-point number.
  • f32, f64: specified float sizes
• decimal: precision decimal number.
  • dec32, dec64, dec128: specified decimal sizes
• byte: an alias to u8.
• char: an alias to u32. For working with Unicode.
• str: a string type. mutable. packed: internally a byte-array, externally a char-array.
• void: an internal type designating the absence of a value. often elided.
• never: a type that denotes functions that do not return. distinct from returning nothing.

bool and int/uint/float and siblings (and subsequently byte and char) are all considered primitive types and are always copied (unless passed as mutable). More on when parameters are passed by value vs. passed by reference can be found in the memory management document.

Primitive types, alongside str, void, and never, form basic types. void and never will rarely be referenced by name: instead, the absence of a type typically implicitly denotes one or the other. Still, having a name is helpful in some situations.

integers

todo

strings

Strings are:

• mutable
• internally a byte array
• externally a char (four bytes) array
• prefixed with their length and capacity
• automatically resize

They are also quite complicated. Puck has full support for Unicode and wishes to be intuitive, performant, and safe, as all languages wish to be. Strings present a problem that much effort has been expended on in (primarily) Swift and Rust to solve.

Abstract Types

Abstract types, broadly speaking, are types described by their behavior rather than their implementation. They are more commonly know as abstract data types: which is confusingly similar to "algebraic data types", another term for the advanced types they are built out of under the hood. We refer to them here as "abstract types" to mitigate some confusion.

iterable types

Iterable types can be one-of:

• array[T, size]: Fixed-size arrays. Can only contain one type T. Of a fixed size size and cannot grow/shrink, but can mutate. Initialized in-place with [a, b, c].
• list[T]: Dynamic arrays. Can only contain one type T. May grow/shrink dynamically. Initialized in-place with [a, b, c]. (this is the same as arrays!)
• slice[T]: Slices. Used to represent a "view" into some sequence of elements of type T. Cannot be directly constructed: they are unsized. Cannot grow/shrink, but their elements may be accessed and mutated. As they are underlyingly a reference to an array or list, they must not outlive the data they reference: this is non-trivial, and so slices interact in complex ways with the memory management system.
• str: Strings. Described above. They are alternatively treated as either list[byte] or list[char], depending on who's asking. Initialized in-place with "abc".

These iterable types are commonly used, and bits and pieces of compiler magic are used here and there (mostly around initialization, and ownership) to ease use. All of these types are some sort of sequence: and implement the Iter interface, and so can be iterated (hence the name).

other abstract types

Unlike the iterable types above, these abstract types do not have a necessarily straightforward or best implementation, and so multiple implementations are provided in the standard library.

These abstract data types can be one-of:

• BitSet[T]: high-performance sets implemented as a bit array.
  • These have a maximum data size, at which point the compiler will suggest using a HashSet[T] instead.
• AssocTable[T, U]: simple symbol tables implemented as an association list.
  • These do not have a maximum size. However, at some point the compiler will suggest using a HashTable[T, U] instead.
• HashSet[T]: standard hash sets.
• HashTable[T, U]: standard hash tables.

These abstract types do not have a natural ordering, unlike the iterable types above, and thus do not implement Iter. Despite this: for utility an elems() iterator based on a normalization of the elements is provided for set and HashSet, and keys(), values(), and pairs() iterators are provided for table and HashTable (based on a normalization of the keys).

Parameter Types

Some types are only valid when being passed to a function, or in similar contexts. No variables may be assigned these types, nor may any function return them. These are monomorphized into more specific functions at compile-time if needed.

Parameter types can be one-of:

• mutable: func foo(a: mut str): Marks a parameter as mutable (parameters are immutable by default). Passed as a ref if not one already.
• constant: func foo(a: const str): Denotes a parameter whose value must be known at compile-time. Useful in macros, and with when for writing generic code.
• generic: func foo[T](a: list[T], b: T): The standard implementation of generics, where a parameter's exact type is not listed, and instead statically dispatched based on usage.
• constrained: func foo(a: str | int | float): A basic implementation of generics, where a parameter can be one-of several listed types. The only allowed operations on such parameters are those shared by each type. Makes for particularly straightforward monomorphization.
• functions: func foo(a: (int, int) -> int): First-class functions. All functions are first class - function declarations implicitly have this type, and may be bound in variable declarations. However, the function type is only terribly useful as a parameter type.
• slices: func foo(a: slice[...]): Slices of existing lists, strings, and arrays. Generic over length. These are references under the hood, may be either immutable or mutable (with mut), and interact non-trivially with Puck's ownership system.
• classes: func foo(a: Stack[int]): Implicit typeclasses. More in the classes section.
  • ex. for above: type Stack[T] = interface[push(mut Self, T); pop(mut Self): T]
• built-in interfaces: func foo(a: struct): Included, special interfaces for being generic over advanced types. These include struct, tuple, union, enum, interface, and others.

Several of these parameter types - specifically, slices, functions, and interfaces - share a common trait: they are not sized. The exact size of the type is not generally known until compilation - and in some cases, not even during compilation! As the size is not always rigorously known, problems arise when attempting to construct these parameter types or compose them with other types: and so this is disallowed. They may still be used with indirection, however - detailed in the section on reference types.

generic types

Functions can take a generic type, that is, be defined for a number of types at once:

# fully generic. monomorphizes based on usage.
func add[T](a: list[T], b: T) = a.push(b)

# constrained generics. restricts possible operations to the intersection
# of defined methods on each type.
func length[T](a: str | list[T]) =
  a.len # both strings and lists have a `len` method

# alternative formulation: place the constraint on a generic parameter.
# this ensures both a and b are of the *same* type.
func add[T: int | float](a: T, b: T) = a + b

The syntax for generics is func, ident, followed by the names of the generic parameters in brackets [T, U, V], followed by the function's parameters (which may then refer to the generic types). Generics are replaced with concrete types at compile time (monomorphization) based on their usage in function calls within the main function body.

Constrained generics have two syntaxes: the constraint can be defined directly on a parameter, leaving off the [T] box, or it may be defined within the box as [T: int | float] for easy reuse in the parameters.

Other constructions like type declarations themselves may also be generic over types. In the future, modules also may be generic: whether that is to be over types or over other modules is to be determined.

Reference Types

Types are typically constructed by value on the stack. That is, without any level of indirection: and so type declarations that recursively refer to one another, or involve unsized types (notably including parameter types), would not be allowed. However, Puck provides several avenues for indirection.

Reference types can be one-of:

• ref T: An owned reference to a type T. This is a pointer of size uint (native).
• refc T: A reference-counted reference to a type T. This allows escaping the borrow checker.
• ptr T: A manually-managed pointer to a type T. (very) unsafe. The compiler will yell at you.

type BinaryTree = ref struct
  left: BinaryTree
  right: BinaryTree

type AbstractTree[T] = class
  func left(self: Self): Option[AbstractTree[T]]
  func right(self: Self): Option[AbstractTree[T]]
  func data(self: Self): T

type AbstractRoot[T] = struct
  left: ref AbstractTree[T]
  right: ref AbstractTree[T]

# allowed, but unsafe & strongly discouraged
type UnsafeTree = struct
  left: ptr UnsafeTree
  right: ptr UnsafeTree

The ref prefix may be placed at the top level of type declarations, or inside on a field of a structural type. ref types may often be more efficient when dealing with large data structures. They also provide for the usage of unsized types (functions, interfaces, slices) within type declarations.

The compiler abstracts over ref types to provide optimization for reference counts: and so a distinction between Rc/Arc/Box is not needed. Furthermore, access implicitly dereferences (with address access available via .addr), and so a * dereference operator is also not needed.

Much care has been given to make references efficient and safe, and so ptr should be avoided if at all possible. They are only usable inside functions explicitly marked with #[safe].

The implementations of reference types are delved into in further detail in the memory management document.

Advanced Types

The type keyword is used to declare aliases to custom data types. These types are algebraic: they function by composition. Such algebraic data types can be one-of:

• struct: An unordered, named collection of types. May have default values.
• tuple: An ordered collection of types. Optionally named.
• enum: Ordinal labels, that may hold values. Their default values are their ordinality.
• union: Powerful matchable tagged unions a la Rust. Sum types.
• class: Implicit type classes. User-defined duck typing.

All functions defined on the original type carry over. If this is not desired, the newtype paradigm is preferred: declaring a single-field struct and copying function declarations over.

Types may be explicitly to and from via the Coerce and Convert classes and provided from and to functions.

structs

Structs are an unordered collection of named types.

They are declared with struct[identifier: Type, ...] and initialized with brackets: { field = "value", another = 500}. Structs are structural: while the type system is fundamentally nominal, and different type declarations are treated as distinct, a struct object initialized with {} is usable in any context that expects a struct with the same fields.

type LinkedNode[T] = ref struct
  previous: Option[LinkedNode[T]]
  next: Option[LinkedNode[T]]
  data: T

let node = { # inferred type: LinkedNode[int], from prints_data call
  previous = None, next = None
  data = 413
}

func pretty_print(node: LinkedNode[int]) =
  print node.data
  if node.next of Some(node) then
    node.pretty_print()

# structural typing!
prints_data(node)

tuples

Tuples are an ordered collection of either named and/or unnamed types.

They are declared with tuple[Type, identifier: Type, ...] and initialized with parentheses: (413, "hello", value: 40000). Syntactic sugar allows for them to be declared with () as well.

They are exclusively ordered - named types within tuples are just syntactic sugar for positional access. Passing a fully unnamed tuple into a context that expects a tuple with a named parameter is allowed (so long as the types line up).

let grouping = (1, 2, 3)

func foo: tuple[str, str] = ("hello", "world")
dbg grouping.foo # prints '("hello", "world")'

func bar(a: (str, str)) = a.1
dbg grouping.bar # prints '"world"'

Tuples are particularly useful for "on-the-fly" types. Creating type declarations to tuples is discouraged - structs are generally a better choice, as they are fully named, support default values, and may have their layout optimized by the compiler.

enums

Enums are ordinal labels that may have associated values.

They are declared with enum[Label, AnotherLabel = 4, ...] and are never initialized (their values are known statically). Enums may be accessed directly by their label, and are ordinal and iterable regardless of their associated value. They are useful in collecting large numbers of "magic values" that would otherwise be constants.

type Keys = enum
  Left, Right, Up, Down
  A = "a"
  B = "b"

In the case of an identifier conflict (with other enum labels, or types, or...) they must be prefixed with the name of their associated type (separated by a dot). This is standard for identifier conflicts: and is discussed in more detail in the modules document.

unions

Unions are tagged type unions. They provide a high-level wrapper over an inner type that must be safely accessed via pattern matching.

They are declared with union[Variant(Type), ...] and initialized with the name of a variant followed by its inner type constructor in brackets: Square(side: 5). Tuples and structs are special-cased to eliminate extraneous parentheses.

type Value = u64
type Ident = str
type Expr = ref union
  Literal(Value)
  Variable(Ident)
  Abstraction(param: Ident, body: Expr)
  Application(body: Expr, arg: Expr)
  Conditional(
    condition: Expr
    then_case: Expr
    else_case: Expr
  )

They take up as much space in memory as the largest variant, plus the size of the tag (one byte).

pattern matching

Unions abstract over differing types. In order to safely be used, their inner types must be accessed via pattern matching: leaving no room for type confusion. Pattern matching in Puck relies on two syntactic constructs: the match statement, forcing qualification and handling of all possible types of a variable, and the of statement, querying type equality while simultaneously binding new identifiers to underspecified portions of variables.

use std.tables

func eval(context: mut HashTable[Ident, Value], expr: Expr): Result[Value]
  match expr
  of Literal(value) then Okay(value)
  of Variable(ident) then
    context.get(ident).err("Variable not in context")
  of Application(body, arg) then
    if body of Abstraction(param, body as inner_body) then
      context.set(param, context.eval(arg)?) # from std.tables
      context.eval(inner_body)
    else
      Error("Expected Abstraction, found {}".fmt(body))
  of Conditional(condition, then_case, else_case) then
    if context.eval(condition)? == "true" then
      context.eval(then_case)
    else
      context.eval(else_case)
  of expr then
    Error("Invalid expression {}".fmt(expr))

The match statement takes exclusively a list of of sub-expressions, and checks for exhaustivity. The expr of Type(binding) syntax can be reused as a conditional, in if statements and elsewhere.

The of operator is similar to the is operator in that it queries type equality, returning a boolean. However, unbound identifiers within of expressions are bound to appropriate values (if matched) and injected into the scope. This allows for succinct handling of union types in situations where match is overkill.

Each branch of a match expression can also have a guard: an arbitrary conditional that must be met in order for it to match. Guards are written as where cond and immediately follow the last pattern in an of branch, preceding then.

classes

Classes can be thought of as analogous to Rust's traits: without explicit impl blocks and without need for the derive macro. Types that have functions defined on them fulfilling the class requirements implicitly implement the associated class.

The class type is composed of a list of function signatures that refer to the special type Self that must exist for a type to be valid. The special type Self is replaced with the concrete type at compile time in order to typecheck. They are declared with class[signature, ...].

type Stack[T] = class
  push(self: mut Self, val: T)
  pop(self: mut Self): T
  peek(self: lent Self): lent T

func takes_any_stack(stack: Stack[int]) =
  # only stack.push, stack.pop, and stack.peek are available, regardless of the concrete type passed

Differing from Rust, Haskell, and many others, there is no explicit impl block. If there exist functions for a type that satisfy all of a class's signatures, it is considered to match and the class typechecks. This may seem strange and ambiguous - but again, static typing and uniform function call syntax help make this a more reasonable design. The purpose of explicit impl blocks in ex. Rust is three-fold: to provide a limited form of uniform function call syntax; to explicitly group together associated code; and to disambiguate. UFCS provides for the first, the module system provides for the second, and type-based disambiguation provides for the third, with such information exposed to the user via the language server protocol.

type Set[T] = class
  in(lent Self, T): bool
  add(mut Self, T)
  remove(mut Self, T): Option[T]

type Foo = struct
  a: int
  b: ref Set[int] # indirection: now perfectly valid

Classes cannot be constructed, as they are unsized. They serve purely as a list of valid operations on a type: no information about their memory layout is relevant. The concrete type fulfilling a class is known at compile time, however, and so there are no issues surrounding the use of classes as parameters, just when attempted to be used as (part of) a concrete type in ex. a struct. They can be used with indirection, however: as references are sized (consisting of a memory address).

## The Display class. Any type implementing `str` is printable.
## Any type that is Display must necessarily also implement Debug.
pub type Display = class
  str(Self): str
  dbg(Self): str

## The Debug class. Broadly implemented for every type with compiler magic.
## Types can (and should) override the generic implementations.
pub type Debug = class
  dbg(Self): str

Classes also cannot extend or rely upon other classes in any way, nor is there any concept of a parameter satisfying two classes. In the author's experience, while such constructions are powerful, they are also an immense source of complexity, leading to less-than-useful hierarchies seen in languages like Java, and yes, Rust. Instead, if one wishes to form an class that also satisfies another class, they must name a new class that explicitly includes all of the other class's associated functions. Given that classes in Puck overwhelmingly only have a small handful of associated functions, and if you're using more than one class you really should be using a concrete type: the hope is that this will provide for explicitness and reduce complexity.

Classes compose well with modules to offer fine grained access control.

Errata

default values

Puck does not have any concept of null: all values must be initialized. But always explicitly initializing types is syntactically verbose, and so most types have an associated "default value".

Default values:

• bool: false
• int, uint, etc: 0
• float, etc: 0.0
• char: '\0'
• str: ""
• void, never: unconstructable
• array[T], list[T]: []
• set[T], table[T, U]: {}
• tuple[T, U, ...]: (default values of its fields)
• struct[T, U, ...]: {default values of its fields}
• enum[One, Two, ...]: disallowed
• union[T, U, ...]: disallowed
• slice[T], func: disallowed
• ref, refc, ptr: disallowed

For unions, slices, references, and pointers, this is a bit trickier. They all have no reasonable "default" for these types aside from null. Instead of giving in, the compiler instead disallows any non-initializations or other cases in which a default value would be inserted.

todo: consider user-defined defaults (ex. structs)

signatures and overloading

Puck supports overloading - that is, there may exist multiple functions, or multiple types, or multiple modules, with the same name - so long as they have a different signature. The signature of a function/type/module is important. Classes, among other constructs, depend on the user having some understanding of what the compiler considers to be a signature. So we state it here explicitly:

• The signature of a function is its name and the types of each of its parameters, in order, ignoring optional parameters. Generic parameters are ???
  • ex. ...
• The signature of a type is its name and the number of generic parameters.
  • ex. both Result[T] and Result[T, E] are defined in std.results
• The signature of a module is just its name. This may change in the future.

structural subtyping

Mention of subtyping has been on occasion in contexts surrounding structural type systems, particularly the section on distinct types, but no explicit description of what the subtyping rules are have been given.

Modules and Namespacing

! This section is incomplete. Proceed with caution.

Puck has a first-class module system, inspired by such expressive designs in the ML family.

What are modules?

pub mod stack =
  pub type Stack[T] = class
    init(static type Self): Stack[T]
    push(mut Self, val: T)
    pop(mut Self): T?
    peek(lent Self): lent T?

  pub mod list =
    type ListStack[T] = list[T]

    pub func init[T](self: static type ListStack[T]): Stack[T] = []
    pub func push[T](self: mut ListStack[T], val: T) = self.push(T)
    pub func pop[T](self: mut ListStack[T]): T? = self.pop
    pub func peek[T](self: lent ListStack[T]): lent T? =
      if self.len == 0 then None else Some(self.last)

use stack.list

let a = ListStack[int].init
print a.len # error: unable to access method on private type outside its module

a.push(5)
print a.pop # Some(5)

Modules package up code for use by others. Identifiers known at compile time may be part of a module: these being constants, functions, macros, types, and other modules themselves. Such identifiers may be made accessible outside of the module by prefixing them with the pub keyword.

Importantly, files are implicitly modules, public and named with their filename. The mod keyword followed by an identifier and an indented block of code explicitly defines a module, inside of the current module. Modules are first class: they may be bound to constants (having the type : mod) and publicly exported, or bound to local variables and passed into functions for who knows what purpose.

Using modules

The use keyword lets you use other modules.

The use keyword imports public symbols from the specified module into the current scope unqualified. This runs contrary to expectations coming from most other languages: from Python to Standard ML, the standard notion of an "import" puts the imported symbols behind another symbol to avoid "polluting the namespace". As Puck is strongly typed and allows overloading, however, we see no reason for namespace pollution to be of concern. These unqualified imports have the added benefit of making uniform function call syntax more widely accessible. It is inevitable that identifier conflicts will exist on occasion, of course: when this happens, the compiler will force qualification (this then does restrict uniform function call syntax). We discuss this more later.

Nonetheless, if qualification of imports is so desired, an alternative approach is available - binding a module to a constant. Both the standard library and external libraries are available behind identifiers without use of use: std and lib, respectively. (FFI and local modules will likely use more identifiers, but this is not hammered out yet.) A submodule - for example, std.net - may be bound in a constant as const net = std.net, providing all of the modules' public identifiers for use, as fields of the constant net. We will see this construction to be extraordinarily helpful in crafting high-level public APIs for libraries later on.

use std.[logs, test]
use lib.crypto, lib.http

Multiple modules can be imported at once. The standard namespaces deserve more than a passing mention. There are several of these: std for the standard library, lib for all external libraries, pkg for the top-level namespace of a project, this for the current containing module... In addition: there are a suite of language namespaces, for FFI - rust, nim, and swift preliminarily - that give access to libraries from other languages. Recall that imports are unqualified - so use std will allow use of the standard library without the std qualifier (not recommended: several modules have common names), and use lib will dump the name of every library it can find into the global namespace (even less recommended).

Implicit Modules

A major goal of Puck's module system is to allow the same level of expressiveness as the ML family, while cutting down on the extraneous syntax and boilerplate needed to do so. As such, access modifiers are written directly inline with their declaration, and the file system structure is reused to form an implicit module system for internal use. This - particularly the former - limits the structure a module can expose at first glance, but we will see later that classes recoup much of this lost specificity.

We mentioned that the filesystem forms an implicit module structure. This begets a couple of design choices. Module names must be lowercase, for compatibility with case-insensitive filesystems. Both a file and a folder with the same name can exist. Files within the aforementioned folder are treated as submodules of the aforementioned file. This again restricts the sorts of module structures we can build, but we will see later that this restriction can be bypassed.

The this and pkg modules are useful for this implicit structure...

Defining interfaces

...

Defining an external API

The filesystem provides an implicit module structure, but it may not be the one you want to expose to users.

...

Memory Management

Puck's memory management system takes heavy inspiration from Rust.

The core borrowing semantics are copied 1:1. Puck's lent T is Rust's &T, mut T is &mut T, and T is T. There are several changes, however:

• T is coerced into lent T and mut T whenever possible
• all lifetimes are elided: indeterminate lifetimes are disallowed
• Copy types are implicit and automatically derived
• Box is ref and built-in
• Rc and Arc are unified under one compiler-optimized refc type

I personally think that Rust's system of ownership is quite good and a joy to use. And more the matter: I can't think of anything better? Every tweak I try to make to Rust's system falls apart when I work through what it looks like in practice. Making ownership rather than borrowing annotated complicates things, because most of the time ownership is desired. Making references second class simplifies things, but not by all that much, and greatly restricts the expressiveness of the language.

The exceptions here being coercing to references (which is trivial) and eliding lifetimes. I have seen nothing to suggest total lifetime elision in Rust is impossible: rather, it seems to mostly be an intentional choice for the purpose of explicitness (and not needing to inspect function bodies). I don't like this, and will be attempting to see just how doable total elision is.

refc and ownership elision

The refc type is special. Being a compiler built-in, it can take advantage of heavy optimizations: notably the count-eliding Perceus algorithm, used in Nim and Koka. The basic idea is to trace ownership throughout the program graph - and wherever the compiler can statically determine an object is moved, omit counts. And Puck has just the ownership framework to reuse!

implicit copy types

There is a distinction between types that can be transparently copied - that is, directly duplicating bytes to form a new object is acceptable - and those not. Pointers to heap memory -

cannot be copied,

stack-oriented

#![allow(unused)]
fn main() {
fn foo(a:
  &T → lent T
  &mut T → mut T
  T → T
):
  &T → lent T
  &mut T → mut T
  T → T
}

#![allow(unused)]
fn main() {
let t: T
foo(
  &t → lent t
  &mut t → mut t
  t → t
)

let u =
  &t → lent t
  &mut t → mut t
  t → t

&&t → lent lent t
*t → deref t

type Foo[T] = struct
  x: lent T
}

I don't know what I want out of a memory management system!

There are quite a deal of interesting avenues to pursue here, and they all profoundly impact the rest of the language (particularly iteration). In no particular order, here are the ones I've been thinking a lot about.

reference counting with ownership elision

Copy Nim/Koka. This is very easy to do: you implement standard reference counting, add a static ownership pass to track places of last use, and remove checks where possible. The benefits of this is that it's simple, deterministic, and performant. It leaves some performance on the table as opposed to Rust, but the usability benefits are high - in particular, you can use it just as you would use any other GC'd language. It's also well-documented by Koka, and lends itself decently well to further optimizations. I will probably end up going with it.

first-class references

Copy Rust. This means you need to support stacked borrows and such. That is a bit unfortunate, both syntax and semantics-wise. You inherit all the benefits of Rust-style ownership - and the downsides, too. In particular, you now need to worry about lifetime elision, and whether that's universally possible.

I am actually really tempted to do this though. Rust + whitespace oriented syntax + better metaprogramming + better modules + overloading + seems fun.

#![allow(unused)]
fn main() {
fn foo(a:
  &T → lent T
  &mut T → mut T
  T → T
):
  &T → lent T
  &mut T → mut T
  T → T
}

#![allow(unused)]
fn main() {
let t: T
foo(
  &t → lent t
  &mut t → mut t
  t → t
)

let u =
  &t → lent t
  &mut t → mut t
  t → t

&&t → lent lent t
*t → deref t

type Foo[T] = struct
  x: lent T
}

o fuk this is really tempting

second-class references

Or, simplified borrowing. Much - nigh all - the complexity of ownership in Rust comes from references being first-class: you can return them, you can set aliases to them, you can create them on-the-fly. But what if they weren't first class? What if you couldn't return references? How much expressive power would you lose?

#![allow(unused)]
fn main() {
fn foo(a:
  &T → lent T
  &mut T → mut T
  T → T
):
  &T → n/a
  &mut T → n/a
  T → T
}

#![allow(unused)]
fn main() {
let t: T
foo(
  &t → lent t // this could probably be elided
  &mut t → mut t // this could probably also be elided
  t → t
)

let u =
  &t →
  &mut t → mut t
  t → t
}

reference by default

...

#![allow(unused)]
fn main() {
fn foo(a:
  &'a T → T
  &mut T → mut T
  T → sink T
):
  &'a T → lent T
  &'a mut T → ???
  T → T
}

#![allow(unused)]
fn main() {
let t: T
foo(
  &t → t
  &mut t → t
  t → t
)

let u =
  &t → lent t
  &mut t → mut t
  t → t
}

Puck's T: Rust's &T Puck's mut T: Rust's &mut T Puck's sink T / owned T: Rust's T Puck's ref T: ??? Puck's ptr T: ???

Types get coerced: no need to say consumes(owned T) (when called).

Puck's memory management system takes heavy inspiration from Lobster and Nim. It is deterministic, requires no reference or lifetime annotations, and has zero runtime overhead (no garbage collector). It achieves this by a sophisticated ownership algorithm building off of ownership implementations in Rust, Nim, Lobster, Koka, and others, but also by restricting the actions of the programmer - in particular, there is no concept of returning a reference from a function or placing a reference within a datatype.

First, a recap of the problems and perils of memory management:

# Data is explicitly heap-allocated
type Data = ref struct[x: int, y: int, z: str]

# use-after-free


# double-free
func consumes_ptr(data: mut Data) =
  let more_data: Data = {x: 5, y: 6, z: "heap allocated!"}
  data = more_data
  #

let heap_ptr: Data = malloc(Data.sizeof)
consumes_ptr(heap_ptr)
free(heap_ptr) # heap_ptr was already freed! segfault

The above pseudo-code showcases the two fundamental errors that a safe memory management system aims to solve: double frees and use-after-frees.

Garbage collected languages solve this by adding runtime overhead to programs: by injecting a check for other existing references before an object would be deallocated, or by periodically tracing references to the heap to determine what memory is not live and may be deallocated. But this overhead comes with a problem: there is a class of stubborn programmers who will take minmaxing performance over 70% fewer vulnerabilities any day of the week.

Thus, Rust's system of ownership - restricting...

Puck's primary

The cheapest form of memory management is not having to manage it at all.

The best memory management is.. not to have to manage it at all!

small structs

You can have references to heap-allocated memory in types, of course - but it's owned by that...

The whole language is subject to change. But the memory management system, particularly more so - I can sit here and describe how I think it works all day, but I'm inexperienced in the implementation of such systems, and so it's likely I will hit some implementation snags - and although I hope for the best, there is the possibility that some would prove to be fatal.

should provide for the closure of such systems i mean destructors. god

TL;DR: I think the best form of memory management is reference counting: it is deterministic and can be optimized down.

TL;DR: Puck uses semi-optimized reference counting that can be manually optimized into an ownership system.

First, a summary of existing memory systems:

• manual memory management: extremely unsafe!
  • extremely unsafe, again! but it can be useful/necessary to turn off safety...
• tracing garbage collection
  • scans live memory
  • typically inconsistent i.e. performance spikes
• reference counting
  • scans dead memory
  • typically slow: querying and updating reference counts is actually a fairly heavy operation
• ownership / move semantics
  • inflexible: implementing say, linked lists is really hard and has to be done in what amounts to another system
• regions?

Examples of problems:

x = 5
y = x
free(x)

puck's memory management should copy Lobster, and take inspiration from Rust.

We should be able to copy the same memory management algorithm as Rust, only instead of failing to compile throw an error

goal: avoid lifetime annotations: https://doc.rust-lang.org/book/ch10-03-lifetime-syntax.html

Access errors: invalid read/write of a pointer Buffer overflow - out-of-bound writes can corrupt the content of adjacent objects, or internal data (like bookkeeping information for the heap) or return addresses. Buffer over-read - out-of-bound reads can reveal sensitive data or help attackers bypass address space layout randomization. Race condition - concurrent reads/writes to shared memory Invalid page fault - accessing a pointer outside the virtual memory space. A null pointer dereference will often cause an exception or program termination in most environments, but can cause corruption in operating system kernels or systems without memory protection, or when use of the null pointer involves a large or negative offset. Use after free - dereferencing a dangling pointer storing the address of an object that has been deleted. Uninitialized variables - a variable that has not been assigned a value is used. It may contain an undesired or, in some languages, a corrupt value. Null pointer dereference - dereferencing an invalid pointer or a pointer to memory that has not been allocated Wild pointers arise when a pointer is used prior to initialization to some known state. They show the same erratic behavior as dangling pointers, though they are less likely to stay undetected. Memory leak - when memory usage is not tracked or is tracked incorrectly Stack exhaustion - occurs when a program runs out of stack space, typically because of too deep recursion. A guard page typically halts the program, preventing memory corruption, but functions with large stack frames may bypass the page. Heap exhaustion - the program tries to allocate more memory than the amount available. In some languages, this condition must be checked for manually after each allocation. Double free - repeated calls to free may prematurely free a new object at the same address. If the exact address has not been reused, other corruption may occur, especially in allocators that use free lists. Invalid free - passing an invalid address to free can corrupt the heap. Mismatched free - when multiple allocators are in use, attempting to free memory with a deallocation function of a different allocator[20] Unwanted aliasing - when the same memory location is allocated and modified twice for unrelated purposes.

look into: koka/nim (nim i believe is equivalent to koka? but koka has way more documentation) rust, of course

Lifetimes

no general specification overview, because i don't actually know how thee would work entirely

notes

you do not need lifetime annotations. there is a minimally-correct set of lifetime parameters for any given program: rust doesn't do this because it doesn't inspect inside function bodies i guess?

lifetimes are probably closely tied to move semantics

so lifetimes come up with things like constants, where everything has a static lifetime

this seems to me like a bit of a hack and really seems ugly, i'm sure you could do with much better with const blocks

in fact i simply must use const exprs i love them

You cannot return a reference from a function.

The unsafe parts of the language are ptr and unsafe.

Size of types

The only types where the size is not known at compile time are:

• interfaces

types can become re-sized when used with ref

Ownership and Threads

The only types that are not Send is Rc. The only types that are not Sync are

Copy and Clone/DeepCopy

Metaprogramming

Puck has rich metaprogramming support, heavily inspired by Nim. Many features that would have to be at the compiler level in most languages (error propagation ?, std.fmt.print, ?, !, -> type sugar, => closure sugar, async/await) are instead implemented as macros within the standard library.

Macros take in fragments of the AST within their scope, transform them with arbitrary compile-time code, and spit back out transformed AST fragments to be injected and checked for validity. This is similar to what the Lisp family of languages do. It has a number of benefits: there is no separate metaprogramming language, it is syntactically and semantically hygienic, and the underlying framework can be reused for all kinds of compile-time code execution.

By keeping an intentionally minimal AST, some things not possible to express in literal code may be expressible in the AST: in particular, bindings can be injected in many places they could not be injected in ordinarily. (A minimal AST also has the benefit of being quite predictable.)

Scope

Macros may not change Puck's syntax: the syntax is flexible enough. They have the same scope as other routines, that is:

function scope: takes the arguments within or following a function call

macro print(params: varargs) =
  var res = Call("write", [stdout])
  for param in params do
    res.params.add(param)

print(1, 2, 3, 4)
print "hello", " ", "world", "!"

block scope: takes the expression following a colon as a single argument

macro my_macro(body)

my_macro
  1
  2
  3
  4

operator scope: takes one or two parameters either as an infix (two parameters) or a postfix (one parameter) operator

# operators are restricted to punctuation
macro +=(a, b) =
  Call("=", [a, Call("+", [a, b])])

a += b

Usage

Macros typically take a list of parameters without types, but they optionally may be given a type to constrain the usage of a macro. Regardless: as macros operate at compile time, their parameters are not instances of a type, but rather an Expr expression representing a portion of the abstract syntax tree. Similarly, macros always return an Expr to be injected into the abstract syntax tree despite the usual absence of an explicit return type, but the return type may be specified to additionally typecheck the returned Expr.

As macros operate at compile time, they may not inspect the values that their parameters evaluate to. However, parameters may be marked const: in which case they will be treated like parameters in functions: as values. (note constant parameters may be written as const[T] or const T.)

macro ?[T, E](self: Result[T, E]) =
  quote
    match `self`
    of Okay(x) then x
    of Error(e) then return Error(e)

func meow: Result[bool, ref Err] =
  let a = stdin.get()?

Quoting

The quote macro is special. It takes in literal code and returns that code as the AST. Within quoted data, backticks may be used to break out in order to evaluate and inject arbitrary code: though the code must evaluate to an expression of type Expr. Thus, quoting is structured: one cannot simply quote any arbitrary section. Quoting is very powerful: most macros are implemented using it.

The Expr type is available from std.ast, as are many helpers, and combined they provide the construction of arbitrary syntax trees (indeed, quote relies on and emits types of it). It is a union type with its variants directly corresponding to the variants of the internal AST of Puck.

Construction of macros can be difficult: and so several helpers are provided to ease debugging. The Debug and Display interfaces are implemented for abstract syntax trees: dbg will print a representation of the passed syntax tree as an object, and print will print a best-effort representation as literal code. Together with quote and optionally with const, these can be used to quickly get the representation of arbitrary code.

Error Handling

Puck's error handling is heavily inspired syntactically by Swift and semantically by the underlying effects system. It uses a combination of monadic error handling and effectful error propagation, with much in the way of syntactic sugar for conversion between the two, and leans somewhat heavily on Puck's metaprogramming capabilities. In comparison to Rust, it is considerably more dynamic by default.

There are several ways to handle errors in Puck. If the error is encoded in the type (as an Option or Result type), one can:

1. match on the error
2. compactly match on the error with if ... of
3. propagate the error with ?
4. throw the error with !

If the error is thrown (encoded as an effect), one can:

1. ignore the error, propagating it up the call stack
2. recover from the error in a try block
3. convert the error to a Result[T] (monadic form)

If an error is thrown, one must explicitly handle it at some level of the stack, or risk runtime failure. This method of error handling may feel more familiar to Java programmers. The compiler will warn on - but not enforce catching - such unhandled errors.

Errors as monads

Puck provides Option[T] and a Result[T, E] types, imported by default. These are union types under the hood and so must be pattern matched upon to be useful: but the standard library provides a bevy of helper functions. Two in particular are of note. The ? operator unwraps a Result or propagates its error up a function call (and may only be used in type-appropriate contexts). The ! operator unwraps an Option or Result directly or throws an exception in the case of None or Error.

pub macro ?[T, E](self: Result[T, E]) =
  quote
    match `self`
    of Okay(x) then x
    of Error(e) then return Error(e)

pub func ![T](self: Option[T]): T =
  match self
  of Some(x) then x
  of None then raise "empty value"

pub func ![T, E](self: Result[T, E]): T =
  match self
  of Okay(x) then x
  of Error(e) then raise e

The utility of the provided helpers in std.options and std.results should not be understated. While encoding errors into the type system may appear restrictive at first glance, some syntactic sugar goes a long way in writing compact and idiomatic code. Java programmers in particular are urged to give type-first errors a try, before falling back on unwraps and try/with.

A notable helpful type is the aliasing of Result[T] to Result[T, ref Err], for when the particular error does not matter. This breaks match exhaustion (as ref Err denotes a reference to any Error), but is particularly useful when used in conjunction with the propagation operator.

Errors as checked exceptions

Some functions do not return a value but can still fail: for example, setters. This can make it difficult to do monadic error handling elegantly. One could return a type Success[E] = Result[void, E], but such an approach is somewhat inelegant. Instead: we treat an assert within a function as having an effect: a possible failure, that can be handled and recovered from at any point in the call stack. If a possible exception is not handled within a function body, the function is implicitly marked by the compiler as throwing that exception.

pub type list[T] = struct
  data: ptr T
  capacity: uint
  length: uint

@[safe]
pub func set[T](self: list[T], i: uint, val: T) =
  if i > self.length then
    raise IndexOutOfBounds
  self.data.set(offset = i, val)

var foo = ["Hello", "world"]
foo.set(0, "Goodbye") # set can panic
# this propagates an IndexOutOfBounds effect up the call stack.

Despite functioning here as exceptions: errors remain types. An error thrown from an unwrapped Result[T, E] is of type E. with statements, then, may pattern match upon possible errors, behaving semantically and syntactically similarly to of branches: though notably not requiring exhaustion.

try
  foo.set(0, "Goodbye")
with IndexOutOfBounds(index) then
  dbg "Index out of bounds at {}".fmt(index)
  panic
finally
  ...

This creates a distinction between two types of error handling, working in sync: functional error handling with Option and Result types, and object-oriented error handling with algebraic effects. These styles may be swapped between with minimal syntactic overhead. It is up to libraries to determine which classes of errors are exceptional and best given the effect treatment and which should be explicitly handled monadically. Libraries should tend towards using Option/Result as this provides the best support for both styles (thanks to the ! operator).

Unrecoverable exceptions

There exist errors from which a program can not reasonably recover. These are the following:

• Assertation Failure: a call to an unhandled assert function has returned false at runtime.
• Out of Memory: the executable is out of memory.
• Stack Overflow: the executable has overflowed the stack.
• any others?

They are not recoverable, and not handled within the effects system, but the user should be aware of them as possible failure conditions.

References

• Error Handling in Swift
• Algebraic Effects for the rest of us

Asynchronous Programming

! This section is a draft. Many important details have yet to be ironed out.

Puck has colourless async/await, heavily inspired by Zig's implementation.

pub func fetch(url: str): str = ...

let a: Future[T] = async fetch_html()
let b: T = a.await
let c: T = await async fetch_html()

Puck's async implementation relies heavily on its metaprogramming system.

The async macro will wrap a call returning T in a Future[T] and compute it asynchronously. The await function takes in a Future[T] and will block until it returns a value (or error). The Future[T] type is opaque, containing internal information useful for the async and await routines.

pub macro async(self): Future[T] =
  ... todo ...

pub func await[T](self: Future[T]): T =
  while not self.ready do
    # block
  self.value! # apply callbacks?

This implementation differs from standard async/await implementations quite a bit. In particular, this means there is no concept of an "async function" - any block of computation that resolves to a value can be made asynchronous. This allows for "anonymous" async functions, among other things.

This (packaging up blocks of code to suspend and resume arbitrarily) is hard, and requires particular portable intermediate structures out of the compiler. Luckily, Zig is doing all of the R&D here. Some design decisions to consider revolve around APIs. The Linux kernel interface (among other things) provides both synchronous and asynchronous versions of its API, and fast code will use one or the other, depending if it is in an async context. Zig works around this by way of a known global constant that low-level functions read at compile time to determine whether to operate on synchronous APIs or asynchronous APIs. This is... not great. But what's better?

Threading

It should be noted that async is not the same as threading, nor is it solely useful in the presence of threads...

How threads work deserves somewhat of a mention...

References:

• What color is your function?
• What is Zig's "colorblind" async/await?
• Zig Learn: Async
• Rust async is colored and that's not a big deal
• Why is there no need for async/await in Elixir?
• Async/await on Wikipedia
• nim-chronos
• nim-cps
• tokio
• Zig-style async/await for Nim

Is async worth having separate from effect handlers? I think so...

Interop with Other Languages

! This section is a draft. Many important details have yet to be ironed out.

A major goal of Puck is minimal-overhead language interoperability while maintaining type safety.

The problems of interop

There are three issues that complicate language interop:

1. The language of communication, i.e. the C ABI.
2. Conflicting type systems, i.e. Python vs. Rust
3. Conflicting memory management systems, i.e. tracing / reference counting vs. ownership

For the first, Puck is being written at the same time as the crABI ABI spec is in development. crABI promises a C-ABI-compatible, cross-language ABI spec: which would dramatically simplify the task of linking to object files produced by other languages (so long as languages actually conform to the ABI). It is being led by the Rust language team, and both Nim and Swift developers have expressed interest in it, which bodes quite well for its future.

For the second, Puck has a type system of similar capability to that of Rust, Nim, and Swift: and thus interop with those languages should be a straightforward exchange of types. Its type system is strictly more powerful than that of Python or C, and so interop requires additional help. Its type system is equally as powerful as but somewhat orthogonal to Java's, and so interop will be a little more difficult.

For the third: Puck uses what amounts to a combination of ownership and reference counting: and thus it is exchangeable in this regard with Rust. Nim and Swift, by contrast, use reference counting: which is not directly compatible with ownership, as attempting to use an owned type as a GC'd reference will immediately lead to a use-after-free. Puck may have to explore some form of gradual typing at linking-time to accommodate making its functions available for use. Using functions from GC'd languages, however, is perfectly doable with the refc type: though this may necessitate copying object graphs over the call boundary.

There is additional significant work being put into the use of Wasm as a language runtime. Wasm allows for - among other things - the sharing of garbage collectors, which means that any garbage-collected language compiling to it can simply use the primitive refc type to denote a garbage-collected reference. This does not, however, immediately work off the bat with ownership: as ownership necessitates certain invariants that garbage collection does not preserve. There is active research into fixing this: notably RichWasm, which retrofits a structural type system with ownership atop Wasm. Such extensions necessitate the runtime environment to implement them, however, and so Puck may have to explore some form of gradual typing for the broader Wasm ecosystem.

Usability

use std.io
use rust.os.linux
use nim.os.sleep
...

Languages often focus on interop from purely technical details. This is very important: but typically little thought is given to usability (and often none can be, for necessity of compiler support), and so using foreign function interfaces very much feel like using foreign function interfaces. Puck attempts to change that.

@[form(this-function)]
pub func this_function() = ...

A trivial concern is that identifiers are not always the same across languages: for example, in Racket this-function is a valid identifier, while in Puck the - character is disallowed outright. Matters of convention are issues, too: in Puck, snake_case is preferred for functions and PamelCase for types, but this is certainly not always the case. Puck addresses this at an individual level by attributes allowing for rewriting: and at a language level by consistent rewrite rules.

...todo...

Existing systems to learn from:

• The Rust ABI
• rust-interop
• CBindGen
• swift-bridge
• Kotlin C interop
• rust-lang-interop
• extern in Rust
• NimPy
• JNim
• Futhark
• Haxe's callfunc

Example Programs

These are taken directly from the (work-in-progress) stdlib.

std.options

## std.options: Optional types.
## This module is imported by default.

use std.format

## The `Option` type.
## A type that represents either the presence or absence of a value.
pub type Option[T] = union
  Some(T)
  None

## Syntactic sugar for optional type declarations.
pub macro ?(T: type) =
  quote Option[`T`]

## Directly accesses the inner value. Throws an exception if None.
pub func ![T](self: T?): T =
  if self of Some(x) then x
  else raise "empty"

## Indirect access. Propagates `None`.
pub macro ?[T](self: Option[T]) =
  quote
    match `self`
    of Some(x) then x
    of None then return None

## Checks if a type is present within an `Option` type.
pub func is_some[T](self: T?): bool =
  self of Some(_)
## Checks if a type is not present within an `Option` type.
pub func is_none[T](self: T?): bool =
  self of None

## Converts an `Option[T]` to a `Result[T, E]` given a user-provided error.
pub func err[T, E](self: T?, error: E): Result[T, E] =
  if self of Some(x) then
    Okay(x)
  else
    Error(error)

## Applies a function to `T`, if it exists.
pub func map[T, U](self: T?, fn: T -> U): U? =
  if self of Some(x) then
    Some(fn(x))
  else
    None
## Converts `T` to a `None`, if `fn` returns false and it exists.
pub func filter[T](self: T?, fn: T -> bool): T? =
  if self of Some(x) and fn(x) then
    Some(x)
  else
    None

## Applies a function to T, if it exists. Equivalent to `self.map(fn).flatten`.
pub func flatmap[T, U](self: T?, fn: T -> U?): U? =
  if self of Some(x) then
    fn(x)
  else
    None
## Converts from Option[Option[T]] to Option[T].
pub func flatten[T](self: T??): T? =
  if self of Some(Some(x)) then
    Some(x)
  else
    None

## Returns the inner value or a default.
pub func get_or[T](self: T?, default: T): T =
  if self of Some(x) then x
  else default

## Overloads the `==` operation for use on Options.
pub func ==[T](a, b: T?): bool =
  if (a, b) of (Some(x), Some(y)) then
    x == y
  else
    false

## Overloads the `str()` function for use on Options.
pub func str[T: Display](self: T?): str =
  if self of Some(x) then
    "Some({})".fmt(x.str)
  else
    "None"

# references:
# https://nim-lang.github.io/Nim/options.html
# https://doc.rust-lang.org/std/option/enum.Option.html

std.results

## std.results: Result types.
## This module is imported by default.

use std.[options, format]

## The Result type. Represents either success or failure.
pub type Result[T, E] = union
  Okay(T)
  Error(E)

## The Err class. Useful for dynamically dispatching errors.
pub type Err = class
  str(Self): str
  dbg(Self): str

## A `Result` type that uses dynamically dispatched errors.
## The `Error` may be any type implementing `Err`.
pub type Result[T] = Result[T, ref Err]
## A `Result` type that only checks for success.
## Does not contain a value.
# pub type Success[E] = Result[void, E]
## A `Result` type that only checks for success.
## Does not contain a value. Dynamically dispatched.
# pub type Success = Result[void]

## Syntactic sugar for dynamic result type declarations.
pub macro !(T: type) =
  quote Result[`T`]

## Indirect access. Propagates `Error`.
pub macro ?[T, E](self: Result[T, E]) =
  quote
    match `self`
    of Okay(x) then x
    of Error(e) then return Error(e)

## Checks if a `Result` type was successful.
pub func is_ok[T, E](self: Result[T, E]): bool =
  self of Okay(_)
## Checks if a `Result` type was not successful.
pub func is_err[T, E](self: Result[T, E]): bool =
  self of Error(_)

## Converts from a `Result[T, E]` to an `Option[T]`.
pub func ok[T, E](self: Result[T, E]): T? =
  if self of Okay(x) then
    Some(x)
  else
    None
## Converts from a `Result[T, E]` to an `Option[E]`.
pub func err[T, E](self: Result[T, E]): E? =
  if self of Error(x) then
    Some(x)
  else
    None

## Applies a function to `T`, if self is `Okay`.
pub func map[T, E, U](self: Result[T, E], fn: T -> U): Result[U, E] =
  match self
  of Okay(x) then
    Okay(fn(x))
  of Error(e) then
    Error(e)
## Applies a function to `E`, if self is `Error`.
pub func map_err[T, E, F](self: Result[T, E], fn: E -> F): Result[T, F] =
  match self
  of Error(e) then
    Error(fn(e))
  of Okay(x) then
    Okay(x)

## Applies a function to `T`, if it exists. Equivalent to `self.map(fn).flatten`.
pub func flatmap[T, E, U](self: Result[T, E], fn: T -> Result[U, E]): Result[U, E] =
  match self
  of Okay(x) then
    fn(x)
  of Error(e) then
    Error(e)
## Converts from a `Result[Result[T, E], E]` to a `Result[T, E]`.
pub func flatten[T, E](self: Result[Result[T, E], E]): Result[T, E] =
  match self
  of Okay(Okay(x)) then
    Okay(x)
  of Okay(Error(e)), Error(e) then
    Error(e)

## Transposes a `Result[Option[T], E]` to an `Option[Result[T, E]]`.
pub func transpose[T, E](self: Result[T?, E]): Result[T, E]? =
  match self
  of Okay(Some(x)) then
    Some(Okay(x))
  of Okay(None), Error(_) then
    None
## Transposes an `Option[Result[T, E]]` to a `Result[Option[T], E]`. Takes a default error.
pub func transpose[T, E](self: Result[T, E]?, error: E): Result[T?, E] =
  match self
  of Some(Okay(x)) then Okay(Some(x))
  of Some(Error(e)) then Error(e)
  of None then Error(error)

## Returns the inner value or a default.
pub func get_or[T, E](self: Result[T, E], default: T): T =
  if self of Okay(x) then x
  else default

## Directly accesses the inner value. Throws an exception if `Error`.
pub func ![T, E](self: Result[T, E]): T =
  match self
  of Okay(x) then x
  of Error(e) then raise e
## Directly accesses the inner error. Throws an exception of type T if `Okay`.
pub func get_err[T, E](self: Result[T, E]): E =
  match self
  of Error(e) then e
  of Okay(x) then raise x

## Overloads the `==` operation for use on Results.
pub func ==[T, E, F](a: Result[T, E], b: Result[T, F]): bool =
  if (a, b) of (Okay(x), Okay(y)) then
    x == y
  else
    false

## Overloads the `str()` function for use on Results.
pub func str[T: Display, E: Display](self: Result[T, E]): str =
  match self
  of Some(x) then
    "Okay({})".fmt(x.str)
  of Error(e) then
    "Error({})".fmt(e.str)

# references:
# https://doc.rust-lang.org/std/result/enum.Result.html
# https://github.com/arnetheduck/nim-results
# https://github.com/codex-storage/questionable

std.format

## std.format: Niceties around printing and debugging.
## This module is imported by default.

## The Display class. Any type implementing `str` is printable.
## Any type that is Display must necessarily also implement Debug.
pub type Display = class
  str(Self): str
  dbg(Self): str

## The Debug class. Broadly implemented for every type with compiler magic.
## Types can (and should) override the generic implementations.
pub type Debug = class
  dbg(Self): str

## Prints all of its arguments to the command line.
pub func print(params: varargs[Display]) =
  stdout.write(params.map(x => x.str).join(" "), "\n")

## Prints all of its arguments to the command line, in Debug form.
##
## Note: this function is special! It does not count as a side effect.
## This breaks effect tracking, of course: but `dbg` is for debugging.
## It will produce a warning in code compiled for release.
@[pure]
pub func dbg(params: varargs[Debug]) =
  stdout.write(params.map(x => x.dbg).join(" "), "\n")

## A dummy implementation of the Display class for strings.
pub func str(self: str): str = self
## An implementation of the Debug class for strings.
pub func dbg(self: str): str = "\"" & self & "\""

## An implementation of the Debug class for all structs.
## Uses the special `struct` typeclass.
pub func dbg[T: Debug](self: struct[T]): str =
  "{{}}".fmt(self.fields.map((key, val) => key & ":" & val.dbg))

## An implementation of the Debug class for all tuples.
## Uses the special `tuple` typeclass.
pub func dbg[T: Debug](self: tuple[T]): str =
  "({})".fmt(self.fields.map((key, val) =>
    key.map(x => x & ":").get_or("") & val.dbg).join(", "))

## An implementation of the Debug class for all arrays and lists.
pub func dbg[T: Debug](self: Iter[T]): str =
  "[{}]".fmt(self.map(x => x.dbg).join(", "))

## The fmt macro. Builds a formatted string from its arguments.
pub macro fmt(self: const str, args: varargs[Display]): str =
  let parts = self.split("{}")
  if parts.len != args.len + 1 then
    macro_error("wrong number of arguments")
  use std.ast
  var res = parts.get(0)!
  for i, arg in args do
    res &= quote(`parts` & str(`arg`) &) # fixme
  res &= parts.last()!
  res

std.debug

## std.debug: Useful functions for debugging.
## This module is imported by default.

## The `assert` macro checks that a provided assertation is true,
## and panics and dumps information if it is not.
## Asserts remain in release builds. If not desired, see `dbg_assert`
pub macro assert(cond: bool) =
  quote
    if not `cond` then
      panic "assertation failed!\n {}".fmt(dbg(`cond`))

## The `dbg_assert` function provides an assert that is compiled out in release builds.
## This is useful for debugging performance-critical code.
pub macro dbg_assert(cond: bool) =
  quote
    when debug then # fixme: where is this constant coming from?
      assert `cond`

## The `discard` function consumes an object of any type.
## Useful for throwing away the result of a computation.
pub func discard[T](self: T) =
  return

## The `panic` function prints a message to `stderr` and quits.
pub func panic(message: str): never =
  stderr.write(message, "\n")
  std.os.exit(1)

## The special ... syntax is used to mark unimplemented parts of code.
## Such code will compile, but panic upon being called at runtime.
## It is usable almost anywhere, including in type declarations, thanks to compiler magic.
@[magic]
pub func ...: never =
  panic("unimplemented")

std.lists

## std.lists: Dynamic arrays.
## This module is imported by default.

## The fundamental list type. Heap-allocated.
## Equivalent to Vec<T> in other languages.
@[opaque] # opaque on a struct tells us raw field access breaks invariants.
pub type list[T] = struct
  data: ptr T
  capacity: uint
  length: uint

## A transparent, common alias for a list of bytes.
pub type bytes = list[byte]

## Initialize and return an empty list with inner type T.
pub func init[T]: list[T] =
  { data = nil, capacity = 0, length = 0 } # fixme: nil!!!!!

## Gets the length of a list.
@[inline] # idk what to do with attributes
pub func len[T](self: lent list[T]): uint =
  self.length

pub func empty[T](self: lent list[T]): bool =
  self.length == 0

## Gets the internal capacity of a list.
func cap[T](self: lent list[T]): uint =
  self.capacity

## Expands the capacity of a list.
@[safe]
func grow[T](self: mut list[T]) =
  self.capacity = max(self.length + 1, self.capacity * 2)
  self.data = self.data.realloc(self.capacity * sizeof(T))

## Pushes a new element to the end of a list.
@[safe]
pub func push[T](self: mut list[T], val: T) =
  if self.capacity == self.length then self.grow()
  self.data.set(val, offset = self.length)
  self.length += 1

## Takes ownership of and pushes all the values of a list into another list.
pub func push[T](self: mut list[T], values: list[T]) =
  for val in values do
    self.push(val)

## Removes & returns an element from the end of a list, if it exists.
@[safe]
pub func pop[T](self: mut list[T]): T? =
  if self.length == 0 then
    None
  else
    self.length -= 1
    Some(self.data.get(offset = self.length))

## Returns a reference to an element of a list, if in range.
@[safe]
pub func get[T](self: lent list[T], i: uint): lent T? =
  if i > self.length then
    None
  else # fixme: interior mutability
    Some(lent self.data.get(offset = i))
## Returns a mutable reference to an element of a list, if in range.
@[safe]
pub func get[T](self: mut list[T], i: uint): mut T? =
  if i > self.length then
    None
  else # fixme: interior mutability
    Some(mut self.data.get(offset = i))

## Sets the element of a list to a value.
@[safe]
pub func set[T](self: mut list[T], i: uint, val: T) =
  assert i <= self.length, "index out of bounds"
  Okay(self.data.set(offset = i, val))

## Inserts a value at a location and shifts elements of the list accordingly.
@[safe]
pub func insert[T](self: mut list[T], i: uint, val: T) =
  assert i <= self.length, "index out of bounds"
  if self.capacity == self.length then self.grow()
  self.data.offset(i).copy(self.data.offset(i + 1), self.length - i)
  self.data.set(i, val)
  self.length += 1
## Inserts a list of values at a location and shifts elements of the list accordingly.
pub func insert[T](self: mut list[T], i: uint, vals: list[T]) =
  for val in vals.rev: # inserting backwards avoids counting
    self.insert(val, i)

## Removes a value at a location and shifts elements of the list accordingly.
@[safe]
pub func remove[T](self: mut list[T], i: uint): T? =
  if index < self.length then None
  else
    self.length -= 1
    let res = self.data.get(i)
    self.data.offset(i + 1).copy(self.data.offset(i), self.length - i)
    res

## Gets the last element of a list, if it exists.
pub func last[T](self: lent list[T]): lent T? =
  self.get(self.len - 1)
## Gets the last element of a list mutably, if it exists.
pub func last[T](self: mut list[T]): mut T? =
  self.get(self.len - 1)

# reference: https://doc.rust-lang.org/nomicon/vec/vec.html

std.strings

## std.strings: The standard implementation of strings.
## This module is imported by default.

## A primitive string type.
##
## We do not want methods defined on `list[byte]` to carry over,
## so we define `str` as a newtype.
@[opaque]
pub type str = struct
  data: list[byte]

## Initialize and return an empty string.
pub func init: str = { data = [] }

## Gets the length of a string.
## This is an O(n) operation, due to UTF-8 encoding.
pub func len(self: lent str): uint =
  var res: uint
  for _ in self do
    res += 1
  res

## Pushes a character to the end of a mutable string.
pub func push(self: mut str, val: char) =
  self.data.push(val.byte) # todo: obsolete by from/to conversion??

## Pushes an owned string to the end of a mutable string.
pub func push(self: mut str, val: str) =
  self.data.push(val.bytes) # todo: obsolete by from/to conversion??

## Removes and returns the last character of a string, if it exists.
##
## SAFETY: We return early upon an empty string.
## And decrement by one char for a non-empty string.
@[safe]
pub func pop(self: mut str): char? =
  let char = self.chars.rev.next?
  self.data.set_len(self.len - char.len) # this is normally unsafe.
  Some(char)

## Returns the character at the provided index, if it exists.
pub func get(self: str, i: uint): char? =
  ...

## Sets the character at the provided index, if it exists.
## As strings are packed, this may call str.grow and reallocate.
## oh fuck we have to insert + remove anyway
pub func set(self: mut str, i: uint, val: char) =
  ...

## Inserts a character at an arbitrary position within a string.
## Panics on failure. (todo: can we do better?)
pub func insert(self: mut str, i: uint, val: char) =
  ...

## Removes and returns a character at an arbitrary position within a string.
## Panics on failure. (todo: can we do better?)
pub func remove(self: mut str, i: uint): char? =
  ...

## Syntactic sugar for string appending.
pub func &=(a: mut str, b: str) =
  a.push(b)

## The concatenation operator. Consumes two strings.
pub func &(a: str, b: str): str =
  a.push(b)
  a

## Conversion from a string to a list of bytes. Zero-cost.
pub func to(self: str): list[byte] = self.data
## Conversion from a str to a list[char]. Reallocates.
pub func to(self: str): list[char] =
  var res: list[char]
  for char in self do res.push(char)
  res
## Conversion from a char to an array of bytes. Zero-cost.
@[safe] # possibly unsafe?? depends on repr of arrays
pub func to(self: char): array[byte, 4] =
  self.cast[array[byte, 4]]

# reference: https://doc.rust-lang.org/std/string/struct.String.html

std.compare

## std.compare: Classes for comparable types.

## The Eq class. For types with some notion of equivalence.
pub type Eq = class
  ==(Self, Self): bool

## A blanket implementation of a corresponding not-equal function.
pub !=[T: Eq](a: T, b: T): bool =
  not(a == b)

## The Compare class. For a type comparable with itself.
pub type Compare = class
  <(a: Self, b: Self): bool

## A blanket implementation of a corresponding greater-than function.
## Note to self: do NOT inline!
pub func >[T: Compare](a: T, b: T): bool =
  b < a

## The Ord class. For types with some notion of equivalence and comparision.
##
## Note: This is *not* a mathematical notion of an order!
## No invariants on `<` nor `==` are guaranteed to hold, as classes
## are implicitly implementable.
pub type Ord = class
  <(a: Self, b: Self): bool
  ==(a: Self, b: Self): bool

## A blanket implementation of a corresponding less-than-or-equal function.
pub func <=[T: Ord](a: T, b: T): bool =
  a < b or a == b

## A blanket implementation of a corresponding greater-than-or-equal function.
pub func >=[T: Ord](a: T, b: T): bool =
  a > b or a == b

# reference: https://doc.rust-lang.org/std/cmp

std.convert

## std.convert: Classes for type coersion and conversion.
## This module is imported by default.

## The Coerce class is used for type conversion that will not fail.
## Its associated methods, `from` and `into`, are used internally
## by the compiler for implicit type conversion (coersion).
pub type Coerce[T] = class
  to(Self): T
  # from(T): Self

## The `from` function is automatically implemented for all types that
## implement `to`: that is, all types T that are convertable to U.
pub func from[T: Coerce[U], U](self: U): T =
  to(self)

## The Convert class is used for type conversion that may fail.
# We'll see what this breaks.
pub type Convert[T, E] = class
  to(Self): Result[T, E]

std.ranges

## std.ranges: Ranges of integers and other things. For iteration.
## This module is imported by default.

type Range[T] = struct
  start: T
  end: T

type RangeIncl[T] = struct
  start: T
  end: T
  done: bool

## Exclusive ranges. Useful for iteration.
## Includes `from`, does not include `to`.
pub func ..(from: int, to: int): Range[int] = { from, to }

## Inclusive ranges. Useful for ranges.
## Includes `from` and `to`.
pub func ..=(from: int, to: int): RangeIncl[int] = { from, to, done = false }

# todo: implement for all types that can increment or smth idk
pub func next[T: int](self: mut Range[T]): T? =
  if self.start < self.end then
    self.start += 1
    Some(self.start - 1)
  else
    None

# todo: We don't need a mutable Range here to peek.
# How does this interact with classes?
pub func peek[T: int](self: mut Range[T]): T? =
  self.peek_nth(0)

pub func peek_nth[T: int](self: mut Range[T], i: uint): T? =
  let res = self.start + i
  if res < self.end then
    Some(res)
  else
    None

pub func next[T: int](self: mut RangeIncl[T]): T? =
  if self.done then
    None
  elif self.start < self.end then
    let res = self.start
    self.start += 1
    Some(res)
  elif self.start == self.end then
    self.done = true
    Some(self.start)
  else
    self.done = true
    None

pub func peek[T: int](self: mut RangeIncl[T]): T? =
  self.peek_nth(0)

pub func peek_nth[T: int](self: mut RangeIncl[T], i: uint): T? =
  let res = self.start + i
  if res <= self.end
  then Some(res)
  else None

std.ast

## std.ast: Exposes the AST for building and operating on with macros.

## The `Expr` type represents the abstract syntax tree of Puck itself.
## It notably lacks type information. It is also not necessarily syntactically
## correct-by-construction: Cond, Try, and Match expressions must have at least
## one branch in their branches (yet this is not expressible here).
pub type Expr = union
  # Terms
  Ident(str)
  Number(int)
  Float(float)
  Char(char)
  String(str)
  Struct(list[(field: str, value: Expr)]) # {...}
  Tuple(list[(field: str?, value: Expr)]) # (...)
  List(list[Expr]) # [...]
  # Bindings
  Let(id: Pattern, kind: Type?, value: ref Expr)
  Var(id: Pattern, kind: Type?, value: ref[Expr]?)
  Constant(public: bool, id: Pattern, kind: Type?, value: ref Expr)
  FuncDecl(
    public: bool,
    id: str,
    generics: list[(id: str, kind: Type?)],
    params: list[(id: str, kind: Type)],
    kind: Type?,
    body: list[Expr])
  MacroDecl(
    public: bool,
    id: str,
    generics: list[(id: str, kind: Type?)],
    params: list[(id: str, kind: Type?)],
    kind: Type?,
    body: list[Expr])
  TypeDecl(
    public: bool,
    id: str,
    generics: list[str],
    body: Type)
  Module(
    public: bool,
    id: str,
    generics: list[str], # always empty for now
    body: list[Expr])
  Use(modules: list[(path: str, alias: str?)])
  # Control Flow
  Call(id: str, params: list[Expr])
  Cond(
    branches: list[(cond: Expr, body: list[Expr])],
    else_body: list[Expr])
  Try(
    try_body: list[Expr],
    catches: list[(exceptions: list[str], body: list[Expr])],
    finally_body: list[Expr]) # todo: throw this out
  Match(
    item: ref Expr,
    branches: list[(pattern: Pattern, guard: Expr?, body: list[Expr])])
  Block(id: str?, body: list[Expr])
  Static(body: list[Expr])
  For(binding: Pattern, range: ref Expr, body: list[Expr])
  While(cond: ref Expr, body: list[Expr])
  Loop(body: list[Expr])
  Attribute(on: ref Expr)
  Quote(body: ref Expr)
  Unquote(body: ref Expr)

pub type Type = ref union
  Never
  Int(size: uint)
  Dec(size: uint)
  Float(size: uint)
  Func(from: list[Type], to: Type)
  Struct(list[(id: str, kind: Type)])
  Tuple(list[(id: str?, kind: Type)])
  Union(list[(id: str, kind: Type)])
  Class(list[(id: str, from: list[Type], to: Type?)])
  Array(size: uint, kind: Type)
  List(Type)
  Slice(Type) # todo: plus ownership
  Alias(str) # todo: params?? huh?
  Const(Type)
  Lent(Type)
  Mut(Type)
  Ref(Type)
  Refc(Type)
  Ptr(Type)

pub type Pattern = union
  Ident(str)
  Number(int), Float(float), Char(char), String(str)
  Struct(name: str, params: list[Pattern])
  Tuple(list[Pattern])
  List(list[Pattern])

@[magic]
pub func quote(body): Expr