Merge remote-tracking branch 'origin/master' into debug

7 files changed, 181 insertions, 62 deletions

diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md
index 56f50e21..8cadb663 100644
--- a/book/src/SUMMARY.md
+++ b/book/src/SUMMARY.md
@@ -8,5 +8,7 @@
   - [Keymap](./keymap.md)
   - [Key Remapping](./remapping.md)
   - [Hooks](./hooks.md)
+  - [Languages](./languages.md)
 - [Guides](./guides/README.md)
+  - [Adding Languages](./guides/adding_languages.md)
   - [Adding Textobject Queries](./guides/textobject.md)
diff --git a/book/src/guides/adding_languages.md b/book/src/guides/adding_languages.md
new file mode 100644
index 00000000..446eb479
--- /dev/null
+++ b/book/src/guides/adding_languages.md
@@ -0,0 +1,60 @@
+# Adding languages
+
+## Submodules
+
+To add a new langauge, you should first add a tree-sitter submodule. To do this,
+you can run the command
+```sh
+git submodule add -f <repository> helix-syntax/languages/tree-sitter-<name>
+```
+For example, to add tree-sitter-ocaml you would run
+```sh
+git submodule add -f https://github.com/tree-sitter/tree-sitter-ocaml helix-syntax/languages/tree-sitter-ocaml
+```
+Make sure the submodule is shallow by doing
+```sh
+git config -f .gitmodules submodule.helix-syntax/languages/tree-sitter-<name>.shallow true
+```
+
+or you can manually add `shallow = true` to `.gitmodules`.
+
+## languages.toml
+
+Next, you need to add the language to the [`languages.toml`][languages.toml] found in the root of
+the repository; this `languages.toml` file is included at compilation time, and
+is distinct from the `language.toml` file in the user's [configuration
+directory](../configuration.md).
+
+These are the available keys and descriptions for the file.
+
+| Key           | Description                                                   |
+| ----          | -----------                                                   |
+| name          | The name of the language                                      |
+| scope         | A string like `source.js` that identifies the language. Currently, we strive to match the scope names used by popular TextMate grammars and by the Linguist library. Usually `source.<name>` or `text.<name>` in case of markup languages |
+| injection-regex | regex pattern that will be tested against a language name in order to determine whether this language should be used for a potential [language injection][treesitter-language-injection] site. |
+| file-types    | The filetypes of the language, for example `["yml", "yaml"]`  |
+| shebangs      | The interpreters from the shebang line, for example `["sh", "bash"]` |
+| roots         | A set of marker files to look for when trying to find the workspace root. For example `Cargo.lock`, `yarn.lock` |
+| auto-format   | Whether to autoformat this language when saving               |
+| comment-token | The token to use as a comment-token                           |
+| indent        | The indent to use. Has sub keys `tab-width` and `unit`        |
+| config        | Language server configuration                                 |
+
+## Queries
+
+For a language to have syntax-highlighting and indentation among other things, you have to add queries. Add a directory for your language with the path `runtime/queries/<name>/`. The tree-sitter [website](https://tree-sitter.github.io/tree-sitter/syntax-highlighting#queries) gives more info on how to write queries.
+
+## Common Issues
+
+- If you get errors when building after switching branches, you may have to remove or update tree-sitter submodules. You can update submodules by running
+    ```sh
+    git submodule sync; git submodule update --init
+    ```
+- Make sure to not use the `--remote` flag. To remove submodules look inside the `.gitmodules` and remove directories that are not present inside of it.
+
+- If a parser is segfaulting or you want to remove the parser, make sure to remove the submodule *and* the compiled parser in `runtime/grammar/<name>.so`
+
+- The indents query is `indents.toml`, *not* `indents.scm`. See [this](https://github.com/helix-editor/helix/issues/114) issue for more information.
+
+[treesitter-language-injection]: https://tree-sitter.github.io/tree-sitter/syntax-highlighting#language-injection
+[languages.toml]: https://github.com/helix-editor/helix/blob/master/languages.toml
diff --git a/book/src/guides/textobject.md b/book/src/guides/textobject.md
index 50b3b574..dd726b7c 100644
--- a/book/src/guides/textobject.md
+++ b/book/src/guides/textobject.md
@@ -5,7 +5,7 @@ require an accompanying tree-sitter grammar and a `textobjects.scm` query file
 to work properly. Tree-sitter allows us to query the source code syntax tree
 and capture specific parts of it. The queries are written in a lisp dialect.
 More information on how to write queries can be found in the [official tree-sitter
-documentation](tree-sitter-queries).
+documentation][tree-sitter-queries].
 
 Query files should be placed in `runtime/queries/{language}/textobjects.scm`
 when contributing. Note that to test the query files locally you should put
diff --git a/book/src/keymap.md b/book/src/keymap.md
index 5a6aee41..c88ed767 100644
--- a/book/src/keymap.md
+++ b/book/src/keymap.md
@@ -1,5 +1,8 @@
 # Keymap
 
+- Mappings marked (**LSP**) require an active language server for the file.
+- Mappings marked (**TS**) require a tree-sitter grammar for the filetype.
+
 ## Normal mode
 
 ### Movement
@@ -58,15 +61,19 @@
 | `.`         | Repeat last change                              | N/A                   |
 | `u`         | Undo change                                     | `undo`                |
 | `U`         | Redo change                                     | `redo`                |
+| `Alt-u`     | Move backward in history                        | `earlier`             |
+| `Alt-U`     | Move forward in history                         | `later`               |
 | `y`         | Yank selection                                  | `yank`                |
 | `p`         | Paste after selection                           | `paste_after`         |
 | `P`         | Paste before selection                          | `paste_before`        |
 | `"` `<reg>` | Select a register to yank to or paste from      | `select_register`     |
 | `>`         | Indent selection                                | `indent`              |
 | `<`         | Unindent selection                              | `unindent`            |
-| `=`         | Format selection                                | `format_selections`   |
+| `=`         | Format selection (**LSP**)                      | `format_selections`   |
 | `d`         | Delete selection                                | `delete_selection`    |
 | `c`         | Change selection (delete and enter insert mode) | `change_selection`    |
+| `Ctrl-a`    | Increment object (number) under cursor          | `increment`           |
+| `Ctrl-x`    | Decrement object (number) under cursor          | `decrement`           |
 
 #### Shell
 
@@ -85,6 +92,7 @@
 | `s`      | Select all regex matches inside selections                        | `select_regex`                       |
 | `S`      | Split selection into subselections on regex matches               | `split_selection`                    |
 | `Alt-s`  | Split selection on newlines                                       | `split_selection_on_newline`         |
+| `_`      | Trim whitespace from the selection                                | `trim_selections`                    |
 | `;`      | Collapse selection onto a single cursor                           | `collapse_selection`                 |
 | `Alt-;`  | Flip selection cursor and anchor                                  | `flip_selections`                    |
 | `,`      | Keep only the primary selection                                   | `keep_primary_selection`             |
@@ -98,9 +106,10 @@
 | `%`      | Select entire file                                                | `select_all`                         |
 | `x`      | Select current line, if already selected, extend to next line     | `extend_line`                        |
 | `X`      | Extend selection to line bounds (line-wise selection)             | `extend_to_line_bounds`              |
-|          | Expand selection to parent syntax node TODO: pick a key           | `expand_selection`                   |
+|          | Expand selection to parent syntax node TODO: pick a key (**TS**)  | `expand_selection`                   |
 | `J`      | Join lines inside selection                                       | `join_selections`                    |
 | `K`      | Keep selections matching the regex                                | `keep_selections`                    |
+| `Alt-K`  | Remove selections matching the regex                              | `remove_selections`                  |
 | `$`      | Pipe each selection into shell command, keep selections where command returned 0 | `shell_keep_pipe`     |
 | `Ctrl-c` | Comment/uncomment the selections                                  | `toggle_comments`                    |
 
@@ -127,25 +136,23 @@ key to return to normal mode after usage (useful when you're simply looking
 over text and not actively editing it).
 
 
-| Key       | Description                                               | Command             |
-| -----     | -----------                                               | -------             |
-| `z` , `c` | Vertically center the line                                | `align_view_center` |
-| `t`       | Align the line to the top of the screen                   | `align_view_top`    |
-| `b`       | Align the line to the bottom of the screen                | `align_view_bottom` |
-| `m`       | Align the line to the middle of the screen (horizontally) | `align_view_middle` |
-| `j`       | Scroll the view downwards                                 | `scroll_down`       |
-| `k`       | Scroll the view upwards                                   | `scroll_up`         |
-| `f`       | Move page down                                            | `page_down`         |
-| `b`       | Move page up                                              | `page_up`           |
-| `d`       | Move half page down                                       | `half_page_down`    |
-| `u`       | Move half page up                                         | `half_page_up`      |
+| Key           | Description                                               | Command             |
+| -----         | -----------                                               | -------             |
+| `z` , `c`     | Vertically center the line                                | `align_view_center` |
+| `t`           | Align the line to the top of the screen                   | `align_view_top`    |
+| `b`           | Align the line to the bottom of the screen                | `align_view_bottom` |
+| `m`           | Align the line to the middle of the screen (horizontally) | `align_view_middle` |
+| `j` , `down`  | Scroll the view downwards                                 | `scroll_down`       |
+| `k` , `up`    | Scroll the view upwards                                   | `scroll_up`         |
+| `f`           | Move page down                                            | `page_down`         |
+| `b`           | Move page up                                              | `page_up`           |
+| `d`           | Move half page down                                       | `half_page_down`    |
+| `u`           | Move half page up                                         | `half_page_up`      |
 
 #### Goto mode
 
 Jumps to various locations.
 
-> NOTE: Some of these features are only available with the LSP present.
-
 | Key   | Description                                      | Command                    |
 | ----- | -----------                                      | -------                    |
 | `g`   | Go to the start of the file                      | `goto_file_start`          |
@@ -156,13 +163,14 @@ Jumps to various locations.
 | `t`   | Go to the top of the screen                      | `goto_window_top`          |
 | `m`   | Go to the middle of the screen                   | `goto_window_middle`       |
 | `b`   | Go to the bottom of the screen                   | `goto_window_bottom`       |
-| `d`   | Go to definition                                 | `goto_definition`          |
-| `y`   | Go to type definition                            | `goto_type_definition`     |
-| `r`   | Go to references                                 | `goto_reference`           |
-| `i`   | Go to implementation                             | `goto_implementation`      |
+| `d`   | Go to definition (**LSP**)                       | `goto_definition`          |
+| `y`   | Go to type definition (**LSP**)                  | `goto_type_definition`     |
+| `r`   | Go to references (**LSP**)                       | `goto_reference`           |
+| `i`   | Go to implementation (**LSP**)                   | `goto_implementation`      |
 | `a`   | Go to the last accessed/alternate file           | `goto_last_accessed_file`  |
 | `n`   | Go to next buffer                                | `goto_next_buffer`         |
 | `p`   | Go to previous buffer                            | `goto_previous_buffer`     |
+| `.`   | Go to last modification in current file          | `goto_last_modification`   |
 
 #### Match mode
 
@@ -172,7 +180,7 @@ and [textobject](./usage.md#textobject) usage.
 
 | Key              | Description                                     | Command                    |
 | -----            | -----------                                     | -------                    |
-| `m`              | Goto matching bracket                           | `match_brackets`           |
+| `m`              | Goto matching bracket (**TS**)                  | `match_brackets`           |
 | `s` `<char>`     | Surround current selection with `<char>`        | `surround_add`             |
 | `r` `<from><to>` | Replace surround character `<from>` with `<to>` | `surround_replace`         |
 | `d` `<char>`     | Delete surround character `<char>`              | `surround_delete`          |
@@ -195,49 +203,76 @@ This layer is similar to vim keybindings as kakoune does not support window.
 | `k`, `Ctrl-k`, `up`    | Move to split above            | `jump_view_up`    |
 | `l`, `Ctrl-l`, `right` | Move to right split            | `jump_view_right` |
 | `q`, `Ctrl-q`          | Close current window           | `wclose`          |
+| `o`, `Ctrl-o`          | Only keep the current window, closing all the others            | `wonly`           |
 
 #### Space mode
 
 This layer is a kludge of mappings, mostly pickers.
 
-| Key     | Description                                                           | Command                             |
-| -----   | -----------                                                           | -------                             |
-| `k`     | Show documentation for the item under the cursor                      | `hover`                             |
-| `f`     | Open file picker                                                      | `file_picker`                       |
-| `b`     | Open buffer picker                                                    | `buffer_picker`                     |
-| `s`     | Open symbol picker (current document)                                 | `symbol_picker`                     |
-| `a`     | Apply code action                                                     | `code_action`                       |
-| `'`     | Open last fuzzy picker                                                | `last_picker`                       |
-| `w`     | Enter [window mode](#window-mode)                                     | N/A                                 |
-| `p`     | Paste system clipboard after selections                               | `paste_clipboard_after`             |
-| `P`     | Paste system clipboard before selections                              | `paste_clipboard_before`            |
-| `y`     | Join and yank selections to clipboard                                 | `yank_joined_to_clipboard`          |
-| `Y`     | Yank main selection to clipboard                                      | `yank_main_selection_to_clipboard`  |
-| `R`     | Replace selections by clipboard contents                              | `replace_selections_with_clipboard` |
-| `/`     | Global search in workspace folder                                     | `global_search`                     |
-
-> NOTE: Global search display results in a fuzzy picker, use `space + '` to bring it back up after opening a file.
+
+| Key     | Description                                                             | Command                             |
+| -----   | -----------                                                             | -------                             |
+| `f`     | Open file picker                                                        | `file_picker`                       |
+| `b`     | Open buffer picker                                                      | `buffer_picker`                     |
+| `k`     | Show documentation for item under cursor in a [popup](#popup) (**LSP**) | `hover`                             |
+| `s`     | Open document symbol picker (**LSP**)                                   | `symbol_picker`                     |
+| `S`     | Open workspace symbol picker (**LSP**)                                  | `workspace_symbol_picker`           |
+| `r`     | Rename symbol (**LSP**)                                                 | `rename_symbol`                     |
+| `a`     | Apply code action  (**LSP**)                                            | `code_action`                       |
+| `'`     | Open last fuzzy picker                                                  | `last_picker`                       |
+| `w`     | Enter [window mode](#window-mode)                                       | N/A                                 |
+| `p`     | Paste system clipboard after selections                                 | `paste_clipboard_after`             |
+| `P`     | Paste system clipboard before selections                                | `paste_clipboard_before`            |
+| `y`     | Join and yank selections to clipboard                                   | `yank_joined_to_clipboard`          |
+| `Y`     | Yank main selection to clipboard                                        | `yank_main_selection_to_clipboard`  |
+| `R`     | Replace selections by clipboard contents                                | `replace_selections_with_clipboard` |
+| `/`     | Global search in workspace folder                                       | `global_search`                     |
+
+> TIP: Global search displays results in a fuzzy picker, use `space + '` to bring it back up after opening a file.
+
+##### Popup
+
+Displays documentation for item under cursor.
+
+| Key      | Description |
+| ----     | ----------- |
+| `Ctrl-u` | Scroll up   |
+| `Ctrl-d` | Scroll down |
  
 #### Unimpaired
 
 Mappings in the style of [vim-unimpaired](https://github.com/tpope/vim-unimpaired).
 
-| Key       | Description                        | Command             |
-| -----     | -----------                        | -------             |
-| `[d`      | Go to previous diagnostic          | `goto_prev_diag`    |
-| `]d`      | Go to next diagnostic              | `goto_next_diag`    |
-| `[D`      | Go to first diagnostic in document | `goto_first_diag`   |
-| `]D`      | Go to last diagnostic in document  | `goto_last_diag`    |
-| `[space`  | Add newline above                  | `add_newline_above` |
-| `]space`  | Add newline below                  | `add_newline_below` |
+| Key      | Description                                  | Command             |
+| -----    | -----------                                  | -------             |
+| `[d`     | Go to previous diagnostic (**LSP**)          | `goto_prev_diag`    |
+| `]d`     | Go to next diagnostic (**LSP**)              | `goto_next_diag`    |
+| `[D`     | Go to first diagnostic in document (**LSP**) | `goto_first_diag`   |
+| `]D`     | Go to last diagnostic in document (**LSP**)  | `goto_last_diag`    |
+| `[space` | Add newline above                            | `add_newline_above` |
+| `]space` | Add newline below                            | `add_newline_below` |
 
 ## Insert Mode
 
-| Key      | Description           | Command                |
-| -----    | -----------           | -------                |
-| `Escape` | Switch to normal mode | `normal_mode`          |
-| `Ctrl-x` | Autocomplete          | `completion`           |
-| `Ctrl-w` | Delete previous word  | `delete_word_backward` |
+| Key                     | Description                 | Command                 |
+| -----                   | -----------                 | -------                 |
+| `Escape`                | Switch to normal mode       | `normal_mode`           |
+| `Ctrl-x`                | Autocomplete                | `completion`            |
+| `Ctrl-r`                | Insert a register content   | `insert_register`       |
+| `Ctrl-w`                | Delete previous word        | `delete_word_backward`  |
+| `Alt-d`                 | Delete next word            | `delete_word_forward`   |
+| `Alt-b`, `Alt-Left`     | Backward a word             | `move_prev_word_end`    |
+| `Ctrl-b`, `Left`        | Backward a char             | `move_char_left`        |
+| `Alt-f`, `Alt-Right`    | Forward a word              | `move_next_word_start`  |
+| `Ctrl-f`, `Right`       | Forward a char              | `move_char_right`       |
+| `Ctrl-e`, `End`         | move to line end            | `goto_line_end_newline` |
+| `Ctrl-a`, `Home`        | move to line start          | `goto_line_start`       |
+| `Ctrl-u`                | delete to start of line     | `kill_to_line_start`    |
+| `Ctrl-k`                | delete to end of line       | `kill_to_line_end`      |
+| `backspace`, `Ctrl-h`   | delete previous char        | `delete_char_backward`  |
+| `delete`, `Ctrl-d`      | delete previous char        | `delete_char_forward`   |
+| `Ctrl-p`, `Up`          | move to previous line       | `move_line_up`          |
+| `Ctrl-n`, `Down`        | move to next line           | `move_line_down`        |
 
 ## Select / extend mode
 
@@ -262,7 +297,9 @@ Keys to use within picker. Remapping currently not supported.
 | `Escape`, `Ctrl-c`           | Close picker      |
 
 # Prompt
+
 Keys to use within prompt, Remapping currently not supported.
+
 | Key                     | Description                                                             |
 | -----                   | -------------                                                           |
 | `Escape`, `Ctrl-c`      | Close prompt                                                            |
@@ -270,15 +307,18 @@ Keys to use within prompt, Remapping currently not supported.
 | `Ctrl-b`, `Left`        | Backward a char                                                         |
 | `Alt-f`, `Alt-Right`    | Forward a word                                                          |
 | `Ctrl-f`, `Right`       | Forward a char                                                          |
-| `Ctrl-e`, `End`         | move prompt end                                                         |
-| `Ctrl-a`, `Home`        | move prompt start                                                       |
-| `Ctrl-w`                | delete previous word                                                    |
-| `Ctrl-k`                | delete to end of line                                                   |
-| `backspace`             | delete previous char                                                    |
-| `Ctrl-s`                | insert a word under doc cursor, may be changed to Ctrl-r Ctrl-w later   |
-| `Ctrl-p`, `Up`          | select previous history                                                 |
-| `Ctrl-n`, `Down`        | select next history                                                     |
-| `Tab`                   | slect next completion item                                              |
-| `BackTab`               | slect previous completion item                                          |
+| `Ctrl-e`, `End`         | Move prompt end                                                         |
+| `Ctrl-a`, `Home`        | Move prompt start                                                       |
+| `Ctrl-w`                | Delete previous word                                                    |
+| `Alt-d`                 | Delete next word                                                        |
+| `Ctrl-u`                | Delete to start of line                                                 |
+| `Ctrl-k`                | Delete to end of line                                                   |
+| `backspace`, `Ctrl-h`   | Delete previous char                                                    |
+| `delete`, `Ctrl-d`      | Delete previous char                                                    |
+| `Ctrl-s`                | Insert a word under doc cursor, may be changed to Ctrl-r Ctrl-w later   |
+| `Ctrl-p`, `Up`          | Select previous history                                                 |
+| `Ctrl-n`, `Down`        | Select next history                                                     |
+| `Tab`                   | Select next completion item                                             |
+| `BackTab`               | Select previous completion item                                         |
 | `Enter`                 | Open selected                                                           |
 
diff --git a/book/src/languages.md b/book/src/languages.md
new file mode 100644
index 00000000..cef61501
--- /dev/null
+++ b/book/src/languages.md
@@ -0,0 +1,14 @@
+# Languages
+
+Language-specific settings and settings for particular language servers can be configured in a `languages.toml` file placed in your [configuration directory](./configuration.md). Helix actually uses two `languages.toml` files, the [first one](https://github.com/helix-editor/helix/blob/master/languages.toml) is in the main helix repository; it contains the default settings for each language and is included in the helix binary at compile time. Users who want to see the available settings and options can either reference the helix repo's `languages.toml` file, or consult the table in the [adding languages](./guides/adding_languages.md) section.
+
+Changes made to the `languages.toml` file in a user's [configuration directory](./configuration.md) are merged with helix's defaults on start-up, such that a user's settings will take precedence over defaults in the event of a collision. For example, the default `languages.toml` sets rust's `auto-format` to `true`. If a user wants to disable auto-format, they can change the `languages.toml` in their [configuration directory](./configuration.md) to make the rust entry read like the example below; the new key/value pair `auto-format = false` will override the default when the two sets of settings are merged on start-up:
+
+```
+# in <config_dir>/helix/languages.toml
+
+[[language]]
+name = "rust"
+auto-format = false
+```
+
diff --git a/book/src/remapping.md b/book/src/remapping.md
index 3369f031..532f502a 100644
--- a/book/src/remapping.md
+++ b/book/src/remapping.md
@@ -15,6 +15,7 @@ a = "move_char_left" # Maps the 'a' key to the move_char_left command
 w = "move_line_up" # Maps the 'w' key move_line_up
 "C-S-esc" = "extend_line" # Maps Control-Shift-Escape to extend_line
 g = { a = "code_action" } # Maps `ga` to show possible code actions
+"ret" = ["open_below", "normal_mode"] # Maps the enter key to open_below then re-enter normal mode
 
 [keys.insert]
 "A-x" = "normal_mode" # Maps Alt-X to enter normal mode
@@ -38,6 +39,7 @@ Control, Shift and Alt modifiers are encoded respectively with the prefixes
 | Left         | `"left"`       |
 | Right        | `"right"`      |
 | Up           | `"up"`         |
+| Down         | `"down"`       |
 | Home         | `"home"`       |
 | End          | `"end"`        |
 | Page         | `"pageup"`     |
diff --git a/book/src/usage.md b/book/src/usage.md
index 71730fa8..6b7cbc41 100644
--- a/book/src/usage.md
+++ b/book/src/usage.md
@@ -62,6 +62,7 @@ Currently supported: `word`, `surround`, `function`, `class`, `parameter`.
 | Key after `mi` or `ma` | Textobject selected      |
 | ---                    | ---                      |
 | `w`                    | Word                     |
+| `W`                    | WORD                     |
 | `(`, `[`, `'`, etc     | Specified surround pairs |
 | `f`                    | Function                 |
 | `c`                    | Class                    |