rework positioning/rendering and enable softwrap/virtual text (#5420)

* rework positioning/rendering, enables softwrap/virtual text

This commit is a large rework of the core text positioning and
rendering code in helix to remove the assumption that on-screen
columns/lines correspond to text columns/lines.

A generic `DocFormatter` is introduced that positions graphemes on
and is used both for rendering and for movements/scrolling.
Both virtual text support (inline, grapheme overlay and multi-line)
and a capable softwrap implementation is included.

fix picker highlight

cleanup doc formatter, use word bondaries for wrapping

make visual vertical movement a seperate commnad

estimate line gutter width to improve performance

cache cursor position

cleanup and optimize doc formatter

cleanup documentation

fix typos

Co-authored-by: Daniel Hines <d4hines@gmail.com>

update documentation

fix panic in last_visual_line funciton

improve soft-wrap documentation

add extend_visual_line_up/down commands

fix non-visual vertical movement

streamline virtual text highlighting, add softwrap indicator

fix cursor position if softwrap is disabled

improve documentation of text_annotations module

avoid crashes if view anchor is out of bounds

fix: consider horizontal offset when traslation char_idx -> vpos

improve default configuration

fix: mixed up horizontal and vertical offset

reset view position after config reload

apply suggestions from review

disabled softwrap for very small screens to avoid endless spin

fix wrap_indicator setting

fix bar cursor disappearring on the EOF character

add keybinding for linewise vertical movement

fix: inconsistent gutter highlights

improve virtual text API

make scope idx lookup more ergonomic

allow overlapping overlays

correctly track char_pos for virtual text

adjust configuration

deprecate old position fucntions

fix infinite loop in highlight lookup

fix gutter style

fix formatting

document max-line-width interaction with softwrap

change wrap-indicator example to use empty string

fix: rare panic when view is in invalid state (bis)

* Apply suggestions from code review

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>

* improve documentation for positoning functions

* simplify tests

* fix documentation of Grapheme::width

* Apply suggestions from code review

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>

* add explicit drop invocation

* Add explicit MoveFn type alias

* add docuntation to Editor::cursor_cache

* fix a few typos

* explain use of allow(deprecated)

* make gj and gk extend in select mode

* remove unneded debug and TODO

* mark tab_width_at #[inline]

* add fast-path to move_vertically_visual in case softwrap is disabled

* rename first_line to first_visual_line

* simplify duplicate if/else

---------

Co-authored-by: Michael Davis <mcarsondavis@gmail.com>

2 files changed, 23 insertions, 2 deletions

diff --git a/book/src/configuration.md b/book/src/configuration.md
index ab229f77..528fafd0 100644
--- a/book/src/configuration.md
+++ b/book/src/configuration.md
@@ -307,4 +307,25 @@ Currently unused
 
 #### `[editor.gutters.spacer]` Section
 
-Currently unused
\ No newline at end of file
+Currently unused
+
+### `[editor.soft-wrap]` Section
+
+Options for soft wrapping lines that exceed the view width
+
+| Key                 | Description                                                  | Default |
+| ---                 | ---                                                          | ---     |
+| `enable`            | Whether soft wrapping is enabled.                            | `false` |
+| `max-wrap`          | Maximum free space left at the end of the line.              | `20`    |
+| `max-indent-retain` | Maximum indentation to carry over when soft wrapping a line. | `40`    |
+| `wrap-indicator`    | Text inserted before soft wrapped lines, highlighted with `ui.virtual.wrap` | `↪ `    |
+
+Example:
+
+```toml
+[editor.soft-wrap]
+enable = true
+max-wrap = 25 # increase value to reduce forced mid-word wrapping
+max-indent-retain = 0
+wrap-indicator = ""  # set wrap-indicator to "" to hide it
+```
diff --git a/book/src/languages.md b/book/src/languages.md
index ff06dc00..0646b9af 100644
--- a/book/src/languages.md
+++ b/book/src/languages.md
@@ -61,7 +61,7 @@ These configuration keys are available:
 | `config`              | Language Server configuration                                 |
 | `grammar`             | The tree-sitter grammar to use (defaults to the value of `name`) |
 | `formatter`           | The formatter for the language, it will take precedence over the lsp when defined. The formatter must be able to take the original file as input from stdin and write the formatted file to stdout |
-| `max-line-length`     | Maximum line length. Used for the `:reflow` command           |
+| `max-line-length`     | Maximum line length. Used for the `:reflow` command and soft-wrapping           |
 
 ### File-type detection and the `file-types` key