Fix unicode errors

Changes `write()` method of native pipe and terminal to accept a
PoolByteArray rather than String. This means that `get_string_from_utf8()`
is no longer called on data coming from PTY and being sent to Terminal.

The terminal state machine already has a UTF8 parser which maintains
its state across calls to `write()`. This means that we can write half
the bytes of a single unicode character in one call and the remaining half
in the next call and the state machine will parse it correctly.

On the other hand, the `get_string_from_utf8()` method of Godot's
PoolByteArray requires that the array contains completely valid UTF8,
otherwise we get errors such as "Unicode error: invalid skip".

The data coming from PTY can be arbitrarily split in the middle of a
unicode character meaning that we will sometimes get errors when calling
`get_string_from_utf8()` on it. This is more likely to occur when there
is a large amount of output (i.e. it's more likely to be split). In other
cases, the data might intentionally contain invalid unicode such as when
printing binary files or random data (e.g. `cat /bin/sh`, `cat /dev/random`).

We avoid these errors by passing the PoolByteArray data directly to the terminal
state machine.

In addition to fixing unicode errors, this commit:
- Prevents repeated calls to pipes `_read_cb()` method that would block Godot
  and result in a crash with the message "ERROR: All memory pool allocations
  are in use" that resulted from writing data to an ever-increasing number of
  PoolByteArrays before any of them could be freed. This could be triggered by
  running the `cat /dev/urandom` command after making the change to `write()`
  mentioned above.
- Prevents memory leaks by freeing libuv buffers after they have been copied
  to PoolByteArrays.

Fixes #55.
This commit is contained in:
Leroy Hopson 2022-08-14 19:45:17 +12:00
parent 054c7c9ad4
commit 9ed6750b83
No known key found for this signature in database
GPG key ID: D2747312A6DB51AA
8 changed files with 80 additions and 40 deletions

View file

@ -68,13 +68,12 @@ var _exit_cb: FuncRef
# Writes data to the socket.
# data: The data to write.
func write(data) -> void:
assert(data is String or data is PoolByteArray)
if data is PoolByteArray:
data = data.get_string_from_utf8()
assert(
data is PoolByteArray or data is String,
"Invalid type for argument 'data'. Should be of type PoolByteArray or String"
)
if _pipe:
_pipe.write(data)
_pipe.write(data if data is PoolByteArray else data.to_utf8())
func resize(cols: int, rows: int) -> void: