std/string/encode

Standard Library documentation

Character-encoding conversions between Strings and BinaryStrings.

Back to Stdlib View Source

Module

Name
std/string/encode
Area
Standard Library
Source
modules/std/string/encode.zzm

NAME

std/string/encode - Character-encoding conversions between Strings and BinaryStrings.

SYNOPSIS

  from std/string/encode import *;

  let bytes := encode( "héllo", ENCODING_UTF16 );
  let text := decode( bytes, "UTF-16" );

IMPLEMENTATION SUPPORT

This module is supported by all implementations of ZuzuScript.

DESCRIPTION

This module converts between text (String) and encoded bytes (BinaryString).

All implementations support UTF-8, UTF-16, UTF-32, and ISO-8859-1 (Latin-1). Implementations are encouraged to support additional encodings where the host platform makes that practical; programs that need to run on every implementation should restrict themselves to the four required encodings.

Encoding names are matched case-insensitively, so "utf-8" and "UTF-8" are equivalent.

For UTF-16 and UTF-32, encode produces the canonical form: big-endian with no byte order mark. This is deterministic and identical across implementations. decode honours a leading byte order mark (consuming it and switching to little-endian where it says so) and otherwise assumes big-endian input.

Invalid input raises an exception: unknown encoding names, bytes that do not form valid text in the requested encoding, and characters that the target encoding cannot represent (for example, encoding "😀" as ISO-8859-1) all throw.

EXPORTS

Functions

  • encode(String text, String encoding)

    Parameters: text is the text to encode; encoding names the target encoding and defaults to "UTF-8". Returns: BinaryString. Encodes text as bytes. Throws a TypeException if text is not a String, and an exception if the encoding is unknown or cannot represent a character in text.

  • decode(BinaryString bytes, String encoding)

    Parameters: bytes is the encoded input; encoding names the source encoding and defaults to "UTF-8". Returns: String. Decodes bytes into text. Throws a TypeException if bytes is not a BinaryString, and an exception if the encoding is unknown or the bytes are not valid for it.

Constants

  • ENCODING_UTF8

    Type: String. The value "UTF-8".

  • ENCODING_UTF16

    Type: String. The value "UTF-16".

  • ENCODING_UTF32

    Type: String. The value "UTF-32".

  • ENCODING_LATIN

    Type: String. The value "ISO-8859-1".

COPYRIGHT AND LICENCE

std/string/encode is copyright Toby Inkster.

It is free software; you may redistribute it and/or modify it under the terms of either the Artistic License 1.0 or the GNU General Public License version 2.