Errow (errow v0.1.0)

Copy Markdown View Source

Errow - Zero-copy, Apache Arrow-compliant packed arrays for Elixir and the BEAM.

Errow provides columnar in-memory arrays backed by continuous BEAM binaries, fully integrated with standard Elixir protocols (Enumerable, Collectable, Access), and 100% byte-for-byte compliant with the Apache Arrow Columnar Specification and Arrow C Data Interface.

Modules

  • Errow.String: UTF-8 variable-length string arrays (Arrow Utf8 / VARCHAR).
  • Errow.Binary: Opaque raw byte blob arrays (Arrow Binary / BLOB).
  • Errow.Numeric: Fixed-width primitive arrays (:s8..:s256, :u8..:u256, :f32, :f64).
  • Errow.CData: Exporters for Arrow C Data Interface (ArrowArray / ArrowSchema metadata and buffers).

Quick Examples

Strings & Comprehensions (Collectable + Enumerable)

iex> arr = for name <- ["Elixir", "Arrow", "BEAM"], into: Errow.string() do
...>   String.upcase(name)
...> end
iex> arr[0]
"ELIXIR"
iex> arr[-1]
"BEAM"
iex> arr[0..1]
["ELIXIR", "ARROW"]

Numerics with 128-bit / 256-bit BigInt Support

iex> num = Errow.numeric([10, 20, 30], :s32)
iex> Enum.sum(num)
60
iex> put_in(num[1], 99)[1]
99

Zero-Copy Arrow Export

iex> arr = Errow.string(["Elixir", "Arrow"])
iex> schema = Errow.CData.export_schema(arr)
iex> schema.format
"u"
iex> %{buffers: [_validity, _offsets, _data]} = Errow.CData.export_buffers(arr)

Summary

Functions

binary()

Creates a new empty Errow.Binary array.

binary(enumerable)

Builds an Errow.Binary array from an enumerable.

f32()

Creates a 32-bit float array (:f32).

f32(enum)
f64()

Creates a 64-bit float array (:f64).

f64(enum)
join(arr, separator \\ "", opts \\ [])

Joins string elements with the given separator.

max(arr)
max(arr, opts)

Returns the maximum value in a numeric or string array.

mean(arr, opts \\ [])

Calculates the arithmetic mean of elements in a numeric array.

min(arr)
min(arr, opts)

Returns the minimum value in a numeric or string array.

min_max(arr, opts \\ [])

Returns {min, max} in a single pass.

numeric(type \\ :s64)

Creates a new empty Errow.Numeric array of the given type (default :s64).

numeric(enumerable, type)

Builds an Errow.Numeric array from an enumerable of numbers and optional nils.

numeric_8()

Creates an 8-bit integer array (:s8 by default, or :u8). Usable directly in comprehensions: into: Errow.numeric_8 or into: Errow.numeric_8()

numeric_8(type)
numeric_8(enum, type)
numeric_16()
numeric_16(type)
numeric_16(enum, type)
numeric_32()
numeric_32(type)
numeric_32(enum, type)
numeric_64()
numeric_64(type)
numeric_64(enum, type)
numeric_128()
numeric_128(type)
numeric_128(enum, type)
numeric_256()
numeric_256(type)
numeric_256(enum, type)
numeric_f32()
numeric_f32(enum)
numeric_f64()
numeric_f64(enum)
numeric_u8()
numeric_u8(enum)
numeric_u16()
numeric_u16(enum)
numeric_u32()
numeric_u32(enum)
numeric_u64()
numeric_u64(enum)
numeric_u128()
numeric_u128(enum)
numeric_u256()
numeric_u256(enum)
product(arr, opts \\ [])

Calculates the product of elements in a numeric array.

s8()

Creates an empty signed 8-bit integer array (:s8). Usable directly in comprehensions: into: Errow.s8 or into: Errow.s8().

s8(enum)
s16()

Creates a signed 16-bit integer array (:s16).

s16(enum)
s32()

Creates a signed 32-bit integer array (:s32).

s32(enum)
s64()

Creates a signed 64-bit integer array (:s64).

s64(enum)
s128()

Creates a signed 128-bit integer array (:s128).

s128(enum)
s256()

Creates a signed 256-bit integer array (:s256).

s256(enum)
string()

Creates a new empty Errow.String array.

string(enumerable)

Builds an Errow.String array from an enumerable.

sum(arr, opts \\ [])

Calculates the sum of elements in a numeric array.

u8()

Creates an empty unsigned 8-bit integer array (:u8). Usable directly in comprehensions: into: Errow.u8 or into: Errow.u8().

u8(enum)
u16()

Creates an unsigned 16-bit integer array (:u16).

u16(enum)
u32()

Creates an unsigned 32-bit integer array (:u32).

u32(enum)
u64()

Creates an unsigned 64-bit integer array (:u64).

u64(enum)
u128()

Creates an unsigned 128-bit integer array (:u128).

u128(enum)
u256()

Creates an unsigned 256-bit integer array (:u256).

u256(enum)

Functions

binary()

Creates a new empty Errow.Binary array.

binary(enumerable)

Builds an Errow.Binary array from an enumerable.

f32()

Creates a 32-bit float array (:f32).

f32(enum)

f64()

Creates a 64-bit float array (:f64).

f64(enum)

join(arr, separator \\ "", opts \\ [])

Joins string elements with the given separator.

max(arr)

max(arr, opts)

Returns the maximum value in a numeric or string array.

mean(arr, opts \\ [])

Calculates the arithmetic mean of elements in a numeric array.

min(arr)

min(arr, opts)

Returns the minimum value in a numeric or string array.

min_max(arr, opts \\ [])

Returns {min, max} in a single pass.

numeric(type \\ :s64)

Creates a new empty Errow.Numeric array of the given type (default :s64).

numeric(enumerable, type)

Builds an Errow.Numeric array from an enumerable of numbers and optional nils.

numeric_8()

Creates an 8-bit integer array (:s8 by default, or :u8). Usable directly in comprehensions: into: Errow.numeric_8 or into: Errow.numeric_8()

numeric_8(type)

numeric_8(enum, type)

numeric_16()

numeric_16(type)

numeric_16(enum, type)

numeric_32()

numeric_32(type)

numeric_32(enum, type)

numeric_64()

numeric_64(type)

numeric_64(enum, type)

numeric_128()

numeric_128(type)

numeric_128(enum, type)

numeric_256()

numeric_256(type)

numeric_256(enum, type)

numeric_f32()

numeric_f32(enum)

numeric_f64()

numeric_f64(enum)

numeric_u8()

numeric_u8(enum)

numeric_u16()

numeric_u16(enum)

numeric_u32()

numeric_u32(enum)

numeric_u64()

numeric_u64(enum)

numeric_u128()

numeric_u128(enum)

numeric_u256()

numeric_u256(enum)

product(arr, opts \\ [])

Calculates the product of elements in a numeric array.

s8()

Creates an empty signed 8-bit integer array (:s8). Usable directly in comprehensions: into: Errow.s8 or into: Errow.s8().

s8(enum)

s16()

Creates a signed 16-bit integer array (:s16).

s16(enum)

s32()

Creates a signed 32-bit integer array (:s32).

s32(enum)

s64()

Creates a signed 64-bit integer array (:s64).

s64(enum)

s128()

Creates a signed 128-bit integer array (:s128).

s128(enum)

s256()

Creates a signed 256-bit integer array (:s256).

s256(enum)

string()

Creates a new empty Errow.String array.

string(enumerable)

Builds an Errow.String array from an enumerable.

sum(arr, opts \\ [])

Calculates the sum of elements in a numeric array.

u8()

Creates an empty unsigned 8-bit integer array (:u8). Usable directly in comprehensions: into: Errow.u8 or into: Errow.u8().

u8(enum)

u16()

Creates an unsigned 16-bit integer array (:u16).

u16(enum)

u32()

Creates an unsigned 32-bit integer array (:u32).

u32(enum)

u64()

Creates an unsigned 64-bit integer array (:u64).

u64(enum)

u128()

Creates an unsigned 128-bit integer array (:u128).

u128(enum)

u256()

Creates an unsigned 256-bit integer array (:u256).

u256(enum)