Errow.Numeric (errow v0.1.0)

An immutable, continuous-memory numerical array implementing the Apache Arrow Primitive columnar format.

Under the hood, data is organized in two buffers:

Data buffer (data): a continuous binary of fixed-width numbers in little-endian.
Validity bitmap (validity): a bitstring tracking null (nil) values (1 bit per element).

Supported Types

Signed Integers: :s8, :s16, :s32, :s64, :s128, :s256
Unsigned Integers: :u8, :u16, :u32, :u64, :u128, :u256
Floating Point: :f32, :f64

Notice that :s128, :u128, :s256, and :u256 are natively supported in little-endian binary format, mapping directly to Arrow Decimal128, Decimal256, and FixedSizeBinary.

Summary

Types

numeric_type()

t()

Functions

append(arr, val)

Appends a number or nil to the array.

buffer_byte_size(numeric)

Returns the byte size of the continuous data buffer.

delete_at(arr, idx)

Deletes the element at index idx.

fetch(arr, idx)

Fetches the element at the given index or a slice specified by a range. Supports negative indices (e.g. -1 for the last element).

from_list(enumerable, type \\ :s64)

Builds an Errow.Numeric array from a list or enumerable of numbers and optional nils.

get(arr, idx, default \\ nil)

Gets the value at index idx or returns default if out of bounds.

max(arr, opts \\ [])

Finds the maximum value in the array.

mean(arr, opts \\ [])

Calculates the arithmetic mean (average) of the elements as a float.

min(arr, opts \\ [])

Finds the minimum value in the array.

min_max(arr, opts \\ [])

Returns a {min, max} tuple in a single pass over the array.

new(type \\ :s64)

Creates a new empty Errow.Numeric array of the given type (defaults to :s64).

null?(numeric, idx)

Checks if the value at the given index is null (nil).

null_count(numeric)

Returns the count of null values.

product(arr, opts \\ [])

Calculates the product of elements in the array. Options

replace_at(arr, idx, new_val)

Replaces the element at the given index with a new number or nil.

size(numeric)

Returns the number of elements in the array.

sum(arr, opts \\ [])

Calculates the sum of elements in the array. Options

to_list(arr)

Converts the Errow.Numeric array into a standard Elixir list.

Types

numeric_type()

@type numeric_type() ::
  :s8
  | :u8
  | :s16
  | :u16
  | :s32
  | :u32
  | :s64
  | :u64
  | :s128
  | :u128
  | :s256
  | :u256
  | :f32
  | :f64

t()

@type t() :: %Errow.Numeric{
  data: binary(),
  item_size: pos_integer(),
  length: non_neg_integer(),
  null_count: non_neg_integer(),
  type: numeric_type(),
  validity: bitstring()
}

Functions

append(arr, val)

@spec append(t(), number() | nil) :: t()

Appends a number or nil to the array.

buffer_byte_size(numeric)

@spec buffer_byte_size(t()) :: non_neg_integer()

Returns the byte size of the continuous data buffer.

delete_at(arr, idx)

@spec delete_at(t(), integer()) :: t()

Deletes the element at index idx.

fetch(arr, idx)

@spec fetch(t(), integer() | Range.t()) :: {:ok, number() | nil | list()} | :error

Fetches the element at the given index or a slice specified by a range. Supports negative indices (e.g. -1 for the last element).

from_list(enumerable, type \\ :s64)

@spec from_list(Enumerable.t(), numeric_type()) :: t()

Builds an Errow.Numeric array from a list or enumerable of numbers and optional nils.

get(arr, idx, default \\ nil)

@spec get(t(), integer(), any()) :: number() | nil | any()

Gets the value at index idx or returns default if out of bounds.

max(arr, opts \\ [])

@spec max(t(), keyword()) :: number() | nil

Finds the maximum value in the array.

mean(arr, opts \\ [])

@spec mean(t(), keyword()) :: float() | nil

Calculates the arithmetic mean (average) of the elements as a float.

min(arr, opts \\ [])

@spec min(t(), keyword()) :: number() | nil

Finds the minimum value in the array.

min_max(arr, opts \\ [])

@spec min_max(t(), keyword()) :: {number(), number()} | nil

Returns a {min, max} tuple in a single pass over the array.

new(type \\ :s64)

@spec new(numeric_type()) :: t()

Creates a new empty Errow.Numeric array of the given type (defaults to :s64).

null?(numeric, idx)

@spec null?(t(), integer()) :: boolean()

Checks if the value at the given index is null (nil).

null_count(numeric)

@spec null_count(t()) :: non_neg_integer()

Returns the count of null values.

product(arr, opts \\ [])

@spec product(t(), keyword()) :: number() | nil

Calculates the product of elements in the array. Options:

:skip_nulls - boolean, defaults to true.

replace_at(arr, idx, new_val)

@spec replace_at(t(), integer(), number() | nil) :: t()

Replaces the element at the given index with a new number or nil.

size(numeric)

@spec size(t()) :: non_neg_integer()

Returns the number of elements in the array.

sum(arr, opts \\ [])

@spec sum(t(), keyword()) :: number() | nil

Calculates the sum of elements in the array. Options:

:skip_nulls - boolean, defaults to true.

to_list(arr)

@spec to_list(t()) :: [number() | nil]

Converts the Errow.Numeric array into a standard Elixir list.