Skip to content

stards_translate

stards_translate converts between .stards files and other formats (JSON, MessagePack, CSV), and can reorganize a file for ISDS (ISIS Dataset) optimization.

Built when STARDS_BUILD_TRANSLATE=ON (the default); see Installation.

Usage

stards_translate [OPTIONS] <input_file> <output_file>

# STAR ↔ JSON (format auto-detected from the file extensions)
stards_translate data.stards data.json
stards_translate data.json data.stards

# CSV to STAR (2D arrays only)
stards_translate data.csv data.stards

# Force the output format explicitly
stards_translate -f json data.stards out.json

# MessagePack (if built with msgpack support)
stards_translate data.stards data.msgpack

The output format is auto-detected from the output file's extension (.stards, .json, .csv, .msgpack/.mp), or you can force it with -f.

Options

Flag Long form Description
-h --help Show the help message
-f <fmt> --format <fmt> Output format: json, msgpack, or isds (otherwise auto-detected from the extension)
-c <alg> --compression <alg> Compression for STAR output: none, gzip, lz4, gzip-shuffle, lz4-shuffle. Default: lz4.
-b <bytes> --block-size <bytes> Block size for STAR compression. Default: 1048576 (1 MB).
-t <n> --threshold <n> ISDS element threshold (default: 100)

The *-shuffle codecs add a byte-shuffle prefilter that greatly improves compression of numeric arrays (e.g. float64); shuffled arrays are read whole rather than sliced.

JSON format

{
  "arrays": {
    "my_array": {
      "dtype": "float64",
      "shape": [10, 20],
      "data": [0.0, 1.0, "..."]
    }
  }
}

ISDS optimization

The ISDS (ISIS Dataset) mode reorganizes a .stards file by size: large arrays go to efficient block storage, while small values are kept in the metadata block for quick access. This suits ISIS camera-state files that mix large arrays (quaternions, ephemeris) with small scalars (focal length, detector center).

  • Arrays with more elements than the threshold → array storage (compressed blocks)
  • Arrays with the threshold count or fewer → metadata storage
  • Default threshold: 100 elements
  • All data is preserved — only its storage location changes
# Reorganize by array size (default threshold of 100 elements)
stards_translate -f isds input.stards output.stards

# Custom threshold
stards_translate -f isds -t 50 input.stards output.stards

Batch conversion

# STAR → JSON for every file in a directory
for file in *.stards; do
    stards_translate "$file" "${file%.stards}.json"
done

# JSON → STAR
for file in *.json; do
    stards_translate "$file" "${file%.json}.stards"
done

See also