C++ Serialization & Reflection

Cista++ is a simple, open source (MIT license) C++17 compatible way of (de-)serializing C++ data structures.
Single header. No macros. No source code generation.

  • Raw performance - use your native structs.
    Supports modification/resizing of deserialized data!
  • Supports complex and cyclic data structures including cyclic references, recursive data structures, etc.
  • Save 50% memory: serialize directly to the filesystem if needed, no intermediate buffer required.
  • Fuzzing-checked though continuous fuzzing using LLVMs LibFuzzer.
  • Comes with a serializable high-performance hash map and hash set implementation based on Google's Swiss Table technique.
  • Reduce boilerplate code: automatic derivation of hash and equality functions.
  • Optional: built-in automatic data structure versioning through recursive type hashing.
  • Optional: check sum to prevent deserialization of corrupt data.
  • Compatible with Clang, GCC, and MSVC
compile & run
namespace data = cista::raw;
struct my_struct {  // Define your struct.
  int a_{0};
  struct inner {
      data::string b_;
  } j;
};

std::vector<unsigned char> buf;
{  // Serialize.
  my_struct obj{1, {data::string{"test"}}};
  buf = cista::serialize(obj);
}

// Deserialize.
auto deserialized = cista::deserialize<my_struct>(buf);
assert(deserialized->j.b_ == data::string{"test"});
compile & run
namespace data = cista::offset;
constexpr auto const MODE =  // opt. versioning + check sum
    cista::mode::WITH_VERSION | cista::mode::WITH_INTEGRITY;

struct pos { int x, y; };
using pos_map =  // Automatic deduction of hash & equality
    data::hash_map<data::vector<pos>,
                   data::hash_set<data::string>>;

{  // Serialize.
  auto positions =
      pos_map{{{{1, 2}, {3, 4}}, {"hello", "cista"}},
              {{{5, 6}, {7, 8}}, {"hello", "world"}}};
  cista::buf mmap{cista::mmap{"data"}};
  cista::serialize<MODE>(mmap, positions);
}

// Deserialize.
auto b = cista::mmap("data", cista::mmap::protection::READ);
auto positions = cista::deserialize<pos_map, MODE>(b);
Use Cases

Reader and writer should run on the same architecture (e.g. 64 bit little endian). Examples:

  • Asset loading for all kinds of applications (i.e. game assets, GIS data, large graphs, etc.)
  • Transferring data over network
  • Shared memory applications

Currently, only C++17 software can read/write data. It is possible to generate accessors for other programming languages, too.

Benchmarks

Have a look at the benchmark repository for more details.

Library Serialize Deserialize Fast Deserialize Traverse Deserialize & Traverse Size
Cap’n Proto 105 ms 0.002 ms 0.0 ms 356 ms 353 ms 50.5M
cereal 239 ms 197.000 ms - 125 ms 322 ms 37.8M
Cista++ offset 72 ms 0.053 ms 0.0 ms 132 ms 132 ms 25.3M
Cista++ raw 3555 ms 68.900 ms 21.5 ms 112 ms 133 ms 176.4M
Flatbuffers 2349 ms 15.400 ms 0.0 ms 136 ms 133 ms 378.0M

Alternatives

If you need to be compatible with other programming languages or require protocol evolution (downward compatibility) you should look for another solution. The following libraries provide some/all of those features.

Alternative libraries:

Human Readable to String

compile & run
struct a {
  CISTA_PRINTABLE(a)
  int i_ = 1;
  int j_ = 2;
  double d_ = 100.0;
  std::string s_ = "hello";
};

int main() {
  a instance;
  std::cout << instance;
  // "{1, 2, 100, hello}"
}

Modify Struct as Tuple

compile & run
struct a {
  int i_ = 1;
  int j_ = 2;
  double d_ = 100.0;
  std::string s_ = "hello";
};

int main() {
  using cista::to_tuple;
  using std::get;
  a i;
  get<0>(to_tuple(i)) = 5;
  get<1>(to_tuple(i)) = 7;
  get<2>(to_tuple(i)) = 2.0;
  get<3>(to_tuple(i)) = "yeah";
}

The Cista++ Serialization is based on a powerful reflection concept made possible by the C++17 structured bindings feature.

This page presents some other utilities enabled by this technique.

Credits: based on the idea presented here.
Comparable: Generate Operators

compile & run
struct a {
  CISTA_COMPARABLE()
  int i_ = 1;
  int j_ = 2;
  double d_ = 100.0;
  std::string s_ = "hello";
};

int main() {
  a inst1, inst2;

  CHECK(inst1 == inst2);

  inst1.j_ = 1;

  CHECK(!(inst1 == inst2));
  CHECK(inst1 != inst2);
  CHECK(inst1 <= inst2);
}

Iterate Each Field

compile & run
struct a {
  int i_ = 1;
  int j_ = 2;
  double d_ = 100.0;
  std::string s_ = "hello";
};

int main() {
  a i;
  cista::for_each_field(
    i, [](auto&& m) { m = {}; });
  CHECK(i.i_ == 0);
  CHECK(i.j_ == 0);
  CHECK(i.d_ == 0.0);
  CHECK(i.s_ == "");
}

Generate SQL Statements

compile & run
struct row {
  sql_col<
    int, name("id"),
    primary, not_null> user_id;
  sql_col<
    std::string, name("first"),
    not_null> first_name;
  sql_col<
    std::string, name("last"),
    not_null> last_name;
} r;

int main() {
  std::cout <<
      create_table_statement(r);
  // CREATE TABLE (
  //   id INT PRIMARY NOT NULL,
  //   first TEXT NOT NULL,
  //   last TEXT NOT NULL
  // );
}