Published 2022-02-05.
What is BonsaiDb?
BonsaiDb is new database aiming to be the most developer-friendly Rust database. BonsaiDb has a unique feature set geared at solving many common data problems. We have a page dedicated to answering the question: What is BonsaiDb?.
It's been over six months since we announced PliantDb 0.1.0-dev.4. Beyond the name change, a lot has changed. We've been focusing our efforts on stabilizing BonsaiDb's core architecture so that we can start building applications with BonsaiDb. Today, we're excited to announce our first alpha release: version 0.1.0.
There is no upgrade path from 0.1.0-dev.4. Prior to this release, we warned against using this project outside of experimentation, so we hope this is not an issue for anyone. However, if this is devastating news to anyone, please reach out to us, and we can try to help.
The biggest barrier to adoption was our own making: our README actively discouraged early-adopters except out of experimentation. We held off releasing this first alpha until after we were confident in the foundation we've laid. We expect bugs will be found, and we are certain that at least one user will lose data. It's just a fact of life. The alpha phase enables adventurous users to begin using BonsaiDb to help us find these issues before we label BonsaiDb "stable."
We encourage users that attempt to use this in an environment where data loss would be detrimental to utilize and test the backup and restore functionality regularly.
We are still planning to add features during the alpha phase, but those features will focus on the higher-level database functionality. Changes to how data is stored will be minimal or non-existent unless bugs or performance issues are uncovered.
While the fundamental ideas powering BonsaiDb are still the same as previous releases, this release left almost no code untouched. The GitHub stats show 29,933 additions and 11,678 across 450 commits -- and those stats don't include the new crates introduced. The release on GitHub covers all of the changes, but we wanted to draw attention to a few notable entries.
Under the hood, this alpha unveils a new storage engine: Nebari. While the storage layer we were using before (Sled) is incredibly fast, it also is quite complex and didn't quite fit BonsaiDb's needs perfectly.
Nebari maintains a fairly straightforward design while still achieving fairly competitive benchmarks for being such a new library. Additionally, its implementation can be custom-tailored to the workloads we need to support BonsaiDb. Nebari performs quite well in our initial testing.
Part of our design from the beginning was to allow users to store arbitrary chunks of data as a document. In other words, users shouldn't be forced into a specific serialization or encoding format for their data. Unfortunately, a side effect of this was the API's usability suffered.
This alpha addresses the API shortcomings with functionality designed around Transmog. Transmog is a fairly simple concept: a universal set of traits to describe a basic serialization and deserialization interface. An example of Transmog in cation can be found in our new example: view-histogram.rs. It demonstrates storing a hdrhistogram::Histogram as a view's value by implementing the needed Transmog traits, allowing these types of fluid queries:
// Retrieve an `hdrhistogram::Histogram` with all samples recorded.
let total_histogram = db.view::<AsHistogram>().reduce().await?;
println!(
"99th Percentile overall: {} ({} samples)",
total_histogram.value_at_quantile(0.99),
total_histogram.len()
);
Using Transmog as a foundation, BonsaiDb provides streamlined APIs for interacting with serialized collections.
We have a vision that BonsaiDb will be a complete application development platform, including building web applications. Therefore, we have created a way to theoretically use any HTTP framework and mount the BonsaiDb WebSockets at a specific route.
We have a new example (axum.rs) which demonstrates how to accomplish this using the Axum framework.
Not all hosting environments offer encrypted filesystems, and almost all applications store some Personally Identifiable Information (PII). Therefore, it is a good practice (and often a legal requirement) to ensure that all sensitive information is stored encrypted on the disk.
BonsaiDb now optionally supports transparent at-rest encryption using externally stored keys.
New and updated crates in the Rust ecosystem power this release:
These are existing crates we've worked on as part of developing this release of BonsaiDb:
We removed OPAQUE support for now, but will be reintroducing it in the future. @daxpedda has been hard at work helping bring the ecosystem up to the current draft spec:
opaque-ke.Our new homepage has basic setup instructions and a list of examples. We have started writing a user's guide, and we have tried to write good documentation.
We would love to hear from you if you have questions or feedback. We have community Discourse forums and a Discord server, but also welcome anyone to open an issue with any questions or feedback.
We dream big with BonsaiDb, and we believe that it can simplify writing and deploying complex, data-driven applications in Rust. We would love additional contributors who have similar passions and ambitions.
Lastly, if you build something with one of our libraries, we would love to hear about it. Nothing makes us happier than hearing people are building things with our crates!