Introducing: Symbiont Assembly™

I’m proud to announce the release of Symbiont Assembly™, the secure, reliable, and performant distributed ledger at the heart of Symbiont’s Smart Securities® Platform.

There are two main parts to Symbiont’s Smart Securities® platform: a smart contracts system purpose-built for institutional finance, and a distributed ledger, designed for permissioned networks. Today, we are releasing Symbiont Assembly™, our distributed ledger. Assembly plays a crucial role as the foundation of Symbiont’s platform, but can also be used independently.

As part of the release, we’re publishing the following documentation and open-source code for public consumption:

This allows anyone to begin developing blockchain applications that work with Assembly or any other ledger with a compatible API. Just as Symbiont’s smart contract system is ledger-agnostic, and able run on any distributed ledger, Assembly provides a foundation on which any permissioned blockchain application may be built. With Assembly handling consensus, fault tolerance, network performance, replicated data storage and proof-of-publication, application developers are free to focus on their own application, without having to deal with these challenging problems.

Why Assembly?

The Importance of Modularity

Modular, layered designs with simple APIs endure. Take the TCP/IP protocol: each layer has specific, well-defined responsibilities, and builds upon the layer below. TCP/IP has powered the Internet since its inception, while usage of the Internet has changed dramatically over the years.

Assembly focuses on being a distributed ledger—and being amazing at it—while leaving the other concerns of a Smart Contract platform to the higher layers of the system.

t would’ve been easy to run our smart contracts system on top of other ledger protocols, because we’ve put in the effort to design our system to be extremely modular. Indeed, we’ve proven that we can even run our smart contracts on Bitcoin if the occasion calls for it.

But none of the alternatives comes close to meeting our needs (or those of the rest of the industry). The requirements themselves are very simple:

  1. True Byzantine Fault-Tolerance
  2. Appropriate for use in permissioned networks
  3. Capable of handling thousands of transactions per second

Assembly is the only distributed ledger to fulfill all of these requirements.

It’s an additional advantage that Assembly was created as a complete, standalone system with a super simple API, rather than as a mere feature of a larger software monolith, whose bloat would have to be stripped or abstracted away for it to be useful in other settings.

A consequence of all the hype around blockchain technology is that much development is done without acknowledgement of prior art. By applying tried and true software development principles, combined with a modern microservice architecture, we are the first to bring a distributed ledger to the market that meets all the properties required for use in financial markets.

Byzantine Fault-Tolerance

As the foundation on which our smart contracts system is built, Assembly handles all communication with other nodes. Our system relies on it to securely and reliably store and publish transactions and to ensure that all nodes see these transactions in the same order.

To serve this function, Assembly has been architected as a distributed log. That means that Assembly is “append-only”; anything written to it will be added to what is already there; history is unchanging. It’s very similar to, and was in fact inspired in part by, Apache Kafka, which is a blazingly fast and scalable message broker in wide use today (able to scale to millions of transactions per second).

The main difference between Assembly and a traditional distributed log, like Apache Kafka, is that whereas Kafka is “crash fault-tolerant”, Assembly is fully Byzantine fault-tolerant. That means that it can tolerate nodes failing in arbitrary ways, not just crashing. This allows the system to stay operational in the presence of purposefully disruptive and malicious nodes, a critical requirement for a decentralized system where nodes are controlled by multiple, disparate organizations.

We implemented Assembly using a Byzantine Fault-Tolerant consensus protocol called BFT-SMaRt, which was designed from the ground up with simplicity and modularity in mind, an approach which fits well with our overall design philosophy. It also has improved performance and elegance over competing protocols.

The API

The design of Assembly’s API makes it extremely easy to integrate into any system for use as distributed ledger storage

It’s not enough to build a secure and performant protocol. It also needs to be easy for developers—both inside and outside of Symbiont—to work with. Assembly has a clean, minimalistic API, built using the same principles that make well-established non-BFT distributed systems so successful.

Assembly’s API is simple and easy to use, and belies the advanced technology that lies beneath it. The Assembly API is ReSTful, standard JSON over HTTP. So standard, in fact, that if you have a ledger node running on your machine and you go to http://localhost:4000/transactions/1 in your favourite web browser, you’ll be able to read transactions directly from the ledger:

{
  "first_index": 1,
  "last_index": 1,
  "transactions": [
    {
      "type": "symbiont/example",
      "tx_index": 1,
      "timestamp": 1461614515676834000,
      "data": "dHgxIGRhdGE=",
      "hash": "a6aea047a8040359d315419484b62be02c3e481d985315245ef75597f77fdbfb",
      "state_hash": "2985804be2e6b1bd4454774e94a3d69fe2f88d3e5399a6a0906c7202f83bc8d6"
    }
  ]
}

The design of Assembly’s API makes it extremely easy to integrate into any system for use as distributed ledger storage, recording anything from the movement of cash to complex legal documents and PDFs in a single location. Assembly can thus provide a single (but distributed and decentralized) depository of records for regulatory reporting, compliance and audit.

You can find the full specification of the API here. To assist further, we’re open-sourcing the API with simple server code equivalent to a single, network-less node of our ledger, as well as some example applications using it. If your system works with this API, then it will also work with Assembly.

Performance

Many attempts at building distributed ledgers are currently suffering from performance and scalability that won’t suffice for production deployments in financial markets.

As it stands, Assembly is able to achieve performance orders of magnitude greater than its competitors. For four nodes in a single region, we consistently process 80,000 transactions per second.

1—Symbiont.io, Assembly Performance Tests, September 2016. Testing was performed with null data payloads on a four-node network of c4.8xlarge machines in the Frankfurt region of AWS.

2—Based on testing conducted in August 2014.

3—Theoretical maximum for a private network. Buterin, Vitalik. “Ethereum Platform Review: Opportunities and Challenges for Private and Consortium Blockchains.” r3cev.com, 2 June 2016, p. 21

4—Testing reported October 11, 2016, with four nodes, each transaction including a simple ChainCode Invoke call.

Assembly’s performance is thus in line with what the academic implementations of BFT-SmaRt and similar consensus protocols demonstrate:

A consequence of Assembly’s excellent performance is that arbitrary data (such as scanned legal documents supplementing smart contracts) can be stored directly on-ledger without the bloat or slow transaction processing that troubles less performant distributed ledgers. That means that no centrally controlled, third-party data store is required to use our system. Users of Assembly are free to make the decision that’s right for their overall product when deciding whether any given data should be stored on- or off-ledger. This capability has been enormously helpful for Symbiont’s smart contracts platform in gaining traction in traditional financial markets.

Security and Privacy

The unique threat to decentralized trust-less networks is that one or more nodes is malicious or compromised, potentially colluding against the honest nodes. This is what we have earlier described as a Byzantine Fault.

Our defense against this category of failures is our BFT protocol, which allows for honest nodes to continue to communicate securely and reach consensus on what’s being written to the ledger despite the presence of such faults. Does this affect our performance? Yes, of course. BFT is the major reason we don’t see the millions of transactions per second that Apache Kafka can manage, but our security model is much stricter than Kafka’s, and security is our highest priority.

Of course we don’t expect our customers to store confidential data unencrypted on Assembly. In the spirit of a layered system, where each layer has limited responsibilities to allow it to excel at what it does, our system handles privacy in a layer on top of Assembly. In that layer we use encryption to create a private ledger within the ledger. This results in far superior data security relative to competing hub-and-spoke and trusted third-party systems, without sacrificing the strong security of Assembly’s consensus protocol. This solution is already available to users of our smart contract system, and the details will be shared in an upcoming blog post.

Conclusion

In the coming months you’ll hear a lot about Symbiont’s Smart Securities® platform and everything it can do. Now you know that its foundation is a distributed ledger, the security, reliability and performance of which are all best-in-class. We call it Assembly, and it offers a simple API that makes it extremely easy to build distributed ledger applications. Contact us to get started building your next product with Assembly!


Also published on Medium.