Sincerity Is the Way of Heaven

To hold a low position and yet not win the confidence of one’s superiors, and still hope to govern the people, is impossible. There is a way to win the confidence of superiors: if one is not trusted by friends, one will not win the confidence of superiors. There is a way to be trusted by friends: if one does not please one’s parents, one will not be trusted by friends. There is a way to please one’s parents: if one is not sincere when examining oneself, one will not please one’s parents. There is a way to be sincere with oneself: if one does not understand what is good, one cannot be sincere with oneself. Therefore sincerity is the way of Heaven; to think about sincerity is the way of man. There has never been anyone completely sincere who did not move others; nor anyone insincere who could move others. — Mencius

At this year’s Code Beam, my talk is titled: Build a decentralized, public-verifiable DB. Over the past two years, blockchain has run hot and cold, hastening the ripening of many meteor-like buzzwords and producing quite a few concepts worth exploring in depth—for example, public verifiability. Public verifiability is about trust: how can we trust the data itself when we do not trust the identity of the party providing it? In the Internet age, trust is based on identity—or rather, on authority. We trust the data of friends on WeChat and search results on Google for no other reason than that the data comes from giants such as WeChat and Google. We “trust” these giants not because we truly believe they do no evil, but because we have no choice; from an economic perspective, we can only hope that the cost of doing evil is greater than the cost of not doing evil.

Take Code Beam tickets. We are willing to trust Eventbrite and buy tickets from it because 1) it is an authority in this domain, and 2) Code Beam sells tickets on its platform. For ticket buyers, we place our trust in the following: 1) Eventbrite will honestly store my purchase record for Code Beam to verify; 2) Eventbrite’s database is secure enough that data will not be corrupted, lost, or maliciously tampered with; 3) even if data is corrupted or lost, Eventbrite has sufficient backups to restore it to a reasonable state.

As engineers, we all know how fragile this trust is. For a database system, although we set up layer upon layer of access permissions and exhaust ourselves considering disaster recovery, the sad truth is that the foundation of all our operations is the assumption that the data itself is trustworthy. Therefore every security measure is about ensuring that a data accessor can only handle data for which they have been granted permission—for example, we allow user Tyr to add data to the asset table, but not modify or delete it. Then, as long as Tyr logs into the database, even if he scribbles arbitrarily into the asset table (as long as it conforms to the defined data types), the database accepts it wholesale.

Yes, in every database system we use—whether Postgres or Riak, Aurora or DynamoDB—as long as the user writing the data has write permission, they can write any true or false data.

In other words, the trustworthiness of data is not built on the data itself but on the safety of the environment in which the data lives. Because we trust the environment in which the data resides, we trust the data itself—what perfect logic! This reminds me of an episode of Mission: Impossible in which Tom Cruise jumps into a data center hidden beneath a dam, replaces a data card, and successfully alters Benji’s identity.

Is there a way to guarantee data integrity and immutability at the moment of input?

The answer is simple—digital signatures.

For convenience, let us assume there is a database Cyberbase that anyone can access—anyone can write data related to themselves into it without approval. The database accepts only correctly signed data and updates itself according to certain rules.

For example, I want to change my WeChat age to 18. In today’s systems, I tell WeChat: “Please change my age to 18,” and WeChat tells the database: update user set age = 18 where id = 'a008374856...'; In Cyberbase, I sign a fact with my private key: Tyr is 18 this year. For this fact, our rule for updating the database is: if the sender’s signature is correct, then execute sender.age = 18. Thus a fact about Tyr can be stated only by Tyr. If a third party—Alice—wants to change Tyr’s age, because she does not have Tyr’s private key and cannot produce a correct signature, she cannot forge a fact about Tyr.

Here we see the first important difference between a traditional database and the imagined Cyberbase: a traditional database does not care about the correctness and integrity of data (the application must guarantee that); as long as the client initiating the update is an authorized user, it blindly updates state. Cyberbase, by contrast, verifies the authenticity of incoming events at the source; only facts with valid signatures are conditionally applied.

Having facts as a foundation does not guarantee that data in Cyberbase has not been tampered with. Even if Tyr’s age is updated to 18 according to the fact Tyr provided, we cannot guarantee that it will not be tampered with to 81. Therefore we need some mechanism to ensure that any illegal update to Tyr’s age will be discovered. Note that here we do not try to ensure that illegal updates never happen—because there is no absolutely secure system or environment; if something can happen, it will happen. What we need to control is the impact after it happens.

This is not difficult. We can use a Merkle tree to repeatedly merge and hash scattered raw data, building a tree that ultimately aggregates into a root hash. In this tree, any change to the data changes its hash, and that change affects the hashes after merging; all the way up, the root hash must change. This poses a challenge to data tampering: under current mathematical models, no algorithm can alter the data itself without reflecting it layer by layer into the root hash. Therefore, as long as we store the root hash properly after each update, any tampering with the data will definitely be discovered. It is like all of Tyr’s data being aggregated into Tyr’s face: one look at that face and the lie about being 81 is exposed.

Ah, store the root hash properly. If a hacker has already changed Tyr’s age to 81, they can naturally modify the hashes layer by layer, and naturally they can also modify the root hash we stored properly. If the root hash is changed too, then forged data is also trusted by Cyberbase—which is unacceptable. So storing the root hash together in the same place as the data is a false solution.

Someone may say: just put Tyr’s face in a bank vault. Then to know whether Tyr’s age has been tampered with, we only need to go to the bank and open the vault. This has two problems: 1) such an operation is expensive and cannot keep up with the frequency of updates in a system; 2) how do we guarantee the security of the communication channel and verify that what comes back from the other side of the channel is truly the real data, not a counterfeit?

A bit stuck.

Let us change perspective. If storing it here is insecure and storing it there is untrustworthy, then why not proclaim it to the whole world, making the root hash generated by every data change known across the entire network? Then, as long as the hacker has not brainwashed the majority of everyone who knows this (more than one third), they cannot successfully tamper with the data.

This solves the problem of storing the root hash without worrying that data tampering will go undetected.

Let us summarize Cyberbase a little: by signing events that occur, each person independently generates a series of verifiable, trustworthy facts related to themselves. These facts are written into Cyberbase according to certain rules and become state in the database. The state in the database is organized with a Merkle tree, yielding a complete trust tree whose root is the root hash. By broadcasting the root hash across the network, we greatly raise the threshold for hackers and make data tampering so costly as to be infeasible.

So far we have not depended on any environmental security—whether physical security, network security, or operating system security. We depend on cryptographic algorithms rooted in mathematics and on predefined rules to guarantee end-to-end that the data is trustworthy.

One important link is still missing: who guarantees that the rules are executed correctly? Cyberbase may run on faulty disks, so that even if the rules are executed correctly, the data ultimately written to disk and the data read back are inconsistent; or a hacker may modify the code that describes the rules, so that trustworthy facts are committed to trustworthy state by untrustworthy code, ultimately yielding untrustworthy state.

So we need to let anyone run Cyberbase. Any node in the network running Cyberbase can continuously accept facts and independently execute them according to the predefined rules. As long as everyone’s initial values are the same (genesis data), and the facts received and their order are the same, then under the same rules they will arrive at the same final state—the same root hash. In this way, a single point of failure or tampering with the rules does not affect the big picture—for a hacker to successfully tamper with the data, they must tamper with enough nodes—and the larger the network and the more dispersed the nodes, the greater the cost of attack.

This is decentralization and public verifiability. If we build such a Cyberbase, we do not need to trust anyone; we only need to trust the cornerstone of everything in the universe—mathematics. On this basis, it constructs a beautiful equality: everyone is equal before data—we treat services provided by giants and services provided by individuals alike along the dimension of trust. If printing and the Renaissance broke the monopoly on knowledge held only by the upper classes (aristocrats, monks), making everyone equal before knowledge; and if the information revolution and the Internet broke the monopoly on creating and distributing information held by a few (TV stations, publishers, magazines and newspapers and other media), making everyone equal before information; then the blockchain technology behind Cyberbase has the chance to break the monopoly on trustworthy data held by a few (Internet giants such as Google, WeChat, and Facebook), making everyone equal before data—in other words, each individual’s competitive advantage will lie in analyzing and processing data and in grasping user experience, not in monopolizing data.