Foresight Ventures: The Present and Future of Web3 Decentralized Database Storage

The field of decentralized database has high attention value and urgent demand, but there is no widely accepted and used product yet.

Written by: Maggie

**1. Why do we need a decentralized database? **

There are two basic methods of data storage for Web2 applications, file system (File) and database (Database). Due to the lack of database products in Web3, most DApps still use centralized databases to store structured data in addition to storing a small amount of important data in expensive smart contracts. As decentralized file systems such as IPFS are gradually used to store NFT data of Web3 applications, decentralized file systems are recognized and accepted by Web3, and the technology of decentralized databases has also undergone a round of iterations, and a variety of new product.

Compared with traditional centralized databases, decentralized databases have unique advantages. It can reduce the risk of single point failure of Web3 projects and make Dapp completely decentralized.

Decentralized databases are suitable for storing hot data with high access frequency, and storing Dapp non-financial data, such as:

• NFT metadata
• DAO voting data
• DEX order book
• Decentralized social data, blog data, mail.
• Complex relational database data required by Dapp.

**2. What types of decentralized database storage systems are there? **

In the past two years, many decentralized database projects have emerged, and some innovative projects have received widespread attention.

• **Ceramic:**Ceramic is a project started in 2019. Data is stored and managed as streams, and formatted event logs are added to the streams. Logs will be documented and uploaded to IPFS. Provides GraphQL API queries. Ceramic does not have an incentive model like IPFS and supports data creation, reading and updating (CRU).
• **OrbitDB: **OrbitDB is an earlier project than Ceramic, and also uses the IPFS file system for file storage. It supports storage of NoSQL databases and files.
• Tableland: Launched in 2022, the project is currently in public beta. The production version of Tableland will be released in 2023. Data storage requires the use of smart contracts, which define SQL statements and set usage permissions. Reading data is done off-chain and does not require payment. Currently, the contract has been deployed on L2 such as ETH and OP.
• Polybase: The project is now live on the testnet. It is a NoSQL database that supports CRUD operations, and each operation requires payment. In addition, Polybase also supports various file systems to store database files, including local disk, IPFS, Filecoin, Polystore, and even AWS S3. Polybase also uses payment channels for data query payments, reducing the frequency of on-chain transactions and avoiding query delays caused by payments.
• Web3Q: The project will be launched in 2022, and the test network has already been launched. A new URL pattern Web//access protocol is proposed for accessing data. Its charging model is very special, and you can get a refund for deleting data.
• **Kwill: **Kwill is an Arweave-based SQL database system that uses smart contracts for payments.
• **KYVE: **KYVE is an Arwave-based database system.

Technically:

• Both SQL and NoSQL can be used as databases, SQL is more mature and efficient, NoSQL is richer and more flexible. The data structure of SQL needs to be highly consistent, with stronger joint query capability, mature and efficient; the KV form of NoSQL is more in line with the design pattern of Ethereum, can support rich data types, and is flexible and easy to expand.
• Functionally, supporting CRUD is the best, but supporting UD will bring complexity to the system. If the system uses local storage, historical value query may not be supported. If you use IPFS and Arweave, you need the database to be append-only, otherwise a piece of data has multiple versions, and the storage cost doubles.
• There are 2 options for the underlying file system.
• Files stored locally are more flexible, and the retrieval logic can be customized, which is more efficient, and avoids the unreliability and complexity caused by using decentralized file systems such as Arweave. For example: users use TokenA to pay database miners, and miners need to pay Arweave coin to store data. The superposition of two layers of networks brings complexity.
• Database files are stored in decentralized file systems such as IPFS and Arweave;
• Stored locally on the node or on the S3 cloud.
• Similar to decentralized storage, improving storage data retrieval speed, incentive models and token economics, and guarantee algorithms for ensuring data availability are key factors in determining whether a protocol will be widely used.
• A good incentive model and token model can not only mobilize the participation enthusiasm of nodes, but also motivate nodes to do the right thing. For example: provide effective retrieval functions, instead of only storing data to obtain storage rewards.
• The data availability guarantee algorithm will check the storage of data by nodes at intervals, requiring nodes to provide data availability certificates, which complement the incentives of nodes to prevent data loss.
• The retrieval of data affects the user experience, which is very important for the convenience and fluency of Dapp.

Summarize

• The field of decentralized databases has high attention value and urgent needs, but there are no widely accepted and used products yet.
• The maturity of decentralized database technology is lower than that of decentralized file storage system. Because the decentralized database technology is based on the distributed file system. Many projects are launched in 2022.
• Improving storage data retrieval speed, incentive model and token economics, and guarantee algorithms used to guarantee data availability are key factors in determining whether a protocol will be widely used. The focus of the protocol will be to reduce retrieval time, which is critical to the ease and fluency of Dapp usage.