Somnia 백서 전문

| | Arbitrum | 944 | 0.25s | 16m | | Ethereum | 62.34 | 12.08 | 16m | | Optimism | 67.41 | 2s | 16m | | Avalanche | 92.74 | 2.03s | 0s | More recent benchmarks look as gas per second. <figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXd4aIfcAxT5HfbxKL_DV2lm3p87K61jbnnU3WY1H47J0h1yyj8aUKiS-zDmk78hR-Z9KuqNAT4ziqTUa9Dybf3LXhTqVBl-J7pF_TPzWhBMGcEGUhoU86S1gsHWEatmoJemJCQjiJAamRlmcgo865Ubwruv?key=Pd6FhcOWiUEbzWyN_-YJmg" alt=""><figcaption></figcaption></figure> These numbers are taken from [this article](https://www.paradigm.xyz/2024/04/reth-perf). To translate this into DPS, it’s about 6.5 DPS for every 1mg/s. So even the fastest chain on this list has a theoretical limit of 650 DPS (about 3700 TPS). If you go outside of the EVM ecosystem, you can do a lot better. Note that this TPS is not the same as above, as the tests differ. | | TPS | Finality | | ----------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------- | | Aptos | 30,000 ([cite](https://aptoslabs.medium.com/previewnet-ensuring-scalability-and-reliability-of-the-aptos-network-48f0d210e8fe)) | 0.9s ([cite](https://twitter.com/Aptos/status/1632801717937922052)) | | SUI | 11,000 – 297,000 ([cite](https://blog.sui.io/sui-performance-update/)) | 480m/s ([cite](https://blog.sui.io/sui-performance-update/)) | | Solana | 1608 ([cite](https://chainspect.app/dashboard)) | 12.8s ([cite](https://chainspect.app/dashboard)) | Aptos and SUI's real-world live tests have not hit these levels yet. These numbers are based on their benchmarks. The Solana numbers differ from those on the Solana webpage as we are using the real-world observed numbers. Using the same methodology as the article above, Solana’s numbers are significantly worse at 273.34 swaps per second. Despite this performance increase, we believe the EVM is important. There is already a rich ecosystem of developers and content around the EVM. Not being able to access that ecosystem seems like a missed opportunity. These chains had to make that tradeoff to create the needed performance. But what if we could have similar performance on the EVM? The bottlenecks we see today with approaches on blockchain implemented with the EVM today are: * **Execution speed** – The rate at which smart contract code can be executed, and block creation can occur. * **Storage** — Retrieval of storing historical data of a chain. Ethereum [EIP-4844](https://www.eip4844.com/) was a recent upgrade that significantly improved the cost of storage for ETH and other L2s. However, we still need improvements in reading and writing data to blockchains. * **Bandwidth** — The amount of bandwidth needed to send data between nodes on the network when running at high transaction levels.\ # Overview The Somnia blockchain has many innovations that enable it to increase performance by several orders of magnitude compared to other EVM chains: * [MultiStream consensus](/concepts/somnia-blockchain/multistream-consensus) - a proof-of-stake, partially synchronous BFT protocol inspired by [Autobahn BFT](https://arxiv.org/pdf/2401.10369). * Independent Data Chains - Each validator operates its own blockchain, or “data chain,” which allows for independent block production. This unique approach eliminates the need for a consensus mechanism within individual data chains, streamlining the data processing workflow. * Consensus Chain - A separate blockchain aggregates the heads of all data chains, employing a modified PBFT algorithm for proof of stake consensus. This structure decouples data production from the consensus process, significantly enhancing overall efficiency. * [Compiled Bytecode](/concepts/somnia-blockchain/accelerated-sequential-execution) - By translating EVM bytecode to highly optimised native code, Somnia achieves execution speeds close to hand-written C++ contracts, facilitating the execution of millions of transactions per second on a single core. * [Faster and predictable database performance](/concepts/somnia-blockchain/somnias-icedb) - Somnia has a custom database called IceDB. It employs performance reports for predictable read and write performance as well as a custom database architecture that enables average read/write operations 15-100 nanoseconds with built in snapshotting. * [Advanced Compression Techniques](/concepts/somnia-blockchain/advanced-compression-techniques) - The Somnia data chain architecture is designed to enable streaming compression in order to maximise data throughput. Somnia combines this with [BLS signature](https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signature-05.html) aggregation in order to achieve extremely high compression ratios, allowing for massive transaction data throughput. This allows theoretical performance above other preported [“limits due to bandwidth”](https://revelointel.com/lightspeed-how-monad-is-superscaling-the-evm-with-keone-and-kevin-g/). # MultiStream Consensus <figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXe7Wf4hn7ozZg4s3oMjgJp-CTVgxPCrlCwcnvYhOX9o4yn89hhsPOBrGFqrTj7IN8jtcknxsqDmvFnkylytq9UfICLGSU5ZH1DPZyL9WcX0HDFtUUXGtrvd4N39NKUCTgLR2zBh_Vda4N_nUBF9fqC2cTzL?key=kDSKjjghsV5HdVIUV1RbMw" alt=""><figcaption>Image showing conesus algorithm</figcaption></figure> In Somnia, every validator publishes their own blockchain, their data chain. This innovation was inspired by the 2024 whitepaper “[Autobahn: Seamless high speed BFT](https://arxiv.org/pdf/2401.10369)”. Every data chain is completely independent, and each block in a data chain contains a blob of bytes. Only the owning validator ever adds blocks to their data chain, and there are no safety mechanisms in place to avoid them forking their data chain or proposing invalid blocks. In other words, the data chains have no consensus mechanism at all. Somnia then includes a **consensus chain**, with each consensus block including the current head of every data chain. This chain uses a modified PBFT consensus algorithm and is a typical proof of stake consensus setup. The consensus chain, including the tip of each data chain, provides full security against validators forking their own data chain. Each consensus block then semantically includes all of the transactions in all of the data chains whose tip it advanced. A deterministic pseudorandom ordering of these data chains then provides a single globally ordered stream of bytes to be executed across all data chains. This setup completely decouples the production and distribution of new data (advancements of the data chains) with the consensus algorithm. It has a number of crucial benefits described below in the Advanced Compression Techniques section (e.g. streaming compression), which let this data frontend reach almost a gigabit per second of published transaction data. \ # Accelerated Sequential Execution ## **The Problem With Parallel Execution** A large number of modern blockchains have attempted to scale their execution using parallelism. This means they execute transactions which are unrelated, on different cores. This can work well when the transactions in a block are unrelated, but breaks down when those transactions modify the same state. The reality, however, is that load spikes mostly happen when there is some event which has caused the spike. This implies a massive correlation between the transactions, which is not normally there in day to day execution. For example: One of Ethereum's biggest load spikes was the Otherside Otherdeed mint. When this happened, the vast majority of all transactions in each block were all modifying the same state ([as they were minting Otherdeeds](https://decrypt.co/99219/otherside-nft-mint-burned-more-157m-ethereum)). Parallel execution would not have worked here. A DEX will contain trading on many unrelated asset pairs throughout each day, but their real load spikes will come from volatility on a specific asset pair. These trades will all modify the same state, meaning parallel execution would not have helped. <figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXdrbW10aNo4JpAuSDrJdce6xQInL1bEASAzpBr-ZOtxCVxjJyEWKZHJPciH1mpjbq5t5yBMD9tTUO_1pUlEUQQvRhhzdiWjO2Oxwhe8zJJT36uz2Q8uQaqLRg99kCafYu25X1qcpAgFUyVDpygGIIRsahNV?key=kDSKjjghsV5HdVIUV1RbMw" alt=""><figcaption>Parallel works well for many individual apps or accounts, swapping tokens</figcaption></figure> <figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXcfVU2BkXvoO29xCqKfv4eZ4u0NrpRs4xM0aAdbco6EtwuQFgUtyUdnNA4WJ_5cY2hYogEzejCNJmq4lLTO09vDOh4rL5vwkuIWguVCBycnbwOCC9au0l7NMAae0FE6-5ki5c_tj2C0uqFAmIJPm5l9bML4?key=kDSKjjghsV5HdVIUV1RbMw" alt=""><figcaption>When you have a hot path with many threads touching the same state parallel breaks down</figcaption></figure> In other words, **parallel execution breaks down exactly when you need it.** This observation is why Somnia has opted to make a single core go extremely fast **instead of relying on parallel execution.** <figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXewGQiGKlqSgXIyXd4c7zX2F_2_PQDqjF6yF9mlPM3fp0NYLhidvVutP-qK7zgoielDph3jClhmbwy8cHk8Cus3TX555vzowXs-gQqbXSu868WWL29qyHWBZmyQTcBD_90SYZ8KCaDzXuL0_91hXwzwWSgp?key=kDSKjjghsV5HdVIUV1RbMw" alt=""><figcaption></figcaption></figure> So, how does Somnia make a single core go this fast? ## **EVM Compilation** Somnia executes EVM transactions. It scales by reaching a very high single-core speed instead of attempting to parallelise over multiple cores. The EVM is a relatively simple stack-based architecture. Due to the implicit stack, stack-based bytecodes are often naturally smaller in size than register-based bytecodes, but they contain many redundant operations. There are broadly two ways to execute a VM's bytecode, interpreted or native (which itself can be split into JIT or AOT compilation). The former effectively simulates the VM itself in software, while the latter translates the bytecode to native instructions, which your CPU then executes directly. Ethereum and most EVM blockchains run an interpreted VM to execute EVM bytecode. This is often a relatively naive implementation which keeps its own stack, loops through each operation, and looks up the functionality of each operation in a lookup table. This is very slow compared to native execution. Further, Solidity and other Ethereum compilers often optimise for bytecode size over gas. For example, they will include code to generate large constants instead of including them inline. This ends up including a large amount of runtime computation that can actually be statically resolved at compile time. To exploit these redundancies, Somnia includes its own EVM compiler, which translates EVM bytecode to x86. This approaches near-native speed (native speed being the equivalent functionality handwritten in C++). In benchmarks, this can execute ERC-20 transfers in hundreds of nanoseconds, achieving millions of TPS on a single core. However, there is no free lunch as the compilation process is relatively expensive. For this reason, you would only do this on contracts that are called frequently, falling back to standard interpreted EVM on the rest. ## **Hardware Level Parallelism** As mentioned above, "parallel execution", attempting to run unrelated transactions on different cores, breaks down when you need it the most because it cannot handle correlated transactions, which are much more common during load spikes. This type of parallelism is also known as "software parallelism". For this reason, Somnia does not attempt to parallelise unrelated transactions. It does, however, enable the CPU to hardware parallelise each individual transaction. This harnesses the single core parallelism available in modern CPUs to speed up every individual transaction, meaning this technique also works for transactions which modify the same state, helping during the load spikes as well. So, what is hardware-level parallelism? <figure><img src="https://content.gitbook.com/content/hEUf5l6MCgn0iJsSVgmd/blobs/k6AbvfKfTZrEImgWrqhS/image.png" alt=""><figcaption>Diagram showing the difference between sequential execution (left) and hardware parralised execution (right)</figcaption></figure> Modern CPU cores give the appearance that they execute assembly instructions in order, implementing control flow primitives by jumping around the assembly. The reality is that they are actually executing these instructions completely out of order, and often in parallel with each other. For example, when reading a value from memory, the CPU will run ahead and work on future computation while waiting for the result. This is all completely invisible to the developer. This can cause massive speedups. Consider the case of an ERC-20 token swap. The program, at a high level, completes the following steps in the first half of it's execution: 1\. Hash the sender account. 2\. Lookup the sender balance using this hash. 3\. Hash the receiver account. 4\. Lookup the receiver balance using this hash. 5\. ... Let's say the speed to hash an account is 150ns, and the speed to lookup the sender balance is 100ns (a single RAM read). If the CPU ran these steps in serial, this would take 150 + 100 + 150 + 100 = 500ns. However, in practice, if these steps are compiled to native code, your CPU hardware would execute 1 and 2 completely in parallel to 3 and 4, causing the execution speed to be 250ns, double the speed. The problem is, interpreting the EVM stops your CPU core from executing assembly in parallel. To really harness the available silicon in your CPU, you need to execute native assembly. Somnia's compiler and database are built to enable hardware-level parallelism of smart contracts. \ \ \ # Somnia's IceDB Somnia also includes its own database called IceDB. It has three key properties: 1\. Deterministic performance 2\. In-memory cache with read promotions 3\. Built-in snapshotting ## **Deterministic Performance** Most blockchains use an embedded database like [LevelsDB](https://github.com/google/leveldb) or [RocksDB](https://rocksdb.org/). These databases are often optimised for write throughput and eventually flush all written data to disk to maintain a steady RAM load. This means that reads from these databases can hit RAM or many different places on disk. Depending on where the value is stored, the latency difference is enormous, differing by up to 1000x. So, how much gas should a user be charged for a read? Should we assume the worst-case scenario that every read hits disk multiple times? Or should we assume some amount will hit memory? The former drastically limits the speed of your blockchain, and the latter allows an attacker to critically slow down your chain. Could we time the read on each node and charge the user based on how long the request took? Unfortunately not, as this would not be deterministic across nodes (LevelsDB, RocksDB, etc., make no attempt to store data in deterministic places). Therefore, we can not change the gas based on this information. Somnia's IceDb has been built from the ground up to have fully deterministic performance. Every read and write to IceDb returns the result and a "performance report". This report details exactly how many cold cache lines were read from RAM, and exactly how many disk pages were read from the SSD. Due to this information being deterministic, we can charge the user based on the actual load they put on the system. Reads that accessed frequent data (and therefore hit RAM) use less gas, and we can then fit more into the block. ## **Improved read/write cache** As mentioned above, databases that persist to the disk often have an in-memory cache which attempts to store frequently accessed data in RAM, to serve frequent data with a much lower latency. Most databases tend to optimise for either read or write. With Somnia, we have created a cache that can do both. This enables the average read/writes of IceDB to be between 15-100 nanoseconds. ## **Built-in Snapshotting** Blockchains require all nodes to periodically agree on exactly which state is in the blockchain after a given block. Most blockchains use a data structure such as a Merkle tree to retrieve a single-state hash that encapsulates all of the data in the blockchain. Merkle trees allow this hash to be updated with every write with reasonable performance while allowing proofs of a particular piece of state to be generated for parties without access to the database. Most blockchains then store each node in this Merkle tree as standard keys into their database. This makes reads or writes from this tree very expensive, as every single node can result in massive latencies from these embedded databases. The reality, however, is the data structure that these databases use under the hood (a log structured merge tree) already lends itself to a tree formation, with vast sections stored immutably on disk. IceDB utilises this underlying immutable structure to support first class state snapshots, without needing the user space execution engine to store a Merkle tree using the databases' key value abstraction. This massively accelerates the performance of IceDB, and reduces the amount of overhead for each value. # Advanced Compression Techniques When you have a lot of transactions, a lot of data needs to go between nodes <figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXc-j1fYvX9rhOUfFhGdomgdhKQyv_X0sWX8hBeOQIWo3HNMGEqBQF6B0TkR33gRYdLvSpsd5iJxeNGazVXtnCW_BQuzlPud6Gf8tgM_UZTU-Cldil3P0G89Ed0rtRj1xm0PV0gUXSAlmOyLvboAziklpmqz?key=kDSKjjghsV5HdVIUV1RbMw" alt=""><figcaption></figcaption></figure> Once you are getting to the world of 100,000’s or even millions of transactions per second you start creating a lot of data. A standard ERC-20 transfer is about 200 bytes when you consider all its parts. Now, imagine we are doing 1 million ERC-20 swaps per second. That’s 190 MBytes/s or 1.5 GBits/s. It's not going to pass over the public Internet, so you are either going to have a very centralised chain or limit your transaction rate. ## **Power law distribution** <figure><img src="https://lh7-rt.googleusercontent.com/docsz/AD_4nXeRqry-9IPGzaY3Ujfo612zu6AwYh3DaUWCrT0-DesSsxOUkHmEiNaA4UKnvzDJfVL_sewKnZXy0xrSMd-RCAF_O9PZAfP3dTYMIFDs0oHrnCI4ZD0zb-jii9vonf8vBb-su2W7WopjgbnI2M71l3UyO8k?key=kDSKjjghsV5HdVIUV1RbMw" alt=""><figcaption>Image showing distribution of the power law</figcaption></figure> However, there is a lot of redundant information in those bytes. The number of bits theoretically required to send a piece of information is the logarithm of its probability of occuring. In practice, the probability distribution of which account is making a transaction, or which contract is being executed, or the arguments to the method being called, is very sharp (often a power law). This means that a minority of those accounts or contracts are highly likely to occur in the data to be sent. For example, if a particular contract was being called by 10% of transactions, its address can be encoded in 3.3 bits. That is a 48x compression ratio on the uncompressed 20 byte address. ## **Streaming Compression** This leads to the question of how we can effectively compress the transaction data moving between nodes. There are broadly two forms of compression: * Block compression * Streaming compression Block compression is given a single block of data, and compresses in a way where the receiver only needs that block of data to decompress it. This is the typical form of compression, such as zip or tar files. It is very convenient because the compressor doesn't need to make assumptions about what other information the client has, or the order that information was sent. Streaming compression is able to assume that the sender and receiver both sha