DoubleZero 백서

DoubleZero

원문

bitpro가 원문에서 글자만 모은 내용이에요. 표·그림·수식은 빠지거나 깨질 수 있어요. 전체 3쪽 중 1쪽

DoubleZero Protocol December 2, 2024 Accelerating communication in high-performance distributed systems to Increase Bandwidth and Reduce Latency Austin Federa, Andrew McConnell, Mateo Ward austin@doublezero.xyz , andrew@malbeclabs.com , mateo@malbeclabs.com Special Thanks to Dr. Nihar Shah, Chris Remus, Malbec Labs, Dr. Kevin Bowers, and the Firedancer team Abstract The DoubleZero protocol is a decentralized framework for creating and managing high-performance permissionless networks, optimized for distributed systems like blockchain. It enables permissionless contributions of underutilized private fiber links to create a dynamic and expansive network. The DoubleZero network has a low-level permissionless architecture capable of seamlessly bridging heterogeneous networks. Together, it creates a meshed transport layer that filters and serves traffic across low-latency and high-bandwidth routes, allowing distributed systems to send and receive information efficiently. Through a permissionless controller that lives on a public blockchain, the protocol attracts new contributions with incentives and manages the network configuration and routing in response to demand spikes, outages, and other disruptions. This network infrastructure layer enables systems like Layer 1 blockchains to be freed of communication bottlenecks and approach the maximum performance that physics allows. In short, the DoubleZero network is a new internet optimized for distributed systems. 0 Introduction The current pace of improvement in throughput and latency of decentralized systems is unable to keep up with the needs of developers and users. This is most noticeable in Layer 1 blockchains, but also true for Layer 2 networks and non-blockchain distributed systems. This is despite substantial improvements in the computing capabilities of individual validators. 1 However, these systems are increasingly bottlenecked by bandwidth limitations and variable latency in communication between validators, rather than compute capabilities within validators. The key to unlocking higher performance for these systems is to optimize data flows. The DoubleZero protocol is designed to do this by combining individually-contributed fiber links into a synchronized network that filters spam, increases bandwidth, lowers latency, and removes jitter from communication. In short, it is designed to improve the performance of blockchains – and all decentralized distributed systems – by advancing reliability and scalability. The physical network layer that blockchains use has not changed since the advent of Bitcoin. This underdevelopment limits the scalability and decentralization of performant blockchains, making them both more centralized and more inefficient. The hardware requirements needed for validators to consume the firehose of inbound transactions escalate quickly. Consensus is reached slowly, as validators propose blocks and vote over public internet paths prone to high jitter and inconsistent routing. Compared to the public internet, the DoubleZero network offers two improvements to blockchains. First, inbound transactions can be edge-filtered (i.e. removed of spam and duplicates) by specialized hardware prior to being sent over the DoubleZero network. This allows blockchains to benefit from shared system-wide filtration resources rather than needing each individual validator to provision sufficient resources. Second, outbound messages can be explicitly routed, tracked, and prioritized to improve efficiency. Validators can control latency and reduce jitter (i.e. random variation) in that latency in reaching consensus. Taken together, the DoubleZero network can achieve levels of performance that are otherwise unobtainable without the centralized co-location of systems. The DoubleZero network architecture maps conceptually to two concentric rings: an outer ingress/egress ring, and an inner data flow ring. The outer ring of the network interfaces with the public internet, and DoubleZero’s network devices here use performant hardware (such as FPGAs) to mitigate distributed denial-of-service attacks, verify signatures, and filter duplicate transactions. Servers on the inner ring build consensus with this filtered traffic over optimally-routed dedicated bandwidth lines. Together, these two rings provide a scalable and permissionless information service. Indeed, the DoubleZero network can be used for optimizing any distributed system. Blockchain is not unique in this bottleneck, as nearly every application of time-sensitive, high-performance compute is limited by ingress and egress of data flow rather than the act of doing the work. For instance, within the blockchain world, RPC nodes, MEV systems, and even Layer 2 chains can 1 For instance, the blockchain space has seen a new generation of high-performance validator clients (e.g. Firedancer and Reth) and a new generation of high-performance chains (e.g. Sui and Monad) that improve on old systems by a literal order of magnitude. 1 benefit from the filtered and low-latency links provided by the network. Outside of the blockchain world, content delivery networks, gaming applications, large language models, and enterprise users can benefit from the network for similar reasons. For many of these applications, their high-priority traffic otherwise flows over the public internet, indiscriminately co-mingled with non-time sensitive traffic. The DoubleZero network offers an alternate and improved model. Figure 1. The DoubleZero Conceptual Network Diagram The DoubleZero protocol is not just a technical achievement – it is a new economic model for bandwidth and communication. On the provider side, it allows private businesses to monetize underutilized private links they may have purchased or leased from a telco or network service provider. Rather than sitting idle, entities can contribute their links into the DoubleZero system and open new revenue streams for themselves. On the user and operator side, it allows distributed systems to enjoy the benefits of private networking without relying on centralized systems and long-term contracts. Instead, these systems can be flexible and continuously fine-tune their throughput requirements, network topology, or communication prioritization. Most importantly, the DoubleZero protocol not only matches suppliers and users, but also turns individual contributions into a unified global network. Individual links from suppliers – which are fixed in their routes, limited in their reach, and offered without ubiquitous global redundancy – may line up poorly with 2 demand needs from modern systems that demand flexibility, reach, and resiliency. The DoubleZero protocol stitches these individual links into a robust and expansive system, leveraging the same network effects that made the internet what it is today. Like the public internet, the distributed ethos of the DoubleZero network is a strength rather than a weakness, with multiple independent contributors creating something more valuable than the sum of its parts. Ultimately, the protocol is the first of its kind as an “N1” – a base layer of neutral and performant physical infrastructure. It is upon this N1 that distributed systems and applications (network-related, such as N2s or otherwise) can be built. A core belief of the DoubleZero project is that abundance breeds innovation. Every step-function change in availability or corresponding reduction in latency opens up entirely new types of applications and uses for distributed systems. From this first principle, DoubleZero sets a north star of increasing bandwidth and reducing latency for distributed systems worldwide to support a more fair and open future for software. 3 System Architecture DoubleZero begins with a simple but profound idea – what if each node operator (validator, RPC, etc.) was not solely responsible for data ingest and egress? We propose that by decoupling filtration and verification from transaction inclusion, block production, and execution, we can create a parallel and protected transaction flow that enables node operators to optimize their network operations. Figure 2. Existing Proof-of-Stake Blockchain Validation Process. While a validator's role is commonly understood as a single operation, there are two underlying steps: producing a block and finding consensus for a block. Block production begins when a flood of unfiltered inbound data arrives at the validator. The validator performs deduplication, filtration, and signature verification operations, reducing the possible pool. The validator then selects a subset of viable transactions to include, constructs the block, and propagates its proposed block to the other validators for consensus. The other validators either approve the block and append it to the chain, or they reject the block and continue onwards. 2 2 While there is a difference between the mechanics of leader and leaderless blockchains, particularly with respect to ordering of blocks, this subtlety can be safely ignored here. 4 Figure 3. Data Flow as Rings. On most networks today, these processes are handled by one x86 machine’s worth of resources. 3 That CPU’s cores are linked by an on-chip network within the machine. We propose to expand this to a global network: the DoubleZero network. By placing FPGA appliances to filter inbound traffic at key ingress points at the network edge, the DoubleZero network acts as a common shield for all local validators (i.e. validators that are downstream of the devices connected to the inner ring of the DoubleZero network). Compared to validators, these FPGA appliances can handle orders of magnitude more traffic as they perform the specific tasks of removing spam, de-duplicating a transaction set, and verifying signatures – and this is all done via open-source software that can be inspected to ensure fair treatment of inbound transactions. Downstream validators receive a 3 Even PBS uses only one x86 machine’s resources at a time. 5 substantially smaller transaction set and only have to re-verify signatures on the transactions included in the final block, to maintain the blockchain's security. Their resources – typically a single x86 machine – are significantly freed to perform tasks related to block production, transaction execution, indexing, and so on. If we disintermediate the filter and execution rings to provide more flexibility, we can distribute ingress resources more efficiently and more resiliently. Through resource sharing, we no longer need to provision each individual validator with enough resources to meet global demand; rather, global demand can be met through infrastructure sharing which requires far fewer resources in aggregate. Figure 4. Separating the Filter and Execution Ring Further optimization of traffic flow is achieved by support for multicast traffic on the inner ring of the DoubleZero network, which functions as a single network domain under common control of protocol participants. Value-added functions like block or shred propagation can be included via multicast. 4 4 Some networks make use of shreds, or pieces of a block, which are transmitted before the full block is built. 6 Having low-latency access to the newest state can provide significant advantages for high-performance applications. A denial-of-service attack, for example, would be much more difficult to successfully pull off under this model – it would require many terabits per second of traffic hitting many geographically-distributed sites simultaneously. It would no longer be an attack against an individual validator or even a single blockchain; it would be an attack against hundreds of data centers and ISPs around the world. A successful attack would be several orders of magnitude more difficult and resource-intensive – and underlines that using infrastructure sharing creates a more robust and resilient network architecture for all. Figure 5. A DDoS Attack on the DoubleZero Network 7 The Physical DoubleZero Network Permissionless Global Connectivity for Blockchain Traffic The physical layer of the DoubleZero network is its foundation. It consists of two key components: network devices at key ingress/egress points of the network and provisioned bandwidth across the network. Together, they form a meshed network that comprises the DoubleZero network. While individual contributions power the DoubleZero network, the network makes use of orchestration technology to aggregate those heterogeneous contributions. This allows the network to leverage the strengths of private networks (which are limited in reach but operate on a unified standard) and public networks (which are expansive in reach but negotiate multiple standards). DoubleZero combines the best of both worlds. Permissionless global software abstracted networks are nothing new – look no further than BitTorrent and the Tor browser. Software abstraction allows these protocols to operate over the internet, but therein lies their greatest limitations – they are software networks running on top of the public internet and are still governed by the same fundamental limitations of all public internet traffic. The architecture of the DoubleZero network is designed to connect validators, RPCs, and other systems at the network edge to a core underlay connected at DoubleZero Exchange points (DZXs). DZXs enable efficient connectivity between multiple data centers within a metro area in a similar way as public internet exchanges connect different networks together. These underlay links are provided by network contributors, and are the core transit layer of the protocol. The inner ring exchange points allow ubiquitous interconnectivity to happen on a global scale, regardless of how many contributors participate in DoubleZero. Network Devices The network devices on the DoubleZero network serve two functions. First, they enable data links contributed by individuals and organizations to operate as one cohesive network with prioritization. Second, they implement filtration, verification, and spam protection. Network contributors on the DoubleZero network are free to utilize any suitable hardware for these tasks, but the network holds itself to high standards. For instance, at launch, the DoubleZero network uses commercially available FPGAs (to balance performance, speed, and cost), and to run integrated appliances that can handle both filtering and routing. One sample deployment alone can handle deduplication, filtering, and signature verification for multiple Gbps of inbound data. These devices also manage connectivity to downstream validators and other client devices. While the deployment of advanced filtration and signature verification technology to data centers and other validator facilities would improve the state of blockchain networks on its own, DoubleZero proposes to go a step further and create a new permissionless transport layer for blockchains and distributed systems that is fully integrated into the filtration service. This can run across a variety of hardware and will expand as the network grows. 8 Fiber Links The fiber links on the DoubleZero network provide the low-latency high-bandwidth connections between different points on the globe. Network contributors provision the connections and they encode a service level agreement for the quality of that connection. The fiber links are either directly owned by the network contributor, or leased by them from the major network service providers. Network contributors add them to the network and commit to a service-level agreement for each link. This commitment establishes the main characteristics of the link, including endpoint locations, bandwidth, latency, and compliant MTU size. (As an illustration, a network contributor for a link from Los Angeles to Singapore may commit to 10 GBps at 85ms latency with 1600 MTU.) Performance of the links is monitored and tested by the DoubleZero network to ensure compliant links are eligible to earn rewards, while non-compliant links may be disqualified for rewards and potentially removed from the network. The DoubleZero project participants believe that there is substantial spare capacity in the fiber market. First, a modern fiber link, whether terrestrial or subsea, can support hundreds of terabits of data per second. Second, there is substantial “dark fiber,” or fiber that has been installed but does not operate. For instance, the Federal Communications Commission [2007] estimates in its last publication that only 35% of the fiber installed in the United States is being utilized, and 65% remains unutilized. And since then, fiber capacity has only grown dramatically; rising from reaching 15% to 46% of US locations from 2015 to 2023 (per the Fiber Deployment Annual Report of 2023) and reaching 70% of EU39 households (per the FTTH Council Europe in 2024). Third, many enterprises, in buying or leasing fiber, tend to over-provision capacity, building it for worst-case rather than routine outcomes. This suggests that the DoubleZero network could benefit from contributors repurposing already-installed links. Contrary to popular belief, data centers within a city or region are not isolated from one another. A vast network of short-haul fiber and other routing services connect data centers within a metropolitan area, allowing for direct connections between them and thus allowing the expansion of the DoubleZero network. This helps reduce incentives for co-location and expands the geographic reach of DoubleZero. The DoubleZero network's core focus is on fiber. However, network contributors are free to provision other communications services, such as microwave or satellite technology, and attach it to the network as an application (an “N2”). Community Governed and Operated Networks Even the most performant physical infrastructure networks see the occasional hiccup. Network-wide issues are where the final component of the DoubleZero system comes into play — a streamlined operational process for bringing together network contributors to diagnose and repair the problem, while users fall back to the public internet. (Failures with a single network contributor’s infrastructure are the responsibility of that network contributor and do not need community-level operations.) 9 Centralized systems generally enjoy advantages over decentralized systems in times of network failure or issues. While centralized systems have more single points of failure, they can quickly muster technical resources to resolve issues (By contrast, decentralized networks such as blockchains are more resilient, but require coordination among many unaffiliated stakeholders to troubleshoot issues.) In its mature state, DoubleZero contributors will be able to commit similar resources towards routing around disruptions and maintaining network cohesion. As the network grows, the DoubleZero network aims to minimize the gap by implementing best practices, developing comparable operational procedures, and simulating failure situations from the start. There are many existing and successful models of multiple network operators working together to operate and maintain international networks – for example, consortium sub-sea cable systems. Network Creation The DoubleZero project is a new economic model for filtration and routing services over global networks. In the DoubleZero protocol, they can be both provisioned and utilized permissionlessly through a blockchain-based smart contract that verifies the integrity of the interactions. Existing solutions today operate on behalf of a specific customer in B2B relationships and do not act to increase the aggregate performance of the distributed system. This flexible model is key to unlocking network effects. The DoubleZero protocol automatically turns individual contributions into a holistic network that is more valuable than the sum of its parts. A provider can contribute a single and even redundant link – it might
프로젝트로 돌아가기