bitpro가 원문에서 글자만 모은 내용이에요. 표·그림·수식은 빠지거나 깨질 수 있어요. 전체 3쪽 중 1쪽
DoubleZero
Protocol
December
2,
2024
Accelerating
communication
in
high-performance
distributed
systems
to
Increase
Bandwidth
and
Reduce
Latency
Austin
Federa,
Andrew
McConnell,
Mateo
Ward
austin@doublezero.xyz
,
andrew@malbeclabs.com
,
mateo@malbeclabs.com
Special
Thanks
to
Dr.
Nihar
Shah,
Chris
Remus,
Malbec
Labs,
Dr.
Kevin
Bowers,
and
the
Firedancer
team
Abstract
The
DoubleZero
protocol
is
a
decentralized
framework
for
creating
and
managing
high-performance
permissionless
networks,
optimized
for
distributed
systems
like
blockchain.
It
enables
permissionless
contributions
of
underutilized
private
fiber
links
to
create
a
dynamic
and
expansive
network.
The
DoubleZero
network
has
a
low-level
permissionless
architecture
capable
of
seamlessly
bridging
heterogeneous
networks.
Together,
it
creates
a
meshed
transport
layer
that
filters
and
serves
traffic
across
low-latency
and
high-bandwidth
routes,
allowing
distributed
systems
to
send
and
receive
information
efficiently.
Through
a
permissionless
controller
that
lives
on
a
public
blockchain,
the
protocol
attracts
new
contributions
with
incentives
and
manages
the
network
configuration
and
routing
in
response
to
demand
spikes,
outages,
and
other
disruptions.
This
network
infrastructure
layer
enables
systems
like
Layer
1
blockchains
to
be
freed
of
communication
bottlenecks
and
approach
the
maximum
performance
that
physics
allows.
In
short,
the
DoubleZero
network
is
a
new
internet
optimized
for
distributed
systems.
0
Introduction
The
current
pace
of
improvement
in
throughput
and
latency
of
decentralized
systems
is
unable
to
keep
up
with
the
needs
of
developers
and
users.
This
is
most
noticeable
in
Layer
1
blockchains,
but
also
true
for
Layer
2
networks
and
non-blockchain
distributed
systems.
This
is
despite
substantial
improvements
in
the
computing
capabilities
of
individual
validators.
1
However,
these
systems
are
increasingly
bottlenecked
by
bandwidth
limitations
and
variable
latency
in
communication
between
validators,
rather
than
compute
capabilities
within
validators.
The
key
to
unlocking
higher
performance
for
these
systems
is
to
optimize
data
flows.
The
DoubleZero
protocol
is
designed
to
do
this
by
combining
individually-contributed
fiber
links
into
a
synchronized
network
that
filters
spam,
increases
bandwidth,
lowers
latency,
and
removes
jitter
from
communication.
In
short,
it
is
designed
to
improve
the
performance
of
blockchains
–
and
all
decentralized
distributed
systems
–
by
advancing
reliability
and
scalability.
The
physical
network
layer
that
blockchains
use
has
not
changed
since
the
advent
of
Bitcoin.
This
underdevelopment
limits
the
scalability
and
decentralization
of
performant
blockchains,
making
them
both
more
centralized
and
more
inefficient.
The
hardware
requirements
needed
for
validators
to
consume
the
firehose
of
inbound
transactions
escalate
quickly.
Consensus
is
reached
slowly,
as
validators
propose
blocks
and
vote
over
public
internet
paths
prone
to
high
jitter
and
inconsistent
routing.
Compared
to
the
public
internet,
the
DoubleZero
network
offers
two
improvements
to
blockchains.
First,
inbound
transactions
can
be
edge-filtered
(i.e.
removed
of
spam
and
duplicates)
by
specialized
hardware
prior
to
being
sent
over
the
DoubleZero
network.
This
allows
blockchains
to
benefit
from
shared
system-wide
filtration
resources
rather
than
needing
each
individual
validator
to
provision
sufficient
resources.
Second,
outbound
messages
can
be
explicitly
routed,
tracked,
and
prioritized
to
improve
efficiency.
Validators
can
control
latency
and
reduce
jitter
(i.e.
random
variation)
in
that
latency
in
reaching
consensus.
Taken
together,
the
DoubleZero
network
can
achieve
levels
of
performance
that
are
otherwise
unobtainable
without
the
centralized
co-location
of
systems.
The
DoubleZero
network
architecture
maps
conceptually
to
two
concentric
rings:
an
outer
ingress/egress
ring,
and
an
inner
data
flow
ring.
The
outer
ring
of
the
network
interfaces
with
the
public
internet,
and
DoubleZero’s
network
devices
here
use
performant
hardware
(such
as
FPGAs)
to
mitigate
distributed
denial-of-service
attacks,
verify
signatures,
and
filter
duplicate
transactions.
Servers
on
the
inner
ring
build
consensus
with
this
filtered
traffic
over
optimally-routed
dedicated
bandwidth
lines.
Together,
these
two
rings
provide
a
scalable
and
permissionless
information
service.
Indeed,
the
DoubleZero
network
can
be
used
for
optimizing
any
distributed
system.
Blockchain
is
not
unique
in
this
bottleneck,
as
nearly
every
application
of
time-sensitive,
high-performance
compute
is
limited
by
ingress
and
egress
of
data
flow
rather
than
the
act
of
doing
the
work.
For
instance,
within
the
blockchain
world,
RPC
nodes,
MEV
systems,
and
even
Layer
2
chains
can
1
For
instance,
the
blockchain
space
has
seen
a
new
generation
of
high-performance
validator
clients
(e.g.
Firedancer
and
Reth)
and
a
new
generation
of
high-performance
chains
(e.g.
Sui
and
Monad)
that
improve
on
old
systems
by
a
literal
order
of
magnitude.
1
benefit
from
the
filtered
and
low-latency
links
provided
by
the
network.
Outside
of
the
blockchain
world,
content
delivery
networks,
gaming
applications,
large
language
models,
and
enterprise
users
can
benefit
from
the
network
for
similar
reasons.
For
many
of
these
applications,
their
high-priority
traffic
otherwise
flows
over
the
public
internet,
indiscriminately
co-mingled
with
non-time
sensitive
traffic.
The
DoubleZero
network
offers
an
alternate
and
improved
model.
Figure
1.
The
DoubleZero
Conceptual
Network
Diagram
The
DoubleZero
protocol
is
not
just
a
technical
achievement
–
it
is
a
new
economic
model
for
bandwidth
and
communication.
On
the
provider
side,
it
allows
private
businesses
to
monetize
underutilized
private
links
they
may
have
purchased
or
leased
from
a
telco
or
network
service
provider.
Rather
than
sitting
idle,
entities
can
contribute
their
links
into
the
DoubleZero
system
and
open
new
revenue
streams
for
themselves.
On
the
user
and
operator
side,
it
allows
distributed
systems
to
enjoy
the
benefits
of
private
networking
without
relying
on
centralized
systems
and
long-term
contracts.
Instead,
these
systems
can
be
flexible
and
continuously
fine-tune
their
throughput
requirements,
network
topology,
or
communication
prioritization.
Most
importantly,
the
DoubleZero
protocol
not
only
matches
suppliers
and
users,
but
also
turns
individual
contributions
into
a
unified
global
network.
Individual
links
from
suppliers
–
which
are
fixed
in
their
routes,
limited
in
their
reach,
and
offered
without
ubiquitous
global
redundancy
–
may
line
up
poorly
with
2
demand
needs
from
modern
systems
that
demand
flexibility,
reach,
and
resiliency.
The
DoubleZero
protocol
stitches
these
individual
links
into
a
robust
and
expansive
system,
leveraging
the
same
network
effects
that
made
the
internet
what
it
is
today.
Like
the
public
internet,
the
distributed
ethos
of
the
DoubleZero
network
is
a
strength
rather
than
a
weakness,
with
multiple
independent
contributors
creating
something
more
valuable
than
the
sum
of
its
parts.
Ultimately,
the
protocol
is
the
first
of
its
kind
as
an
“N1”
–
a
base
layer
of
neutral
and
performant
physical
infrastructure.
It
is
upon
this
N1
that
distributed
systems
and
applications
(network-related,
such
as
N2s
or
otherwise)
can
be
built.
A
core
belief
of
the
DoubleZero
project
is
that
abundance
breeds
innovation.
Every
step-function
change
in
availability
or
corresponding
reduction
in
latency
opens
up
entirely
new
types
of
applications
and
uses
for
distributed
systems.
From
this
first
principle,
DoubleZero
sets
a
north
star
of
increasing
bandwidth
and
reducing
latency
for
distributed
systems
worldwide
to
support
a
more
fair
and
open
future
for
software.
3
System
Architecture
DoubleZero
begins
with
a
simple
but
profound
idea
–
what
if
each
node
operator
(validator,
RPC,
etc.)
was
not
solely
responsible
for
data
ingest
and
egress?
We
propose
that
by
decoupling
filtration
and
verification
from
transaction
inclusion,
block
production,
and
execution,
we
can
create
a
parallel
and
protected
transaction
flow
that
enables
node
operators
to
optimize
their
network
operations.
Figure
2.
Existing
Proof-of-Stake
Blockchain
Validation
Process.
While
a
validator's
role
is
commonly
understood
as
a
single
operation,
there
are
two
underlying
steps:
producing
a
block
and
finding
consensus
for
a
block.
Block
production
begins
when
a
flood
of
unfiltered
inbound
data
arrives
at
the
validator.
The
validator
performs
deduplication,
filtration,
and
signature
verification
operations,
reducing
the
possible
pool.
The
validator
then
selects
a
subset
of
viable
transactions
to
include,
constructs
the
block,
and
propagates
its
proposed
block
to
the
other
validators
for
consensus.
The
other
validators
either
approve
the
block
and
append
it
to
the
chain,
or
they
reject
the
block
and
continue
onwards.
2
2
While
there
is
a
difference
between
the
mechanics
of
leader
and
leaderless
blockchains,
particularly
with
respect
to
ordering
of
blocks,
this
subtlety
can
be
safely
ignored
here.
4
Figure
3.
Data
Flow
as
Rings.
On
most
networks
today,
these
processes
are
handled
by
one
x86
machine’s
worth
of
resources.
3
That
CPU’s
cores
are
linked
by
an
on-chip
network
within
the
machine.
We
propose
to
expand
this
to
a
global
network:
the
DoubleZero
network.
By
placing
FPGA
appliances
to
filter
inbound
traffic
at
key
ingress
points
at
the
network
edge,
the
DoubleZero
network
acts
as
a
common
shield
for
all
local
validators
(i.e.
validators
that
are
downstream
of
the
devices
connected
to
the
inner
ring
of
the
DoubleZero
network).
Compared
to
validators,
these
FPGA
appliances
can
handle
orders
of
magnitude
more
traffic
as
they
perform
the
specific
tasks
of
removing
spam,
de-duplicating
a
transaction
set,
and
verifying
signatures
–
and
this
is
all
done
via
open-source
software
that
can
be
inspected
to
ensure
fair
treatment
of
inbound
transactions.
Downstream
validators
receive
a
3
Even
PBS
uses
only
one
x86
machine’s
resources
at
a
time.
5
substantially
smaller
transaction
set
and
only
have
to
re-verify
signatures
on
the
transactions
included
in
the
final
block,
to
maintain
the
blockchain's
security.
Their
resources
–
typically
a
single
x86
machine
–
are
significantly
freed
to
perform
tasks
related
to
block
production,
transaction
execution,
indexing,
and
so
on.
If
we
disintermediate
the
filter
and
execution
rings
to
provide
more
flexibility,
we
can
distribute
ingress
resources
more
efficiently
and
more
resiliently.
Through
resource
sharing,
we
no
longer
need
to
provision
each
individual
validator
with
enough
resources
to
meet
global
demand;
rather,
global
demand
can
be
met
through
infrastructure
sharing
which
requires
far
fewer
resources
in
aggregate.
Figure
4.
Separating
the
Filter
and
Execution
Ring
Further
optimization
of
traffic
flow
is
achieved
by
support
for
multicast
traffic
on
the
inner
ring
of
the
DoubleZero
network,
which
functions
as
a
single
network
domain
under
common
control
of
protocol
participants.
Value-added
functions
like
block
or
shred
propagation
can
be
included
via
multicast.
4
4
Some
networks
make
use
of
shreds,
or
pieces
of
a
block,
which
are
transmitted
before
the
full
block
is
built.
6
Having
low-latency
access
to
the
newest
state
can
provide
significant
advantages
for
high-performance
applications.
A
denial-of-service
attack,
for
example,
would
be
much
more
difficult
to
successfully
pull
off
under
this
model
–
it
would
require
many
terabits
per
second
of
traffic
hitting
many
geographically-distributed
sites
simultaneously.
It
would
no
longer
be
an
attack
against
an
individual
validator
or
even
a
single
blockchain;
it
would
be
an
attack
against
hundreds
of
data
centers
and
ISPs
around
the
world.
A
successful
attack
would
be
several
orders
of
magnitude
more
difficult
and
resource-intensive
–
and
underlines
that
using
infrastructure
sharing
creates
a
more
robust
and
resilient
network
architecture
for
all.
Figure
5.
A
DDoS
Attack
on
the
DoubleZero
Network
7
The
Physical
DoubleZero
Network
Permissionless
Global
Connectivity
for
Blockchain
Traffic
The
physical
layer
of
the
DoubleZero
network
is
its
foundation.
It
consists
of
two
key
components:
network
devices
at
key
ingress/egress
points
of
the
network
and
provisioned
bandwidth
across
the
network.
Together,
they
form
a
meshed
network
that
comprises
the
DoubleZero
network.
While
individual
contributions
power
the
DoubleZero
network,
the
network
makes
use
of
orchestration
technology
to
aggregate
those
heterogeneous
contributions.
This
allows
the
network
to
leverage
the
strengths
of
private
networks
(which
are
limited
in
reach
but
operate
on
a
unified
standard)
and
public
networks
(which
are
expansive
in
reach
but
negotiate
multiple
standards).
DoubleZero
combines
the
best
of
both
worlds.
Permissionless
global
software
abstracted
networks
are
nothing
new
–
look
no
further
than
BitTorrent
and
the
Tor
browser.
Software
abstraction
allows
these
protocols
to
operate
over
the
internet,
but
therein
lies
their
greatest
limitations
–
they
are
software
networks
running
on
top
of
the
public
internet
and
are
still
governed
by
the
same
fundamental
limitations
of
all
public
internet
traffic.
The
architecture
of
the
DoubleZero
network
is
designed
to
connect
validators,
RPCs,
and
other
systems
at
the
network
edge
to
a
core
underlay
connected
at
DoubleZero
Exchange
points
(DZXs).
DZXs
enable
efficient
connectivity
between
multiple
data
centers
within
a
metro
area
in
a
similar
way
as
public
internet
exchanges
connect
different
networks
together.
These
underlay
links
are
provided
by
network
contributors,
and
are
the
core
transit
layer
of
the
protocol.
The
inner
ring
exchange
points
allow
ubiquitous
interconnectivity
to
happen
on
a
global
scale,
regardless
of
how
many
contributors
participate
in
DoubleZero.
Network
Devices
The
network
devices
on
the
DoubleZero
network
serve
two
functions.
First,
they
enable
data
links
contributed
by
individuals
and
organizations
to
operate
as
one
cohesive
network
with
prioritization.
Second,
they
implement
filtration,
verification,
and
spam
protection.
Network
contributors
on
the
DoubleZero
network
are
free
to
utilize
any
suitable
hardware
for
these
tasks,
but
the
network
holds
itself
to
high
standards.
For
instance,
at
launch,
the
DoubleZero
network
uses
commercially
available
FPGAs
(to
balance
performance,
speed,
and
cost),
and
to
run
integrated
appliances
that
can
handle
both
filtering
and
routing.
One
sample
deployment
alone
can
handle
deduplication,
filtering,
and
signature
verification
for
multiple
Gbps
of
inbound
data.
These
devices
also
manage
connectivity
to
downstream
validators
and
other
client
devices.
While
the
deployment
of
advanced
filtration
and
signature
verification
technology
to
data
centers
and
other
validator
facilities
would
improve
the
state
of
blockchain
networks
on
its
own,
DoubleZero
proposes
to
go
a
step
further
and
create
a
new
permissionless
transport
layer
for
blockchains
and
distributed
systems
that
is
fully
integrated
into
the
filtration
service.
This
can
run
across
a
variety
of
hardware
and
will
expand
as
the
network
grows.
8
Fiber
Links
The
fiber
links
on
the
DoubleZero
network
provide
the
low-latency
high-bandwidth
connections
between
different
points
on
the
globe.
Network
contributors
provision
the
connections
and
they
encode
a
service
level
agreement
for
the
quality
of
that
connection.
The
fiber
links
are
either
directly
owned
by
the
network
contributor,
or
leased
by
them
from
the
major
network
service
providers.
Network
contributors
add
them
to
the
network
and
commit
to
a
service-level
agreement
for
each
link.
This
commitment
establishes
the
main
characteristics
of
the
link,
including
endpoint
locations,
bandwidth,
latency,
and
compliant
MTU
size.
(As
an
illustration,
a
network
contributor
for
a
link
from
Los
Angeles
to
Singapore
may
commit
to
10
GBps
at
85ms
latency
with
1600
MTU.)
Performance
of
the
links
is
monitored
and
tested
by
the
DoubleZero
network
to
ensure
compliant
links
are
eligible
to
earn
rewards,
while
non-compliant
links
may
be
disqualified
for
rewards
and
potentially
removed
from
the
network.
The
DoubleZero
project
participants
believe
that
there
is
substantial
spare
capacity
in
the
fiber
market.
First,
a
modern
fiber
link,
whether
terrestrial
or
subsea,
can
support
hundreds
of
terabits
of
data
per
second.
Second,
there
is
substantial
“dark
fiber,”
or
fiber
that
has
been
installed
but
does
not
operate.
For
instance,
the
Federal
Communications
Commission
[2007]
estimates
in
its
last
publication
that
only
35%
of
the
fiber
installed
in
the
United
States
is
being
utilized,
and
65%
remains
unutilized.
And
since
then,
fiber
capacity
has
only
grown
dramatically;
rising
from
reaching
15%
to
46%
of
US
locations
from
2015
to
2023
(per
the
Fiber
Deployment
Annual
Report
of
2023)
and
reaching
70%
of
EU39
households
(per
the
FTTH
Council
Europe
in
2024).
Third,
many
enterprises,
in
buying
or
leasing
fiber,
tend
to
over-provision
capacity,
building
it
for
worst-case
rather
than
routine
outcomes.
This
suggests
that
the
DoubleZero
network
could
benefit
from
contributors
repurposing
already-installed
links.
Contrary
to
popular
belief,
data
centers
within
a
city
or
region
are
not
isolated
from
one
another.
A
vast
network
of
short-haul
fiber
and
other
routing
services
connect
data
centers
within
a
metropolitan
area,
allowing
for
direct
connections
between
them
and
thus
allowing
the
expansion
of
the
DoubleZero
network.
This
helps
reduce
incentives
for
co-location
and
expands
the
geographic
reach
of
DoubleZero.
The
DoubleZero
network's
core
focus
is
on
fiber.
However,
network
contributors
are
free
to
provision
other
communications
services,
such
as
microwave
or
satellite
technology,
and
attach
it
to
the
network
as
an
application
(an
“N2”).
Community
Governed
and
Operated
Networks
Even
the
most
performant
physical
infrastructure
networks
see
the
occasional
hiccup.
Network-wide
issues
are
where
the
final
component
of
the
DoubleZero
system
comes
into
play
—
a
streamlined
operational
process
for
bringing
together
network
contributors
to
diagnose
and
repair
the
problem,
while
users
fall
back
to
the
public
internet.
(Failures
with
a
single
network
contributor’s
infrastructure
are
the
responsibility
of
that
network
contributor
and
do
not
need
community-level
operations.)
9
Centralized
systems
generally
enjoy
advantages
over
decentralized
systems
in
times
of
network
failure
or
issues.
While
centralized
systems
have
more
single
points
of
failure,
they
can
quickly
muster
technical
resources
to
resolve
issues
(By
contrast,
decentralized
networks
such
as
blockchains
are
more
resilient,
but
require
coordination
among
many
unaffiliated
stakeholders
to
troubleshoot
issues.)
In
its
mature
state,
DoubleZero
contributors
will
be
able
to
commit
similar
resources
towards
routing
around
disruptions
and
maintaining
network
cohesion.
As
the
network
grows,
the
DoubleZero
network
aims
to
minimize
the
gap
by
implementing
best
practices,
developing
comparable
operational
procedures,
and
simulating
failure
situations
from
the
start.
There
are
many
existing
and
successful
models
of
multiple
network
operators
working
together
to
operate
and
maintain
international
networks
–
for
example,
consortium
sub-sea
cable
systems.
Network
Creation
The
DoubleZero
project
is
a
new
economic
model
for
filtration
and
routing
services
over
global
networks.
In
the
DoubleZero
protocol,
they
can
be
both
provisioned
and
utilized
permissionlessly
through
a
blockchain-based
smart
contract
that
verifies
the
integrity
of
the
interactions.
Existing
solutions
today
operate
on
behalf
of
a
specific
customer
in
B2B
relationships
and
do
not
act
to
increase
the
aggregate
performance
of
the
distributed
system.
This
flexible
model
is
key
to
unlocking
network
effects.
The
DoubleZero
protocol
automatically
turns
individual
contributions
into
a
holistic
network
that
is
more
valuable
than
the
sum
of
its
parts.
A
provider
can
contribute
a
single
and
even
redundant
link
–
it
might프로젝트로 돌아가기