bitpro가 원문에서 글자만 모은 내용이에요. 표·그림·수식은 빠지거나 깨질 수 있어요. 전체 5쪽 중 2쪽
ially important in cases of
server failure. In contrast, for centralized CDN-based delivery, high concurrent users instead
place scalability pressures on the CDN servers.
However, the shortcoming of pure peer-to-peer streaming is availability. Peers come and go at
anytime, which makes it difficult to predict the availability of any given peer node. There are
also uncontrollable differences of nodes, such as upload and download capacities. On the
other hand, a CDN service is more reliable and robust, and hence it can serve as a reliable
“backup” when the stream is not available from peer nodes.
Our goal is to achieve maximum CDN bandwidth reduction without sacrificing the
quality-of-service (QoS) which is critical to established streaming platforms such as Netflix,
YouTube, Twitch, Facebook and others. This means whenever possible we want the peer nodes
to pull the stream from each other instead of from the CDN. To achieve this goal, it’s crucial for
the peer nodes to be able to identify neighboring nodes effectively. If a node can identify
multiple peers in close proximity, chances are that it can find peers that can provide the video
stream segments much more consistently. On the contrary, if the identified peers are “further
away” in terms of network hops, nodes might not be able to pull stream from peers
consistently and cause degraded user experience like stuttering, frequent rebuffering, etc.
To address this problem, Theta has designed and is currently implementing a strategy which
combines both a hyper-optimized tracker server and player client-side intelligence. Essentially,
the tracker server provides high level guidance (e.g. a list of candidate peers) for the player
client, while the player client implements a peer filtering algorithm at a finer granularity based
on multiple variables to find the neighboring nodes that can best serve them.
11
Figure 4 . Interactions between the tracker servers and player clients
Geo-Optimized Tracker Server
In order to provide a list of candidate peer nodes to each client, the tracker server records the
location information whenever a peer joins the network, including its IP address,
latitude/longitude, and a number of other performance parameters. With this information the
server can organize the nodes in a spatial database. Theta’s “hyper-optimized” spatial
database is optimized for storing and querying data that represents objects defined in
geometric space. As a peer node joins the network, the server can perform a spatial query to
retrieve a list of candidate peers that are in the close proximity very quickly and efficiently, see
Figure 4. The tracker servers and the spatial databases can be maintained by video streaming
sites that use the Theta network and/or by community peers for content delivery.
As we mentioned earlier, a peer node might leave the network at anytime. Hence the tracker
server also needs to be aware of which nodes are active. To achieve this, an active peer node
needs to maintain a socket connection with the server and send heartbeat signals
consistently. If the server doesn’t receive a heartbeat for a certain amount of time, it considers
that peer node as having left the network, and updates the spatial database accordingly.
An important distinction is that the “distance” between two peer nodes is measured by the
number of router hops between them rather than the geographical distance. Typically network
distance and geographical distance are highly correlated, but aren’t necessarily equivalent. For
example, two computers could sit next to each other physically, but connect to different ISPs
so there might be many hops between them. Hence, aside from geographical information, the
tracker server also utilizes the connectivity between the IP addresses collected in the past to
analyze and select neighbor candidates. For example, candidates returned by the spatial query
can go through another filter to exclude those that are not connected to the same ISP as the
viewer’s.
12
Intelligent Player Client
Each peer node may act both as a viewer, a caching node or both. As the node launches,
during the handshake step, it retrieves a list of candidate peers from the tracker server for the
livestream it’s playing. Then, it performs a speed and availability test to select a subset that
has optimized performance, connectivity and can reliably provide the video stream segments.
The client performs the speed and availability tests regularly during a live stream session and
continuously refines its neighbor list.
Figure 5 . Player stream data buffer handling
To avoid QoS degradation, local buffer management is critical. The client player maintains a
local cache to buffer the downloaded stream data as in Figure 5. If the duration of the cached
stream data is less than a certain threshold, the player checks with the neighboring peers to
see if they have the desired video stream segment. In the event when none of the neighbors
has that segment, the player intelligently falls back to the CDN. To achieve the best QoS
possible, the player fetches an updated candidate list from the tracker server on a regular
basis during the stream session.
The first version of the client video player is a web/HTML5 based player which employs the
WebRTC protocol for stream delivery among peers. Deploying web-based players requires
minimal effort. Streaming sites and platforms simply embed the player onto their webpages,
and it instantly has access and “launches” millions of end user nodes in the Theta mesh
network. Thus, the deployment of Theta’s mesh streaming technology is very light-weight and
frictionless.
Theta also plans to release desktop and mobile client support. The advantage of a desktop
client app over the web/HTML5 player is that it can run in the background to facilitate video
stream relay (with the consent of the user) even when the end user is not watching any video
streams. Further, Theta is investigating dedicated hardware, IOT devices, SmartTVs and related
approaches that are specifically designed for stream relay and re-broadcast. Such devices can
provide potentially better availability and bandwidth.
13
Theta Blockchain Ledger
The Theta Ledger is a decentralized ledger designed for the video streaming industry. It
powers the Theta token ecosystem which incentives end users to share their redundant
bandwidth and storage resources, and enables them to engage more actively with video
platforms and content creators. To realize these goals, a number of challenges, many of which
are unique for video streaming applications, need to be tackled.
One of such challenges is to support ultra high transaction throughput . Although many
blockchain projects are facing transaction throughput problems, scaling for live video
streaming is different and possibly even more complex. Typically, video segments are a couple
of seconds long. To achieve the finest granularity of a token reward — one micropayment per
video segment — even a live stream with a moderate ten thousand concurrent viewers could
generate a couple of thousand microtransactions per second, which far exceeds the maximum
throughput of today’s public chains, such as Bitcoin and Ethereum. Popular live streams like
major esport tournaments can attract more than one million viewers watching one stream
simultaneously, not to mention multiple concurrent live streams, which could potentially push
the required transaction throughput to the range of millions per second.
A byproduct of the high throughput is rapidly growing storage consumption . Storing the
micropayment transactions is highly storage demanding. With tens of thousands of
transactions added to the ledger every second, the storage space of an ordinary computer
could run out quickly.
Video streaming applications typically require fast consensus . For bandwidth sharing rewards,
the users that contribute redundant bandwidth typically want the payment to be confirmed
before sending the next one. Other use cases, such as virtual gift donations to live stream
hosts, also require short confirmation times to enable to real-time interactions between the
hosts and audience.
Finally, as in any blockchain, security of the ledger is critical. Security is highly correlated with
the level of decentralization . In a Proof-of-Stake (PoS) based consensus mechanism,
decentralization means an even stake distribution among consensus participants. Ideally, the
consensus mechanism should allow thousands of independent nodes, each with similar
amounts of stake and each possessing a local copy of the blockchain, to participate in the
block finalization process. To compromise such a system, a significant amount of independent
nodes would need to be controlled by the attackers, which is difficult to achieve.
To achieve these goals, we have designed our PoS consensus algorithm based on the
Byzantine Fault Tolerance (BFT) protocols, which offers good guarantees such as consistency
(a.k.a. safety) when more than 2/3 of nodes running the ledger software are honest. However,
the traditional BFT algorithms do not allow a high level of decentralization. They typically incur
O ( n 2 ) messaging complexity even for the normal (non-faulty proposer) case, where n is the
number of nodes participating in the consensus protocol. When we have thousands of nodes,
it will take considerable amount of time to reach agreement. In this paper, we present a novel
multi-level BFT consensus mechanism that allows mass participation, and still achieves
1000+ TPS throughput with the transaction confirmation time as short as a few seconds.
Such level of transaction throughput, although already much higher than Bitcoin and
Ethereum, is still not sufficient to handle the micropayments for the “pay-per-byte”
14
granularity. To further increase the throughput, the Theta Ledger provides native support for
off-chain scaling, with a “resource oriented micropayment pool” which further amplifies the
supportable throughput by several order of magnitudes.
We note that the off-chain payment support not only boosts the throughput, but also
decreases the number of the transactions that need to be stored in the blockchain. On top of
that, we introduce the technique of state and block history pruning to further reduce the
storage space requirement. Moreover, we have adopted the microservice architecture for the
storage system, which can adapt to different types of machines and storage backends, be it
powerful server clusters running in data centers, or commodity desktop PCs.
The Consensus Mechanism
Multi-Level BFT
The Theta Ledger is built on a novel multi-level BFT consensus mechanism which allows 11
thousands of nodes to participate in the consensus process, while still supporting very high
transaction throughput (1000+ TPS).
The core idea is to have a small set of nodes, which forms the validator committee , produce a
chain of blocks as fast as possible using a PBFT-like process. With a sufficient number of 12
validators (e.g. 10 to 20), the validator committee can produce blocks at a fast speed, and still
retain a high degree of difficulty to prevent an adversary from compromising the integrity of
the blockchain. Hence, it is reasonable to expect that there is a very high probability the
validators will produce a chain of blocks without forks. Then, the thousands of consensus
participants, called guardians , can finalize the chain generated by the validator committee.
Here “finalization” means to convince each honest guardian that more than 2/3 of all the other
guardians see the same chain of blocks.
Since there are many more guardians than validators, it could a take longer time for the
guardians to reach consensus than the validator committee. In order for the guardians to
finalize the chain of blocks at the same speed that the validator committee produces new
blocks, the guardian nodes can process the blocks at a much coarser grain. To be more specific,
they only need to agree on the hash of the checkpoint blocks, i.e. blocks whose height are a
multiple of some integer T (e.g. T = 100 ). This “leapfrogging” finalization strategy leverages
the immutability characteristic of the blockchain data structure — as long as two guardian
nodes agree on the hash of a block, with overwhelming probability, they will have exactly the
same copy of the entire blockchain up to that block. Finalizing only the checkpoint blocks gives
sufficient time for the thousands of guardians to reach consensus. Hence, with this strategy,
the two independent processes, i.e., block production and finalization, can advance at the
same pace.
Under the normal condition, finalizing a checkpoint block is similar to the "commit" step of the
celebrated PBFT algorithm since each guardian has already stored the checkpoint block locally.
Moreover, the checkpoint block has been signed by the validator committee, and hence it is
highly likely that all the honest guardians have the same checkpoint. Thus, we only need a
protocol for the honest guardians to confirm that indeed more than 2/3 of all guardians have
the same checkpoint hash.
11
For a more in depth discussion, please refer to our multilevel BFT technical report:
https://github.com/thetatoken/thetaprotocolledger/blob/master/docs/multilevelbfttechreport.pdf
12
Castro et al . Practical Byzantine Fault Tolerance
15
To implement this protocol, a naive all-to-all broadcasting of the checkpoint block hash could
work, but it yields quadratic communication overhead, and so cannot scale to large numbers of
guardians. Instead we propose an aggregated signature gossip scheme which could
significantly reduce messaging complexity. The core idea is rather simple. Each guardian node
keeps combining the partially aggregated signatures from all its neighbors, and then gossips
out the aggregated signature, along with a compact bitmap which encodes the list of signers.
This way the signature share of each node can reach other nodes at exponential speed utilizing
the gossip protocol. Within O (log n ) iterations, with high probability, all the honest guardian
nodes should have a string which aggregates the signatures from all other honest nodes if
there is no network partition. In addition, the signature aggregation keeps the size of the
node-to-node messages small, and thus further reduces the communication overhead.
As mentioned above, the validator committee is comprised of a limited set of validator nodes,
typically in the range of ten to twenty. They can be selected through an election process, or a
randomized process, and may be subject to rotation to improve security. To be eligible to join
the validator committee, a node needs to lock up a certain amount of stake for a period of
time, which can be slashed if malicious behavior is detected. The blocks that the committee
reaches consensus on are called settled blocks , and the process to settle the blocks is called
the block settlement process .
The guardian pool is a superset of the validator committee, i.e. a validator is also a guardian . The
pool contains a large number of nodes, which could be in the range of thousands. With a
certain amount of tokens locked up for a period of time, any node in the network can instantly
become a guardian. The guardians download and examine the chain of blocks generated by the
validator committee and try to reach consensus on the the checkpoints with the above
described “leapfrogging” approach. By allowing mass participation, we can greatly enhance the
transaction security. The blocks that the guardian pool has reached consensus on are called
finalized blocks , and the process to finalize the blocks is called the block finalization process .
The name multi-level BFT consensus mechanism reflects the fact that the validator/guardian
division provides multiple levels of security guarantee. The validator committee provides the
first level of protection — with 10 to 20 validators, the committee can come to consensus
quickly. Yet it is resistant enough to attacks — in fact, it already provides a similar level of
security compared to the DPoS mechanism if each validator nodes is run by an independent
entity. Thus, a transaction can already be considered safe when it has been included in a
settled block, especially for low stake transactions. The guardian pool forms the second line of
defense. With thousands of nodes, it is substantially more difficult for attackers to
compromise blockchain integrity, and thus provides a much higher level of security. In the
unlikely event that the validator committee is fully controlled by attackers, the guardians can
re-elect the validators, and the blockchain can restart, advancing from the most recent block
finalized by the guardians. A transaction is considered irreversible when it is included in a
finalized block. We believe this mechanism achieves a good balance among transaction
throughput, consistency, and level of decentralization, the three corners of the so-called
“ impossible triangle ”.
The multi-level security scheme suits video streaming applications well. For streaming
platforms, most of the transactions are micropayments (e.g. payment for peer bandwidth,
virtual gifts to hosts, etc.) which typically have low value, but require fast confirmation. For
such low stake payments, the users only need to wait for block settlement, which is very fast,
in a matter of seconds. For high stake transfers, the user can wait longer until the block
16
containing the transaction is finalized, which could take slightly longer time, but is still in the
range of minutes.
System Model
Before diving into the details of the block settlement and finalization process, we first list our
assumptions of the system. For ease of discussion, without loss of generality, below we
assume each node (be it a validator or a guardian) has the same amount of stake . Extending
the algorithms to the general case where different nodes have different amount of stake is
straightforward.
Validator committee failure model : There are m validator nodes in total. Most of the time, at
most one-third of them are byzantine nodes. They might be fully controlled by attackers, but
this happens only rarely. We also assume that between any pair of validator nodes there is a
direct message channel (e.g. a direct TCP connection).
Guardian pool failure model : There are n guardian nodes in total. At any moment, at most
one-third of them are byzantine nodes. We do not assume a direct message channel between
any two guardians. Messages between them might need to be routed through other nodes,
some of which could be byzantine nodes.
Timing model : We assume the “weak synchrony” model. To be more specific, the network can
be asynchronous, or even partitioned for a bounded period of time. Between the asynchronous
periods there are sufficiently long periods of time where all message transmissions between
two honest nodes arrive within a known time bound . As we will discuss later in the paper, Δ
during the asynchronous period, the ledger simply stops producing new blocks. It will never
produce conflicting blocks even with network partition. During synchronous phases, block
production will naturally resume, and eventual liveness can be achieved.
Attacker model : We assume powerful attackers. They can corrupt a large number of targeted
nodes, but no more than one-third of all the guardians simultaneously. They can manipulate
the network at a large scale, and can even partition the network for a bounded period of time.
Yet they are computationally bounded. They cannot forge fake signatures, and cannot invert
cryptographic hashes.
The Block Settlement Process
Block settlement is the프로젝트로 돌아가기