Allora 백서

Allora

원문

bitpro가 원문에서 글자만 모은 내용이에요. 표·그림·수식은 빠지거나 깨질 수 있어요. 전체 5쪽 중 2쪽

y the contribution of the forecasting task to the network accuracy, which in turn sets the reward distribution between the inference and forecasting tasks. 2. A ‘one-out’ network inference I−li , which omits a single inference or forecast-implied inferencel from the weighted average in Equation 9. If an inference from the inference task is omitted (Iij), the forecast-implied inferences (Iik) are updated accordingly before calculating I−li . The one-out network inferences represent an approximation of Shapley (1953) values and are used to quantify the individual contributions of workers to the network accuracy, which in turn sets the reward distribution between workers. 3. A ‘one-in’ naive network inference I+ki, which adds only a single forecast-implied inference Iik to the inferences from the inference taskIij. As such, it is used to quantify how the naive network inferenceI−i changes with the ad- dition of a single forecast-implied inference, which in turn is used for setting the reward distribution between work- ers for their forecasting tasks. The one-in naive network inference better differentiates between forecast-implied inferences than their associated one-out inferences, because there exists some redundancy between multiple fore- casting tasks and omitting a single one need not negatively impact the network inference. After all, the forecasting tasks can only draw from a finite number of original inferences and forecast-implied inferences may sometimes be mutually exchangable. This redundancy is desirable from a decentralization perspective and should not be disincen- tivized by exclusively using the one-out network inference to reward workers for the forecasting task. Therefore, we additionally use the one-in naive network inference to quantify each worker’s individual contribution. All inferences generated by the network are collected as Ii ={Ii,Iij,Iik,I−i ,I−li,I +ki}, (12) and are evaluated by the reputers. The reputers obtain a ground truth of the target variable Y when it becomes available (for simplicity, we assume this is delayed by one time step) and compare it with each of the inferences by calculating the topic’s loss functionQ: Lim = Q(Ii−1,Yi−1), (13) Lijm = Q(Ii−1,j,Yi−1), Likm = Q(Ii−1,k,Yi−1), L−im = Q(I−i−1,Yi−1), L−lim = Q(I−l,i−1,Yi−1), L+kim = Q(I+k,i−1,Yi−1). 1Recall that a worker may choose to engage (or not) in any of the tasks, so 2Nw inferences represents an upper limit to the total number. For simplicity, we assume here that all workers perform all tasks, but the design outlined in this paper does not require this assumption to be satisfied. ADI 1, 1–19 (2024) 6 Kruijssen et al. It is left to the individual reputer to decide whether the loss reported at time step i also includes some record of the historical loss, e.g. by using an exponential moving average. Doing so introduces a free parameter αm that controls the relative weights of the historical and current losses. Because reputers are rewarded based on their consensus (see§4), this introduces a game-theoretical aspect to the reputer task. However, forecasting workers require access to the instantaneous loss to achieve context awareness. Their positive contribution to the network accuracy therefore implies that αm = 1 should be expected in practice. Each reputer has a stakeSim, and the losses reported by the reputers are combined through a stake-weighted average of the logarithm of the loss: logLi = ∑ mSim logLim∑ mSim , (14) where we have avoided specifying all six variations listed in Equation 13 for brevity. The resulting losses are used by the network to calculate the corresponding regrets, which set the weights of the network inference as specified in Equation 9– Equation 11. The regrets are calculated according to an exponential moving average with a fiducial parameterα∈ (0, 1]: Ril =α (logLi− logLil) + (1−α)Ri−1,l. (15) We adoptα = 0.1 as the fiducial value, which provides a reasonable balance between historical performance and recency. Corresponding regrets are also calculated for the losses of the secondary inferences listed in Equation 13, to enable evaluating Equation 10 for each of these. 3.2 Confidence Intervals on the Network Inference In addition to providing network inferences Ii, the network also provides confidence intervals (CIs) on the network inference that reflect the spread of the inferences provided by the workersIil. The CIs are based on percentiles P , which are chosen to be P∈{ 2.28, 15.87, 84.13, 97.72}% (with corresponding quantiles q≡ P/100) to mimic the 1σ and 2σ limits of a Gaussian distribution, even if there is no guarantee that the distribution of worker inferences is Gaussian. When calculating the CIs, the weight wil assigned to each worker inference is accounted for to ensure that inferences with low weight do not inflate the CIs. Givenq, the weighted quantile inferenceIqi is computed by interpolating a weighted cumulative distribution function (CDF). The application of a weighted average to calculate the network inference in Equation 9 implies that the variance of the network inference is smaller than the variance across the sample of worker inferences. To account for this variance reduction, we adjust the worker inferences before constructing the CDF. Specifically, we scale the deviations of each worker inference from the weighted mean by a factor of 1/ √ N. The adjusted worker inferenceI′il is computed as I′il =Ii + Iil−Ii√N , (16) where Ii represents the individual elements of the network inference from Equation 9. This adjustment reflects the decrease in the standard error of the mean with increasing sample size, ensuring that the confidence intervals appropriately reflect the uncertainty in the network inference. Next, the worker inferences and their corresponding weights are sorted over dimension l such that Iil≤ Ii,l+1. For each sorted worker inferencel, the cumulative sum of the weights is calculated as cil = l∑ l′=1 wil′. (17) These cumulative weights are then normalized to create a weighted cumulative distribution function (CDF): Cil = cil− 0.5wilmaxlcil , (18) where maxlcil is the total sum of the weights. The term 0.5wil is subtracted to locate the quantile value at the middle of the weight associated withIil. Finding the inference valueIqi corresponding to the quantileq then requires a simple linear interpolation within the sorted list of data points using the weighted CDF. We define l = 𝓁 as the largest index where Cil≤q. We then obtainIqi as Iqi =I′i𝓁 + q−Ci𝓁Ci,𝓁+1−Ci𝓁(I′i,𝓁+1−I′i𝓁). (19) This calculation of CIs accounts for the relative weight of each worker inference and only reflects the degree of internal disagreement among the workers providing inference to the network. It does not provide any insight into historical network performance in relation to the ground truth. ADI 1, 1–19 (2024) Allora: a Self-Improving, Decentralized Machine Intelligence Network 7 3.3 Numerical Simulation of Allora’s Inference Synthesis and Performance We now quantify the expected performance of the Allora architecture, obtained through a numerical simulation of the logic described in§3.1. The experiment is designed to closely mimic the live network performance without needing to run any actual inference models. Here we describe how the worker and reputer functionality is mimicked, and how the simulation is set up. We consider a simple numerical experiment using mock data, where three workers generate inferences that predict the ground truth with some specified accuracy. The worker inference accuracy is characterized by an error (randomly drawn for each worker from a log-normal distribution with log-mean −0.700 and log-dispersion 0.176) and a bias (randomly drawn for each worker from a normal distribution with mean zero and dispersion0.05). Each inference-providing worker gains experience over time, which is modelled by multiplying its error and bias by a time-dependent factorfxp,i, defined as fxp,i = 12(1 + e−0.03i), (20) reflecting an exponential decay from unity to 0.5. At each time step, the error and bias of a different randomly-drawn worker is temporarily decreased by a factor 0.3 to model context-dependent outperformance. With these parameters, the inference provided by each worker during each time step is obtained by taking the ground truth and adding a difference drawn from a Gaussian distribution with mean equal to the worker’s current inference bias and standard deviation equal to the worker’s current inference error. The workers also forecast each other’s losses with different degrees of context-sensitivity, i.e. their ability to anticipate this temporary outperformance. In the simulation, we adopt a standard mean squared error (MSE) to described the loss. The worker forecasting accuracy is characterized by a logarithmic error (randomly drawn for each worker from a log- normal distribution with log-mean−0.222 and log-dispersion 0.176), a logarithmic bias (randomly drawn for each worker from a normal distribution with mean zero and dispersion0.3), and a context sensitivity parameterfcontext that is obtained by randomly drawing a numberx∈ [0, 1] and applying a sigmoid function: fcontext =σ[a(x−b)], (21) to increase the incidence of values near zero and unity, with a = 10 andb = 0.5. As before, each forecast-providing worker is assumed to gain experience over time by multiplying the error and bias withfxp,i from Equation 20. With these parameters, the forecasted losses provided by each worker during each time step are obtained by taking the logarithm of the actual loss for each worker’s inference and adding a logarithmic difference drawn from a Gaussian distribution with mean equal to the worker’s current forecasting bias and standard deviation equal to the worker’s current forecasting error. We consider two such forecasted losses. The first forecasted loss is a perturbation of the actual loss including context- dependent outperformance, modelled as described above by randomly decreasing the error and bias of a worker each time step. The second forecasted loss is a perturbation of the loss corrected for any context-dependent outperformance, obtained by dividing the actual loss by a factor0.32 (appropriate for the MSE loss adopted here). The final forecasted loss is obtained as a linear combination of the logarithms of these losses, where the logarithm of the actual loss is weighted by a factorfcontext, the logarithm of the context-unaware loss is weighted by a factor1−fcontext, and the result is converted back into linear space. This accurately reflects the individual context awareness of each forecast-providing worker. Finally, a set of five reputers is initialized by defining three parameters that describe the accuracy and definition of their reported losses. Each reputer is characterized by an error (randomly drawn for each reputer from a log-normal distribution with log-mean−1.000 and log-dispersion 0.097) and a bias (randomly drawn for each worker from a normal distribution with mean zero and dispersion 0.05). Additionally, we assume that reputers do not report the instantaneous loss, obtained by simply comparing each inference to the ground truth, but factor in a worker’s historical performance by applying an exponential moving average to the instantaneous loss (analogously to e.g. the regret calculation of Equation 15), each adopting an individual value αm (randomly drawn for each reputer from a log-normal distribution with log-mean −1.523 and log-dispersion 0.301). With these parameters, the losses reported by each reputer during each time step are obtained by taking the logarithm of the actual loss for each inference and adding a logarithmic difference drawn from a Gaussian distribution with mean equal to the reputer’s bias and standard deviation equal to the reputer’s error. The exponential moving average using each reputer’s individual value of αm is then applied to the logarithm of the losses, before converting these back to linear space. Figure 2 shows the losses of the resulting forecast-implied inferences, together with the losses of the original infer- ences, the loss of the naive network inference, and the loss of the complete network inference. On average, the naive network is as good as or better than the best inference. However, the forecast-implied inferences are even more accurate, and the network-wide inference outperforms all other inferences on average. We find that the addition of the forecasting task greatly improves the network accuracy, even in cases where the accuracy of the workers performing the forecasting task is lower than during the inference task. Even a moderate contextual awareness of when inferences are typically more accurate seems to be sufficient to yield a net improvement of the network accuracy. With the presented design, Allora optimally combines the inferences produced by the network participants through its Inference Synthesis mechanism. This is achieved by recognizing both the historical and context-dependent accuracy of the inferences. The key ingredient is the introduction of the forecasting task, which correlates performance and context by letting workers forecast each other’s losses under current conditions. ADI 1, 1–19 (2024) 8 Kruijssen et al. 2021-04 2021-07 2021-10 2022-01 2022-04 2022-07 2022-10 2023-01 2023-04 2023-07 Time 10 4 10 3 10 2 Losses Inferences Forecast-implied inferences Naive network loss Full network loss Figure 2: Demonstration of Allora’s self-improving intelligence and the accuracy improvement due to its context-aware Inference Synthesis mechanism. The dotted black line shows the naive network loss as a function of time, which is obtained by combining individual inferences (blue) without context awareness. The solid black line shows an order of magnitude improvement in loss thanks to the introduction of the forecasting task (cyan), which correlates performance and context by letting workers forecast each other’s losses under the current conditions. 4 Allora’s Differentiated Incentive Structure 4.1 Reward Distribution among Individual Network Participants The second critical hurdle to achieving decentralized machine intelligence is creating custom incentive structures that appropriately reward different actions within the network. Workers should be rewarded for their inference and forecasting tasks according to their unique contribution to the network. Fundamentally, there is no reason why this reward should depend on a monetary commitment such as a stake; in fact, stake-dependent rewards distract from their single objective of maximizing the network accuracy. By contrast, reputers must reach a form of consensus on their reported losses and should be rewarded for their proximity to that consensus. Because the network should reward consensus among reputers, their rewards can follow the common practice in decentralized systems of depending on the stake. By reporting on the performance of workers and thereby influencing their reward allocation too, reputers’ stakes provides the economic security for the entire topic. Common ways to quantify the unique contribution of participants to an end result include the Shapley (1953) value, Fisher (1922) information score, Banzhaf (1965) Power Index, and many others. The computational cost of these metrics is often high due to their reliance on large permutation sets, which can be prohibitive in a decentralized network setting. Therefore, Allora adopts a simple approximation of the Shapley values to score worker performance. For the inference task, we define the performance score as the difference of the logarithm of the loss between the one-out inference and the network inference, where the inference provided by a worker during the inference task is omitted from the network inference: Tij = logL−ji− logLi, (22) where j in the subscript indicates that we only consider one-out losses of inferences from the inference task. Recall that these one-out losses include the secondary impact of omitting an inference on the forecast-implied inferences by recalculating these (see §3). The performance score Tij is positive if the removal of an inference would increase the network loss, and is negative if its removal would decrease the network loss. The worker performance during the forecasting task can be scored similarly, but requires additional information from the one-in inferences introduced above. The forecasting task is comparatively redundant, i.e. in order to function well, a topic requires only one worker with reasonable context awareness to provide forecasted losses. As a result, removing any individual worker from the forecasting task may not noticeably impact the network inference loss, but the redundancy between workers is desirable (and should be rewarded) from a network perspective. A complete Shapley value calculation would remedy this problem by considering all possible permutations of workers, but at a prohibitive computational cost. Allora sidesteps this issue by adding only a single, information-rich permutation per worker, where the forecast-implied inference of that worker is added to the naive network inference to quantify its individual impact. The worker performance ADI 1, 1–19 (2024) Allora: a Self-Improving, Decentralized Machine Intelligence Network 9 score then becomes a combination of the one-out score and the one-in score: Tik = (1−f+)T−ik +f+T +ik, (23) where we define the one-out score analogously to Equation 22: T−ik = logL−ki− logLi, (24) and the one-in score as T +ik = logL−i − logL+ki. (25) It is easy to verify that these definitions satisfy the required directionality of the scores, i.e. the one-out score increases if the removal of a forecast-implied inference would increase the network inference lossLi, and the one-in score increases if the addition of a forecast-implied inference would decrease the naive network inference loss L−i . The weight of both terms in Equation 23 is parameterized using f+, which represents the fraction of permutations in a binomial experiment in which a worker appears solo: f+ = 12Nf, (26) whereNf is the number of workers providing forecasted losses during the forecasting task. The scores obtained in Equation 22 and Equation 23 facilitate the distribution of rewards to workers for inference and forecasting tasks. For a total reward allocated to each of these tasks per time step ( Ui andVi, both specified in§4.2), we use the scores to calculate the fraction of the rewards allocated to the two respective worker tasks that is received by each individual worker. For the inference task, these are defined as uij = M(Tij)∑ jM(Tij), (27) and for the forecasting task, these are vik = M(Tik)∑ kM(Tik). (28) Here,M is a mapping function that maps scores to reward fractions and the division by the sum ensures normalization of the reward fractions to unity. The mapping function must satisfy a number of simple requirements. It must reward positive scores and attribute negligible reward to negative scores. It must also be agnostic to the absolute scale of the scores, so that the inference and forecasting tasks are compensated according to a similar differentiation between contributions. Finally, it must accept a free parameter that can be used to control the spread in reward fractions, because this allows the network to influence the (de)centralization of the rewards if needed. The simplest functional form satisfying these requirements is M(T ) =φp,c [ T σ(T ) +ϵ ] , (29) whereσ(T ) represents the standard deviation of all scores over the ∆N most recent time steps. The use of the potential functionφp,c (see Equation 6) ensures that only positive scores receive significant rewards, while negative scores receive a small reward to acknowledge the contribution to decentralization. Dividing by the standard deviation of the scores ensures a similar differentiation between contributions, irrespective of the absolute scale of the score. The second term in the denominator is a
프로젝트로 돌아가기