🌐 Context We currently have a retrievability assurance protocol (Retriev.org, details here) which works as follows:
- There is a consortium of providers that sign up to participate in our system;
- There is a fixed set of referees for which we can say that half of them are honest (eg, 5 referees and we trust 3 of them);
- Client and Provider agree on a retrievability deal (aka the assurance policy for the retrieval service); providers lock down a collateral for each deal and client locks down a payment for the service;
- When a client does not get the requested file, it appeals to the referees. One of them tries to retrieve the file asking it again to the provider, if this works the file is forwarded to the client (the provider gets paid); If not, we try again. After some failed attempts, the provider is penalized: the collateral is taken away (and no payment is provided).
Referee Network and its Invocation
While the deal is active, the client can appeal to the referees (n referees in total) at any instant on time for requesting the assured storage. The appeal is considered valid if all the following is true:
- The deal is active
- For this deal, the number of appeals already created is less the the maximum allowed by the protocol (eg, 5)
- There is no active appeal already
- It includes the payment of an appeal fee. This is the amount of token that the client pays to the referees for their service.
Start and Process appeal
If an appeal is created, any referee that is free can start the appeal process.
The protocol to process an appeal is made by k (eg, 12) rounds each. In each rounds the referees try to retrieve the file in this way:
- Step 1: a retrieval leader is chosen at random.
- Step 2: the leader asks the file to the provider.
- Step 2.a: If the leader does not get the correct file in time, he sends a signed “failure msg” on chain; the other referees see the message and do nothing for this round; in the next round all starts from Step 1;
- Step 2.b: If the leader gets the file it forwards it to the client and to the other referees. Each referee does the following:
- If the correct file is received, forward it to the client;
- If the correct file is not received before the end of the round, sign a “failure vote” and broadcast it to the other referees;
- Count the number of failure votes received, if they are ≥n/2, then send a “failure msg” on chain; at the next round, start from Step 1. Otherwise, do nothing for the remaining rounds. Note that only the first “failure msg” goes through.
After the k rounds are over, the contract checks the total number of “failure messages”.
- If it is greater or equal to a pre-determined slashing threshold, then the provider is “slashed” (ie, the collateral is deposited to the contract owner’s account), the payment is returned to the client and the deal is deactivated.
- Otherwise nothing happen (in particular, the deal stays active).
Given the current design, the protocol is not enforcing any retrievability “budget” nor frequency.
We want to introduce here the possibility of introducing a Retrieval Budget within the retrievability deal duration, which would be beneficial both for Client ad Provider.
- Clients will negotiate in advance how many retrievals are granted by the protocol, ensuring this budget covers his needs through time
- Providers will have a precise idea of the bandwidth requirement they will have to allocate bandwidth to serve the file in a given window of time
We stress that we envision this feature to be opt in.
📌 Protocol Overview
Step 1: Retrieval Policy Proposal
When proposing a retrievability deal, a client can specify additional asks with respect to retrievals (note that all these asks are optional)
- What is the retrieval budget he wants to have (that is, how many retrievals he want to be granted during the deal duration). This is in essence a bandwidth ask for the provider. We refer to a retrieval budget unit as a “pass”
- The frequency of passes within the deal duration
- The price which he is willing to pay for each pass (aka retrieval)
- A fault tolerance on the asks above (e.g. 10%)
- The pass issuance policy (see below)
At this point, the Client’s funds are locked into the smart contract. Deal acceptance now works the same way as in the standard retrievability deal.
Pass Issuance Policy
We envision two options for Pass Issuance:
- Option 2: The Provider generate the passes
- Provider generates passes and signs them
- Passes are given off-chain to the Client
- Clients’ signatures are used by the Provider to redeem the payment later
- Option 2: Client Generated the passes
- Client generates the passes and commits to the MT root of them
Step 2: Retrievals
According to the retrievability policy, at some point the Client wants to use a pass to download some data for the Provider.
- If the retrieval happens smoothly, the Client gets the data and the Provider gets the signed pass which ca be redeemed later.
- If the Provider is not serving the data or she is not compliant with the Retrievability Policy, the Client forward a “bad pass” notification to the Referee Network and starts an Appeal
- If the Provider is serving the data according to the Retrieval Policy but the client is not providing a signed pass accordingly, she can start a Pass Appeal with the Referee Network
Failure rates are part of the retrievability policy and can be used by the Referees as a reason for (partially) slashing. An high level idea on how to deal with failures at the Referees level is then following
- If according to the Referee Network the Provider’s failure rate is high, then the Provider looses its staked money
- If according to the Referee Network the Provider’s failure rate is low (below agreed rate, so bad clients) - then auditors will grant tokens to SP at inverse of that rate without using additional bandwidth. The idea here is that not serving the file to Clients but serving files to the committee is not profitable for the Provider.
In general economics we should have is:
- In case of misbehaving Clients: Providers end up at a neutral place (get money, ends up serving a multiple of overall total content)
- In case of misbehaving Providers: Clients lose money, but the Provider looses a multiple x stake
- WS: how exactly auditors work, referring to hackmd.io/@irenegia/retriev
- WS: flesh out pass negotiation protocol