A proposal to implement indefinite terms for FIL+ data cap allocations, independent of storage deals, realising a controlled version of

Terminology

Allowance: data cap provided to a verifier by the root, or to a client by a verifier.

Allocation: data cap allocated by a client to a specific piece of data.

Term: period of time for which a sector/deal/allocation is active or valid.

Claim: a provider’s assertion they are storing all or part of an allocation

A FIL+ data cap allocation doesn’t have any intrinsic term. An allocation is currently bounded in time only because (a) allocation claims are made via deals, (b) we don’t support deals longer than a sector’s life, and (c) a sector has a maximum lifespan . When we either decouple FIL+ from the market actor (to enable alternative markets) or enable renewal/extensibility/transfer of deals, deal terms will no longer constrain the data cap term.

By leaning in to this idea we can both (1) simplify the concepts and implementation of quality-adjusted power, resolving most of the challenges that motivate the proposal; and (2) take a simple path for implementation of

Related:

These ideas build toward the

This proposal is a mechanism for removing FIL+ term limits in the context of the existing built-in actors, especially the storage market. This makes it a change that we could release to the network prior to the larger re-architecture for programmable markets. Alternatively, we could bundle this along with a larger architectural change (probably less total effort).

Storing a piece of verified data can be viewed as a network-subsidised deal (in addition to the implicit subsidy of block reward for all storage). One party to the deal is the network itself. While a verified client must bless a particular piece of data, the network will pay for its ongoing storage. This deal is independent of any other “normal” client←→provider deal, where a client can offer to pay an additional amount.

Thus, accounting for verified pieces is performed by the verified registry actor, rather than by the built-in (or any other) storage market actor. The verified registry actor maintains a record for each allocation and claim. These are somewhat analogous to a proposal and deal in the storage market, but simpler.

A verified client makes an allocation of data cap for a particular piece of data to (initially) a particular provider. An allocation specifies:

Due to the implementation of term enforcement (details below), clients should leave a fairly large buffer between the minimum and maximum term to make the allocation practical for a provider. The maximum term may be arbitrarily large, subject to policy set in the verified registry actor (details below).

When a provider claims an allocation, this obligation and benefit is also stored in the verified registry actor. A claim records:

When a claim is made, the allocation record is removed from state. Claim records persist until the provider cleans them up (to reduce their on-chain accounting costs).

A data cap allocation and claim are independent of any storage deal. So, for example:

The verified registry actor can enforce a policy on the minimum and maximum terms. Policy values are set by the verified registry root actor. Term policies include:

For example, we might set a policy minimum of 180 days (matching the sector minimum commitment) and maximum of five or ten years, or effectively indefinite. The policy constrains the values that a client may set when allocating data cap. Policy cannot change the minimum term for an already-committed piece of verified data.

We could support per-client policies. This would let the root key set different term policies for different data cap holders.

The current implementation of quality-adjusted power spreads out the power and reward boost from an arbitrary deal term over a sector’s lifespan. This behaviour introduces challenges, as the new QAP can be hard to recalculate when updating sector data. A deal term rarely matches a sector’s lifespan exactly, because the client and provider can’t coordinate that well. This proposal fixes that behaviour by aligning verified data terms with sector lifespans.

The term for a verified piece starts when the data is committed. There is no separate “deal start date”, only an allocation expiration epoch by which the data must be committed to be eligible. The sector immediately gains 10x power according to the fraction of deal space occupied by verified data.

The term for a verified piece ends when the sector expires or data is replaced. The term cannot end during a sector’s life except through explicit replacement of data, in which case the sector immediately loses the power boost from the verified data.

Aligning the boosted period with the term exactly in this way removes the spreading-out of QA power. Providers gain 10x power for each byte of verified data exactly while that data is proven. In general, providers will gain slightly more power, and hence reward, for verified pieces due to the improved alignment of term with sector lifespan. This will be more pronounced where the deal term was much shorter than the lifespan of the sector into which it was committed.

A sector’s pledge requirement is proportional to its share of power, just as today.

After the minimum term for a verified piece expires, the pledge requirement does not change, just as the power and reward boost are retained. The provider’s commitment to storing the verified piece extends until the sector’s scheduled expiration. If the sector fails or is manually terminated early, the termination penalty includes the boosted power.

If the verified piece is sealed into a new sector in order to continue the commitment beyond the original sector’s life, the initial pledge for the new sector is calculated using the network conditions at the time. As pledge requirements are generally expected to decrease, this will decrease the required lockup for continuing the commitment beyond the minimum term.

The miner actor is trusted code and can enforce the verified minimum term and the continual storage of the data.

An allocation’s minimum term is enforced by the miner actor by requiring that the sector into which the piece is sealed must have a remaining lifespan that is sufficient to fulfil the minimum term. While slightly limited in flexibility, this avoids any need for the miner actor to track the claims that are active in each sector in order to enforce penalties at otherwise-normal sector expiration events. When a sector terminates at its scheduled expiration epoch, no penalties are due.

Similarly, an allocation’s maximum term is enforced by requiring that the sector into which the piece is sealed must have a remaining lifespan no longer than that which would reach the maximum term. That is, a sector’s expiration epoch must fall between the minimum and maximum term’s of any verified claim it holds.

The longest practical minimum term for a verified data cap allocation today is 540 days, being the maximum initial sector lifespan. There is no shortest practical maximum term, as data could be snapped-in to a sector expiring arbitrarily soon (if it meets the minimum term).

There is no change to termination penalty calculations. If a sector is terminated early (continual faults, or manual termination), the provider will pay a penalty related to the expected reward earned by the sector (up to some limit).

The built-in storage market actor does not support resumption of a deal after termination, but the verified registry can support re-commitment of the same data. If the claim’s maximum term has not been reached, the provider may commit the same piece into a new sector and regain the quality-adjusted power. This resumption ignores the original expiration epoch for the allocation, but the new sector’s committed lifespan is still constrained to fit within the claim’s term.

Prior to a sector’s scheduled expiration, a provider may commit the same data into a new sector and retain the boosted power and rewards. The new sector must have an expiration epoch no sooner than the sector the data is coming from, and no later than the term limit for the verified piece. Upon a transfer like this, the new sector’s power is increased and the old sector’s power decreased by the amount corresponding to the size of the verified data, thus keeping the miner’s total power constant.

The old sector’s pledge requirement remains at its original value, even though the sector’s power is reduced. This follows existing practise of maintaining sector pledge at a high water mark, and discourages moving data around between sectors solely as a mechanism to reduce total pledge. The new sector’s pledge is calculated according to the network requirements at the time.

Verified pieces may be transferred from a sector that is faulty, as a means of preserving data stored in a broken sector. After verified data is transferred out from it, the fault and termination penalties for the old sector are reduced accordingly. If the original sector is terminated, the verified data cannot be sealed into a new sector without a new data cap allocation.

The client that initially made an allocation can increase the maximum term for that allocation or claim, up to the verified registry’s policy maximum. Increasing the term maximum doesn’t require agreement from the provider.

A verified client can extend the term for a data cap claim beyond the initial maximum by spending new data cap on it. The term maximum can then be extended by up to the policy maximum again beyond its initial maximum. [Optional] The client extending the allocation need not be the one that originally made it.

In this way, the notaries can support the indefinite extension of FIL+ power and rewards without that necessarily being the default behaviour.

A provider cannot extend the scheduled expiration for a sector past the maximum term of a piece of verified data that it holds. The provider must give up the verified power boost when processing such a sector extension. This does not reduce the sector’s pledge requirement, but does reduce penalties to be paid on any future fault to reflect the new, lower power.

In order to prevent a provider using a trivial extension as a means of escaping a commitment to store the piece for the sector’s full originally-scheduled lifetime, this facility to drop a verified claim can only be exercised in the final TBD epochs of the sector’s scheduled lifespan, and only for non-faulty sectors.

The workflows described above are independent of any market actor, and take place through interactions by the client and provider with the verified registry actor directly. Transitioning fully to this model thus requires reworking of deal client and provider workflows. However, we can proxy these mechanisms through the built-in storage market actor, thus leaving client and provider workflows untouched. The market actor will make some default selections for the additional capabilities.

At present, the built-in storage market actor directly manages data cap accounting (even though the client’s allowances are stored in the verified registry actor’s state). When a deal proposal is published with VerifiedDeal=true, data cap is deducted from the client’s allowance and attached to a specific piece of data and provider (like an allocation), but is revoked if the provider does not commit the data before some deadline. When a deal is activated, the allocation becomes irrevocable and the provider gains benefit (like a claim).

We can proxy explicit allocations and claims through existing workflows:

The market actor is also responsible for computing the verified deal weight, an input to sector power. Deal weight is calculated as deal size times duration. With data cap claim terms exactly aligned with sector lifetime, the “space-time” aspect is no long relevant, only the total deal space as a fraction of sector capacity.

The miner actor can then compute a sector’s power as SectorSize + (9 * VerifiedSize).

This proxy pattern will initially raise the gas cost of deals, but this is a transitional state.

FIL+ data cap may be viewed as a fungible but restricted-transfer token. Proxying data cap allocations through the built-in market actor is a special case of a more general delegated authority to transfer tokens, common in, e.g. the ERC-20 specification. The built-in storage market actor is approved to lock data cap allowance tokens on allocations, which are then burnt when the allocation is claimed. This smooths the client experience when making deals, removing the need for a two-step process to both allocate data cap and create a deal.

We can and should extend this approval to third party market contracts to be developed, so that they may enjoy the same benefits. A client can approve any address to allocate its data cap tokens on its behalf. Such a delegation mechanism also opens up potential for more automated allocation workflows in the future.

Similarly, a provider’s claim on the benefits and obligations of specific data cap allocation can be viewed as a non-fungible, restricted-transfer token. We can and should implement standard NFT interfaces as they emerge on the FVM (and this use case can inform those interfaces). Initially, verified piece claims are not transferrable between providers but, as we implement transfer in the future, the token transfer concept provides a good mechanism for representing this.

As for delegated data cap allowance, we might also implement delegated authority to transfer verified piece claims. This would permit the development of marketplaces and other piece liquidity mechanisms. Transfer restrictions would still require the transferee to prove commitment of the relevant data and, usually, the client to consent to the transfer.

Delegated authority mechanisms like this support optionally coupling FIL+ behaviour with market behaviour.

As an example, with FIL+ decoupled from markets it would be difficult to implement a negatively-priced deal (where the provider pays the client), because a provider could claim a verified piece allocation directly with the verified registry actor, and ignore the costly deal. Delegated authority allows a market actor to require commitment to a deal as a condition of making the data cap allocation, because the market itself will make the allocation only after the deal is published (at which point the market can require a deposit or pre-payment of funds).

We could support data cap allocations identifying pieces that are larger than a single sector. This would allow a scalable representation within the verified registry actor.

Rather than claim an entire allocation at once, a provider claims a range of the allocation for each sector by providing a sub-piece CID and an inclusion proof into the root piece CID. The sectors with sub-pieces gain power individually.

All ranges from a single allocation must be claimed by the same provider. The client could split a dataset into shards if they wish to spread it among multiple providers.

If a sector hosting one range of an allocation is lost, the provider will pay a termination penalty but can seal the same range into a new sector to regain the same power. This maintains the incentive to host a complete replica of the allocation, which is probably what the client values.

An allocation is successfully claimed if at least one range of the allocation is committed before the allocation expires. Other ranges may be committed after the expiration epoch. This provides symmetry with resumption after faults.

This technique could mesh nicely with the supersector technique from project Neutron. A miner could then efficiently claim a contiguous range of the data cap that is many sectors large, with a higher level inclusion proof.

We could migrate the verified status of all existing deals into the verified registry actor. This would prevent the verified piece benefits from expiring with the corresponding deal, and permit their term to be extended up until the maximum lifetime of the sector into which those pieces are sealed (5y from commitment). We would need to select a default policy for the data cap claim term, such as:

After migration, verified clients could raise the maximum term of each claim up to the verified registry’s policy maximum. This would require explicit messages from the clients, but no action from providers beyond extending their sectors’ expirations.

Other components and effects of such a migration include:

We could add support for the transfer of verified piece claims to other providers. In order for the original client to retain discretion over the providers suitable for storing their data, this would involve a mechanism for the client to assent to a particular transfer: a transaction with the verified registry actor, or a signed token authorizing the transfer.

In the case of allocations that did not specify a particular provider, we could allow transfer without involving the client (the client essentially declared up front that any provider is acceptable).

Similar to transferring data between sectors at one provider, the destination provider would commit the data into a sector, and then claim a transfer from the source provider. Similar restrictions would exist on the committed term of the sector in the destination provider. The obligation and power would transfer to the new provider immediately.

The pledge requirement for the verified data in the new sector should match that required for an internal transfer.

The verified registry actor tracks data cap allocations in state explicitly. Verified clients allocate data cap to a specific piece of data. A verified data allocation may be claimed by a provider that has committed matching data without regard to any storage market arrangements.

A client may make multiple allocations for the same piece (for distinct replicas), but each consume additional data cap.

The sketch below includes the optional large-piece representation.

TODO: Replace verified registry state/methods with

TODO Add methods to approve a delegate market actor to allocate data cap on behalf of a client.

No necessary public API change for basic functionality.

Exploiting the range information for larger-than-sector allocations requires a storage deal to specify the FIL+ allocation range covered, and an inclusion proof. This is a change to market API, or possibly to miner API.

Change behaviour of PublishStorageDeals and ActivateDeals to allocate and claim data cap if deal verified is set true.

Changes to the return value of ActivateDeals to return the right information for the miner actor to calculate power.

No public API change.

Changes to interactions with the market actor to pass any additional data for claiming data cap allocations, and to calculate sector power according to data cap allocations that exactly match sector lifespan.

Change extend sector expiration to check FIL+ allocations and fail if taking one past its maximum.

More disruptive changes will be introduced with https://github.com/filecoin-project/FIPs/discussions/298

Workflows: