Alex NorthSince December 2021, Filecoin protocol designers have been pursuing a vision for a platform that offers basic, composable primitives for smart contracts to interact with sector storage.
The final essential step is to eliminate constraints on the data stored in sectors: to free CommD. This will pave the way for unrestricted innovation in data & storage contracts. However, motivation for this has waned, partly due to concerns about losing control over deal-making and transparency. This post outlines the costs and limitations of the built-in market actor’s privileged position, and shows how the same goals (or even stronger ones) can be achieved without constraining sector content or deal terms. This would provide a strong and simple foundation for many and varied storage & data applications.
- Big picture
- The built-in storage market limits development and use
- Deal collateral can be done better
- Simpler by decoupling
- Potential improvements
- Free CommD
- Impact and objections
- No impact on use of FIL for deals
- No impact on transparency
- No impact on Filecoin Plus
- Other objections
- Summary
Big picture
Filecoin is a storage network. Success and impact mean abundant storage capacity and strong utilisation (these are the first two steps of Juan’s Filecoin Master Plan). Maintaining that impact requires a long-term economically sustainable protocol.
High storage utilisation needs simplicity, flexibility, and low costs. There are many different potential uses, with different needs and constraints: data DAOs, perpetual storage, automated replication/repair, novel markets, capacity deals, insurance, futures… no-one can know the full potential today. Protocol designers could attempt to build all desirable features into the network core, but that would be very expensive and extremely slow, being subject to prudent conservatism and network governance processes. Development will be much faster if the Filecoin protocol provides core storage and data commitment functionality, and then gets out of the way. (This doesn’t preclude convenient higher-level wrappers, but avoids mandating them where they aren’t fit for purpose).
Developers want a better platform on which to build storage-related applications, but the built-in storage market is inflexible, expensive, and hard to change. Its privileged control over sector data commitments stands in the way of innovation. Its friction limits onboarding. Filecoin either needs to build all features into the built-in market, or needs to move it out of the way. Removing the built-in market from critical network code will lead to a simpler and more efficient core and greater innovation potential sooner. This need not compromise any economic sustainability mechanism. Indeed, the greater flexibility and lower costs provide a foundation for far greater utility, and in turn far greater economic value.
The built-in storage market limits development and use
The built-in storage market actor acts as the gatekeeper for what data an SP can seal into a sector. Sector content is constrained to match a set of deals from the built-in market, or be zero.
A deal is conceptually similar to a contract in the traditional sense: agreement between parties to a clear set of terms. The built-in market implements exactly one set of terms, which specifies:
- a mechanism for payment (full escrow),
- a facility for client and provider collateral (owed to the network, not the counterparty), and
- behaviour in case the deal is not activated or fulfilled.
Just as important is what it doesn’t specify:
- any mechanism for renewal,
- any mechanism for transfer by either party,
- any mechanism to change the term or price (extend, re-negotiate),
- any notion to apply the same terms to more than one piece of data,
- any ability to spread large data across multiple sector,
- requirement for the provider to pay the client rather than vice-versa,
- etc.
These are the only terms that are allowed.
Some functionality can, in principle, be built on top of today’s built-in market. In most cases this is achieved by making a new deal every time something changes (e.g. extension, renewal, transfer). But this is very complicated and inefficient, both because the on-chain state bookkeeping is expensive, and because a provider must re-seal (or snap) data every time to demonstrate to the market that it has been committed.
Some functionality cannot be implemented in terms of the built-in market. For example, re-negotiation of an active deal is impossible, since the client cannot choose to release the storage provider from their obligation. Some storage-related applications just aren’t markets at all.
The built-in storage market is also expensive, and has poor scalability prospects. Chain state manipulation makes deal activation expensive for the storage provider. Slightly worse-than-linear per-deal state means that deals impose a large and growing burden of state size on all validators. Both the cost and burden will accelerate as deal-making activity increases, creating a limit on utilisation.
It is possible to invest in improving the built-in storage market in some dimensions through FIPs and network upgrades. Adding functionality will generally increase complexity and hence cost. Reducing cost will usually mean reducing functionality (e.g. FIP-0060). It is not possible to improve the market in all desirable dimensions, if efficiency is one of them. Participants would be better served by specialised storage & data applications (including DAOs, pools, derivatives, markets, etc.) that have the features they desire without overheads associated with those they don’t. And they would be served much sooner if those applications were not held back by core dev capacity and network governance.
Deal collateral can be done better
The built-in storage market sets a minimum provider deal collateral equivalent to 1% of the circulating token supply, in proportion to the deal data’s share of network raw byte power. This may be compared to sector initial pledge of 30% of the circulating supply, in proportion to the sector’s share of quality-adjusted power. The deal collateral is burnt (benefitting token holders) if the provider either fails to activate the deal in time, or terminates the sector hosting it before the deal completes. No collateral is given to the wronged client.
Deal collateral has no impact on consensus security. It does have two positive effects:
- it functions as (mandatory) disincentive to SPs for early-terminating sectors with deals (in addition to sector termination penalty), and
- it removes some tokens from circulation for the deal duration, which benefits existing token holders.
Both of these are a cost to the storage provider: an expected loss of collateral from some accidental terminations, and the cost/time-value of leasing/holding tokens for the deal duration.
From one point of view, disincentives to termination are a matter for negotiation between a client and provider. This is most relevant for non-verified deals, where the client is paying full price and provider costs increase that price. Clients and providers have no ability to reduce this, regardless of any use case or user preference for low price over high reliability. Nor can a client instead specify a collateral to be paid to them in case of failure. For applications that aren’t deals, mandatory collateral is quite out of place.
Another point of view is that general reliability is an asset to the Filecoin network as a whole; that most participants benefit from the enforcement of a minimum standard, overriding user sovereignty. This is a bit like city building codes or a state minimum wage.
As a share of locked tokens, deal collateral is trivial and could only even approach 1/30th of pledge collateral if all sectors carried data—this impact is negligible.
Simpler by decoupling
The same ends can be achieved more simply by capturing deal collateral at the miner directly, without depending on any storage market (including the built-in one). This would more simply and directly express the goals, as well as perform the only critical function of the built-in market actor.
To implement this, the miner actor could require additional pledge collateral for any sector with a non-zero data commitment (any non-CC sector). This entire “data collateral” could be burnt along with the termination fee if a sector is terminated early. (There are some complications here to eventually support re-snapping new data into sectors, which could invalidate deals; these complications affect the built-in market too).
Update: now explored in Data termination penalty—design sketch.
With this change, the built-in market would serve no network-essential function and could be safely bypassed.
Potential improvements
A simpler mechanism makes potential improvements more apparent—can the goals of deal collateral be met in other ways, or even improved?
If users and providers are free to negotiate their own deal collateral amounts via storage applications, the built-in actors need not be concerned. The token-locking benefits to holders could be replaced with with a deal fee: tokens that are burnt (paid to the network) for any non-CC sector commitment or update. This would fit in very cleanly with explicit sector onboarding fees (proposed in FIP discussion #557).
Free CommD
The essential change to enable efficient, flexible user-programmed storage & data applications is to free CommD: let an SP specify a data commitment without reference to the built-in storage market. This immediately enables alternative markets, and other applications, including off-chain. Such applications can be much better for participants by implementing the features they want, at lower cost.
Alternative markets will generally be opaque to network code: built-ins won’t necessarily be able to see or influence what data is stored, the terms of deals, the payments made etc. This can seem scary, but the protocol can be improved to grow utility, capture value, and provide a strong platform with only the minimal necessary constraints on usage.
To do this, Filecoin can capture value at the storage commitment layer, which it will always control. This can be done with pledges and fees levied per-sector, not per deal; see proposals #557 and #587) . Sectors with data are still distinguished from CC, and can pay different fees. Filecoin can capture revenue however the sectors are used, without needing to know about or constrain details of that how.
Impact and objections
Discussions in July 2021 opposed permitting SPs to specify arbitrary data commitments. Both the situation and our understanding of it have changed. Contrary to positions expressed 20 months ago, it’s now apparent that Filecoin can be a network that supports uncontrolled use of the sector capacity, and capture value anyway.
No impact on use of FIL for deals
One objection to freeing CommD is that participants may then settle deals in stablecoin, or tokens other than FIL. This objection is misguided, because participants can already do this, and the network cannot fundamentally prevent it. The idea that the built-in market controls the use of FIL for deals is an illusion.
Today, a stablecoin-denominated on-chain deal can be made on in the FVM by wrapping a zero-priced built-in deal with a different smart contract. The same has always been possible with an off-chain contract. The same goes for collaterals that might be pledged by the parties to each other. The built-in market only adds cost and friction to deals.
This objection to allowing trade in non-FIL assets arises from a view that long-term health of the Filecoin economy depends on trade being settled in FIL. Having some amount of FIL in use for deals does support demand for the token, but this isn’t the same as being critical for long-term economic health. The network can’t enforce it anyway. The protocol must be designed to foster a healthy economy regardless of settlement assets used.
There are already good reasons participants might choose to use FIL. Part of an SP’s costs (gas and protocol fees) are denominated in FIL, regardless of settlement currency. FIL might become more attractive as an asset for collateral by growing the overall economic activity and stability.
No impact on transparency
Another objection is that long-term economic health depends on economic activity being transparent to participants; opaque are bad for Filecoin as a product. Like settlement assets, the idea that the built-in market offers control here is an illusion.
The built-in market doesn’t offer much meaningful transparency today. Consider the NFT use case: a third-party aggregator pulls data into large chunks and then makes deals for these large chunks with selected SPs. The data and terms an end-user cared about are not tracked, and the off-chain “deal” is invisible. In other cases, on-chain there is no strong distinction between SPs “self-dealing” to store arbitrary data and “real” paying client deals. Many arrangements are struck with off-chain contracts.
Note that Filecoin Plus usage metrics are tracked on-chain, independently of markets. The built-in verified registry actor already records the data commitment, size, duration, and parties involved for every verified deal. This is completely independent of control over sector CommD. Since, today, almost all deals are FIL+ verified, observers would lose nothing by inspecting the verified registry instead of the built-in market. Outside FIL+, the meaningful and observable data today is whether or not a sector-level data commitment is zero, and the size & duration of a sector’s commitment. These would remain available.
Transparency is nice for metrics. But fundamental product utility like low cost and flexible features are far more important product needs. Like the commercial lease analogy, on-chain deals in the built-in market reduce the value of the storage capacity; unlike the lease analogy, they only appear to add legibility.
Much activity will end up effectively transparent anyway, if chain explorers and analytics platforms for Ethereum and other ecosystems are any guide.
No impact on Filecoin Plus
Filecoin Plus is ready for this. The built-in verified registry actor already accounts for what it needs to administer data cap and quality-adjusted power without depending on the built-in storage market. It’s ready to interoperate with any storage applications, and can even be used to directly allocation DataCap without any market application deal. Filecoin Plus is programmable.
Other objections
The block reward shouldn’t subsidise a deal market outside the Filecoin economy.
The block reward subsidises storage capacity, and use of that capacity. Filecoin can extract fees from that directly. There is no market “outside the Filecoin economy”: they all must ultimately prove a data commitment, and Filecoin can capture value at this point.
Inefficient on-chain markets are better because they burn more gas, which is network revenue.
High fees and inefficiency are a shoot-yourself-in-the-foot strategy for network revenue. Far better would be to improve functionality and reduce accidental costs, capturing revenue instead through explicit functionality-based fee mechanisms. Execution gas may become an abundant and deflationary Web3 commodity anyway; being expensive-by-design can’t last. See .
Summary
- High utilisation of the Filecoin storage network can be promoted by simplicity, flexibility, and low costs. These enable faster easier iteration on programmable storage & data applications (data DAOs etc) both by PL internal teams and ecosystem companies.
- The built-in storage market acts as a gatekeeper for data storage, but has only basic functionality, making it expensive and not scalable. It limits development and adds cost and complexity (e.g. Deal onboarding actor interactions). Participants would be better served by specialised applications, which can have desired features without unnecessary overheads.
- The only essential thing the built-in market achieves is to enforce minimal deal collateral. This could be done this simpler and better directly with the miner actor. Alternative mechanisms might achieve even better goals.
- The critical change to support rapid and open development is to remove constraints on sector data commitments. This sounds scary because it appears to give up some control and transparency. It shouldn’t be because (a) the control and transparency was illusory and (b) the network still controls the essential functionality of data & storage commitments, and can observe storage utilisation. Filecoin Plus is unaffected, and remains highly transparent.
- Filecoin can capture value directly at the sector commitment layer, without constraining market features. This would give a simpler and more flexible platform, and protocol revenue based on the core value provided by Filecoin as a storage network.
- Participants can’t be forced use FIL as a settlement currency. The economy and token value can be strengthened through driving utilisation and improving Filecoin as a platform for development.