Problem
The FIL+ allocation strategy is slow, costly and does not scale onchain.
Let’s think as the datacap as a non-transferable token for simplicity. Currently, on a high level the FIL+ allocation looks like the following:
- Notary “mints” tokens to the client
- Client “burns” tokens to create an allocation (on-chain)
- The SP claims the allocation (which in turns, creates a “claim”) - on-chain
Each allocation is for only one sector at a time.
Note that all of these operations can be done in batch, but there is still per-sector overhead for each allocation and claim.
Goal
We want to change the process so that clients can create a single allocation for data that will span multiple sectors.
The primary goal is to reduce the workload/cost on the client (the second step mentioned in the above section). This proposal also enables to reduce the workload/cost on the SP as well by claiming multiple allocations at once but it is not required.
Proposal
We propose the client use a commitment to represent a set of Piece CIDs (CommPs) that the client wishes to put on Filecoin.
The client can then do a single allocation for this set of CommPs.
The SP can claim set of commPs either incrementally (one by one) or by batches.
A proposed workflow for the allocation is as follow:
- The client computes all the CommPs they wish to store
- The client computes a commitment to these CommPs, let’s call it C
- The client sends all the CommPs to the SP
- The client creates an FIL+ allocation using the commitment C
- SP checks the commitment C from all the commPs, and checks against the FIL+ allocation’s commitment
Once that is done, the claims process works in the following way:
- SP creates an opening proof for each CommP they will seal into a sector. The proof is with respect to the commitment C.
- Note that the SP can create an opening proof for multiple commP at once !
- SP submits the opening proof onchain when proving a sector. The opening proof is checked against C.
Commitment
We propose to use a simple commitment called KZG commitment. It is an algebraic commitment enabling to create a constant size proof of opening (80B).
These commitments are becoming more and more used, and library support is already present. For example, danksharding or the new state trie in Eth2 will be using KZG commitments all over the place.
Constant size proofs is one major advantage, regardless of the number of allocation the SP is claiming, because it supports batch opening.
This is not the case with Merkle Tree for example.
Another reason is that having algebraic commitment in Filecoin opens the door to much better scalability and interoperability perspective. In particular, these commitment are cheaply aggregatable and verifiable.
A Merkle Tree would be cheaper on execution from the client perspective but it will be using more non indexed storage. In other word, the gas usage will be higher.
Cost Comparison:
This table shows an example for an opening proof representing 64 commP:
Onchain Proof Size
(non indexed) | 20 KB | 48B |
Merkle Tree | KZG | |
Client Commitment
Cost | 2048 hash inv.
(<< 1ms) | 1 MSM (size 1024) in 1ms (w/ blst) |
Onchain Commitment
Size | 320B | 48B |
Onchain Proof Verification | 64 * 10 hash calls | 1 pairing check + 1 MSM size 64 + O(64) field ops
→ less than 1ms |
As you can see, it’s a trade-off between onchain footprint and verification cost. We believe that for small sizes like this, onchain verification cost is still reasonable and bring the best trade-offs.
Implementation
A KZG commitment would require a precompile to cheaply verify it onchain. This precompile would use blst as the backend which already provides many of the building block to verify kzg commitment (+ many other libraries building on DankSharding on Eth2 are popping up with their KZG implementation). More work is needed to explore the space of libraries and decide if we want to integrate or use our own implementation (a whole impl. fits in < 500 lines).
Note that we could also decide to use another curve, bn254, which is faster than bls12-381 and more importantly, is available on any EVM chains ! However, this curve have less implementations out there.
Discussion
Checking commitment between client and SP
There is an alterntive process we describe here for completeness which essentially make sure the client and SP agree on C offchain first.
- Client computes the commitment C
- Client sends all the CommPs to the SP
- The SP computes the commitment C locally
- They make sure the Commitment is the same
- Client creates an FIL+ allocation using the commitment C
Because the client and SP have an incentive to work together, it might be better to make them check the commitment C off-chain first. This is up for discussion.
Interoperability with other chains
Because this proposal does not use any specific encoding (IPLD etc), a client can create his commitment on another chain or IPC subnets, as long as the precompile is enabled there.
Given we are “creating a new state” in some sort, the ability to use KZG commitment scheme (with known curves) would allow us to use it and refer to it in a different context than Filecoin. For example, it would be “possible” to create these commitments on any Ethereum-related chains (when this EIP is included or if we use BN254, already available today).
Different Poreps
In the case Porep changes, this proposal would still be compatible as it does not interpret the commP in anyway (i.e. we could change it to using blake3 for example).
If the size of the sector changes, for example with smaller sector sizes, this proposal can work as well on a larger set of commPs (e.g. pass from 1024 commP to 2048 smaller commP). In sector sizes grow larger, this proposal would work on a smaller set of commPs (e.g. pass from 1024 to 512 set size of commP).