✨

FIP-0059 (Synthetic PoRep)

fip
title
author
discussion-to
status
type
category
created
0059
Synthetic PoRep
@Kubuxu @Luca @ @Nicola @Irene
#649
draft
technical
core

Simple Summary

This proposal presents a new PoRep protocol (Synthetic PoRep) that reduces the size of the temporary data stored between PreCommit and ProveCommit (150 epochs) from ~400GiB to ~25GiB, with no impact on security.

Abstract

Synthetic PoRep achieves reduction in used up space by reducing the set of challenges that might be chosen during the interactive Commit step from all possible challenges to some predetermined number that is feasible to precompute.

  • A Storage Provider can complete the challenge generation and vanilla proof computation before performing PreCommit on-chain thus removing layers data before the sector is pre-committed on-chain;
  • The GPU cost for SNARK generation during Commit is not significantly increased.

image

Motivation

The current interactive PoRep protocol (PreCommit + ProveCommit) requires to seal and keep a buffer of 12 layers (11 SDR layers + 1 data layer) between PreCommit and ProveCommit. This is the cost that we are targeting to reduce with this FIP.

Specification

Protocol Overview

Differences between currently deployed PoRep and Synthetic PoRep are limited to challenge generation and additional capabilities for the Storage Provider.

  1. Starting point
    1. We assume there is a sector S for which the Storage Provider completed PreCommit1 and PreCommit2 computations.
    2. This sector is not listed on-chain.
    3. The Storage Provider possesses knowledge or CommR and CommD of that sector (acquired in PreCommit2 step) and the layers needed to generate CommR
  2. Storage Provider generates “Synthetic” challenges from CommR
    1. Based on the CommR, the Storage Provider generates a list of N_synchallenges
    2. The Storage Provider computes responses for all the N_syn challenges, which take the form of N_syn vanilla proofs and saves them for future use.
  3. Storage Provider can remove layers data
    1. As the Storage Provider knows responses to all possible challenges that will be asked in the interactive step, he can remove the layers data which is needed to respond to challenges.
  4. Storage Provider publishes “PreCommitsSector”
    1. Using the same flow as today Storage Provider submits the sector for PreCommit.
    2. This establishes when in the future the randomness for interactive response will be known (PreCommitChallengeDelay)
  5. Storage Provider generates and publishes ProveCommitSectors proof
    1. Storage Provider waits PreCommitChallengeDelay (150 epochs).
    2. The randomness revealed at PreCommitEpoch+PreCommitChallengeDelay selects N_verified challenges to be verified on-chain from the N_syn challenges generate in step 2.
    3. Storage Provider takes the N_verified vanilla proofs which were generated earlier corresponding to selected challenges and computes SNARK proofs of these challenges.
    4. Storage Provider publishes the ProveCommit either in individual or aggregated form.
  6. Chain verifies proof
    1. Using interactive randomness as a seed chain generates N_verified challenges by selecting N_verified indices out of N_syn and computing them.
    2. Generated challenges are fed into proof verification.

Actor changes

  • Add two new proof types to the list of proof types that can be used when pre-committing a new sector
    • RegisteredSealProof_SynthStackedDrg32GiBV1
    • RegisteredSealProof_SynthStackedDrg64GiBV1
  • The allowable delay of the new proof types is the same as the StackedDRG proof types V1.1;
  • No changes in the PreCommit and ProveCommit methods used today
  • The ProveCommit passes the new prove type to the proof verification syscall.
  • In ProveCommitAggregate verify that all precommits share the same proof type.

Proof changes

  • New parameters
    • N_syn set to 2^18;
    • N_verified set to 176 (same as N_porep_challenges/k) ;
  • Two new registered seal proofs:
    • RegisteredSealProof_SynthStackedDrg32GiBV1
    • RegisteredSealProof_SynthStackedDrg64GiBV1
  • New challenge generation functions. For example
      • image
    1. Note that we are evaluation using ChaCha20 in the place of Sha256 (we will updated this FIP accordingly)

  • Proof construction and verification can use the same functions as today.

Design Rationale

Synthetic PoRep is a PoRep optimization which has basically no downside with respect to the status. Indeed, it would allow for more than 90% storage cost savings between PreCommit and ProveCommit. Additionally, we would have

  • No impact on the current on-chain flow,
  • No need of new Trusted Setup
  • No proving overhead on the StorageProvider side
  • No impact on PoRep security
💡
- Point out that NI can be an alternative. - More PoRep challanges gives a trade of with reduced number of synth challgenes but requires trusted setup and more proving.

Backwards Compatibility

Synthetic PoRep would become a new proof type with the same on-chain flow as current PoRep.

Test Cases

Will be included with implementation to be presented.

Security Considerations

In the current PoRep protocol if more that 3.9% of nodes in a layer are wrongly encoded, then the ProveCommit step will fail with large probability (larger than 1-2^(-10)). In the new protocol, the SP first samples a set of N_syn positions and then proceeds to sample the 176 challenges from there. In order to be able to keep the same security as before, we need that the distribution of errors in the synthetic challenge set is as close as possible to the original distribution of errors in the layer. However the adversary can try different sets to get one where the fraction of wrongly encoded nodes is smaller than 3.9%. Say for example that adversary wants a fraction of 3.49% (this will allows it to pass with probability (1-0.0349)^176 > 2^{-9} > 2^{-10}), we can show that if N = N_syn is large enough, then this is not possible. More in details, the probability that the number of wrongly encoded in the sythetic set is ≤ 0.0349*N is given by the bynomial probabilty:

P=∑i=00.0349N(Ni)pi(1−p)N−i with p≥0.039P= \sum_{i=0}^{0.0349 N} \binom{N}{i}p^i (1-p)^{N-i} \text{ with } p ≥ 0.039

and with N = N_syn ≥ 225000, then P < 2^{-80}.

Incentive Considerations

This proposal does not affect the current incentive system of the Filecoin network.

Product Considerations

This proposal reduces the hardware usage for the PoRep and therefore represents a cost saving opportunity for Storage Providers.

Moreover, Synthetic PoRep can be also beneficial in terms of sealing throughput. Today SPs need to have ~500 GiB SSD for sealing a sector. After PC1 and PC2 this storage capacity is mostly filled with the 11 layers of SDR which need to stay there for 150 epochs, before being proved at ProveCommit. With Synthetic PoRep, only a small buffer of less than 25GiB need to be kept around until ProveCommit. This means that with less than 5% more SSD storage available, SPs can start sealing a new sector right after completing PC1 and PC2 of the old sector, without need to wait ProveCommit to be over. Note that, assuming PC1 takes almost 3h and we have 150 epochs between PreCommit and ProveCommit, this result in a possible 25% additional sealing throughput.

Implementations

Implementation in progress.

Copyright Waiver

Copyright and related rights waived via CC0.

CryptoNet is a Protocol Labs initiative.