Logo
    Synthetic PoRep: Implementation breakdown and timeline
    ๐Ÿ“Š

    Synthetic PoRep: Implementation breakdown and timeline

    ๐Ÿ‘จโ€๐Ÿš€
    This is a short outline of features and changes needed to ship Synthetic PoRep

    Intro

    • Between PreCommit and ProveCommit SPs have to bring over almost 400 GB for 150 epochs.
    • We want to lower down this storage requirement by 90%+ reducing it to less than 25 GB.

    Relevant docs/links:

    • https://github.com/filecoin-project/FIPs/discussions/603
    No Buffer PoRep: Removing 10 layers storage between PreCommit and ProveCommit ยท Discussion #603 ยท filecoin-project/FIPs

    @Kubuxu @lucaniz @irenegia @nicola @rosariogennaro Problem The current interactive PoRep protocol (PreCommit + ProveCommit) requires to seal and keep a buffer of 12 layers (11 SDR layers + 1 data l...

    github.com

    No Buffer PoRep: Removing 10 layers storage between PreCommit and ProveCommit ยท Discussion #603 ยท filecoin-project/FIPs
    • โœจFIP-0059 (Synthetic PoRep)
    • Embed GitHubEmbed GitHub

    Component implementation breakdown

    Actors

    • Add a new type of PoRep (Synthetic one) co-existing with the current one

    Proofs

    • New: Synthetic Challenge generation
      • Add a separate step after PC2 which generate the Synthetic Challenges Superset (245k vanilla proofs)
      • Audit
      • Implementation
    • New: write new challenge selection method for challenges to be selected among the Synthetic Challenges Superset
      • Audit
      • Implementation
    • C1 with the synthetic challenge needs to select challenges from the Synthetic Challenge Superset and not accessing the 11 layers buffer
    • Exposed APIs via Filecoin Proofs and FFI

    Lotus

    • Allow SPs to select which type of PoRep proof they want to use
    • Run the Synthetic Challenge Generation step if SP opts into it
    • Delete ~11 layers of the SDR graph and keep the Synthetic Challenge Superset only
    • Handle the transfer of synthetic challenges between sealing (i.e. challenge generation) and proof creation (ProveCommit)

    Audit

    • We need an internal audit/review of both theory and implementation - It can be done right after the proofs implementation

    Implementation Roadmap items

    Q1/Q2 Plan
    ETA
    Date
    Dependency
    Effort
    DRI
    Comments/Flags
    Current status
    Req gathering and Proof updates

    3-4w

    February 21, 2023
    Lotus

    TBD

    FilCrypto/nemo

    Backlog
    Write tests and validate the implementation - Proofs

    1w

    TBD

    FilCrypto/CNL

    Backlog
    Filecoin Proofs API and FFI updates

    1w

    TBD

    FilCrypto/nemo

    Actors updates

    1w

    TBD

    TBD

    Backlog
    Write tests and validate the implementation - Actors

    3d

    TBD

    TBD

    Backlog
    Lotus proofs integration

    1w

    TBD

    TBD

    Backlog
    Lotus actors integration

    2d

    TBD

    Backlog
    Write tests and validate the implementation - Lotus

    3d

    TBD

    TBD

    Backlog
    Gas estimation for Proof verification

    5d (if test vectors are available)

    TBD

    TBD

    Challenge generation changes the verification performance carachteristics

    Backlog
    Audit (internal only)

    1w

    TBD

    TBD

    We can start the audit as soon as we have the proofs updates done

    Backlog
    Testnet validations, hot fixes and delivery of final Release candidate

    1w

    TBD

    TBD

    Backlog
    Network upgrade + support

    2w

    TBD

    TBD

    Backlog

    Next steps

    CryptoNet will review Jake's notes @ @
    When to ship this? talk to jennifer/Lotus โ‡’ April? is lotus onboard? (@ initially, let's coordinate on slack)
    Open a channel on fil slack for Synthetic PoRep project (@)

    CryptoNet is a Protocol Labs initiative.