Creator
Nicola
Created
Nov 20, 2023 3:41 PM
- Project
- Map of Efforts
- Design Space (opinionated)
- Architecture
- Parties
- High level Overview
- High level APIs
- Client
- Aggregators Market
- Storage Provider (or aggregator)
- Buffer
- Repair
- Plan
Project
Problem
- It is very complex to store data smaller than 32GiB
- There is no standardized way of doing this
- Several projects work via centralized servers, centralized credentials (username and pass) and centralized means of payment (credit cards)
Goal
Anyone should be able to experience storing data into Filecoin using a wallet and a non-proprietary protocol.
High level plan
Design a standard interface for deal-making that operates via a layer 2.
Map of Efforts
Design Space (opinionated)
Here we will collect our design goals and requirements
Protocol Design requirements
- It is not possible to do this at the layer 1 (see https://youtu.be/kSE1gZVMXHk?t=1328)
- Aggregators shouldn’t have ad-hoc APIs (composability)
- Aggregators take deals from a market with other aggregators (where clients can specify the aggregators)
- …
Design goals
- …
Architecture
Parties
- Client
- Storage Provider
- Aggregator
- Hot Stoage Market L2
- Buffer
- Repair & others
High level Overview
Data preparation
- Client prepares data locally (or delegates it - out of scope)
Buffer
- Client stores their data at the Buffer
Advertising
- Client advertises their subpiece to the Aggregators Market
- Aggregator advertises their subpieces to the Storage Provider
Piece offloading
- Storage Provider/Aggregator picks a set of subpieces from the Aggregators Market
- Storage Provider tries to retrieve the data from the Buffers
- If Storage Provider succeeds, SP generates a commitment and sends the list and the commitment to the Aggregator
- Aggregator verifies the commitment from and sends a confirmation to the SP
Status check
- The client queries the Aggregator to learn about the status of their deal
Repair
- If Storage Provider loses deals, Repair gets the subpiece from available SPs and restart the process from 1 as a client
High level APIs
Client
- Put:
- Prepares the data
- Sends it to the buffer
- Advertises to the aggregators market
- Waits until the deal is made and stores the info
- Get:
- Get the SP from the info
- Starts request to get the file
- List:
- Get the SP from the info
- Starts request to get the list of files
Aggregators Market
- Piece Collection
- (Client-facing) Put /
submit
- (Client-facing) Status /
getAllDeals
- (SP-facing) AcceptCIDs
Storage Provider (or aggregator)
- Piece offload
- (SP-facing) PullCIDs
- Routine checkChain /
complete
- Filecoin Deal Proposer API
Buffer
- (Client-facing) Put
- (SP-facing) Get
Repair
- (Client-facing) RequestService
- (Client-facing) Status
- Routine Check
Plan
- Offchain phase: The first plan is to have Repairs, Buffers and Aggregators as off-chain entities
- Onchain phase: The second plan is to turn Repairs, Buffers and Aggregators into smart contracts