Meeting 11/05

Nicola

we should trust clients

can we sell this ?
I’ve done the commD’ ↔ someone must do this new commitment

comD for computation and comD for storage

The source of the problem is at the client

We currently do sha256 because it’s client efficient

Designing a Designated Verifier Market

Client stores data with a designated verifier proof of retrievability commD (algebraic)

Client is the minimum person set that cares if the file is stored ← designated verifier is at least himself
The proof could be faked by the client (but why would they want to do that)

if your application can’t trust the client → you are not sure that the data is being stored, BUT if you try to get the data back and they give it to you, you can always verify the correctness of the data yourself

An SP knows the IPFS hash of the data as well as the storage commitment (specific to a verifier), when the network asks them for a hash, then they will serve it

In other words the storage commitment is NEVER used for file transfer, but only for storage verification purposes

How do I know that there is a mapping between IPFS hash and storage commitment, you dont need to know! because the only time it matters is when you download it (and you can verify)

Problem: if some has stored a file, I can’t rely on trusting the fact that someone is already paying for that storage.

TL;DR:

There is a storage commitment, only those that want to store the file care about it

There is a file commitment, those that want to download or use it, care about it

Problem: you can’t be sure that a storage commitment is a good mapping to a file commitment

Nicola - Kuba - Nicolas

Single Commitment Vision

The idea is that there is a single PieceCID commitment

Pros:

There is a single commitment that can be used for storage, retrieval and computation
One can know if a FIL PieceCID is effectively stored

Cons:

Retrieving a PieceCID is not ideal for clients using IPFS
Computing over a SHA-256 PieceCID is very expensive for computation
Suffers in case we upgrade PieceCID commitment scheme
Binds us potentially to specific proof scheme or curve
Not backward compatible
Potential new commitment does not meet user-friendly practical requirement

Next steps:

Either, accept current time to prove statements about IPFS CIDs (close to impossible)

Or, change the IPFS CID to be Snark friend and change the Filecoin PieceID to be that (requires research, difficult)

Multiple Commitments Vision

👉

Thesis: There are multiple PieceCID commitments, one optimized for each usage, even more than one for usage.

For example, a streaming merkle tree is better than balanced merkle tree for videos, hence one might opt for two retrieval commitments instead of one.

The link between the two can be:

Trusted
Verifiable

Pros:

One commitment for each usage
Ideally even more than one depending on application

Multiple Commitments (Trusted)

In this world, we trust the client for having computed the right commitments.

Pros:

No much work, it is already working like this

Cons:

Cannot be known with 100% certainty if a CID is stored into FIL, unless it’s the PieceCID, however, if the client is trusted, then they can trust the link between PieceCID and other commitments

📝

Example: Solana stores block data into FIL and writes the retrieval hash and the computation hash in a trusted channel (e.g. deal info label, mapping smart contract). Those that want to use Solana data use the specific trusted hash

Add support for other CIDs into the deal (potentally a new standard for the label, or an external contract that creates the mapping, or multiple contracts)

Multiple Commitments (Verifiable)

There are two types of verifiability:

Perfectly Linked: 100% confidence that the link between PieceCIDs and other commiments are correct (unpractical today, for large sizes - especially for PieceCIDs using SHA)
Weakly Linked: not 100% linked.

Pros:

Anyone can create the links using whatever technology of their preference

Cons:

Highly likely expensive and neither the user nor the SP will create these links

📝

Some users independently store Wikipedia pages, you want to contribute and you don’t know what pages are missing using the Retrieval CID only - so you check on the filecoin network what data is missing.

Next steps:

Potentially change PieceCID commitments to something that is easier to prove.
Find a way to weakly map SHA256 commitments into other commitments that are easier to prove.

Goals

Map IPFS CID into PieceCID (doesn’t have to be SNARK and can be interactive)
Find an example and pick a commitment scheme that we use for provability and generate a proof of linking between this commitment and the pieceCID (ok to be offchain & ok to be weakly linked)