Lotus changes to the storage subsystem + FSM changes

Date

October 24, 2021 → January 4, 2022

Assign

@ToBeRemoved @, @Łukasz Magiera

Status

Completed

First-timer

ToBeRemoved

Property

Definition of done + expectations for this milestone:

The lotus-miner binary's proving logic is organized into roughly two layers, the FSM which handles chain and deals subsystem interaction and implements configurable operation logic, and the sector storage layer which manages sector data and associated files, calling into the proving ffi as needed.

This issue is to add FSM support to the lotus miner for recording all of the application logic around doing snap deals replica updates. This means crafting new states and a path through them in the FSM taking sectors from empty to provably containing snap deals and recording this on-chain.

Done Criteria:

Most fundamental criteria: one integration test checking that snap deals can be added in the happy path

A basic methodology exists for deals to come into the system and be added to snap deals. The basic requirement is that storage providers be able to mark sectors as cc upgrades as exists today to accept snap deals. The debate about the further scope that people want should happen in another issue but should be discussed with all Filecoin stewards

The FSM path should guide sectors through the pipeline from

1) marked for snap deals
2) data added
3) sector-storage system performs an update, proof, and returns proof data
4) communication with the chain
5) cleanup of old sector data.

Of course, we need to handle error cases too for the entire flow

Dependencies:

This PR needs the following things done before it can be completed

change specs-storage interface: Update Interface for snap deals specs-storage#17

integrate ffi backend into lotus: feat: add API and tests for Empty Sector Update support filecoin-ffi#208

Embed GitHub - Update Interface for snap deals

Ongoing work can be tracked by checking:

Issues:

FSM changes that we need to make for Snapdeals implementation · Issue #7515 · filecoin-project/lotus

We need to change the pipeline in order to implement changes for SnapDeal

github.com

FSM changes that we need to make for Snapdeals implementation · Issue #7515 · filecoin-project/lotus

sector-storage supports snap deals · Issue #7525 · filecoin-project/lotus

This issue will be used to track our design + implementation efforts to create a more flexible way to sector storage for SnapDeals Direct DRI to deliver will be: @magik6k will do the design + @ZenGround0 will be involved @ZenGround0 will...

github.com

sector-storage supports snap deals · Issue #7525 · filecoin-project/lotus

Relevant PR's

- PR that will introduce Snap deals support for sector storage by adding a new file type - #7615

Feat/snap deals storage by ZenGround0 · Pull Request #7615 · filecoin-project/lotus

Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes.

github.com

Feat/snap deals storage by ZenGround0 · Pull Request #7615 · filecoin-project/lotus

Notes and remarks for this milestone

This isn't blocking the current implementation path but still needs to be done within the SnapDeals project

we are really behind the original estimation, as we discussed In Madeira call.
@ToBeRemoved and @Łukasz Magiera will pick up this line of work after we integrate lotus and actors work relevant for SnapDeal implementation.

We have been investigating and have a plan at this point for FSM, sector storage is a work in progress

Implement snap deals without supporting updating snap deals with new deals

We are going to do the MVP, which means not going to do this currently

Relevant docs

Deal Inputs for the Design
how we started the discussion + got feedback from Molly as product representative is in this doc:

Snap Deals Lotus FSM Design Options

Last modified by Jennifer Wang 4 years ago

docs.google.com