Onboarding gas, maximum onboarding rates and aggregation

Average Sector onboarding cost without deals: 27.7e6 (PreCommitBatch per sector) + 103e6 (ProveCommit) = 130.7e6 gas

Average Sector onboarding cost with a single deal:

20e6 (PreCommitBatch per sector) + 103e6 (ProveCommit) + 204e6 (PublishStorageDeals per 1 deal) = 334.7e6 gas

Ideal gas available in a single day: 24*60*2 * 5 * 5e9 = 72e12 optimistic empirical correction for imperfect message packing: 72e12 * 21/25 = 60.5e12

Max daily sector onboarding in the current network, assuming no other gas usage than what is necessary for onboarding:

without deals: 60.5e12 / 130e6 = 378e3 which is 14PiB/day assuming 32GiB sectors.

with deals: 60.5e12 / 334e6 = 180e3, which is 5.51PiB/day with 32GiB sectors or 8.26 PiB/day assuming a 50/50 split of 32 and 64GiB sectors (which is the split we observe). Right now, the network onboards ~6PiB/day.

These numbers must be lowered by gas consumed by maintenance (WindowPoSt consumes ~10% of chain bandwidth) and other operations that happen on-chain, lowering the onboarding capacity to 12.6PiB/day and 7.4PiB/day correspondingly (assuming only 10% WindowPoSt).

Aggregation is dysfunctional due to the additional sector and deal activation costs, making aggregating more expensive than single-sector onboarding.

Conclusion

Our onboarding capability is 12.6PiB/day for CC and 7.4PiB/day for sectors with data. The real onboarding capability is probably the ~6PiB/day we observe in the network.

Aggregation is completely unused due to a protocol bug when single onboarding delegates parts of the accounting to cron. This makes aggregation more expensive than single onboarding (170M for aggregate onboarding vs. 100M for single proof commit). This is being worked on as part of DDO.

We can assume it will be fixed as part of DDO, but the fix in DDO makes single onboarding pay the same costs, resulting in even lower onboarding capacity.

At maximum aggregation is capable only of saving the gas for proof verification and proof size (I’m assuming infinite savings of these two)

Gas for proof verification: 34721049 = 34.7e6

Gas for proof size: 192b * 1300 gas/b = 0.25e6

Gas affected by aggregation: 34.7e6 + 0.25e6 = 35e6

Without deals: 35e6 / 130e6 = 26% With deals: 35e6 / 334.7e6 = 10%

but gas for onboarding is lowered by the cron subsidy, which is going away

~30M gas for sector activation

~130M gas for deal and claim activation

Without deals: 35e6 / 160e6 = 20%

With deals: 35e6 / 464e6 = 7.5%

How the numbers in this section look like depends on DDO implementation; this is just an extrapolation.

Despite this, it is highly unlikely that aggregation will enable more than 30% additional onboarding capacity.

FIP-32: pre commit: 3.12x to 18.55M → 57.8M gas

prove commit: 1.25x 52.25M → 65.3M gas

FIP-57:

pre commit: 46352972 to 48949895 1.05

prove commit: 63102326 to 75481067 1.19

Data over last month (2023-08-21 - 2023-09-19) based on Lily data in BigQuery.

Batch PreCommit no deals:

[{
  "sum_batch_size": "51266",
  "sum_gas_limit": "300276061472",
  "avg_gas_per_precommit": "27701571.557284474",
  "avg_batch_size": "44.118760757314988",
  "count": "1162"
}]

Batch PreCommit with deals:

[{
  "sum_batch_size": "120723",
  "sum_gas_limit": "2087925973893",
  "avg_gas_per_precommit": "19930440.065211494",
  "avg_batch_size": "10.53429319371728",
  "count": "11460"
}]

Batch PreCommit overall:

[{
  "sum_batch_size": "171989",
  "sum_gas_limit": "2388202035365",
  "avg_gas_per_precommit": "20645861.931301568",
  "avg_batch_size": "13.626128981144035",
  "count": "12622"

ProveCommit:

[{
  "sum_gas_limit": "370868769202422",
  "avg_gas_limit": "103040368.94348519",
  "count": "3599257"
}]

ProveCommitAggregate:

[{
  "sum_batch_size": "48954",
  "sum_gas_limit": "4461317294337",
  "avg_gas_per_sector": "169999496.26351941",
  "avg_batch_size": "45.160516605166045",
  "count": "1084"
}]

PublishStorageDeals:

[{
  "sum_batch_size": "3453233",
  "sum_gas_limit": "578764162178114",
  "avg_gas_per_deal": "204993231.44796419",
  "avg_batch_size": "8.6474589004219613",
  "count": "399335"
}]

During Hyperdrive:

PreCommit:

[{
  "sum_gas_limit": "1027384137929502",
  "avg_gas_limit": "20829362.392090783",
  "count": "49323840"
}]

Batch PreCommit

Prove Commit:

[{
  "sum_gas_limit": "4466686370450686",
  "avg_gas_limit": "63853414.437638626",
  "count": "69952193"
}]

ProveCommitAggregate:

[{
  "sum_batch_size": "73426287",
  "sum_gas_limit": "542145209670373",
  "avg_gas_per_provecommit": "25148213.833912864",
  "avg_batch_size": "58.3095390113162",
  "count": "1259250"
}]

PSD:

[{
  "sum_batch_size": "428402",
  "sum_gas_limit": "14383186127620",
  "avg_gas_per_deal": "40193626.240020409",
  "avg_batch_size": "5.8693245650089034",
  "count": "72990"
}]