<Section name="Incentivized, decentralized storage" description="">

# Centralized and decentralized storage

In our Content Addressing on the Decentralized Web tutorial, we explored how content addressing enables us to trust peers on the decentralized web. The concepts of content addressing and Content Identifiers (CIDs) are key to understanding how Filecoin verifies storage, so please go check out that tutorial now if you haven't yet done so.

On a decentralized, peer-to-peer network like IPFS, we can use cryptographic hashing to confirm that any number of peers are storing exact copies of our data. However, we're depending on the generosity of those parties and have no guarantee that our data will continue to be replicated over time. Filecoin, on the other hand, adds an incentive layer to promote long term, verifiable storage on the decentralized web.

# Incentivized storage

When we pay a storage provider in a centralized system, we place explicit trust in a specific entity (AWS, etc.) to keep our data safe and store it for a promised period. With the decentralized Filecoin network, on the other hand, we have access to a variety of storage providers (referred to as storage miners) who are competing for our business. We pay for these storage deals with Filecoin tokens (FIL), a cryptocurrency.

Although we may not be familiar with these providers in the way we're acquainted with the well-known centralized storage providers, we can place our trust in the design of Filecoin's protocol and implementations, which in turn provide us with the ability to verify that storage miners are honoring their agreements to store our data for a set period of time. Thanks to Filecoin's blockchain, all participants in the network work together to verify each deal. These consensus mechanisms are how users in a distributed network come to agreement without the need for a central authority.

# Cryptographic proofs

Proving systems, used by all blockchains, are cryptographic protocols where a prover proves something to a verifier. In the case of Filecoin, the storage miner is responsible for proving that they're storing the correct data over time, and verifier duties are shared by all participants in the network.

In this tutorial we'll explore the cryptographic proofs that Filecoin uses to verify storage, called **Proof of Replication (PoRep)** and **Proof of Spacetime (PoSt)**.

It's important to note that cryptographic proofs are different from encryption. The Filecoin protocol does not provide encryption, although you may use it to store encrypted data.

Whereas blockchains often use proofs of work (meaning computation or processing power), Filecoin's proofs are proofs of useful storage (a kind of proof of stake). Because these proofs don't have to run continuously, they're more computationally efficient and less bad for the environment.


<Quiz id={"zkou1"} />

</Section>

<Section name="Preparing and transferring data" description="">

# Preparing data for storage

Before a system file (e.g. `puppy.gif`) can be stored on the Filecoin network, it must first be transformed into a Filecoin Piece.

In the first stage of this transformation, the system file is chunked up with UnixFS to create an IPLD DAG (Directed Acyclic Graph). You can learn more about DAGs (a form of merkle tree) in our Content Addressing on the Decentralized Web tutorial. This IPLD DAG has a payload CID, identical to an IPFS CID, which represents the root of the DAG.

The IPLD DAG is then serialized to a [CAR file](https://github.com/ipld/specs/blob/master/block-layer/content-addressable-archives.md#summary) and bit padded to make a Filecoin Piece. (Bit padding adds extra bits to make the piece conform to a standard size.) This piece has a unique piece CID, also known as a CommP (Piece Commitment).

Since payload CIDs and piece CIDs are cryptographic hashes of the data itself, they're unique, with identical CIDs guaranteeing identical content. Identical IPLD DAGs will produce identical payload CIDs and identical pieces will produce identical piece CIDs, no matter who's going to store or retrieve them.

# Negotiating a storage deal and transferring data

When a client negotiates a storage deal with a miner, they're hiring them to store a piece of data, which might be a whole or partial file. Miners store these pieces from one or more clients in sectors, the fundamental storage unit used by Filecoin. Sectors come in a variety of sizes, and a client can store data up to the largest sector size per deal. (Learn more about [sector sizes and storing large files](https://docs.filecoin.io/how-to/store/prepare-data/#for-files-bigger-than-a-sector).)

A piece CID is wrapped with other deal parameters to create a Deal Proposal. The deal CID contains information about the data itself, in the form of the piece CID, the identities of the miner and client, and other important transaction details.

The client sends this deal proposal to a miner, who agrees to store their data. Once the miner has confirmed, the client transfers their data to the miner. Once the miner has the data and verifies that it matches the piece CID noted in the deal proposal, they publish the deal proposal on Filecoin's blockchain, committing both parties to the deal.


<Quiz id={"5oir0"} />

</Section>

<Section name="Proof of Replication (PoRep)" description="">

In a Proof of Replication, a storage miner proves that they are storing a physically unique copy, or replica, of the data. Proof of Replication happens just once, at the time the data is first stored by the miner.

# Filling sectors and generating the CommD

As the storage miner receives each piece of client data, they place it into a sector. Sectors are the fundamental units of storage in Filecoin, and can contain pieces from multiple deals and clients.

Once a sector is full, a CommD (Commitment of Data, aka UnsealedSectorCID) is produced, representing the root node of all the piece CIDs contained in the sector.

# Sealing sectors and producing the CommR

Next, a process called sealing takes place.

During sealing, the sector data (identified by the CommD) is encoded through a sequence of graph and hashing processes to create a unique replica. The root hash of the merkle tree of the resulting replica is the CommRLast. (Learn more about merkle trees in our Content Addressing on the Decentralized Web tutorial.)

The CommRLast is then hashed together with the CommC (another merkle root output from Proof of Replication). This generates the CommR (Commitment of Replication, aka SealedSectorCID), which is recorded to the public blockchain. The CommRLast is saved privately by the miner for future use in Proof of Spacetime, but is not saved to the chain.

The encoding process is designed to be slow and computationally heavy, making it difficult to spoof. (Note that encoding is not the same as encryption. If you want to store private data, you must encrypt it before adding it to the Filecoin network.)

The CommR offers the proof we need that the miner is storing a physically unique copy of the client's data. If you store the same data with multiple storage miners, or make multiple storage deals for the same data with a single miner, each deal will have a different CommR.

The sealing process also compresses the Proof of Replication using zk-SNARKs to keep the chain smaller so that it can be stored by all members of the Filecoin network for verification purposes. We'll learn more about zk-SNARKs in a future lesson.

<Quiz id={"zi17w"} />

</Section>

<Section name="Proof of Spacetime (PoSt)" description="">

Whereas Proof of Replication is run once to prove that a miner stored a physically unique copy of the data at the time the sector was sealed, Proof of Spacetime (PoSt) is run repeatedly to prove that they are continuing to dedicate storage space to that same data over time.

PoSt builds upon several elements created during Proof of Replication: the replica, the privately saved CommRLast, and the publicly known CommR.

# How it works

First, PoSt randomly selects some leaf nodes of the encoded replica and runs merkle inclusion proofs on them to show that the miner has the specific bytes that should be there. Then, the miner uses the privately stored CommRLast to prove (without revealing its value) that they know of a root for the replica which both agrees with the inclusion proofs and can be used to derive the publicly-known CommR.

The final stage of PoSt compresses these proofs into a single zk-SNARK. (We’ll learn about zk-SNARKs in the next lesson.)

When miners agree to store data for a client, they're required to put down collateral. If they fail a Proof of Spacetime at any point during the contract, they'll be penalized. This is a key part of the incentivization that encourages good behavior amongst all the players in the Filecoin network.

<Quiz id={"oze1z"} />

</Section>

<Section name="zk-SNARKs" description="">

Both the Proof of Replication and Proof of Spacetime processes in Filecoin use zk-SNARKs for compression.

zk-SNARKs stands for "Zero-Knowledge Succinct Non-Interactive Arguments of Knowledge." You can think of them as hashes of computations. They let us prove that a proof has been done correctly without having to reveal the details of the proof itself or the underlying data on which it's based.

# How it works

The process of creating Filecoin's zk-SNARKs is computationally expensive (slow), but the resulting end product is small and the verification process is very fast. Compared to the original proofs, zk-SNARKs are tiny, making them efficient to store in a blockchain. For example, a proof that would have taken up hundreds of kilobytes on the Filecoin chain can be compressed to just 192 bytes using a zk-SNARK.

As mentioned previously, everyone running a Filecoin node maintains an up-to-date version of the chain for verification purposes. Keeping each proof small with the assistance of zk-SNARKs minimizes the storage demands placed on each node in the Filecoin network, as well as the length of time it takes to verify a transaction.

<Quiz id={"4xdzu"} />

</Section>


<Section name="Verifying your deal" description="">

# Applications using libp2p
Once compressed, the key data needed to verify storage is stored on the Filecoin chain, a copy of which is maintained by each user running a node. This allows for Proof of Spacetime to run regularly over time.

As a storage client, you can run the `lotus client list-deals` command to list all of the storage deals proposed from that node. For example, here are the results from a node from which only one deal has been proposed:

```
$ lotus client list-deals

DealCid:    bafyreiefvrrv5j7omqzfersogg4nqzctyzj66rcmkwkbxxx5prvd5sklci
DealId:     2
Provider:   t01000
State:      StorageDealActive
On Chain?:  Y (epoch 59)
Slashed?:   N
PieceCID:   bafk4chzazx6u4luj34azuit37rlylgrcbgkaakqsjt5avsbolxale2igii3q
Size:       1016
Price:      1000000
Duration:   2744
```

Let's explore the data made available by this command:

- **DealCid**: Content identifier (CID) for the deal proposal.
- **DealId**: A unique ID for the deal.
- **Provider**: A unique identifier for the storage provider with whom the deal was made, also known as a storage miner.
- **State**: The state of the deal. This will most often be `StorageDealActive` once the data is stored and sealed. (Note that currently this will stay as `StorageDealActive` even after the duration of the deal expires or the miner fails a Proof of Spacetime, so it's important to refer to the `slashed` field for the latter case.)
- **On Chain?**: A boolean indicating whether the deal has been stored on the chain. If positive, this field will also indicate the epoch in which the data was stored. An epoch is a specific point on the chain. Lower numbers are further back in the history of the chain, while higher numbers are more recent.
- **Slashed?**: A boolean indicating whether the storage provider has failed a Proof of Spacetime. (If the miner stops storing your data, this value will change to Y and the miner will be penalized.)
- **PieceCID**: A CID (Content Identifier) representing the stored data, also known as CommP (Piece Commitment)
- **Size**: The bytes of data being stored.
- **Price**: The price per epoch in Filecoin Token (FIL) for the storage deal.
- **Duration**: The total duration of the agreed deal in epochs (one iteration of the blockchain, currently equivalent to 30 seconds.

# Compare these deals

For the quiz below, consider the following three deals:

```
$ lotus client list-deals

DealCid:    bafyreiefvrrv5j7omqzfersogg4nqzctyzj55rcmkwkbxxx6prvd5sklci
DealId:     32
Provider:   t01000
State:      StorageDealActive
On Chain?:  Y (epoch 59)
Slashed?:   N
PieceCID:   bafk4chzazx6u4luj34azuit37rlylgrcbgkaakqsjt5avsbolxale2igii3q
Size:       1016
Price:      1000000
Duration:   2744

DealCid:    bafyreiefvrrv5j7omqzfersogg4nqzctyzj44rcmkwkbxxx5prvd5sklci
DealId:     45
Provider:   t01000
State:      StorageDealActive
On Chain?:  Y (epoch 22)
Slashed?:   Y
PieceCID:   bafk4chzazx6u4luj34azuit37rlylgrcbgkaakqsjt5avsbolxale2igii3q
Size:       1016
Price:      1000000
Duration:   2744

DealCid:    bafyreiefvrrv5j7omqzfersogg4nqzctyzj33rcmkwkbxxx4prvd5sklci
DealId:     57
Provider:   t01000
State:      StorageDealActive
On Chain?:  Y (epoch 38)
Slashed?:   N
PieceCID:   bafk4chzazx6u4luj34azuit37rlylgrcbgkaakqsjt5avsbolxale2igii3q
Size:       1016
Price:      1000000
Duration:   2744
```

<Quiz id={"mmgtv"} />

</Section>


Learn how Filecoin's built-in verification methods make it possible to trust peers in an incentivized, decentralized file storage system