What is the best storage layout for a Chia mining rig?
The answer probably is "multiple independent drives".
Background
Chia, the latest fad in cryptocurrency (as of May 2021), uses disk space instead of computational work as proof of commitment.
The equivalent of mining in Chia is to produce many gigantic files full of something, let these files fill all the disk
space you are willing to commit, and then sit and wait.
This commitment is called "proof of space and time", proving that you want to sacrifice large amounts of disk space
for an extended period.
The reward probability increases with larger disk space and longer time.
Chia "mining", or "farming", process
While Bitcoin is said to be "mined", Chia is said to be "farmed".
The process consists of two distinct stages.
Plotting
TThe first stage is to generate the data files (called "plots").
The minimum plot size is 100 GB per file.
A single 100 GB plot file generation requires about 2 TB of writes to temporary storage.
The peak temporary space requirement for a plot is about twice the plot size, 200 GB of temporary space for a 100 GB plot.
Because of the I/O requirements, the temporary space must be really fast.
A ramdisk is your best choice if you have at least 256 GB of RAM.
The other option is one or several SSDs.
Farming
Farming Chia consists mainly of waiting.
After creating the plot files, you are supposed to put them onto slow storage for relatively infrequent reads.
If you get lucky, whatever is inside one of the plot files will sometimes produce a small amount of Chia money.
Storage system layout
The requirements for the long-term storage system for a Chia farm are approximately as follows:
- maximum capacity;
- ability to fully utilize disks of different sizes;
- some fault tolerance.
Some fault tolerance would be nice because, while you can regenerate the plots, the regeneration costs a non-trivial amount of time and SSD endurance.
We should also remember that, within reason, read and write speeds are not a factor.
Any single hard drive is fast enough to satisfy the farm requirements.
It would be a bad idea to daisy-chain several hard drives over a single USB 2.0 port,
but a SATA-connected or even an USB3-connected hard drive will work fine.
So, for N disks of equal size, the commonly used options are:
- Multiple individual disks;
- RAID0;
- RAID5;
- Some complicated system like ZFS or Microsoft Storage Spaces, with or without fault tolerance.
If you have a mix of several disks of different sizes, let's say in sets of three or five or so,
choices become slightly more complicated:
- Multiple individual disks;
- Multiple RAID0s, one for each set of the same-size disks;
- Multiple RAID5s, one for each set of the same size disks;
- ZFS or MS Storage Spaces, with or without fault tolerance.
Let's examine these options.
Multiple individual disks
With N multiple individual disks, the resulting system has a capacity of these N disks.
However, there is a slight issue with capacity utilization.
The Chia plot files are relatively big, with a minimum being 101 GB.
Let's say you have a 1000 GB disk and need to put as many 101 GB files on it as possible.
You can only fit 9 for 909 GB, wasting about 91 GB of slack space because you can't fit a 101 GB plot file into 91 GB.
As the number of disks grows and the size of a single plot grows, this may become a problem.
This effect is more pronounced with smaller disks.
There is no fault tolerance in using multiple individual disks.
If one disk fails, its contents are lost.
There is no requirement to have all disks of the same size.
You can mix and match individual disks in any way.
RAID0
RAID0 has the same capacity as N disks, with no overhead.
At most, one plot file worth of capacity is lost to slack space for all N disks instead of one per disk.
Fault tolerance is, however, abysmal.
If one disk fails, all the contents of the entire array are lost.
Different disk sizes can be accounted for by grouping disks of the same size into their separate RAID arrays.
The space loss to slack increases as the number of distinct RAID0 arrays increases.
RAID5
RAID5 uses one of the disks to protect against a single disk failure.
Thus, the total capacity is equal to N-1 disks, and the loss to slack is one plot file worth of capacity.
The fault tolerance seems reasonable for a Chia farm, allowing for a single disk failure.
Different disk sizes can be accounted for by grouping disks of the same size, similar to what one does with a RAID0 setup.
However, the capacity loss is more pronounced because each RAID5 uses one disk worth of capacity.
ZFS or Storage Spaces
ZFS is different from Storage Spaces in many aspects.
However, I group both because of their ability to work with multiple disks,
producing their own equivalent to RAID in their own manner.
You can set up ZFS and Storage Spaces (the latter in combination with NTFS or ReFS)
to produce RAID0 or RAID5 equivalent, depending on the exact configuration.
Therefore, the considerations discussed above for RAID0 and RAID5 also apply here.
The downside of both of these systems is their relative complexity.
ZFS has many bells and whistles you do not need for a Chia farm, like checksumming, snapshots, and such.
A Storage Spaces pool requires precise handling to lay out the pool and spaces,
or else you end up with one-third of the capacity lost to overhead.
The other significant factor is that maybe your existing system has readily available spare
capacity.
The choice
If you have an existing system, go with it.
Utilizing free space on the existing system is the only case to use ZFS,
Storage Spaces, or anything complicated at all.
In almost all other cases, individual disks seem to be the best option.
The fault tolerance is such that if one disk is lost, the plots on it are lost.
The rest of the system continues to function.
However, on the RAID5 system, you would not have had these plots in the first place.
The capacity of one failed drive is just not available in a RAID5, so RAID5 does not provide any benefit over individual disks.
Filed under: RAID.
Created Friday, May 21, 2021