Storage configuration for optimal PeerTube server performance

[unfa]

Hi!

I’m trying to find information about storage configuration recommendations, but the only thing I found was « don’t use a Raspberry Pi with ADSL ».

The server I’m thinking about will have 2 HDDs working in RAID1 for data safetly (though this should also double read throughput).
I am wondering if adding an SSD with Bcache is going to be a good idea, if the server starts seeing dozens or maybe even hundreds of simultaneous playbacks of various videos at once.

I expect the HDDs may get overwhelmed with random I/O volume when multiple videos are played back at once. I am guessing a Bcache would solve that problem by storing most commonly accessed data blocks in solid-state memory that’s random IO is extremely high in comparison, leaving the HDDs to stream less popular videos.

I wonder if anyone has any experience with this, and can share any thoughts on the matter.

Thanks.

[alterak]

An SSD is always a good idea! At the moment I have ordinary HDD disks, in total there are 2 3TB disks installed, I use the ZFS file system. So far, only Mirror is used, but in the near future I will buy another disk and transfer it to RAIDZ1 (this is similar to RAID5), after that I expect a performance increase.

I will never store data on extraneous resources. I always use only mine.

[unfa]

Allright, so my non-scientific test shown that playing 25 1080p videos downloaded from YouTube saturates the CPU (Ryzen 7 1700), but gives like 25% load on the hard drive (5300 RPM HDD).

It may be highly inaccurate, because the video files were recently downloaded, and would all easily fit in my RAM, so it’s possible they weren’t streamed from the disk.

If it was accurate, I’d expect 100 concurrent video playbacks to be doable on a single HDD.
And since there’s gonna be two in RAID1, it should handle 200, and it’s unlikely we’ll reach this much load any time soon anyway, as currently my YouTube channel (~15k subs) gets average of 70 playbacks per hour.

I’ll go without an SSD cache, and if need be, add that later.
I’ve also heard about dm-cache.

[2zu52kUN]

Hi, I’m also in the hw building phase.

I’m in for a 4TB RAID10 configuration with somewhat older (WD SE 7200/64MB) 4x2TB drives(on average they do about ~140MB/s steady reads with sample size 2MB up to 128MB).
I bought a few new some years ago for my NAS, and bought a few more(+spare) used for $20 each recently(although those S.M.A.R.T states are not the cleanest/but no errors, and I do backups, and will mix hose accordantly ).

Also for ZFS use(for that RAM cache benefit, upgradable with SSD cache, but lots of RAM is king, especially if you can fit the most latest/trending video’s in it).
Without any additional cache I could expect 448 MB/s @75% reads in RAID10 according to https://wintelguy.com/raidperf.pl
If you have the SATA ports/HBA for it, it can offer performance for cheap with used disks while providing up to two disks redundancy (six drives with RAID10 ~672 MB/s @75%reads or 840 MB/s @100%reads).

[2zu52kUN]

I would like to edit above message but unable to…
First test results on old HW available a.t.m, 4C/16GB-ECC-1333MT/over SataII on those four old 7200/64 2TB disks.
With ZFS pool in raid10

Network:
10GB file over Samba share to NVME (10GE/MTU: 9000) steady 430MB/s at 100% read.

Local: partial results of iozone default

Local: partial results of iozone 96g