Hello. Why peertube needs S3 object storage, if local copies of files are still in local (server hdd) /storage ?
Hello,
I’m not sure to understand as S3 is an option in PeerTube that is not enabled by default?
Hello, I’m new to using PeerTube and have a question regarding video file management. I recently uploaded a video named ‹ original.mp4 › to my PeerTube instance, which is hosted on my server. I noticed that PeerTube has transcoded the video into an HLS format and uploaded it to my remote ‹ hls › bucket.
I’m curious about what happens to the local ‹ original.mp4 › file after this process. Does it remain on my local server, or is it deleted once the transcoding and uploading to the remote bucket are completed?
Furthermore, if the ‹ original.mp4 › file is indeed deleted, does this mean all streaming traffic will be directed from the remote bucket? Additionally, does PeerTube offer a feature that allows caching (or copying) of popular videos in local storage? I’m interested in knowing if, in the absence of a cached video, PeerTube is capable of downloading it from the remote bucket upon user request.
Thank you for your assistance.
I mean a caching solution similar to what rclone offers with its VFS cache settings. The parameters analogous to rclone’s --vfs-cache-mode full, --vfs-cache-max-size 100G, and --vfs-cache-max-age 120h. We maintain up to 100GB of cached data in the default cache directory for a duration of 120 hours. The cache would operate on a ‹ first in, first out › basis, where older files are removed to make space for new ones, provided they’re not currently in use. Can PeerTube support a similar setup for caching video files?
It’s deleted from your local storage
Yes it is!
No but you can easily setup a server cache in front of S3. You can also create another PeerTube instance and use instance redundancy: Instance follows & redundancy | PeerTube documentation
Thank you very much for your help! 
Excuse me to enter this thread like this, but I’ve been trying to do this for a few weeks and can’t seem to get it right.
Would it be possible to show an example configuration for nginx (ideally working within PeerTube’s docker webserver image) similar to what is found in the Mastodon documentation?
At present, the object storage function only works for Internal, Private or Password protected videos, thanks to the PEERTUBE_OBJECT_STORAGE_PROXY_PROXIFY_PRIVATE_FILES environment variable being set to true.
Example of a video working : https://tube.fedi.quebec/w/53QyGjSXRvgS1nVy1RpHEw (password : s3)
Example of a video not working : https://tube.fedi.quebec/w/wkAGvV2UhMSamf2oVHqy4d
Thank you !
1 « J'aime »
What is your server cache domain name?
Actually, none. I couldn’t make anything work.
PEERTUBE_OBJECT_STORAGE_STREAMING_PLAYLISTS_BASE_URL and
PEERTUBE_OBJECT_STORAGE_WEB_VIDEOS_BASE_URL
are both pointing at the same place as PEERTUBE_WEBSERVER_HOSTNAME (tube.fedi.quebec) to avoid PeerTube to serve URLs of Public/Unlisted videos pointing directly to the bucket. Public egress at OVH is too expensive. But if it goes through a VPS at OVH first, it’s considered as internal egress at no cost. That’s why I want to make this reverse-proxy/cache server in front of my bucket.
I planed to use medias.tube.fedi.quebec for this as soon as I have a config that works.
I think you should setup your cache server behind medias.tube.fedi.quebec, ensure it works correctly and then set PEERTUBE_OBJECT_STORAGE_STREAMING_PLAYLISTS_BASE_URL and PEERTUBE_OBJECT_STORAGE_WEB_VIDEOS_BASE_URL to medias.tube.fedi.quebec.
That’s what I’m trying to do, but I can’t manage it. With only this on the documentation on the PeerTube site, I’m trying configurations that work with other projects, like Mastodon, but they don’t work with PeerTube.
Now I have this file for medias.tube.fedi.quebec :
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=CACHE:10m inactive=7d max_size=10g;
server {
listen 80;
server_name medias.tube.fedi.quebec;
root /var/www/html;
location / { return 301 https://$host$request_uri; }
}
server {
listen 443 ssl;
http2 on;
server_name medias.tube.fedi.quebec;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_certificate /etc/letsencrypt/live/medias.tube.fedi.quebec/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/medias.tube.fedi.quebec/privkey.pem;
root /var/www/html;
keepalive_timeout 30;
location = / {
index index.html;
}
location / {
try_files $uri @s3;
}
set $s3_backend 'https://my-bucket.s3.bhs.perf.cloud.ovh.net';
location @s3 {
limit_except GET {
deny all;
}
resolver 213.186.33.99;
proxy_set_header Host my-bucket.s3.bhs.perf.cloud.ovh.net;
proxy_set_header Connection '';
proxy_set_header Authorization '';
proxy_set_header Range $http_range;
proxy_hide_header Set-Cookie;
proxy_hide_header 'Access-Control-Allow-Origin';
proxy_hide_header 'Access-Control-Allow-Methods';
proxy_hide_header 'Access-Control-Allow-Headers';
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-server-side-encryption;
proxy_hide_header x-amz-server-side-encryption;
proxy_hide_header x-amz-bucket-region;
proxy_hide_header x-amzn-requestid;
proxy_ignore_headers Set-Cookie;
proxy_pass $s3_backend$uri;
proxy_intercept_errors off;
proxy_cache CACHE;
proxy_cache_valid 200 48h;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
proxy_cache_lock on;
expires 1y;
add_header Cache-Control public;
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET';
add_header 'Access-Control-Allow-Headers' 'range';
add_header X-Cache-Status $upstream_cache_status;
add_header X-Content-Type-Options nosniff;
add_header Content-Security-Policy "default-src 'none'; form-action 'none'";
}
}
My CORS on the bucket :
{
"CORSRules": [
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"GET"
],
"AllowedOrigins": [
"*"
]
}
]
}
(P.S. my-bucket is not the real name of my bucket)
You can check back the public video : https://tube.fedi.quebec/w/wkAGvV2UhMSamf2oVHqy4d
Whats wrong in my config?
(Canada trains <3). The video correctly works on my side 
Woa!
I’m totally confused.
I haven’t touched anything since my last failure. I guess there was a delay in propagating my config due to a cache somewhere. Anyway, great!
It works for the vast majority of people I’ve asked to test it. But there’s still this case that a friend has (Tabinol, the one who posted the video in question, I don’t know if it can be linked). He sent me these screenshots :
EDIT :
Another person reported the same problem to me.
Then I managed to reproduce the error with two Firefox (based) Extended Support Release :
- Firefox ESR (115.6.0esr)
- Mercury (115.4.0esr)
Tabinol confirms that he got the error using Firefox ESR 115.6.0esr (default in Debian it seems). After installing the version directly from Mozilla (121.0.1), it works.
The problem comes from Firefox ESR, everything points to that.
1 « J'aime »
Can you paste your nginx configuration?
I think you need to accept OPTIONS request so limit_except GET OPTIONS
You also have to forward Range header. Since you want caching, it requires more configuration: Smart and Efficient Byte-Range Caching with NGINX
I suggest to remove proxy_cache_lock on; and add
slice 1m;
proxy_cache_key $host$uri$is_args$args$slice_range;
proxy_set_header Range $slice_range;
proxy_http_version 1.1;
If the configuration works on your side, can we add it to the documentation?
1 « J'aime »
It seems to work with Firefox ESR now !
Here is my (edited with your proposal) config file for medias.tube.fedi.quebec :
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=CACHE:10m inactive=7d max_size=10g;
server {
listen 80;
server_name medias.tube.fedi.quebec;
root /var/www/html;
location / { return 301 https://$host$request_uri; }
}
server {
listen 443 ssl;
http2 on;
server_name medias.tube.fedi.quebec;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_certificate /etc/letsencrypt/live/medias.tube.fedi.quebec/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/medias.tube.fedi.quebec/privkey.pem;
root /var/www/html;
keepalive_timeout 30;
location = / {
index index.html;
}
location / {
try_files $uri @s3;
}
set $s3_backend 'https://my-bucket.s3.bhs.perf.cloud.ovh.net';
location @s3 {
limit_except GET OPTIONS {
deny all;
}
resolver 213.186.33.99;
proxy_set_header Host my-bucket.s3.bhs.perf.cloud.ovh.net;
proxy_set_header Connection '';
proxy_set_header Authorization '';
#proxy_set_header Range $http_range;
proxy_hide_header Set-Cookie;
proxy_hide_header 'Access-Control-Allow-Origin';
proxy_hide_header 'Access-Control-Allow-Methods';
proxy_hide_header 'Access-Control-Allow-Headers';
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-server-side-encryption;
proxy_hide_header x-amz-server-side-encryption;
proxy_hide_header x-amz-bucket-region;
proxy_hide_header x-amzn-requestid;
proxy_ignore_headers Set-Cookie;
proxy_pass $s3_backend$uri;
proxy_intercept_errors off;
proxy_cache CACHE;
proxy_cache_valid 200 48h;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
#proxy_cache_lock on;
slice 1m;
proxy_cache_key $host$uri$is_args$args$slice_range;
proxy_set_header Range $slice_range;
proxy_http_version 1.1;
expires 1y;
add_header Cache-Control public;
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET';
add_header 'Access-Control-Allow-Headers' 'range';
add_header X-Cache-Status $upstream_cache_status;
add_header X-Content-Type-Options nosniff;
add_header Content-Security-Policy "default-src 'none'; form-action 'none'";
}
}
I’m pretty sure it still needs a bit of improvement, but at least it seems to be functional.
I have no objection to its addition to the documentation, and even less if qualified people could improve it!
Thank you @Chocobozzz !
Just found that it won’t works for public or not listed lives. But it works for private lives (thanks again to PEERTUBE_OBJECT_STORAGE_PROXY_PROXIFY_PRIVATE_FILES) and, not tested, but pretty sure for internal and password protected lives too.
1 « J'aime »
Thanks, I think you should not use a cache for .m3u8 files, and a small cache (10 minutes for example) for .ts files.
This seems to works :
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=CACHE-S3:10m inactive=7d max_size=10g;
server {
listen 80;
server_name {{ peertube_webserver_medias_hostname }};
root /var/www/html;
location / { return 301 https://$host$request_uri; }
}
server {
listen 443 ssl;
http2 on;
server_name {{ peertube_webserver_medias_hostname }};
ssl_protocols TLSv1.2 TLSv1.3;
ssl_ciphers HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_certificate /etc/letsencrypt/live/{{ peertube_webserver_medias_hostname }}/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/{{ peertube_webserver_medias_hostname }}/privkey.pem;
root /var/www/html;
keepalive_timeout 30;
location = / {
index index.html;
}
location / {
try_files $uri @s3;
}
location ~ \.(json|m3u8|ts)$ {
try_files $uri @s3_nocache;
}
set $s3_backend 'https://{{ peertube_object_storage_streaming_playlists_bucket_name }}.{{ peertube_object_storage_endpoint }}';
location @s3 {
limit_except GET OPTIONS {
deny all;
}
resolver {{ resolver }};
proxy_set_header Host {{ peertube_object_storage_streaming_playlists_bucket_name }}.{{ peertube_object_storage_endpoint }};
proxy_set_header Connection '';
proxy_set_header Authorization '';
proxy_set_header Range $slice_range;
proxy_hide_header Set-Cookie;
proxy_hide_header 'Access-Control-Allow-Origin';
proxy_hide_header 'Access-Control-Allow-Methods';
proxy_hide_header 'Access-Control-Allow-Headers';
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-server-side-encryption;
proxy_hide_header x-amz-server-side-encryption;
proxy_hide_header x-amz-bucket-region;
proxy_hide_header x-amzn-requestid;
proxy_ignore_headers Set-Cookie;
proxy_pass $s3_backend$uri;
proxy_intercept_errors off;
proxy_cache CACHE-S3;
proxy_cache_valid 200 48h;
proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
# proxy_cache_lock on;
slice 1m;
proxy_cache_key $host$uri$is_args$args$slice_range;
proxy_http_version 1.1;
expires 1y;
add_header Cache-Control public;
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Range,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
add_header X-Cache-Status $upstream_cache_status;
add_header X-Content-Type-Options nosniff;
add_header Content-Security-Policy "default-src 'none'; form-action 'none'";
}
location @s3_nocache {
limit_except GET OPTIONS {
deny all;
}
resolver {{ resolver }};
proxy_set_header Host {{ peertube_object_storage_streaming_playlists_bucket_name }}.{{ peertube_object_storage_endpoint }};
proxy_set_header Connection '';
proxy_set_header Authorization '';
proxy_set_header Range $http_range;
proxy_hide_header Set-Cookie;
proxy_hide_header 'Access-Control-Allow-Origin';
proxy_hide_header 'Access-Control-Allow-Methods';
proxy_hide_header 'Access-Control-Allow-Headers';
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header x-amz-meta-server-side-encryption;
proxy_hide_header x-amz-server-side-encryption;
proxy_hide_header x-amz-bucket-region;
proxy_hide_header x-amzn-requestid;
proxy_ignore_headers Set-Cookie;
proxy_pass $s3_backend$uri;
proxy_intercept_errors off;
expires 0;
proxy_cache off;
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Methods' 'GET, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Range,DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
add_header X-Cache-Status $upstream_cache_status;
add_header X-Content-Type-Options nosniff;
add_header Content-Security-Policy "default-src 'none'; form-action 'none'";
}
}
1 « J'aime »
Thanks @manuelviens!
Would you like to edit the remote storage page to add a section that explains how to setup a S3 mirror for peertube?