Stretch youtube channel import over time

Is it possible to stretch full youtube channel import somehow over time? To me it looks like all videos are more or less imported at once which is quite challenging. Scaling peertube local storage is one of the biggest problems I have in operating my instance. There is a setting import.video_channel_synchronization.videos_limit_per_synchronization but this seems only to be working for synchronization of new videos in a remote channel and not the initial full import. Any suggestions?

Hi,

Initial import should respect videos_limit_per_synchronization limit :thinking: Can you try to confirm it’s not the case?

I think this value is used for initial import: https://github.com/Chocobozzz/PeerTube/blob/v5.2.1/server/lib/job-queue/handlers/video-channel-import.ts#L41

which is 1000 by default. I guess when I lower this there won’t be a full import but only an amount of full_sync_videos_limit videos and from then on only new videos in remote channel.

This is a channel import from the database. To me it looks like the videos are downloaded sequentially and not staggered (like 10 videos/hour as per default values).

We also have crashed the server with a large number of imports via API, Local storage filled before transcode and move. A GUI setting for a number to hold would be huge benefit for us. We are looking to migrate some people from Vimeo and BrightCove, but we have to do it by hand in batches because of this.

Hi,

Please create an issue on GitHub - Chocobozzz/PeerTube: ActivityPub-federated video streaming platform using P2P directly in your web browser so we can fix the crash :slight_smile: