Move to object storage failed, script not working

PeerTube
1

For some videos on my instance uploads to object storage fails with:

Job: 254 Type: move-to-object-storage Processed on 7/20/23, 5:54:56.155 AM Finished on 7/20/23, 5:55:19.994 AM
--
{   "videoUUID": "3773fafa-6c3b-4791-aea8-57f96660dd53",   "isNewVideo": true,   "previousVideoState": 2 }
Error [ERR_STREAM_WRITE_AFTER_END]: write after end     
at new NodeError (node:internal/errors:387:5)     
at ClientRequest.end (node:_http_outgoing:968:15)     
at writeBody (/app/node_modules/@aws-sdk/node-http-handler/dist-cjs/write-request-body.js:22:21)     
at ClientRequest.<anonymous> (/app/node_modules/@aws-sdk/node-http-handler/dist-cjs/write-request-body.js:9:13)     
at ClientRequest.emit (node:events:513:28)     
at HTTPParser.parserOnIncomingClient [as onIncoming] (node:_http_client:635:11)     
at HTTPParser.parserOnHeadersComplete (node:_http_common:128:17)     
at TLSSocket.socketOnData (node:_http_client:534:22)     
at TLSSocket.emit (node:events:513:28)     
at addChunk (node:internal/streams/readable:315:12)     
at readableAddChunk (node:internal/streams/readable:289:9)     
at TLSSocket.Readable.push (node:internal/streams/readable:228:10)     
at TLSWrap.onStreamRead (node:internal/stream_base_commons:190:23)

After running create-move-video-storage-job the video got uploaded successfully to object store but master playlist is still on local storage. The video is now stuck in TO MOVE TO AN EXTERNAL STORAGE state and can’t be played.

I tried downloading the video files from object store to local store, reset state in database and run the script again - without success. The video is stuck in mentioned state, videos got uploaded and master playlist is still in local store.

Any help with this is highly appreciated.

2

I still have this error from time to time. It looks like it happens more often when there is load on the system/object store. I bumped related dependencies to the latest version:

    "@aws-sdk/client-s3": "^3.414.0",
    "@aws-sdk/lib-storage": "^3.414.0",
    "@smithy/node-http-handler": "^2.1.4",

Let’s see if this brings some improvement.

3

I have exactly the same error ERR_STREAM_WRITE_AFTER_END every time I try to use create-move-video-storage-job command. In my case files are not moved and the video still works correctly from local storage, but peertube starts to display annoying warning that video may work incorrectly as files failed to move.

I do not see any explanation in logs. It’s not related to load of object store as it’s close to zero in my case. I use minio S3 on the server.

Uploading new files works correctly.

4

I haven’t had problems since I updated dependencies. I think they are included in upcoming Peertube release.

1 « J'aime »
5

Thanks! I will patiently wait for new release then.

6

I’ve just upgraded to version 6.0 and all problems with minio/s3 gone. Thanks!!