Vtt captions bounce around all over the place

I am trying to add captions to a video on our PeerTube instance.

I uploaded the vtt file with caption start/end times that overlapped, and noticed that the text appeared in different places when the captions overlapped.

I remade the vtt file with the caption start/end times not overlapping any more, and uploaded it.

But still, some of the captions appear in the wrong place. Sometimes even on top of the video itself.

Any idea what I’m doing wrong?