We help IT Professionals succeed at work.

Understanding Streaming Issue

Today I used Facebook Live with Wirecast's presets of 4 Mbps and 30 fps. In an hour I had over 65000 drops frames. My camera is set to 29.97 which I guess explains some of the dropped frames, but when I viewed my completed video it was only 36 minutes and the audio was at least 3 minutes behind the video. Is this because I don't have enough bandwidth or is there another problem with my computer? I never noticed my processor nor memory maxxing out.
Comment
Watch Question

Consultant
CERTIFIED EXPERT
Commented:
First some questions:
Which compression codec are you using?
What are your encoding options set to (audio bit rate, video bit rate, compression settings, etc.)
What is the format of the video and the audio used in the recording?
Is the audio always the same amount out of sync, or does it become more out of sync over time?

Interesting about the dropped frames, as they equate to approximately 36.1 minutes of lost content. That might explain your 36 minutes left of the approximately 1 hour video. I am wondering where in the process they were lost.  And when you watched the video, was the entire program there, just compressed in time?

3 minutes sync error sound like an early on failure as it should never be that far out to start. Is 3 minutes accurate, or are you exaggerating a bit here? :-)

There are three typical types of failures. 1). Audio ahead of video, or video ahead of audio. 2). A part of the video or audio is damaged, so sync slips. 3). The video's framerate has been changed but the video player thinks it is something else, so playback will skew over time..  There are some programs out there that attempt to fix this problem, but I have never used them.

In general for AV encoding, sync errors can happen right at the beginning of a video, or slowly over time, so the fixes are a bit different. Here are some ideas:
1. Make sure the output framerate setting matches the source's framerate, and pick a constant framerate. For example: if source is 29.97 fps video, put the same frame rate or 29.97 into the desired frame rate setting for the output.  Avoid variable frame rates as they make the problem even worse. Typically audio is a constant rate, but avoid variable rates here too.
2. Set the audio sample rate same as with video frame rate, and be sure the encoding value matches the source audio sample rate.
3. If the audio is going out over time, or slowly loses, then corrects itself every few seconds: Set the keyframes (a full frame of video and audio) value to a lower figure. A typical keyframe rate is 300, which in 30 fps material, is one keyframe every 10 seconds. Try lowering the value to every 3 seconds or 90 frames per second.

VLC player has a video/audio sync correction, but not for 3 minutes. But you might try it.  In VLC, if your audio requires to be slowed down then hit the ‘K‘ key. For the opposite of this hit the ‘J‘ key. For Mac, the keys are ‘G‘ and ‘F‘. Confirmation is displayed on the top-right area of the player’s screen. It fades away quickly. Pressing the key once will speed up or slow down the audio by 50 milliseconds. If you continue to press it, the number of milliseconds will continuously be incremented or decremented.

Author

Commented:
When I say three minutes it is a conservative estimate. For the last three minutes of the stream that speaker was not on stage and the video cuts off when I stopped the stream.
For the first 40 minutes real-time there was no audio at all because it wouldn't stream. When I did get it to finally stream then what I was hearing going out was in sync with the video.
Owen RubinConsultant
CERTIFIED EXPERT

Commented:
That is fine, I was just trying to save doing math to see if I could figure out why.

What player are you using to watch the video? Have you tried VLC?

Author

Commented:
I did it by accident. I reported my comment after I realized that I marked it as solved. I did not discover anything yet.
Owen RubinConsultant
CERTIFIED EXPERT

Commented:
Thanks. Took me by surprise!  :-)

Author

Commented:
Alright, so the video displays as being 34:51 in Facebook Videos and the true length is 1:01:06.  As far as codecs, I used the Wirecast defaults.  I know it defaults to 720p30 and 4 mb.

Author

Commented:
Everything I read says that Wirecast's default settings should work with Facebook Live, but mostly selecting their defaults, I cannot change anything, so I know it is doing H.264, 44.100 on audio, along with 128kbps.

Author

Commented:
This is the streaming video that is messed up.  I know that there are skips in it, but my original estimate was wrong.  He stopped talking over 5 minutes before the video ended.  
https://www.facebook.com/westminstercalifornia/videos/352490802022229/ is the bad video.  I re-streamed the audio with a mostly empty room with full audio: https://www.facebook.com/westminstercalifornia/videos/1994290007544906/
Owen RubinConsultant
CERTIFIED EXPERT

Commented:
But was the audio ever correct, even at the beginning?

I am failry sure that all those dropped frames have added up to a lot of sync errors. I do not know that codec you used, but if it does not do keyframes (a place where a full video frame is saved) often enough, sync can be a big issue, especially with that many dropped frames.

Back of envelope, the difference between 29.97 fps and 30 fps can cause some errors:
29.97 fps in one hour is 107892 frames  (basically, frames per hour)
30.00 fps in one hour is 108000 frames  (basically, frames per hour)
In one hour, the difference is about 62892 frames, not far off from your dropped frames count. So a difference between the 29.97 and 30.00 frames IF the playback speed is wrong can be very big.
Think what 62829 frames at 30 fps = approx 2094 seconds, or 34.9 minutes difference.

I am guessing the playback rate may be wrong, causing you to "loose" that much time, and get way out of sync.

Author

Commented:
The video that ended up on Facebook was never in sync. The video that I saw being sent from the computer was in sync. So, I guess, my question would be, are dropped frames on input or output? In other words, am I dropping frames from the capture or on encoding?
Owen RubinConsultant
CERTIFIED EXPERT

Commented:
I guess I would have to see a diagram of how it is all connected and study the pieces. But fairly sure that the two systems were running at different rates, which caused problems. Any place where the frame rate changed can be a cause of dropped frames. I am not sure in your set up where it went from 30 to 29.97, but I suspect the encoding.

In the future, they should all be at the same video rate and you can always transcode (to a different rate) after you have a good master.

Can you get access to the file on the computer to encode again?
Owen RubinConsultant
CERTIFIED EXPERT

Commented:
As a side note,  I tried to convert some HD video into SD video, and had the creeping sync errors, and for me, it was.not using the same video rate as the source. But what happens when you can’t use the same rate?

Just as an example: Frame rate was a problem when converting film, at 24 fps to video at (approx) 30 fps for TV. It required encoding trick called 3:2 pull down. This is a great example of how things might go badly, and a clever trick to get around it. The conversion really shows up in a panning scene where you can see a small stutter in the video that was not in the film. Otherwise the film would run silly fast.

C28D627F-260D-4D1F-B24F-7239C8877AD.jpeg
Similar problem was seen when British TV (PAL) is 625 lines at 50 fields/s (25 frames per second) is converted to  American TV (NTSC) is 525 lines at 59.94 fields/s (60,000/1,001 fields/s). Aside from the line count being different, converting to a format that requires 60 fields every second from a format that has only 50 fields poses difficulty. Every second, an additional 10 fields must be generated—the converter has to create new frames (from the existing input) in real time. Often, British TV in America runs a bit faster than in Britain.

Just a few examples to show that this has been a problem for some time now.

Author

Commented:
The video was encoded live with no recording existing apart from what was sent to Facebook so I cannot encode it again.

My camera itself runs at either 59.94 or 29.97 fps and Wirecast was configured for 30 fps. The video should have come off my camera, gone to Wirecast, then gone to Facebook. OBS Studio and FMLE do not seem to have any issues, apart from lines in OBS if the people move, but no dropped frames.

Author

Commented:
The bitrate is automatically set at 4128k in Wirecast, which makes sense.  It is using 4000k for video and 128k for audio.  I was also wrong on the audio sample rate, which is 48.000 not 44.100.
wirecastpresets.jpg

Author

Commented:
In the end, I changed Wirecast to my own presets of 29.97 fps and 1 Mbps and everything works with 1 total frame dropped.

I think the combination of 4 Mbps and 30 fps that Wirecast uses simply overloaded the computer.
Owen RubinConsultant
CERTIFIED EXPERT

Commented:
Excellent. I am actually surprised in the setup you described that you should have had such problems, so maybe it happened after it got to Wirecast, but happy that you were able to get things working. These kinds of errors are often a surprise, and actually Wirecast should have been able to detect your incoming stream info and transcode it properly.

Cheers