Sometimes having enough bandwidth isn't enough. There are numerous issues related to the way data flows on the internet that may affect the quality of your live stream. Here is how to diagnose and correct those problems so that you obtain the best possible connection for your live event.
Always test your connection before your event
Testing your internet connection before your event gives you a much better idea of what you're getting into before it's too late. You'll have more options to correct issues before your setup day than once you go live. Here is more information on how to find out if your internet connection is suitable for a webcast.
To summarize, the most obvious identifiable cause of connection issues is that the infrastructure of your internet connection may not be up to the task of streaming. As a rule of thumb, DSL connections only provide--at best--1.5 Mbps upload capacity, which may be enough for a low quality, single bitrate stream of 750 kbps. If you plan on streaming in Adaptive Bit Rate (ABR) HD or to multiple devices, you will need a bonded cable or fiber connection.
Detecting Throttling
While not a problem for most business accounts, cable ISPs (especially Comcast) sometimes throttle consumer connections down when they observe significant amounts of bandwidth transfer, e.g. a webcast. This means that while a speed test will show much more available bandwidth, the ISP will reduce the effective bandwidth significantly after a short time in order to reduce your usage. Obviously, this severely affects your stream. Throttling does not appear immediately. It usually occurs within 20-45 minutes of starting the stream.
If you suspect that your ISP may be throttling your connection, perform the following test:
- Do a "control" measurement of the location's bandwidth using speedtest.net. Note the results.
- Conduct a stream with the same bandwidth that you'll use during the event for a minimum of an hour.
- Stop the stream and conduct a test with Glasnost--a web application that detects ISP throttling--selecting the Flash video option.
- Note the results.
If you suspect throttling, contact your ISP about ways to possibly resolve the issue. Unfortunately, in our experience, most ISP representatives will firmly deny throttling. Your best option may be upgrading the account or finding another ISP.
How to troubleshoot and improve existing connections
While your infrastructure may be up to the task of handling a live stream, it's possible to still experience problems when streaming. Sometimes a connection doesn't perform up to its full potential. You'll notice this when your stream experiences problems in spite of tests showing you have enough bandwidth or when bandwidth tests show large fluctuations. In these cases, to resolve this issue, you'll need to determine whether an issue is occurring on your local area network (LAN) or on the wide area network (WAN).
Detecting the location of the problem
Most often, when you are experiencing streaming issues and you know that you should have enough bandwidth, there could be a bad link in the chain of devices providing the connection to your publishing point. This cause of the problem is typically either a malfunctioning piece of hardware or network congestion clogging the route. In order to narrow down the problem, it's best to do a ping test to detect packet loss and then determine where the packet loss is occurring.
How to do a ping test
Ping is a little command that sends a "return me" message to another IP address and measures how long it took for the round trip. Doing numerous ping tests in a row can detect how much packet loss is occurring if you observe that a number of messages aren't being returned. To ping your publishing point open your terminal on a Mac or your command prompt on a Windows machine. In your terminal or command prompt window, type:
ping xxx
where xxx is either the IP address or the URL of your publishing point and press Enter. The result should look like this:
During the test, if you see a message that states "Request timeout" that means a packet was lost.
At the end of the test, you should see a series of lines telling how packets were sent to that site and how much was returned. The number next to the "ms" is the time in milliseconds. Less than 10% packet loss is acceptable and to be expected. If you are seeing more than 10% packet loss, you are experiencing enough loss to affect your live stream.
Running a traceroute
The next step is to isolate where that loss is occurring by running a traceroute to identify each hop in your stream's path to the publishing point. To do this, type:
traceroute xxx
where xxx is is the ip address or url of the publishing point. You should then see a list of servers, like this:
From there, starting with the first server on the list, ping each hop (ip address) individually. Moving out from your location, if you suddenly observe a large number of packets being lost, then you know the problem occurs before or at that address.
Possible causes of packet loss
Once you've narrowed down the possible locations it's easier to determine what the source of the problem may be.
Malfunctioning Equipment
If a piece of equipment is malfunctioning or improperly configured, it could easily be contributing to packet loss. If that piece of gear is located before your gateway, in other words is within your LAN, you may have a chance to fix, replace or bypass it. If bypassing the gear does not resolve the issue, congestion could be causing the problem.
Causes of congestion
The classic case of LAN congestion happens when you stream live video perfectly right up until the moment when your event starts. Then all hell breaks loose. This is typically caused by event related usage eating up bandwidth. While upload and download bandwidth is usually allocated independently, and viewers downloading your stream should not affect your upload bandwidth, the start of an event often initiates other bandwidth usage, such as POS systems and members of the media uploading photos and on-demand videos. Of course, if you are monitoring your stream on site, sudden increased traffic on your network would affect that experience while not affecting viewers outside of your network. However, due to background internet usage that's not readily apparent when any device is connected to the internet, it's best to limit usage to only essential users.
WAN congestion is caused by traffic outside of your network. In North America, Internet usage peaks between 9PM and 12AM EST. If your broadcast occurs during that window, your stream may be competing with users watching Netflix, Hulu, etc. elsewhere on the web. WAN congestion may occur at any time, which is why it's essential to test your connection at the same time of day, and ideally the same day of the week. Hopefully, these tests will indicate if your streaming path is subject to congestion.
If you're using a cellular connection to broadcast your stream, your ISP may throttle your bandwidth to handle more connected devices seeking service from the local cell tower. The drop in quality may be caused by the growing crowd attending your event.
Solving congestion
If you do have LAN congestion, you have two options, either kick users off your network until the your stream improves or use a dedicated connection where you don't have to share bandwidth with other users. In either case, the goal is to obtain a connection with sufficient overhead that won't fluctuate during your event.
Alleviating WAN congestion is a little more tricky. Your best option is to choose a different publishing point, which you can obtain from your streaming provider. Ideally, you should choose a publishing point closer to your location. However, if the bottleneck is occurring between between you and the closest location, your next best option is to choose a publishing point that avoids the troubled area. For example if you are in Chicago and your publishing point is in New York City, you may want to change your publishing point to Los Angeles. You can check that you've avoided the bottleneck by running another traceroute and seeing if you've avoided the troubled hop.
Lastly, engineers working for the owner of the troubled infrastructure may be able to route traffic differently so that the bottleneck becomes less congested.
The Last Resort
Finally your best option might be bypassing your internet connection altogether. We'll include options for doing that as well as how to get an internet connection to remote locations in a future article. In the meantime, if you have a challenging streaming situation, feel free to contact us about a solution. We have years of experience bringing livestream ready connections to some of the most difficult and remote locations on earth.
