When FOX Sports experienced streaming performance issues and higher infrastructure costs during high-traffic events like the Super Bowl, they realized they needed to accommodate for increased traffic on their streaming services.
With X-Team’s help, FOX Sports reduced the cost of their cloud infrastructure and significantly improved the amount of traffic their streaming services could handle.
FOX Sports, also referred to as FOX Sports Media Group, is the sports programming division of the FOX Broadcasting Company, owned by the Fox Corporation, that is responsible for sports broadcasts on the FOX network and its dedicated national sports cable channels.
The Super Bowl broke all streaming records. An increasing number of Americans watch the Super Bowl on a live stream from a variety of devices instead of watching the event on their TVs. This posed new challenges for the streaming services of broadcasting companies. FOX faced significant streaming performance issues, including server crashes and high infrastructure costs.
FOX Sports needed a solution to optimize both performance and cost efficiency, enabling the platform to handle higher traffic volumes while keeping infrastructure costs manageable.
X-Team’s solution focused on improving scalability, reducing costs, and enhancing the user experience by migrating key services from Node.js to Go, a more efficient language for handling high volumes of traffic. The X-Team developers created a specialized toolkit to streamline development and ensure FOX Sports’ services could scale more effectively. This toolkit included integrations for NewRelic, Splunk, and InfluxDB, as well as default middleware and configuration management tools.
To handle influxes of traffic, X-Team developed a disaster recovery service to identify streaming channels that are overloaded and seamlessly reroute users to alternative streaming providers.
X-Team also designed a decisioning service to direct new viewers to the best streaming path and content delivery network (CDN) based on their IP address. This service was essential for managing traffic across multiple CDNs, ensuring that no single provider became overwhelmed.
Finally, X-Team developed a distributed configuration management (ConfigTool) service to run A/B tests and experiments to optimize performance. For example, it was used to test whether the decisioning service helped reduce load on individual CDNs, with positive results.
Initially, FOX Sports’ Node.js services struggled under 30,000 requests per second, with server CPUs crashing and restarting due to the load. However, after migrating to Go, the new services consistently maintained just 12% CPU usage and consumed only 20MB of RAM, even under the same high request load, allowing FOX Sports to handle far higher traffic volumes without disruption.
The switch to Go not only enhanced performance but also reduced cloud infrastructure costs. With more efficient resource utilization, FOX Sports needed fewer resources to handle peak traffic, leading to lower operational costs, especially during high-demand events like the Super Bowl.
X-Team’s disaster recovery and decisioning services helped ensure a smooth streaming experience. The disaster recovery service rerouted users away from overloaded channels, while the decisioning service directed viewers to the best CDN based on their location, ensuring balanced traffic distribution and minimizing service disruptions.
Using the ConfigTool Service, FOX Sports was able to run A/B tests to fine-tune their CDN traffic distribution, improving load balance and reducing strain on individual providers. This ensured that FOX Sports could efficiently manage traffic and avoid CDN failures during these major events.
FOX Sport’s collaboration with X-Team resulted in a streamlined, cost-effective solution for handling large-scale streaming demands. By migrating critical services to Go and implementing a suite of innovative tools and services, FOX Sports improved performance, reduced infrastructure costs, and enhanced the overall viewer experience.