Imagine you run an online comedy platform, a destination for in-the-moment, live humor video streams. Your users are individuals who find humor in everyday situations out on the street, in the office, at a restaurant…you name it, they find a way to capture it.

When your users spot a laugh-worthy situation, they expect to start streaming live using their mobile devices within seconds of taking their phones out of their pockets, or risk losing the opportunity to stream the moment.

Unfortunately, imagine your app takes 45 seconds on average to allow a user to start live streaming, as your transcoding provider needs the time to “boot up”. The start-up time delay causes your users to miss streaming humorous events, which in turn results in you losing users daily.

When it comes to time-sensitive live events, this transcoding session startup delay is a dealbreaker.

Many live streaming platforms need to get content out immediately to reduce video latency, but starting a new livestream can be a time consuming process.

Though latency can accrue at all different steps of the 

, including the video encoding pipeline, ingest operations, packaging operations, broadband transport speed, content delivery network (CDN), and video player, the process of starting a video’s encoding session can be one of the biggest contributors to end-to-end latency.

Typically, an encoding or transcoding session begins with an API call to a transcoding provider of choice. Even if one has programmatically configured this API call, and it happens instantaneously as the live video starts to be transmitted, it can take up to 40 seconds for encoding providers to assign the video to a working transcoder. The result - a video starting with 40 seconds of delay, or more,

ude latency added in other portions of the live video streaming workflow.

The reason for this delay can be largely attributed to transcoding server “boot up” time. Typically, transcoding providers use Google Cloud or AWS to run their encoding software and pay for virtual server space from said companies only when necessary. This translates to an infrastructure that “boots up” only when a transcoding request comes in, a process that involves setting up internal servers from a “server image” or a copy of the server with all necessary settings, data files, and application configurations to transcode your video.

An imperfect solution to this problem is to pay for and reserve as many servers as you think you might need 24/7. This is not recommended, as it would be a lot more expensive and a waste of computational resources.

Thanks to Livepeer’s global GPU infrastructure, reserving server space 24/7 for video transcoding is not a solution anyone needs to resort to anymore.

Livepeer leverages a readily available global network of always-on, incentivized transcoding providers that eliminate server boot-up latency from the video transcoding pipeline. Transcoding begins the moment a request hits our API, as there are always transcoders waiting and ready to take on transcoding “jobs.”

How are we able to provide instant, pay-as-you go transcoding?

Livepeer has discovered a new, underutilized source of transcoding capacity - cryptocurrency mining equipment. There are thousands of “mining farms” distributed all around the world, each equipped with thousands, or hundreds of thousands, of GPUs typically used for cryptocurrency mining. Within every one of these GPUs exists a video encoder card not currently being used.

Livepeer makes it possible for these data center operators to put their video encoder cards to use and turn on a lucrative new revenue stream with no additional cost or overhead. This new zero-cost revenue stream incentivizes these data center operators, or as they are more colloquially called, miners, to join the Livepeer transcoder network and provide excellent transcoding services to those who request them.

Thanks to this distributed network of always-on transcoders, Livepeer can transcode video segments much faster than realtime in just a few hundred milliseconds. Intelligent segment distribution and redundancies across the Livepeer network, in turn, ensure elegant fall back options in case of bad transcoder performance. In short, transcoding never stops.

Since Livepeer does not compete for the same cloud computing resources as Amazon or other transcoding providers, it can offer unique, cost-effective and flexible pricing based on transcoding usage, not on reserved server space. Cloud provider costs just can’t compete.

For more information on always-on transcoder access, our unique transcoding services, or any other questions you might have, send us a note to 

Livepeer's Always-On Transcoding Network

The Livepeer streaming API is powered by a network with uniquely distributed architectural design. At the core of this network is a live ingest and transcoding engine that tries to maintain a robust live streaming workflow in the face of networking issues or hardware failures. This helps the Livepeer infrastructure scale in unique ways.

In a simple live streaming setup, a broadcaster sends a video stream into a media server. The media server receives the stream, transcodes it into different renditions, packages it up into the delivery format, and sends it to the CDN edge. The video player loads from the edge and plays the video on the user’s device.

This system works well for a small number of concurrent streams. But as we start to scale this system, we will quickly run into performance bottlenecks with the media server and transcoding system – 

 for a live stream to reach a wide audience. As an example, a medium VM on AWS can only transcode 2-3 concurrent streams in a standard bitrate ladder (1080p, 720p, 480p, 360p, 240p). A common solution to this problem is to operate many media server instances and load balance between them. However, this creates another problem – transcoding processes can fail in the middle of a stream if the resources are not carefully managed. The difficult problem of dynamically spinning up and down servers and load balancing long running streams between them make the system brittle. Each small error can cause multiple streams to fail at the same time. Each failure can make the live streams buffer, drop frames, or even completely cut out.

Instead of having each stream handled by a single server, Livepeer takes a distributed approach for ingest and transcoding. Ingest servers create video segments based on incoming keyframe intervals, and each segment is sent to a network of distributed transcoding clusters.

The ingest server keeps track of the latency and performance of each transcoding cluster, so that it can switch to different clusters in the middle of a stream when performance starts to degrade. Since the ingest servers only need to worry about video ingestion and segmentation, they can be strategically placed in high bandwidth environments with geographic diversity, which optimizes for network connection with broadcasters and CDNs. Each ingest server can handle many concurrent streams, because it no longer needs to take on the computation burden of transcoding.

The transcoding cluster has layers of redundancy and failover, so that failures can be quickly retried within the cluster. These transcoding clusters can operate in a variety of different environments – professional data centers, cryptocurrency mining farms, gaming computers to list a few. Most of the current transcoders are GPU based, which reduces transcoding latency by an order of magnitude. Because of the low cost in these environments comparing to public cloud providers, transcoding jobs can even be done multiple times in parallel to ensure high success rate and low latency in the live stream. The Livepeer transcoding software is currently tuned to run in a multi-GPU environment, load balancing across 2-8 GPUs on the same machine. Depending on the types of GPUs, some machines can transcode over 100 concurrent video streams. If internet congestion happens in a certain region, with a specific ISP, or DNS provider, other transcoder clusters will seamlessly pick up the work from the ones that are impacted by the outage. All of this can happen without central coordination, since the ingest servers and transcoders are all making independent routing decisions.

The distributed architecture also poses unique and interesting engineering challenges. For example, when each segment is processed independently, it's challenging for the system to have stream awareness and ensure consistent segment lengths, GOP sizes, and other video format specific configurations. The ingest server / transcoder communication should have proper back-pressure built in, so the performance degradation happens gracefully instead of abruptly. There should also be proper retry policy in place, so a DDoS attacker cannot bring down the whole infrastructure through amplified retries caused by transcoding errors. Livepeer is working on solutions that can address issues like this in both a controlled environment and a decentralized environment where a part of the infrastructure is operated by third-party operators.

Livepeer is a decentralized network, and the operation of the nodes that make up the network depends on independent node operators. In order to maintain baseline capacity to regions of the world with high demand, Livepeer operates a portion of the network. The deployment is on top of a heterogeneous environment, ranging from bare metal servers to Kubernetes namespaces. We use tools like Kubernetes, Ansible, and Consul to manage our deployment. This allows us to gain more control in resource management and automatically recover from hardware/software failures. 

Live streaming on the internet is a difficult task because users demand for 100% reliability, and the internet is not a very stable environment. Livepeer uses robust software with "cheap and fast" hardware to help us with the goal of achieving high reliability, while keeping the cost low. We are constantly improving our 

, and we would love to hear your feedback. If you are interested in learning more about Livepeer, please check out the 

. If you are interested in working on Livepeer, please take a look at the 

"senior video infrastructure engineer" position

"senior software engineer - public network" position.

Blog

Livepeer's Always-On Transcoding Network

Livepeer's Always-On Transcoding Network

Scaling the Livepeer Network

Ready to get started?