Skip to content

Making a track start fast: mirrors, hedged requests and a stream relay

How Melodius turns a tap into audio quickly: cached playable copies, one copy per source first, hedged resolution after 2.5 seconds, a head start for the next tracks, and rate limits that ignore seeks.

4 min read

The moment that decides whether a music app feels good is the gap between tapping a song and hearing it. In Melodius that gap hides several network hops, any of which can be slow or fail. This post walks through the choices that keep it short.

From a tap to audio

A track in Melodius is identified by its catalogue match (see matching songs across catalogues). To play it, the server needs a playable copy: a Deezer, YouTube Music or SoundCloud version of the same recording. Self-hosted NodeLink nodes, four of them across regions, search for copies and resolve each one to an upstream stream URL. The player then fetches audio through Melodius's own stream relay.

So a cold play is: find copies, resolve one, start relaying. Each step got its own shortcut.

Remembering playable copies

A song's copies rarely change, so searching for them on every play is wasted time. Found copies go to Redis for seven days. A song played again within the week skips the search entirely.

typescript
const MIRRORS_TTL_SECONDS = 7 * 24 * 60 * 60;
const NO_MIRRORS_TTL_SECONDS = 30 * 60;

Two details matter. When a search finds nothing, that is cached too, but only for 30 minutes, so a temporary failure does not hide a song for a week. And when every cached copy fails to play, they are forgotten and searched again fresh. Cache reads and writes are best-effort: if Redis is unavailable, playback just does the slower search.

Ordering copies

Not every copy streams equally fast, so each provider can rank its copies by how soon they start. The list is sorted by that rank, then reordered so the best copy from each source comes first:

typescript
// Each source's best copy first, so a source failing as a whole leaves another to play.
const oneFromEachSourceFirst = (mirrors) => {
  const seen = new Set();
  const firsts = mirrors.filter((m) => !seen.has(sourceOf(m)) && seen.add(sourceOf(m)));
  return [...firsts, ...mirrors.filter((m) => !firsts.includes(m))];
};

The reason is correlated failure. If one source starts refusing the server, all of its copies fail together. Trying three copies from the same source in a row would burn the whole time budget on one outage.

Hedged resolution

Resolving a copy is usually quick and occasionally very slow. Waiting for the slow case is what makes an app feel broken. So resolution is hedged: the first copy starts resolving, and if it has not answered within 2.5 seconds, or fails sooner, the second starts in parallel. Whichever answers first plays.

Resolved upstream URLs are kept in the process for ten minutes, up to 500 of them, so seeking within a song never resolves it again.

A head start for the next song

The endpoints that hand the player a list of songs, such as a playlist or a queue, start resolving the first two relayed tracks before the player asks for them. By the time the player's stream request arrives a round trip later, the stream is ready or already on its way.

The relay and seeks

The relay protects the audio nodes with a per-listener limit on how many streams can be opened. One bug taught me what that limit must count. Seeking within one track makes the player send many range requests, and each one was charged as a new stream. After some seeking, the next songs were refused and showed "We could not find a playable source for this track."

The fix was to count opening a track, not each range request inside it. The broader lesson: a rate limit should count the thing that costs you, and in a stream relay that is a new upstream stream, not an HTTP request.

All of this sits behind one tap. If you want to feel the result, play something on melodius.in; the rest of the architecture is in the Melodius case study.

Illustrated avatar of Kumar Manas

Written by Kumar Manas

Full-stack developer in India. I build Melodius and Krypton.

Related projectMelodius: music streaming platform