Phoenix Channels at scale

Phoenix Channels makes real-time look easy. You create a MyAppWeb.SomeChannel module, add a few handle_in/3 clauses, and you're broadcasting. Excellent! The default settings will carry you to a few thousand concurrent connections without effort, which is perfect for a low-traffic site.

What if your site becomes popular, though? The next orders of magnitude, reaching 100,000 connections per node and beyond, expose a different set of problems you'll need to deal with. For example:

• You run out of file descriptors.
• The BEAM port table overflows.
• Phoenix.PubSub broadcasts start showing up in scheduler latency profiles.
• Defaults that were fine at a small scale become a noticeable share of your CPU at 100k connections.

This documentation contains the information I wish had existed when I first hit those problems.

Who this documentation is for

This documentation is for you if:

• You've shipped at least one Phoenix Channel.
• You're either operating a Phoenix Channel at scale today, or you can see the scale curve coming.
• You're comfortable on the BEAM.
• You've read the official Phoenix Channels docs.
• You want the field-guide version of scaling Phoenix Channels that picks up where the official documentation leaves off.

What you'll find here

This documentation will help you learn the following aspects of dealing with Phoenix Channels:

• When to use Channels: a decision-maker's overview. When Channels are the right tool, when they're not, and what they cost compared to the alternatives.
• Your first real-time feature: a tutorial that builds a shared-cursor feature end-to-end in about 30 minutes. Use it as a starting point or a sanity check.
• Handle 100,000 concurrent connections on a single node: the operational how-to. OS, BEAM, and Phoenix tuning that gets you to six-figure scale on one box.
• Scaling beyond one node: clustering, PubSub topology, and the tradeoffs of going horizontal.
• Channel callbacks, annotated: the callback surface, with the behavior the official docs assume you already know.
• How Phoenix.PubSub distributes messages: the architecture, the assumptions, and the places where the model leaks under load.

How to read this information

We've written these pages using the Diátaxis style, which consists of the following types of information:

• Tutorials: you learn by doing
• How-tos: you solve a specific problem
• References: you look something up
• Explanations: you understand the why

Every page tells you, in its opening sentence, what type of document it is.

If you're new to the site, the overview and tutorial are the natural entry points. If you're here because production is on fire, jump straight to the how-to guides.