How can I implement large presence channels on Channels?

How to implement large presence channels on Pusher Channels

Introduction

Pusher Channels has a feature called "presence channels”. When used at scale, this feature leads to a very high message rate. This document shows why that message rate is high, then suggests some ways you can adjust your UX requirements to enable lower message rates, then shows how you can implement these alternatives using tools provided by Pusher Channels.
 
 

The UX requirements

Assume FooBar Inc hosts online multiplayer games. While player 42 is in game 123, player 42 will see an up-to-date list of all other players who are in game 123.
 

The problem: too many presence events

To implement this, FooBar use Pusher Channels’ presence channel feature. For game 123, they have the channel presence-game-123. When player 42 subscribes to this channel, he receives the full current set of other players, and a member_added event is broadcast to all other subscribers. When player 42 unsubscribes (or disconnects), a member_removed event is broadcast to the remaining subscribers.
 
A typical FooBar game has 600 players, and each player stays for an average time of 60 seconds. By Little’s Law, we know that 600/60 = 10 new clients arrive every second. Clients leave at the same rate, so 10 clients also leave every second. Each arrival triggers a member_added event, and each departure triggers a member_removed event, so we have 10+10 = 20 triggered events per second. Each event is broadcast to the entire group of 600, consuming 600 messages from FooBar’s quota. So the presence channel for this game consumes 20 * 600 = 12,000 messages per second! This is over one billion messages per day!
 
This throughput is deemed unacceptable. This is the problem we seek to fix.
 

How to relax the UX requirements

As the above calculation shows, the “high message rate” problem follows directly from the requirements. Therefore, in order to reduce this message rate, one or more requirements must be relaxed. Here are some requirements and how they might be relaxed:
 
  • The “realtime” requirement. It may be unnecessary for the player to have a completely up-to-date view of the other players.
    • Alternative 1: only show the initial value on page load, then have no further updates.
    • Alternative 2: batch updates. Every 10 seconds, send the latest member_added and member_removed events.
  • The “full set” requirement. It may be unnecessary for the player to have access to the full set of other players.
    • Alternative 1: show a subset of the other players, e.g. the oldest 20 players.
    • Alternative 2: show a counter, e.g. “245 people are playing”.
  • The “accuracy” requirement. It may be unnecessary for the player to know the exact set, or the exact number of other players in the set.
    • Alternative: show an approximate counter, with client-side jitter.
 
These adjustments to the requirements can be combined in many ways - too many to list here. Instead, we show the tools that Pusher Channels provides for implementing these combinations, and then show one example set of relaxed requirements, and how these could be implemented using Pusher Channels.

 

Tools provided by Pusher Channels to implement relaxed requirements

Pusher Channels provides three tools you can use to implement these relaxed requirements: the member presence events toggle, and server-side presence and channel existence info. Below, we’ll see how you can combine these three tools to achieve an example set of requirements.
 
  • “Member presence events” toggle. If this is on (the default), Pusher Channels will send member_added and member_removed  events. If this feature is turned off, Channels only sends the initial presence set on subscription, but then does not keep the client up-to-date. Unfortunately, this toggle is currently only exposed to admins - please contact support quoting your app ID, and we’ll turn this off for you. (We plan to expose this toggle in the dashboard in future; see the end of this doc.)
  • Presence HTTP API. From your server, you can fetch a full snapshot of the members for a given presence channel. (Alternatively, you can fetch just the number of members in a given presence channel. This is more efficient than fetching the full list.)
  • Channel existence HTTP API. From your server, you can fetch a full list of channels beginning with a prefix, e.g. presence-game-.
 
(In addition to polling our HTTP APIs, you can use webhooks to receive this data with lower latency. However, implementing this reliably can be more complex, so in this document, we stick to polling HTTP APIs.)
 

Example relaxed UX requirements

FooBar Inc hosts online multiplayer games. While the player is in the game, the player will see two things: a list of up to 5 players, plus an approximate count of the total number of other players. This could look like the UI below. Both are refreshed every 60 seconds, but the counter UI updates every second with simulated client-side jitter. If the player clicks on the “711 other players” counter, they see the full list of players.
 
These requirements lead to a much lower message rate: 600/60 = 10 messages per second, or 0.08% of the original message rate!
 

How to implement these relaxed requirements

  • Contact support with your app ID, and request that “member presence events” are toggled off for your app.
  • When your client subscribes to presence-game-123, it will receive the full initial set of other players. Use this to construct the initial UI, taking an example subset and the total count, and then throw away the rest.
  • On your server, every 60 seconds, fetch the full list of channels with the prefix presence-game-, then for each of these channels, fetch the full member list. For each channel, take up to 5 players from the list, and the total, then trigger a presence-update event with a body of the form {"players": ["p1", "p2", "p3", "p4"], "total": 711}.
  • Your client should bind to the event presence-update on the presence channel. On receipt of such an event, the client will refresh the UI.
  • On your client, every second, either increment or decrement the counter. This provides a realtime experience, without the expense.
  • On your server, expose a /presence/:channel endpoint, which will serve the full presence list to your clients.
  • When the user clicks on the “711 other players” counter, the client should make a request to /presence/presence-game-123 for the full presence list, then display this. (Alternatively, the client could display the list it was originally served on subscription. This is simpler and more efficient, but the list will become stale.)
 
Was this article helpful?
1 out of 1 found this helpful
Haven't found what you were looking for?
Submit a ticket