Speeding Up A Shiny App: Future/Promise and Caching

We build a lot of Shiny apps, and most work fine without a lot of customization.

But special cases require some fine-tuning to get everything working correctly, especially with a lot of simultaneous users.

One recent app was built for a financial trading firm, and needed to be open and responsive for a large set of traders all day long.

But there was a major barrier: every five seconds, data needed to be pulled from a database on another continent back to the US, transformed with some relatively processor-intensive steps, and only then were results displayed on the screen…maybe only a second or two before the next update cycle would begin.

For one user, no problem – but for two, three, four, …, users? When using the open source version of Shiny Server, all sessions are working in a single R process, which means each session has to wait for other sessions to update first.

As soon as the processing for all sessions takes longer than the five-second update interval, you can think of this like a line (for movie tickets, stadium entrance, grocery check-out, etc), where for every person finishing up a the front of the line, more than one person is getting into the back of the line.

There’s no way for the server to catch up!

The first approach is to move long-running steps into separate R processes, which is relatively easy to do with the async functionality added to recent versions of Shiny along with the promises and future packages.

There’s a helpful blog post here from RStudio about how to implement async processing in a Shiny app with a basic example. We found this really helpful as a starting point.

So that helped the app quite a bit – instead of running into timing limits at two or three concurrent sessions, we could get up to the number of processors on the server, but then hit another wall.

That’s when we decided to combine the async approach with another nice feature of the maturing Shiny ecosystem: the in-memory cache.

In this case, the data coming from the database, and the transformed results of that data, are going to be the same for every session. Only minor changes are different across sessions, like filter parameters (e.g. only show orders great than $X).

That means all the heavy lifting can be done once, by whichever session was started first, and then saved in the cache, and all other sessions can simply read from the cache.

How do we keep track of live sessions? We created a global object called session_tokens, initialized as character(0).

Then within the server function:

if (!(session$token %in% session_tokens)) {
  session_tokens <<- c(session_tokens, session$token)
}

After that, every step that should be processed only once looks like this:

if (session$token == session_tokens[1]) {
  message(' -- setting cache: ', session$token)

  # heavy lifting

  cache$set('key', value)
} else {
  message(' -- reading cache: ', session$token)
  value <- (cache$get('key'))
}

This does cause a slight delay for every session reading from the cache, since obviously those sessions have to wait for the pilot session to finish processing first.

However, that slight delay is a small price to pay to avoid an ever-growing session queue, or maxing out all processors on the server for two out of every five seconds.

There is one more thing to do, which is to remove session tokens when each session ends – this way if the current first session ends, the next one can become the session doing the work:

onSessionEnded(function() {
  message(' -- ending session ', session$token)
  session_tokens <<- setdiff(session_tokens, session$token)
})

This was a fun challenge, and yet another reminder of how powerful Shiny apps can be these days, a great improvement over the somewhat limited world of three or four years ago.