Initial Stages
A few years ago we built a pilot project Shiny application that collected and summarized social media network statistics for colleges and universities.
The final product was a little rough around the edges – it didn’t install easily, required a lot of manual API key setup, and just generally didn’t meet the end users’ needs.
After about a year of project dormancy, the client approached us about reworking the application. This time we wouldn’t use Shiny, and our aim would be to provide
Picking a Tool
We thought for a while, and figured this might be a good opportunity to try Firebase. In some ways, it was a little bit of overkill, but it really helped us avoid deliberating over some of the key questions of application design:
- Authentication is about as simple as possible. Use the built-in Firebase sign-up/sign-in (ready in a matter of seconds), or add social sign-ins if you think your users would prefer it. We don’t have to worry about password storage or reset under this system, which is great.
- The database just works, and we don’t have to configure it, manage it, or scale it. It’s a NoSQL JSON tree , which does mean we needed to spent more time than usual planning the data storage layout (denormalization is almost always the answer). The database doesn’t handle complex queries very well, so in general we found it easier to organize it so all of a user’s data could be retrieved in a single query.
- With Firebase hosting, we could put the site on a CDN using https with a custom domain name, and never have to manage anything.
In summary, we could get to the point and start working on the actual application. That’s a huge advantage for a small team.
Getting Busy
We first set up developer accounts with each of Twitter, Facebook, Instagram, Youtube, and Bitly so we could access their APIs.
Thankfully most of these services are pretty easy to use and well-documented these days. Instagram presented the biggest barrier, actually requiring us to post a video screencast of the user experience once the application was ready for production. It wasn’t a big deal in the long run – approval was granted in less than 24 hours.
The Data
So what kind of data did we need to collect and display? There are a lot of parallels across networks: followers for Twitter, likes/followers/subscribers for Facebook, Instagram, and Youtube, respectively.
We track these metrics over time in line graphs, along with the same info for their direct competitors, both in absolute and per-enrollment terms.
Then we have post-specific engagement metrics: each tweet, post, or video has a total of likes/shares/views/etc., so we calculate the proportion of the institution’s follower base that engaged in some way with each post.
That information helps users target their posts to topics that will make their audience pay attention. That’s what social media marketing is all about.
There are a few additional features too: since Twitter follower profiles contain a location, we built a heat map of the most recent set of followers:
We let users search Twitter for terms (for example, the institution name, or specific hashtags they care about) and automatically calculate a sentiment sore.
Students in a good mood about the university? Great!
Bad mood over a recent decision by the athletic director? At least they can handle it proactively, before it gets out of hand.
We also modeled a plot of activity versus engagement by day and time of the week. It might not be perfect yet, but it’s designed to show where activity could be increased to match higher follower engagement. In other words, if your followers are engaging on Friday and Saturday nights, then you might want to move more posts to those times, or you might want to rethink the content of your Tuesday morning posts. Meet your followers where they are, you know?
Lessons Learned
- Firebase is a deployment accelerator. Without it, we’d still be debating which database and server architecture to use – instead, we used what they offered. It’s not technically perfect, but it got us moving, and in a matter of weeks we had a product ready to be sold. That’s rapid development.
- Use D3…but use a library! Wouldn’t homemade D3 graphics be cool? Really something to tell your family about at Thanksgiving. But when can use a library to write clean, predictable, consistent syntax that creates several line and bar plots, you should do that. We used c3.js, and would absolutely use it again. The author deserves a lot of credit.
- Error-proof your API calls. Weird things happen all the time (okay, only in production after you’re sure the test code is working). But if you spend time thinking about what your code should do in the event of various API errors, you’ll be much happier.
Modern Research Consulting 