Sync is a Two-Way Street

Most of what I’ve written so far has been about uploading changes from the client to the server. But we found it is equally important to download changes from the server periodically, too. What use is offline mode when you aren’t viewing fresh data? How can you share perspective with someone if you only ever push data and never pull?

To that end, we also built in background downloading as part of our sync system. Thankfully, downloading fresh data is a lot simpler than making changes.

Background Updates

Part of keeping data fresh is in making sure we download data periodically in the background. We have to be extremely careful with this, though; we don’t want to chew through user bandwidth or battery.

First, we only schedule occasional syncs with the server, at most twice a day.

Another optimization involves skipping downloading unchanged data. Boards and cards store when they last had activity. Before we download full data, we can query whether or not the board or card has had any recent activity. If not, then we don’t need to download it, saving us a bunch of bandwidth.

In addition, we try to intelligently pick what data to download for the user. Starred boards and recently viewed boards will get synced, whereas a board you’ve never viewed may not be synced at all.

Download Request Queue

How do we schedule what data we want to download in the background? Why, with a download queue, of course!

Much like the upload request queue, we created a download request queue where we can register data to be downloaded whenever it becomes convenient. We can queue up anything we’d like, such as “download member boards”, “download current board”, or “download current card.”

Unlike the upload queue, which executes in the order received, the download queue is a priority queue. Some data needs to be synced more urgently than others. If the user just opened the app, we want to download an up-to-date list of the boards immediately. If the user gets a notification, syncing the information related to it can happen a little less urgently. And if the user hasn’t opened the app for a while, then syncing any of their information is low priority.

Also, sometimes we’ll want to remove items from the queue. If the user quickly opens a board and then closes it before we have a chance to sync it, then we don’t want to bother with the sync.

I’ve got to be honest with you, though: our download system is not fully realized yet. The above is only implemented in key places. Much of our data is still coming from direct requests. We’d like to eventually transition everything to being downloaded in the background, though. It makes our overall architecture easier to depend on a single download system, instead of having it be split out all over the place. That way common book-keeping - such as updating sync state - can be done in a standardized way.

This article was originally posted on the Trello engineering blog and has been reproduced here for posterity.