Small Gains: Big Impact
In the transit-planning universe, planners and economists often get excited about 5 minutes shaved off of a 30-minute bus ride. The individual rider might shrug at a mere 50 minutes saved per week, but the planners and economists multiply that 50 minutes by the number of people who ride that route during a typical week and see something much bigger.
There’s a compounding effect: it doesn’t take long before these small gains in efficiency add up to a huge economic impact overall.
What's this have to do with FileMaker, you ask? Well, the new FileMaker 15 is now available, and the inevitable question comes up "should we upgrade?" Sometimes the best discoveries when a new version of FileMaker hits the shelves are the “sleeper” features and under-the-hood improvements. In the most extreme cases, these go entirely undocumented by FileMaker Inc.; such is the case with the improvements in portal performance that accompany the FileMaker 15 release.
Portals being a very important part of the FileMaker user interface, anything affecting their performance, good or bad, is going to have a major impact on the user’s experience. Users of a database application often spend large chunks of their workday in the application, so, like that 5 minutes shaved off the bus ride, small efficiencies add up through the same compounding effect. And, as you’ll see, this one is a "big" small efficiency!
No, I'm not talking here about the new, threaded portal rendering (aka "inline portal progress bar") that is a key public feature point for FileMaker 15; I'm talking raw performance optimization. So impressive is this optimization, in fact, that I never even saw the new inline progress bar during my testing!
We didn't set out to measure differences between FileMaker 14 and FileMaker 15, and the marketing materials gave us no reason to expect them. We had developed a file to assess performance of several different techniques (possible future blog posts!) over the wide-area network (WAN). At Beezwax, we often build large database applications that are hosted in one place and then accessed by clients around the world, so testing and optimizing performance over the WAN is always critical. We hosted this particular file on a server in London, so that we could measure the new techniques’ performance over the WAN from our home office in Oakland, California. It happened that the file included a portal that shows 10,000 records.
Noticing that this 10K-row portal loaded so rapidly and scrolled so smoothly in a FileMaker Pro 15 client prompted us to compare its performance using a FileMaker Pro 14 client. The result was a huge "wow"!
Performance Testing Methods
Our setup was an FMP file hosted in London, on FileMaker Server 15. Our testing was performed in Oakland, California using both FileMaker Pro 14 and FileMaker Pro 15 clients. Server and client machines are both connected to a corporate network via VPN. Additional hardware, software, and network details are provided in the footnote, for technically curious readers.
In order to test portal performance itself, independent of the techniques we initially set out to test, I added a single parent record with a UUID primary key, and created a standard equijoin to 10,002 child records, all with the same UUID value in a foreign key field; that is, one parent record with 10K related records displaying in a portal. For each test, I quit FileMaker to clear cached data, relaunched, opened the hosted file to a no-record Home layout, and navigated directly to a layout showing the single parent record and either an unsorted or sorted portal.
For the scrolling tests, I didn't drag the scrollbar's thumb, which skips records, but scrolled as quickly as I could (without locking up the layout in the FileMaker Pro 14 tests) by swiping on the trackpad, measuring the time it took to scroll the entire length of the portal. The results presented here for each set of tests are the mean of 3 replicate tests for each client (FileMaker Pro 14 or FileMaker Pro 15). All tests were highly reproducible: coefficients of variation were 5% or less for the portal-loading tests, and less than 20% for the scrolling tests.
Time to initially load and render the portal
FileMaker 15 consistently loaded and rendered the portal faster than FileMaker 14 (Figure 1).
FIGURE 1. Time to Load and Render Portal
For unsorted portals, both clients loaded reasonably fast, although there was a highly reproducible 2-fold difference in loading time favoring FileMaker 15. The 2 seconds that version 14 required to render the loaded portal may not seem like much, but remember that each time you navigate to a different parent record, the portal needs to be loaded with fresh data (and FileMaker 14 lacks the inline progress bar feature, so you have to wait for the portal to completely draw and load before you can interact with the layout or move on to the next record): it's that 5 minutes on the bus again.
For sorted portals, the difference was more dramatic, with FileMaker 15 showing only a negligible (0.1 second) penalty for sorting the portal, and a 5-fold improvement over FileMaker 14, which consistently threw up a "records remaining to sort" dialog.
Time to scroll length of portal
Here's where my socks started to really get knocked off!
FIGURE 2. Time to Scroll to End of Portal
Both the sorted and unsorted versions of the 10,000-row portal scrolled quickly and smoothly in FileMaker 15. This was so contrary to our extensive experience with previous FileMaker versions that it is what first prompted me to open the file in FileMaker Pro 14, just to remind myself that I've never seen a large portal like this perform so responsively in a hosted file. As expected, in FileMaker 14, attempting to quickly scroll through the portal was almost painfully choppy and halting.
Time to put some metrics to it (Figure 2). Your eyes are not deceiving you: FileMaker 15 performed 31-fold faster for unsorted portals, and 12-fold better for sorted portals. Scrolling through this long portal in FileMaker 14 was an exercise in patience. The tests were actually a bit hard to conduct; pushing too hard brought the scrolling performance to its knees and the portal would almost freeze up periodically. Interestingly, the sorted portal performed a bit better (133 seconds vs. 154 seconds), presumably due to the sorting operation resulting in some additional caching, but was still unacceptably slow. FileMaker 15, in contrast, raced along with nary a stumble, completing the task in a small fraction of the FileMaker 14 time; there was a small, reproducible penalty for sorting (11.4 seconds vs. 4.9 seconds), but scrolling was extremely responsive nonetheless.
Conclusion
FileMaker engineers have obviously done some serious optimization of portal performance in FileMaker 15, resulting in much faster loading (particularly notable for sorted portals, which almost seem not to need additional time to sort the records) and a remarkably improved scrolling performance with large portals.
What's this mean for you?
If you're an end user, it's back to the bus analogy, where small efficiencies add up. You'll immediately appreciate the more responsive UI and spend less time waiting for your database to catch up to you. While these improvements will be most noticeable in solutions hosted over the WAN, they will be there even in solutions hosted on a local network, and will add up over time through the compounding effect.
If you’re a developer, this opens up the possibility of using portals, even portals displaying a large number of records, as more central components of the UI. In the past we might have avoided UI patterns that are highly portal-dependent, but the results presented here suggest that we have turned a major corner in this regard. Of course, this is really an end-user advantage, because we can provide our users with ever more flexible options in UI design.
For everyone, this means that one of the more compelling reasons to upgrade to FileMaker 15 is one that didn't even make the marketeers' bullet-point lists, but is big enough to bring a smile to all of our faces.
Technical Details
Round-trip ping times: 171.5 ± 1.24 ms [avg ± SD, n=10]
Server machine: Mac Mini with 2.8 GHz chip and 8 GB RAM.
Server memory cache set to default (512 MB)
Client cache set to default (256 MB, for both FMP14 and FMP15 clients)