User’s Sandbox: Performance, data integrity, and UX for FileMaker

The "User’s Sandbox” (aka “Local File") technique builds a sandbox for data editing by users, protecting data integrity and transactionality, without sacrificing usability. After refining this approach to prioritize user experience and security, at Beezwax we’ve used it for years, adapting it to build more performant, more accurate, better looking, and more functional Claris FileMaker applications.

Andrew “Red” Witschonke, Director of Engineering at Beezwax, first presented on the User’s Sandbox technique at Claris Engage 2024 — the premier event for the FileMaker development community, held at Apple’s Austin Campus and hosted by Claris, an Apple company.

Missed my Claris Engage session?

Here’s an outline, with highlights of key points and benefits. Let us know any questions or feedback.

A video presentation is part of the Engage 2024 recordings available to attendees, or for purchase in the virtual package from Claris.

Best,
– RED

User's Sandbox: Opening new performance, data, and UI/UX possibilities.

Speaker: Andrew “Red Witschonke, Director of Engineering, Beezwax
Date: February 7, 2024 at Claris Engage

Summary: Maintaining usability, performance, and data integrity is a difficult balancing act when trying to deliver the best applications for your users. The “local file” technique goes a long way to maximize all three of these areas in applications. In this session, we will share how to implement this technique and some of the novel uses we came up with over the last six years.

This technique is like using super global fields, adding all the power of relationships, calculations, and summaries while still being session-bound. With this added flexibility and power, there are countless things that suddenly become possible. By adopting this technique, we have been able to build more performant, more accurate, better looking, and more functional applications.

HIGHLIGHTS & BENEFITS OF THE USER’S SANDBOX TECHNIQUE WITH FILEMAKER

I. Introducing User's Sandbox

• User's Sandbox Architecture
• User's Sandbox and FileMaker Server
• Leveraging FileMaker’s Toolbox
  • Standard Fields
    • Persistent
    • Searchable/Indexable
    • Complex Schema
  • Global Fields
    • Session Bound
    • Locally Cached

II. "Sandbox File" Benefits: Best of Both Worlds

• Customization through UI/UX Flexibility
  • User-specific
  • Relational
  • Session-bound
• Ensuring Data Integrity
  • Sandboxed record editing
  • Separation of concerns
• The Power of Local Caching
  • Performance
  • Limit network traffic
  • Locally stored/cached data

III. Implementation: Making User's Sandbox Work

• Step-by-Step: Storing and Exporting the Sandbox File
  • Create a new FMP12 file and store it in a container field in your hosted application.
  • OnFirstWindowOpen, export the stored Sandbox File to the user's temp directory using a global variable as the path.
• Adding External Data Source: Integrating Seamlessly with Your Application
  • Add your global variable as the file path for a new external FileMaker data source.

IV. Harnessing the Potential: What User's Sandbox Can Achieve

• Unlocking Unique Record Context for Enhanced User Experience
• Store State Per-Record
  • Filter State
    • Unique filter per hosted record
  • Selection State
    • Selected child record per hosted record
  • Multi-Window
    • Interact with records across multiple windows
• Simplifying Complex Record Editing
• Sandboxed Editing
  • Records Are Local
    • User can revert/undo all pending changes
  • Leverage Unstored Calculations
    • Utilize unstored calculations with minimal performance impact
• Edit Groups of Records
  • Multiple Records
    • Make a "batch" change to multiple records
  • Manage Parent/Child Records
    • Edit attributes of parent and child records together
  • Prepped for Transactions
    • Group of edits can be executed transactionally

SANDBOX FILE EDITING UI

V. Optimizing Network Traffic

• Understanding the Impact of Fewer Server Calls
  • Packaging User Actions Increases Efficiency
    • Interacting with data in the Sandbox File requires no additional communication with the hosted file.
    • Packaging record edits into a single batched PSoS operation minimizes round trips to the server.
    • The more complicated the interaction, the more dramatic the difference between number of calls.

• Why Network Efficiency Matters: High Latency and Its Consequences
  • High Latency Kills Performance
    • High latency (common for remote users)
      has an exponential impact on user experience.
    • Each call to server (explicit or background) incurs an additional performance penalty for the user.
    • Caching record data in the Sandbox File cuts down on volume of network traffic.

VI. Addressing FAQs

• Managing Two Files: Tips for a Smooth Workflow
  • It’s Less Work Than You Think
    • The Sandbox file only contains tables/fields
  • Only Add What You Need
    • There is no need to completely mirror your hosted application. Only add what you need to drive the functionality you want.
  • Minimal Relationships
    • Only the schema needed to support any calculated fields you create needs to live in the Sandbox File.
  • Everything Else is Hosted
    • Scripts, Layouts, and their associated TO's can all live in the hosted file and maintained alongside the rest of the app.
• Dealing with Record Conflicts: Architect a Method that Works for the Client
  • It's the Developer's Choice
  • Architect a Method That Works for the Client
  • UI/UX Only
    • Only use the Sandbox File to add more engaging user experiences in your application.
    • Keep using normal record editing techniques and avoid the issue entirely.
  • Last-in-Wins
    • The last edit from any user is what drives the final record state.
    • In many cases, a record is “owned” by a single user, so simultaneous edits are unlikely.
  • Manage Record State
    • Mark hosted records as "checked out" or compare record state prior to user edits to the current hosted state and prompt the user if changes have occurred.
• Compatibility with FileMaker Go
  • “Yup”
• Compatibility with WebDirect
  • “Not Really”
  • WebDirect is Unique
    • No Temp Directory
    • Since WebDirect is effectively running entirely on server, there is no ability to export or interact with a locally cached file.
    • Workarounds Exist
    • A copy of the Sandbox File could be hosted alongside the hosted file and referenced for users accessing the file via WebDirect. However the user-specific nature of the local file would be lost.
• Best Practices and Lessons Learned: Tips for Smooth Implementation
  • Export Method
    • Use the "Files" script steps to export your Sandbox File to the temp directory as a part of your "OnFirstWindowOpen" script from a layout without any references to fields from the Sandbox File.
  • Automate Dev Workflow
    • Add a script to the Sandbox File to export a clone of itself. Then call that script from an “update Sandbox File” script in the hosted application that re-inserts the resulting FMP12 file.
  • UI + Scripting
    • You can build layouts in the context of Sandbox File tables in the hosted application, as well as write scripts that reference fields in those table occurrences.

VII. Choosing User's Sandbox: Use It Where It Makes Sense

• An Arrow in Your Quiver
  • The Sandbox File technique is more about adding to the collection of tools you can implement to deliver great user experiences, not replacing those tools.
  • Think of it as adding "Global Tables" to the list of other tools in FileMaker.
• vs. Global Fields
  • Global fields are still a super useful tool that can be used in many ways.
  • Using an equivalent Sandbox File field makes sense when you want to easily add record-specific context and persistence.
• vs. WebViewer
  • You can build using web code in a WebViewer. However, the Sandbox File technique is entirely "native" FileMaker, meaning any experienced developer can pick it up and start developing.

Get The Recordings – For This And Other Claris Engage Sessions

Individuals who attended Claris Engage 2024 get session recordings as part of their registration.
But even if you couldn't make it to Engage, Claris offers a virtual package with content from more than 50 recorded sessions.