Vince reacts with details, and his share of amazement, to the experience of pair programming with ChatGPT as co-pilot, searching for deeper meaning in FileMaker scripts using semantic search (using InspectorPro 8), and the overall experience using AI with FileMaker 2024.
Is Co-piloting the new Pair Programming?
There is a certain type of joy in solving a problem with your pair programming partner. There’s a shared feeling of success when you have succeeded in finding a good solution, together. But there is an interesting new version of this experience starting to happen, and that is when your partner is not a human but a machine 🤖. I expect that different people will have different reactions to this. My own reaction has mostly been amazement.
I had one such moment a few weeks ago when I was struggling with some XPATH language and wasn’t getting the results I was looking for. We often tell our team that if you struggle for more than 10 minutes on something, stop and ask for someone to pair program with you.
That is exactly what I did, but in my case it wasn’t a human but ChatGPT. I gave it the chunk of XML and the XSL that I had scraped together so far and told it what I needed. As soon as I hit the return key … it immediately replied with an affirmative response, “Sure, I’l help you with that.” Then, a second later code started to stream on my screen. I copied the code and it ran just fine; no syntax errors. But still not the result I was looking for. Not a problem–as a pair, “we” iterated about 6-7 more times and at the end of this process, I had what I wanted. Whoo hoo! No high fives, but I would offer my new partner a byte sandwich 😀.
Then and Now–Preparing for FileMaker 2024
Yesterday I was looking back through some of the data that I have saved related to InspectorPro and FileMaker development — 30+ years, on my computer, in storage, and in a notebook where I capture ideas that come to mind. Here is a picture from that notebook. Wow, from 2 years ago…
In my research archive, I also found a video from when I shared with my friend Luke to tell him about the new architecture that I was planning on having in the “next” release. Wow, that video was from Dec of 2019! Now almost 4.5 years ago, that “next” release is here: FileMaker 2024. I guess some things just take time. And are worth the wait.
So much has changed and evolved with being a Claris FileMaker Developer over the last 4.5 years. For example, we got built-in native transaction support and the ability to create audit logging. And now we are stepping into a brave new world of AI.
Even if you missed the internet wave, or the mobile wave, you won’t miss this wave. In fact if you are using a modern smart phone you are already using some aspects of AI. And with the release of Claris FileMaker 2024, we have the ability to engage deliberately with AIs in numerous ways, including by generating embeddings (machine learning objects, prepped for AI usage) and performing semantic search.
The Search for Meaning, in FileMaker scripts
In a meetup discussion not too long ago, Chris Moyer asked, “What if we were to send ChatGPT an entire FileMaker script and ask it to summarize it?” The idea was that you could figure out — the machine could learn — how to generate an embedding based on the summarized text. And that should allow you to search on the meaning of the script, assuming the summary and embedding accurately encompass the intent of the script.
Traditionally, when you perform a Find in FileMaker, it will search for a specific string, number, date or maybe a range of values. But now with semantic search you’ll be able to find things that have similar meanings. For example if you performed a Find in FileMaker with the word “form” it would only find values that had the word “form” in the field. With semantic search, however, you can search using the word “form” and it might find records that included the word “form” and also the word “shape”, or even fields that include a shape like “triangle”, “circle”, or “square”.
As I continued my own research into semantic search on FileMaker scripts, I thought “Let’s get the XML representation of the script and ask ChatGPT to generate a summary based on the XML.” Below is the XML for a single Set Field script step, followed by the textual equivalent. Note how much smaller the textual equivalent is. Sending all those extra XML tags isn’t helpful to the AI, and it’s more costly because we are using up more tokens.
Here is a example of a single script step what the XML would be vs. the normal textual representation of the script step in FileMaker Script Workspace:
The challenge has been to generate the textual equivalent of the script and send that instead. We found that it would provide a 90% – 95% savings in overhead. Note that in the past, FileMaker’s Database Design Report (DDR) had an attribute that gave us the script step text. That is not present in the new Save a Copy as XML version – so we had to build our own way.
Let’s look at a small script that is only 35 steps long. This is a template script and pretty easy to understand from the generous comments and white spacing. One could scan the comments of the script and quickly understand what it is generally doing.
We fed the textual representation of this script to ChatGPT and here is the concise summary it returned:
This script serves as a template with a single pass loop pattern to provide a framework for developers. It measures script execution time.
Seeing as we had already sent ChatGPT the text of the script, we next asked it to do two things. First, give us a concise summary, and second give us a more detailed summary, in case the concise one was just too short and did not provide enough details. Here is the more detailed summary of the script above:
This FileMaker script is a template designed for developers. It utilizes a single pass loop pattern, with sections for declaring variables, setting context, and performing work within nested loops. The script includes timing measurements to track the execution duration, with a calculation of total time shown in a custom dialog at the end.
Now let’s say the developer did not add any comments to their script. How well would this work? Here is the concise summary of the script above but with all the step comments taken out:
This script calculates the total time taken for a loop iteration.
And here is the detailed summary of the above script:
The script starts by setting a variable with the current time in milliseconds, then enters a loop. Inside the loop, it retrieves a value from the script parameter, goes to the original layout, sets variables, and performs another loop that exits based on certain conditions. Finally, it sets another variable with the end time, displays a custom dialog showing the total time taken for the script execution.
This is a pretty good summary, but having comments could enable the AI to be more descriptive rather than presenting a narrative that lists the script steps.
These summaries can be indispensable in a number of ways. For example, let’s say you inherit a solution that someone else has worked on, or it might even be a solution that you yourself worked on a number of years back and have only a vague memory of. Having a descriptive summary of a script could be a big time-saver compared to working through the script steps manually.
Let’s look at a much longer script and see how it does. There is a script in InspectorPro that is 382 steps. That script itself performs Global Search. What it does, is that it navigates to … wait a minute, why don’t I let ChatGPT tell you what it does. Here is the detailed summary of what that script does:
The script loops through different sections such as schema, calculations, and history to perform a global search based on the provided parameters. It handles customization of search fields, expanding search results, package data, and unpacking results. Additionally, it performs find operations, calculates duration, and updates fields with the search results.
Yep. That’s a fairly accurate description of what that script does.
And one more example with InspectorPro 8, which itself is built using FileMaker. We have a great feature where you can associate tags with any item. Let’s say that at some point you have over 1,000 analyses and you want to reset your solution and start from scratch because you are beginning another development sprint. InspectorPro will keep the tags and notes so that when you process your first analysis, it can reconnect those tags and notes with items they belonged to previously.
You can imagine that script could be a little bit involved and complicated. Here is a detailed summary of what that 161 step script does:
This script loops through area items, finds corresponding tags, and reassigns them based on set criteria. It involves creating new windows, performing finds, handling tag associations, and cleaning up records after processing.
Search by Meaning = Semantic script searching
You can see that AI was helpful in generating a script summary. But with the release of Claris FileMaker 2024 we now get the built-in ability to perform a semantic search. Basically you can search on the meaning of things. Using the last example, let’s say I did not know to use the word re-associate and instead searched for something like: “Reconnect Tags To Items”.
The key to using semantic search in FileMaker is a new concept called embedding. I am new to this, like many of us FileMaker developers will be, so I asked ChatGPT to give me a very simple explanation of what embeddings are, and here it is:
Sure! Embeddings are a way to represent things like words, images, or other data as numbers so that computers can understand and work with them.
A more detailed explanation is:
– ChatGPT 4o
Imagine you have a bunch of words, and you want to store each one in a way that captures its meaning. Instead of using the actual word, you can use a list of numbers (called a vector) to represent it. Each number in the vector captures some aspect of the word's meaning.
For example, the word "king" might be represented as [0.1, 0.8, 0.3], and the word "queen" might be [0.1, 0.8, 0.4]. These numbers are chosen so that similar words have similar vectors. This way, the computer can compare the vectors and understand that "king" and "queen" are related.
In simple terms, embeddings are like converting words or data into secret codes (numbers) that help computers understand their meanings and relationships.
Once you have your embedding, then you can semantically search (on meaning) using the new Semantic Find script step. In this case, we used summaries of scripts that ChatGPT generated for us, created an embedding, and could then search semantically on it using the Perform Semantic Find script step. The first search result was my “tag re-associate” script, which is great because it is actually the most relevant.
There are some great blog posts by my colleague Will Miro where he dives into the details of this. To learn more, I invite you to check out his blog posts on Semantic Search with FileMaker 2024.
The impact AI will have is already here, and as you can see generating a summary of scripts is going to be extremely helpful to developers in a number of different ways.
At some point in the not too distant future we will have AI assistants that can help us do these kinds of things and much, much more. Because AIs continue to develop their grasp of semantics and the nuances of language, the experience of having a more human-like discussion will be amazing. On some level it still a long ways away from “Please build me the whole solution that does XYZ.” But maybe it doesn’t need to be looked at as needing to build the whole system. Maybe it can be looked at as making sure each piece is right and it’s up to us to pull together the whole system, in a similar way to how we take ChatGPT's responses as a basis for content today.
Whatever it is going to be… I’ll just ask ChatGPT to write a concluding paragraph about the future possibilities that lie ahead with the help of AI. This is what it has to say:
As we look to the future, the integration of AI into FileMaker development holds boundless potential.
With AI's ability to analyze vast amounts of data, automate repetitive tasks, and provide intelligent insights, developers can craft more efficient, innovative, and user-friendly solutions. The continuous advancements in AI technology will open new doors for creativity and problem-solving, allowing developers to push the boundaries of what is possible.– Concluding paragraph by ChatGPT 4o
By embracing AI, FileMaker developers will not only enhance their own productivity but also deliver unparalleled value to their clients, transforming the way businesses operate and thrive in the digital age. The future is bright, and the opportunities are endless for those who dare to innovate with AI.