Perform Semantic Find

Performs a semantic find in the specified field and constrains the specified record set for the given search text and model to use or for the given embedding vectors.

See also 

Options 

  • Query by specifies whether the query is Natural language (text) or embedding Vector data.

  • Record set selects whether to search All records or Current found set.

  • Target field specifies a text or container field to search in. The contents of the field must be embedding vectors, either as a JSON array if a text field, or as binary data if a container field.

  • Return count is a numeric expression for the number of semantically similar records to return as the found set. If not specified, the default is a maximum of 10 records.

  • Cosine similarity condition specifies how the cosine similarity of the data is compared to the Cosine similarity value. If not specified, the found set isn't limited by a cosine similarity condition.

  • Cosine similarity value is the threshold used along with the Cosine similarity condition to determine whether data is semantically similar or dissimilar. This numeric expression must have a value between -1 (dissimilar) and 1 (similar), inclusive.

  • Save result saves the found set in a text field or variable as a JSON array of record IDs and their cosine similarity values sorted in order of descending cosine similarity.

Options available only when Query by is Natural language:

  • Account Name is a text expression of the AI account for this script step to use. In the current file, use the Configure AI Account script step to set up the account and assign it this name any time before this script step runs.

  • Embedding Model is the name of the model to generate embedding vectors. Specify the model name as a text expression, which is available from the model provider.

  • Text is a text expression for the natural language text query. The resulting text is sent to the model for embedding vectors, which are then compared with the embedding vectors in Target field for similarity.

Options available only when Query by is Vector data:

  • Vector is the embedding vectors that represent the text or images to find. This is an expression that returns container data, such as a reference to a container field. Results will be semantically similar to the text or image this vector data represents.

Compatibility 

Product Supported
FileMaker Pro Yes
FileMaker Go Yes
FileMaker WebDirect Yes
FileMaker Server Yes
FileMaker Cloud Yes
FileMaker Data API Yes
Custom Web Publishing Yes

Originated in version 

21.0

Description 

This script step looks for records in which the content of the Target field is semantically similar to the query. When searching for text, this can provide more useful results than looking for literal keyword matches as the Perform Find does.

When searching for images, you can specify natural language text or text embedding vectors that describe the images you're searching for. Or you can specify embedding vectors for an image that's similar to the images you're searching for.

This script step has two modes of operation determined by the Query by option:

  • When set to Natural language, you provide the query as natural language text in the Text option. This script step sends that query text to the specified model for conversion to embedding vectors. This is why this mode requires an AI account name and embedding model.

  • When set to Vector data, you provide the embedding vectors as container data for the query. If you commonly use certain queries, it may be more efficient to get embedding vectors for those queries once and store them rather than frequently get them from the model.

    When specifying embedding vectors for an image to search for, use this option. You must use the same image embedding model to generate the vectors for the Vector option as you do for the image embedding vectors in the Target field.

For image as well as text embedding vectors, you can use the models supported by the open source model server provided separately with FileMaker Pro or installed with FileMaker Server. See the Claris Engineering Blog.

Starting with either all records in the current table or only the records in the current found set (set by the Record set option), this script step compares the data in Target field with the query for semantic similarity (or dissimilarity). The result is a constrained (narrowed) found set of no more than Return count records that meet the cosine similarity conditions. The resulting found set is sorted in order of descending cosine similarity so the most relevant records are first.

Notes 

  • Normalized embedding vectors are required. All embedding vectors must be generated from the same model to ensure compatibility and performance; mixing embedding vectors from different models isn't supported.

Example 1 

Configures an AI account, goes to the Meeting Details layout, then in all records, performs a semantic find for Text (as natural language) in the Meetings::Note_Embedding container field, which contains embedding vectors for the contents of the Meetings::Note field (see Example 1 in Insert Embedding in Found Set). For more useful results, the return count is limited to 10 and the cosine similarity between Text and the contents of Meetings::Note_Embedding field must be greater than 0.4. The resulting found set is sorted with the most relevant records first.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: OpenAI ; API key: "sk-RZCtpWT..." ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]

Perform Semantic Find [ Query by: Natural language ; Account Name: "my-account" ; Embedding Model: "text-embedding-3-small" ; Text: "Recruitment, job definition, training plan" ; Record set: All records ; Target field: Meetings::Note_Embedding ; Return count: 10 ; Cosine similarity condition: greater than ; Cosine similarity value: .4 ]

Example 2 

Configures an AI account, goes to the Meeting Details layout, then asks the user for text to find. The GetEmbedding function sends that text to the model, then Set Field stores the embedding vectors in the global container field Meetings::Input. Assuming the Meetings::Note_Embedding container field contains embedding vectors for the Meetings::Note field's contents (see Example 1 in Insert Embedding in Found Set), Perform Semantic Find compares the vector data in Meetings::Input with Meetings::Note_Embedding in all records and returns a found set that meets the return count and similarity criteria.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: OpenAI ; API key: "sk-RZCtpWT..." ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]
Show Custom Dialog [ "Find meetings with notes about:" ; $Input ]

Set Field [ Meetings::Input ; GetEmbedding ( "my-account" ; "text-embedding-3-small" ; $Input ) ]

Perform Semantic Find [ Query by: Vector data ; Meetings::Input ; Record set: All records ; Target field: Meetings::Note_Embedding ; Return count: 10 ; Cosine similarity condition: greater than ; Cosine similarity value: .5 ; Save result: $$result]

With the Save result option selected, for a given query that returns three records, JSONFormatElement($$result) returns:

Copy 
[
    {
        "recordId" : "7",
        "similarity" : 0.999999947111836
    },
    {
        "recordId" : "1",
        "similarity" : 0.529809641529481
    },
    {
        "recordId" : "5",
        "similarity" : 0.524595621039394
    }
]

Example 3 

Searches a set of images for those that best match a text description. The script configures an AI account for the open source model server installed with FileMaker Server, goes to the Meeting Details layout, shows all records, then gets embedding vectors for the images in the Meetings::Image field using an image embedding model and stores them as binary data in the Meetings::Image_Embedding container field. Perform Semantic Find sends a natural language query text ("People sitting around a table") to a text embedding model using the same account, then compares the result with Meetings::Image_Embedding in all records and returns a found set that meets the similarity criteria.

Copy 
Configure AI Account [ Account Name: "my-account" ; Model Provider: Custom ; Endpoint: "https://my-server.example.com:8080/" ; Verify SSL Certificates ; API key: Global::API_Key ]

Go to Layout [ "Meeting Details" (Meetings) ; Animation: None ]
Show All Records

Insert Embedding in Found Set [ Account Name: "my-account" ; Embedding Model: "clip-ViT-B-32" ; Source Field: Meetings::Image ; Target Field: Meetings::Image_Embedding ]

Perform Semantic Find [ Query by: Natural language ; Account Name: "my-account" ; Embedding Model: "sentence-transformers/clip-ViT-B-32-multilingual-v1" ; Text: "People sitting around a table" ; Record set: All records ; Target field: Meetings::Image_Embedding ; Cosine similarity condition: greater than ; Cosine similarity value: .7 ]

Related topics