Documentation

Populate a Store-specific external search

Elevate, May 20-22-2025, Miami Beach, Florida
The Store and Product Selection data models allow you to build Store-specific search indices that can be used for querying Store-specific product data to power Store-specific frontends. The following guide explains how a commercetools-external search service can be integrated with the commercetools Composable Commerce to build several Store-specific search indices and how to use Subscriptions and Messages to keep the search indices up to date.

This guide assumes that each search index contains the product data available for exactly one Store.

Initial Setup

This section explains which steps need to be completed to set up the integration between commercetools Composable Commerce, a message queuing service, and an external search service. The first four steps are specific to the service providers and therefore not explained here in detail.

  1. Set up Store-specific search indices.
  2. Set up a message queue for notifications about changes on Products.
  3. Set up a message queue for notifications about changes on Product Selections.
  4. Set up a message queue for notifications about changes on Stores.

Once this is complete, you need to set up three subscriptions to get notified upon changes on product data related to a Store.

Set up Subscriptions

The following subscriptions need to be set up on commercetools Composable Commerce:

Subscription for Changes on Products

Create a Subscription to listen to any updates on Product with the following changes field:
{
  ...
  "changes": [
    {
      "resourceTypeId": "product"
    }
  ]
}

Subscription for Changes on Product Selections

By using Message Subscriptions on Product Selection Messages, you get notified whenever a Product was added or removed to/from a Product Selection or there have been a change on a Variant Selection.

Subscription for Changes on Stores

By using Message Subscriptions on Store Messages, you get notified whenever a Product Selection was added or removed to/from a Store or whenever a Product Selection is activated or deactivated for a Store.

If the Stores' configuration does not change, you can skip this step since there is no need to subscribe to those changes.

Initial product data feed

You have now the search indices and message queues in place and you have also set up the Subscriptions to ensure you will not miss any update on Product data from now on. What is left to do, is to fill the search indices with Product data since the indices are still empty.

For each search index you have to find all Products available for the respective Store:

  1. Given a Store, iterate over all Product Selection Assignments to get Product References. You can sort and filter on the product.id field for that purpose. Keep in mind that the response will include duplicate Products whenever more than one active Product Selection of the given Store includes a Product, so you might want to keep them distinct on your side.
  2. For each Product, fetch the respective Product Projection in the Store and add it to your search index.
Loading...

Keep the search indices up to date

Feeding the search indices with initial product data is now complete. Over time it is likely that the Product data or the Product Selection will change. You need to reflect these changes in the search indices to keep them up to date in the store front, too.

React to changes on Products

Any change on a Product triggers a Message to be sent to the Product queue you have set up before. You react upon such messages in the following way:

Check which Stores are impacted

In case you maintain a preconfigured list of Stores, you can avoid this step and Update the search index. Otherwise you need to fetch all the Stores in which the updated Product is available by:
  1. finding out which Product Selections the Product belongs to.
  2. iterating over all returned Product Selections and checking which Stores are impacted.
In the second step you use following where query to filter for Stores that contain activated Product Selections with the changed Product:
productSelections(active=true and productSelection(id in :productSelectionIds))
and you pass productSelectionIds as input variables.

Update the search index

After you found out in which Store(s) the updated Product is available, you can now update the respective search indices with the changed product information.

For each impacted Store,

  • if the Product was deleted, it can be removed from the search index of the corresponding Store.
  • if the Product was updated, fetch the projected Product in the Store.
    • If the Product is available in the Store, put the returned data into the search index of the corresponding Store.
    • If a 404 Not Found error is returned, the Product is not available in this Store and you delete it from the search index of the corresponding Store.

As an optimization, we recommend comparing the version of the updated Product with the version stored in the search index, and to only update in case the version number from the received message is higher than the number in the search index.

React to changes on Product Selections

In the section before you handled changes on the individual Products, but you also want to update our Store's search index whenever a new Product was added or removed to/from the Store's Product Selections. This case is not covered by changes on Products but by messages on the Product Selection queue you set up earlier like this:

Check which Stores are impacted

In case you maintain a preconfigured list of Stores, you can avoid this step and Update the search index. Otherwise you need to fetch all the Stores impacted by the change.
From the ProductSelectionProductAdded Message, the ProductSelectionProductRemoved Message, or the ProductSelectionVariantSelectionChanged Message, you get the ID of Product Selection.
You can check which Stores are impacted by using the following where query to filter for Stores that contain the updated Product Selection:
productSelections(active=true and productSelection(id = :productSelectionId))
and you pass productSelectionId as input variables.

Update the search index

If a Product was added to a Product Selection,

  1. fetch the Product Projection in the Store and
  2. put the returned data into the search index of the corresponding Store.

If the Variant Selection of a Product to a Product Selection was changed,

  1. fetch the Product Projection in the Store and
  2. update the search index of the corresponding Store with the returned data.

If a Product was removed from a Product Selection, delete it from the search index of the corresponding Store.

React to changes on Stores

You also want to update our Store's search index whenever a Product Selection was added or removed to/from a Store or whenever a Product Selection was activated or deactivated in a Store. You handle this by processing messages from the Store queue you set up earlier like this:

Check which Stores are impacted

You get the ID of the impacted Store from the Store Messages.

Update the search index

In case a Store was created, feed all product data into the new search index as described in initial product data feed.
In case a Store was deleted, you can remove the corresponding search index.
In case the Product Selection configuration of a Store was changed, you have two options:
Store configuration is not expected to be changed frequently. As a consequence, the Store configuration is cached up to one minute. You have to respect this delay before fetching any projected product in the Store.

Best practices

Detect unnecessary updates

We recommend you to store the version of the Products in the search indices, and to use this information to detect if some product data is already up to date and does not need any further updates.

Reindex all data from time to time

A search index can become out of date in case an update was missed, or due to changes on a Store's configuration. To keep such inconsistencies to a minimum, we recommend reindexing all product data in a search index, not only on demand, but also from time to time.

The following steps suggest a way how to do that without impacting your live traffic for the search index of one Store:

  1. Create a second search index for this Store.
  2. Let the Product updates now apply to both search indices: the live index and the new index.
  3. Feed all product data into the new search index as described in initial product data feed.
  4. Once the new search index is ready, move your live traffic over to it.
  5. Delete the old live index that is not used anymore.