Configuring an Elasticsearch Instance

Configure SystemLink Enterprise to access a remote Elasticsearch database to enhance scalability and performance.

You must follow these steps under the following conditions.

  • You are upgrading from a SystemLink Enterprise version before 2025-07.
  • You want to improve your search performance.
    Note This feature is currently only available for the FileIngestion service.

Choosing an Elasticsearch Deployment

SystemLink uses Elasticsearch to improve search performance. You can use an Elasticsearch instance in the same Kubernetes cluster as your SystemLink Enterprise installation or an external instance.

Use the following table to choose the Elasticsearch deployment that best suits your use case.

Deployment When to Use Details
SystemLink Elasticsearch Helm chart
  • You need your database in the same Kubernetes cluster as your SystemLink Enterprise installation.
  • Your organization is comfortable managing an Elasticsearch instance.
  • You want user autoprovisioning and user dedicated configurations for SystemLink Enterprise.

You can run this instance on existing Kubernetes worker nodes or dedicated worker nodes using taints and tolerations.

For more information and recommended resources, refer to Sizing Considerations when Deploying an Elasticsearch Instance.
Elastic Cloud

You want to simplify database provisioning, operation, backup, and restore operations.

For more information and recommended resources, refer to Sizing Considerations when Deploying an Elasticsearch Instance.

Configuring the SystemLink Elasticsearch Helm Chart with Enabled Autoprovisioning

To configure Elasticsearch for the first time, you must provision the passwords.

  1. Open the elasticsearch.yaml file.
  2. Set the sl-elasticsearch.usersProvisioning.enabled value to True.
  3. Open the elasticsearch-secrets.yaml file.
  4. Set the password for each index.
    Service User Password
    fileingestioncdc filescdc sl-elasticsearch.secrets.filescdcPassword
  5. Deploy Elasticsearch.

Configuring a Remote Elasticsearch Instance or the SystemLink Elasticsearch Helm Chart with Disabled Autoprovisioning

To configure Elasticsearch for the first time, you must provision the indexes.

  1. Open the systemlink-secrets.yaml file.
  2. Set the password for each index.
    Note Some services require privileges on multiple indexes. For example, if the files,files_* parameter is specified, the service requires privileges for the following indexes:
    • The files index.
    • All indexes that match the files_* pattern (where * is a wildcard).
    Service Database User Password
    fileingestioncdc files,files_* filescdc fileingestioncdc.secrets.elasticsearch.password
  3. Deploy Elasticsearch.

Sizing Considerations When Deploying an Elasticsearch Instance

Configure the Elasticsearch instances to handle the scale of data you have.

Resource requirements are based on service usage. Refer to the following table for tested configurations at a specified scale when configuring resources based on your expected usage.

Note These resource requirements increase as Elasticsearch usage increases.
Service Scale Nodes CPU RAM Persistence Primary shards
FileIngestion 25 million files 2 1 4 GB 50 GB 2
FileIngestion 80 million files 4 1 4 GB 200 GB 4

Based on your scale, select and apply that configuration.

  1. Open the elasticsearch.yaml file.
  2. Set the sl-elasticsearch.elasticsearch.master.replicaCount value to the listed nodes.
  3. Set the sl-elasticsearch.elasticsearch.master.resources.requests.cpu value to the listed CPU.
  4. Set the sl-elasticsearch.elasticsearch.master.resources.requests.memory value and the sl-elasticsearch.elasticsearch.master.resources.limits.memory value to the listed RAM.
  5. Set the sl-elasticsearch.elasticsearch.master.persistence.size value to the listed persistence storage size.
  6. Open the systemlink.yaml file.
  7. Set the fileingestioncdc.job.connectors.sink.elasticsearch.index.primaryShardsCount value to the listed number of shards.
    Note The shards configuration only works when on the initial deployment. To change the configuration after the first deployment, you must manually delete the files index from Elasticsearch and redeploy the FileIngestionCDC application.