Split Synchronizer

Split Synchronizer coordinates feature flag and segment data between Harness Feature Management Experimentation (FME) servers and SDKs that do not have native shared caching. It uses Redis as a shared datastore, allowing multiple SDK instances to stay synchronized without each fetching data independently.

Use Split Synchronizer with PHP, Python, .NET, Node.js, Ruby, Go, or Java SDKs in environments where multiple application processes run concurrently and need consistent feature flag evaluations. The Synchronizer also posts impression data (records of flag evaluations) and event data (custom tracking from SDK .track calls) back to Harness servers for analytics and experimentation reporting.

What you will learn from this topic

• What Split Synchronizer is and when to use it: Understand how Split Synchronizer coordinates data across SDK instances and which SDK languages require it for shared caching.

• How to install Split Synchronizer: Learn installation methods including Docker, command-line (Linux/Mac/Windows), and cloud platforms like Heroku and AWS ECS.

• How to configure for different Redis topologies: Configure Split Synchronizer for standard Redis (single instance), Redis Sentinel (high availability with automatic failover), or Redis Cluster (horizontal scaling).

• How to monitor and troubleshoot: Use the admin dashboard, health check endpoints, and troubleshooting guidance to ensure Synchronizer runs reliably in production.

Before you begin

Before installing Split Synchronizer, ensure you have the following prerequisites:

• Harness FME account: You need access to Harness Feature Management Experimentation. If FME does not appear in your Harness account, contact your account administrator or Harness Support.

• Redis instance: Split Synchronizer requires a Redis instance running and accessible. Supported topologies include standard Redis (single instance), Redis Sentinel (high availability with automatic failover), and Redis Cluster (horizontal scaling). Go to Redis installation documentation to install Redis if you do not have it already. For production deployments, Redis 5.0 or later is recommended.

• SDK API key: You need an SDK-type API key to authenticate Split Synchronizer with Harness FME servers. To find your SDK API key:

  1. In Harness, go to Project Settings > Feature Flags > Environments

  2. Select your environment

  3. Copy the SDK Key (also called Server SDK Key or API Key)

  Note: Use an SDK-type key, not an Admin or Client key. SDK keys allow Synchronizer to fetch feature flag definitions and post impressions (flag evaluation records) and events (custom tracking data from SDK .track calls) back to Harness for analytics.

• Permissions: If your Harness account uses role-based access control (RBAC), ensure you have permissions to view environment settings and SDK keys. Typically, you need View permissions on Feature Flags. Go to RBAC in Harness to understand permissions and Manage roles to configure role assignments.

Supported SDKs

Split Synchronizer works with the following Harness FME server-side SDKs:

• PHP SDK

• Python SDK

• .NET SDK

• Node.js SDK

• Ruby SDK

• Go SDK

• Java SDK

Synchronizer compatibility matrix

Notes:

• “Not implemented” means no Synchronizer support for that language.

• “N/A” means not applicable. The Synchronizer is not used for those platforms.

• SDK versions noted are minimum or recommended versions tested with each Synchronizer version.

If you need Split Synchronizer support for a language not listed here, contact Harness Support to discuss your options.

Architecture

Split Synchronizer performs five primary actions:

• Fetch feature flags: Retrieve feature flag definitions from Harness FME servers.

• Fetch segments: Retrieve segment lists and membership data.

• Post impressions: Send impression data (records of flag evaluations) generated by SDKs to Harness servers for analytics.

• Post telemetry: Send SDK performance metrics such as latencies and operational data.

• Post events: Send custom event data generated by SDK .track method calls to Harness servers.

Synchronizer architecture diagram

Split Synchronizer coordinates data between Redis (shared by SDKs) and Harness FME servers

Setup

Split Synchronizer can be installed using Docker (recommended), command-line binary, or deployed to cloud platforms like Heroku or AWS ECS.

Docker installation (recommended)

Docker is the recommended installation method for most deployments. The Split Synchronizer Docker image is available on Docker Hub.

To install using Docker:

1. Pull the image: docker pull splitsoftware/split-synchronizer

2. Run the container with your configuration:

Command line

To install and run the service from command line, follow the steps below depending of your platform.

Linux

On Linux systems, the Synchronizer service install script is invoked with this.

OSX

On OSX systems, the Synchronizer service install script is invoked with this.

Windows

On Microsoft Windows systems, follow these steps:

1. Download the app from https://downloads.split.io/synchronizer/split_sync_windows.zip.

2. Unzip the downloaded file.

3. Run it!

Run the service

To run the service, paste the snippet below into your command line terminal and add in your SDK-Key.

Linux/Mac

Windows

Open the cmd terminal or the PowerShell terminal, go to (cd) unzipped Split Synchronizer folder, and type:

Recommended configuration for production

You can run the service with the simple steps above, but the system is more stable in your production environment when you run the job with a scheduling system. We recommend starting the synchronizer via supervisord, a daemon that launches other processes and ensures they stay running.

To use supervisord, make sure that it is installed on your machine. You can get help on the installation at the official Supervisord documentation.

When supervisord is installed into your project, copy and paste the program below anywhere into the supervisord.conf file that should now be in your project.

Advanced configuration

The Synchronizer service has a number of knobs for configuring performance. Each knob is tuned to a reasonable default, however, you can override the default values by changing a splitio.config.json file or by setting your customer values as parameters of -config in the command line option. In this section, we lay out all the different knobs you can configure for performance, Redis, and logging.

The splitio.config.json file provided via the -config option lets you control how often the synchronizer fetches data from Harness servers. You can create a sample JSON file automatically with default values by running this command.

Methods to configure the Split Synchronizer

You can configure the Split Synchronizer service using the command line or by directly editing the above mentioned JSON configuration file.

Handling High Impression Rates with Synchronizer

When using a server-side SDK with the Split Synchronizer and Redis, it is important to properly configure the Synchronizer to handle a high load of incoming impressions efficiently.

Key Configuration Parameters

The Split Synchronizer (version 1.6.0 and above) exposes several parameters that control impression processing performance:

• impressionsMaxSize: Maximum size for the impressions queue.

• impressionsRefreshRate: How often impressions are processed and sent (in seconds).

• impressionsThreads: Number of threads handling impressions processing.

• impressionsPerPost: Number of impressions sent per post request.

Adjust these parameters according to your expected impression volume. Because the Synchronizer uses multithreading, increasing impressionsThreads can reduce latency in posting impressions.

Below is an example JSON configuration designed to handle approximately 100,000 impressions per minute. Update the API Key, Redis host, port, and database number to match your environment before applying.

Configuration reference

Split Synchronizer configuration can be specified using command-line flags, JSON configuration file, or environment variables. The following reference lists all available configuration options.

CLI configuration options and equivalents in JSON and environment variables

Deploying Synchronizer on Heroku

Follow these steps to deploy the Synchronizer container on Heroku:

1. Clone the GitHub repository or download the source code and unzip it to a new folder (e.g., mysync).

2. Open a terminal and cd to that folder.

3. Run this command to create a Heroku app:

4. You should see a response like:

5. Set the stack to container and add the Go language buildpack:

6. Open the existing Dockerfile and replace the last line:

   with:

   Heroku only supports CMD for containers.

7. Add the following content to a heroku.yml file:

8. Add a Procfile with the content:

9. On the Heroku dashboard, open your app, go to the Settings tab, and add the following environment variables:

   • SPLIT_SYNC_API_KEY

   • SPLIT_SYNC_REDIS_HOST

   • SPLIT_SYNC_REDIS_PORT

   • SPLIT_SYNC_REDIS_DB

   • SPLIT_SYNC_REDIS_PASS

10. Modify entrypoint.sh to map the admin dashboard to Heroku's $PORT environment variable.

    Replace the last line:

    with:

11. Run these git commands to deploy to Heroku:

12. Check logs using heroku logs.

13. You should see output similar to:

14. Access the Admin Dashboard by visiting https://[heroku-app-name].herokuapp.com/admin/dashboard.

Using the Synchronizer as a proxy service on Heroku

Since Heroku maps only one port, you need to assign the proxy listener port to the $PORT environment variable.

1. Update entrypoint.sh by replacing exec split-sync ${PARAMETERS} with exec split-sync ${PARAMETERS} -proxy-port $PORT.

2. Modify Procfile by changing its content to web: sh entrypoint.sh.

3. Add the following environment variables in the Heroku UI:

   • SPLIT_SYNC_PROXY (value: on)

   • SPLIT_SYNC_PROXY_SDK_APIKEYS (value: your customer API key)

4. Deploy changes:

5. Configure SDKs. An example Java SDK configuration:

Since Heroku maps only a single port, the Admin Dashboard will not be accessible when running in proxy mode.

Listener

The Split Synchronizer provides an impression listener that bulks post impressions to a user-defined HTTP endpoint.

The endpoint should expect a POST request, containing a JSON body with the following format.

Currently, the configuration options are available in the integrations.impressionListener section of the JSON configuration file detailed in Advanced configuration.

Using a network proxy

If you need to use a network proxy, configure proxies by setting the environment variables HTTP_PROXY and HTTPS_PROXY. The internal HTTP client reads those variables and uses them to perform the server request.

Inject a certificate into a Synchronizer Docker image

If the Synchronizer Docker container is running in a network that uses an SSL proxy, the Synchronizer may fail to authenticate the root certificate. This causes errors like the following when trying to connect to Harness FME servers to fetch feature flag definitions:

To resolve this, you need to inject the root certificate into the Synchronizer Docker image by rebuilding it with the proxy certificates included.

1. Clone the Synchronizer public repository:

2. Change to the cloned directory:

3. Copy your proxy certificates (root, intermediate, and proxy certs) into this folder. For example:

4. Open the Dockerfile in the split-synchronizer folder with a text editor, and just before the line containing EXPOSE 3000 3010, add:

5. Save and close the Dockerfile.

6. Build the new Docker image:

7. Run the new image to verify it works (replace environment variables as needed). The http_proxy and https_proxy variables are optional based on your setup:

Admin dashboard and monitoring

Split Synchronizer provides admin endpoints and a dashboard for monitoring synchronization status, cached data, and performance metrics in real-time.

Admin endpoints

The Split Synchronizer service exposes admin endpoints on port 3010 by default. For security, endpoints support HTTP Basic Authentication configured through admin-username and admin-password settings.

/info/ping

A ping endpoint to monitor the service status. If the service is running, it sends the text response pong and the HTTP status code 200.

/info/version

Returns the split-sync version in JSON format.

/info/uptime

Returns the uptime string representation in JSON format.

/info/config Returns a JSON object describing the current configuration of the Synchronizer.

/health/application Returns a JSON object describing whether the synchronizer is healthy or not.

/health/dependencies Returns a JSON object describing whether the servers the synchronizer depends on are healthy or not.

Observability

The Split Synchronizer exposes an observability endpoint that provides visibility into cached data.

You can use this endpoint in versions 5.0.3 or later.

The following metrics are available:

Example response

Use this endpoint to verify that feature flags and segments are synchronized and stored correctly.

Admin Dashboard

Split-sync has a web admin UI out of the box that exposes all available endpoints. Browse to /admin/dashboard to see it.

The dashboard is organized in four sections for ease of visualization:

• Dashboard: Tile-sorted summary information, including these metrics:

  • Uptime: Uptime metric

  • Healthy Since: Time passed without errors

  • Logged Errors: Total count of error messages

  • SDKs Total Hits: Total SDKs requests

  • Backend Total Hits: Total backend requests between split-sync and Harness servers

  • Cached Feature flags: Number of feature flags cached in memory

  • Cached Segments: Number of segments cached in memory

  • Impressions Queue Size: shows the total amount of Impressions stored in Redis (only Producer Mode).

  • Impressions Lambda: shows the eviction rate for Impressions (only Producer Mode).

  • Events Queue Size: shows the total amount of Events stored in Redis (only Producer Mode).

  • Events Lambda: shows the eviction rate for Events (only Producer Mode).

  • SDK Server: displays the status of Split server for SDK.

  • Events Server: displays the status of Split server for Events.

  • Streaming Server*: displays the status of Split streaming service

  • Auth Server*: displays the status of Split server for initial streaming authentication

  • Telemetry Server*: displays the status of Split server for telemetry capturing.

  • Storage: (only Sync mode) displays the status of the storage.

  • Sync: displays the status of the Synchronizer.

  • Last Errors Log: List of the last 10 error messages

• SDK stats: Metrics numbers and a latency graph, measured between SDKs requests integration and proxy.

• Data inspector: Cached data showing feature flags and segments; filters to find keys and feature flag definitions.

• Queue Manager: expose sizes of Impressions and Events queues.

Service shutdown

The split-sync service can catch a kill sig command and start a graceful shutdown, flushing all cached data progressively. Additionally, you can perform graceful stop and force stop (kill -9) with one click from the admin dashboard.

If you have configured a Slack channel and the Slack Webhook URL, an alert is sent to the channel when and initialization or shutdown is performed.

Troubleshooting

Use the following troubleshooting guidance to resolve common Split Synchronizer issues.

Details: When using Synchronizer in proxy mode, the SDK may never become ready and receive a 500 HTTP error. Synchronizer’s debug logs show successful calls to Harness FME servers, but the JSON response contains an empty list of feature flags:

This indicates that no feature flags are present in the environment associated with the SDK API key used by the Synchronizer. Although the upstream HTTP call succeeds, Synchronizer returns a 500 error to the SDK because it cannot compute any treatments without any feature flags

• Ensure that the environment linked to the SDK API key used by Synchronizer has active feature flags configured.

• Alternatively, verify that you are using the correct SDK API key for the intended environment with feature flags.

SDK getTreatment Always Returning 'control' When Using Synchronizer Docker

After installing and running the Split Synchronizer Docker instance with a Redis instance, and configuring an SDK to use Redis, the getTreatment call always returns 'control'.

By default, the Synchronizer Docker instance uses a prefix for Redis keys. If the SDK does not specify the same prefix in its Redis configuration, it cannot read the data stored by Synchronizer.

1. Check if the Synchronizer is using a prefix by running:

   Look for text before "SPLITIO" in the keys.

   Example output showing the prefix myprefix:

2. Update your SDK configuration. Specify the redisPrefix parameter in your SDK configuration so it matches the Synchronizer prefix.

   For example, in the Python SDK:

No Impressions Sent from Python SDK 7.x and Synchronizer 1.x

When using Synchronizer 1.x with Python SDK 7.x, the Python SDK processes treatments correctly and Synchronizer does not report any errors. However, no impressions are sent to the Harness FME servers.

Starting in Python SDK 7.0.0, design changes were made to align with the enhancements introduced in Synchronizer 2.0. Therefore, when using Python SDK 7.x, you must upgrade to Synchronizer 2.x.

Details: After starting Split Synchronizer (version 1.6.0 and above), the debug log and admin dashboard show the error POST method: Status Code: 404 - 404 Not Found on all network POST calls.

This error occurs because an incorrect API key is passed to the Synchronizer, causing the Synchronizer to be unable to find the Account in the Harness FME servers.

1. Verify the API key used by Synchronizer is correct. Synchronizer API key must be of SDK type. API keys can be viewed from Admin settings on the API keys page: https://app.split.io/org/[Your Account ID]/admin/apis.

   
2. Ensure the API key is properly passed to Synchronizer.

   Common ways include:

   • Command line argument: -api-key <APIKEY>

   • JSON configuration file: The apiKey property is used to issue requests against Harness FME servers.

     
   • Proxy mode usage: When Synchronizer is used in proxy mode (not Redis), the "auth" section and "sdkAPIKeys" allow setting custom API keys for internal use, enabling SDKs to use the internal custom API key.

     
   • Docker environment variable: If running Synchronizer within the packaged Docker image, use:

Next steps

You have successfully installed and configured Split Synchronizer. Your SDKs can now share feature flag data through Redis while Synchronizer coordinates updates with Harness FME servers.

• Configure FME SDKs: Update SDK configuration to connect to your Redis instance and use Split Synchronizer.

• Monitor Split Synchronizer health: Use the admin dashboard at http://<host>:3010/admin/dashboard to monitor synchronization status, queue sizes, and performance metrics.