Skip to main content

Validate Your SDK Setup

Last updated on

The SDK validation checklist helps you ensure that the SDK is implemented according to Harness FME best practices. This document describes the guidelines for incorporating the SDK into your software application in all supported languages. The main purpose is to define the general guidelines, checks, and validations that can be useful for developers and software architects to avoid common mistakes or oversights and to ensure optimal performance of the SDK.

This guide covers recommendations in the following areas:

  • Architectural design principles
  • Safety checks for prevention of race conditions
  • Taking advantage of helpful FME features
  • Configuration validation exercises

These areas each reflect best practices that come from our own experience at Harness FME using the SDK, and the experiences of customers like you. In addition, they also convey an understanding of how SDK works beneath the surface.

You can use or adapt them to your needs. The primary objectives are to ensure resource optimization, maximum application responsiveness, appropriate security enforcements, and proactive issue detection in your project, team, organization, or company source code working with the SDK.

Before you begin​

  • Harness FME SDK installed: You need an SDK already integrated in your application. Go to SDK Overview to install and configure an SDK for your platform.
  • Access to Harness FME UI: You need access to the Harness FME web interface to view usage data, live tail, and Data hub. Go to Settings > Usage for usage data, enable live tail in your environment settings, and access Data hub from the main navigation. For account access, see Getting started with Harness Platform.
  • Permissions: You need View access to Usage data, live tail, and Data hub to validate SDK behavior. See RBAC in Harness for permission configuration.
  • Basic SDK concepts: Familiarity with SDK architecture, impressions (records of getTreatment calls), events (custom tracking data), and CONTROL treatments (default fallback when SDK cannot evaluate). Go to SDK Overview for an introduction to these concepts.

All Harness FME SDKs​

The following validation considerations are relevant for all FME SDKs:

  • Ensure that the SDK is implemented in a singleton pattern. Using the SDK as a singleton (a single instance shared across your application) ensures that the minimum number of threads are used to serve your application. If you do not, you can overload your infrastructure with unnecessary network traffic and use up far more application threads than is required. Use multiple clients on the client side from a single factory if you need to get treatments for multiple different traffic type ids (user identifiers for targeting).

  • Ensure that the SDK is blocked until it signals it is ready. All FME SDKs have a method that blocks the thread until the SDK is ready with feature flag and segment definitions. Calling getTreatment (the method to evaluate feature flags) before the SDK is ready returns CONTROL treatments (the default fallback value).

  • Run the SDK with DEBUG enabled and evaluate any errors or warning messages that are thrown. Pay attention specifically to errors or warnings related to multiple factories, missing event listeners, or other incorrect factory and client configuration. Note: Run with debug enabled for only a few minutes.

  • Ensure any code calling getTreatment is able to handle when ‘CONTROL’ is returned. The SDKs return ‘CONTROL’ as a treatment string when there is a connectivity error. Ensure that there is a fall through in the if-statement to support this.

  • Validate SDK Versions are up to date. Review the SDK tab on the Usage data page. Ensure that the SDKs are up to date, or, at the minimum, they are on the same major version. It is helpful to establish and document a regular SDK update cadence, such as quarterly or biannually. Check the SDK CHANGES.txt on github for any SDKs you are using to see if anything may be relevant to your usage of Harness FME.

  • Evaluate if you can take advantage of the SDK .destroy() method. The .destroy() method of the SDK flushes all stored unpublished events and impressions. This is primarily advantageous for the client side SDKs where you have parts of the user journey that explicitly end their session. On the browser, .destroy() returns a promise. If it’s resolved, then you can be sure that all data is pushed to Harness. On the server side it also may be useful in the event that you need to shutdown a service running the FME SDK. Calling .destroy() ensures that data is posted back to Harness.

  • Validate 1 minute of impressions (and events) on the Harness FME live tail. Enable the Query for about a minute and ensure that the number of impressions received by Harness is about what you’d expect from SDK activity.

  • If you have events coming in, validate them with a similar approach. Ensure that events coming in have the event properties that you would expect them to have.

  • Validate that all expected attributes are being passed to the SDK. Our recommendation is to wrap the SDK to ensure that attributes are always passed to the SDK. A consistent attribute set is important to ensure that all targeting rules have access to the same list of attributes. Client Side SDKs also have the ability to bind attributes to the client itself.

  • Validate 24 hours of impressions (and events) from the Data hub. Take a feature flag that has a known high activity and download all impressions for it from the previous 24 hour period. Ensure that the number of treatments and IDs all match with expectations. For events, take the previous full 24 hours of events, if applicable. With impressions, take note if you are seeing any ‘CONTROL’ treatments as those warrant further investigation to understand why those are happening.

  • Evaluate if you can take advantage of Flag Sets You can use Flag Sets for limiting the flags downloaded by an SDK. Flag Sets allow you to control from Harness FME which flags are downloaded by an SDK. This means you can ensure that only the flags needed for a frontend SDK or a backend SDK are downloaded. This reduces the time for the SDK to get ready while also saving memory and bandwidth.

Browser SDKs (including Angular, React, etc.)​

The following items are specific to browser-based SDKs:

  • (React-specific) Ensure that the SDK is only used in a component or higher-order component (HOC). Review the code samples on our help center. Do not create a new factory for each time a subcomponent is rendered.

  • Evaluate if localStorage mode is something you may be interested in. By default, the SDK stores the cache in memory, which means every time the user visits the page, the SDK has to re-download the whole cache again.

    Using this option stores the cache in the browser file system, which improves the SDK performance after the first load. For more information, refer to Why does the JavaScript SDK return Not Ready status in slow networks? Using this option also allows users to view localStorage in their browser to see rollout plans. If you are use multiple factories, ensure that you are setting prefixes explicitly so they don’t overwrite one another’s localstorage objects.

  • Evaluate if you can take advantage of lazy loading. The SDK factory must have the customer key at initialization time. This key might not be available initially though, especially if the key is provided from another tool (e.g., Segment or mParticle). Using the Lazy init allows you to initialize the SDK by passing a dummy key, then create a new client from the same factory object when the actual customer key is obtained.

Mobile SDKs​

The following items are specific to the mobile SDKs:

  • Ensure that the SDK background syncing is enabled if desired. Mobile SDKs have the synchronizeInBackground configuration setting that allows them to synchronize to the Harness FME servers while in the background. By default, this is disabled.

All Client-side SDKs (including iOS, React, JS, etc.)​

The following items apply to all client-side SDKs (browser, mobile, and other client platforms):

The following items are specific to all client-side SDKs. This includes mobile- and browser-based SDKs.

  • Log out and log back in to your app. Ensure your application is using the proper key value for the SDK after logging out and logging back in. It's also helpful to test by fully closing the app, clearing the cache, and logging in fresh. Verify that the factory is created correctly, and—if you're using both—confirm that you have a Split client for both unauthenticated and logged-in user IDs for flags using different traffic types.

  • Evaluate if you can take advantage of additional SDK emitted events. In addition to SDK_READY, client side SDKs also emit the following additional events that may be useful:

    • SDK_READY_FROM_CACHE. The SDK is ready to evaluate using cached data (which might be stale). If conditions are met, this event is emitted almost immediately since access to the cache is synchronous. Otherwise it won't fire.
    • SDK_READY_TIMED_OUT. When this event fires, it doesn't mean the SDK initialization is interrupted. SDK_READY may still fire at a later time if or when the SDK finishes downloading the necessary information from the servers. This may happen with slow connections or environments which have many feature flags, segments, or dynamic configurations.
    • SDK_UPDATE. This event fires whenever a feature flag or segment is changed. Use this if you want to reload your app every time you make a change in the user interface.

  • Evaluate if you need to change the flush rate. The SDK posts impressions on frequency based on the parameter scheduler.impressionsRefreshRate. By default, the parameter is set to 60 seconds in the browser and 30 minutes in the mobile SDKs. This means after the getTreatment function is called, impressions get posted back to the Harness FME servers after that length of time.

    On mobile devices, if the user stays in the app for less than that amount of time, the impressions stay in the SDK cache. However, they are not posted as the posting thread has not run yet. The next time a user opens the app, the impressions are posted but this can be a few days later.

    For browsers, the JS SDKs use the beacon API to post results back to the Harness FME servers when the page is no longer visible.

    For experimentation, it is desired to have the results up to date. It is recommended to set the parameter scheduler.impressionsRefreshRate to a value less than the average time the user stays on the app.

Server-side SDKs (Python, Node.js, Java, etc.)​

The following items are specific to server-side SDKs:

  • Evaluate your traffic needs. You may need to change the impressionsRefreshrate. The SDK has threads that sync the FME definitions from Harness FME servers to the cache, and posts all impressions and events created in the cache. Make sure the SDK can handle the incoming impressions load because the SDK drops impressions if the cap is reached in the impressionsQueue and impressions can’t be evicted.

    The SDK has parameters to control the run frequency for these threads. We recommend to estimate the highest number of impressions created at peak time from incoming user sessions and divide that by the number of app servers that have the SDK to estimate the number of treatments per minute each SDK generates. Roughly, the SDK’s default impressionsQueue can handle 2000 treatments per minute. If the peak time generates higher impressions, we can reduce the value of scheduler.impressionsRefreshRate by half (for example, from 60 to 30 seconds).

This traffic sizing is for pushing data back to Harness FME servers. Even if the impressionsQueue is full and drops impressions, serving treatments is not affected.

Troubleshooting​

SDK remains in not ready state after blocking in Harness FME

Verify network connectivity to Harness FME servers, check SDK API key is correct, and ensure firewall rules allow outbound connections. Review SDK logs with DEBUG enabled for connection errors.

Impressions not appearing in Harness FME live tail after SDK integration

Ensure getTreatment is being called in your application code, verify the SDK has finished initializing (SDK_READY event), check impressionsRefreshRate is set appropriately, and confirm network connectivity to Harness FME servers.

High number of CONTROL treatments in Harness FME experiment results

CONTROL treatments indicate the SDK could not evaluate the flag. Common causes: calling getTreatment before SDK is ready, network connectivity issues, invalid targeting rules, or missing user attributes required by targeting rules.

Dropped impressions due to impressionsQueue overflow in server-side Harness FME SDK

Reduce scheduler.impressionsRefreshRate to flush impressions more frequently (e.g., from 60 to 30 seconds), or increase impressionsQueue size if supported by your SDK version. Estimate peak treatments per minute and adjust accordingly.

Next steps​

You have validated your Harness FME SDK implementation against best practices. Your SDK is now ready for production deployment with optimized performance and reliability.

  • Go to Viewing experiment results to analyze feature flag performance and metrics.
  • Go to SDK Overview to explore advanced SDK features like lazy loading, background syncing, and Flag Sets.
  • Go to Troubleshooting for detailed SDK troubleshooting guidance and validation rules.