Configuring Metric Sources for Warehouse Native Experimentation
Overview
A Metric Source defines where Harness FME reads metric and outcome data used to evaluate experiment results from your data warehouse.
When creating or editing a Metric SourceA data source that defines how metrics are collected and calculated for an experiment. In Warehouse Native, this data is typically stored in your data warehouse., navigate to the Metrics page from the FME navigation menu and click the Metric Sources tab. Then, click + Create metric source.
- Name: Enter a name for the metric source.
- Owners: Assign one or more owners to make clear who is responsible for maintaining the metric source.
- Description: Enter a description for the metric source.
To define the metric source table, select Table name or SQL query in the Source table section.
- Table Name
- SQL Query
Select a table
Recommended if your event data is already modeled into a clean event table.
- Select an existing table name directly from the schema.
- Click Test connection to validate that Harness can query the table successfully before continuing.
Use a custom SQL query
Recommended for light data transformations (e.g., extracting values from JSON), joins across multiple tables, or filtering by event type.
You must have permissions to access all tables referenced in your query, based on the role and credentials configured when setting up your warehouse connection.
- Write a SQL query that outputs the required fields.
- After entering your query, click Run query to validate and preview results before proceeding.
After setting up Metric Sources, you can create metric definitions to aggregate event data by type (i.e., count, sum, or average). With Metric Sources configured, your metrics remain consistent, standardized, and reusable across experiments and analyses.
Harness shows a preview of the data returned from your table so you can validate that the expected rows and columns are present.
Configure your environments
- Column Mapping
- Hardcoded Value
Select an environment column and map its values to Harness FME environments.
For example, select the ENV_NAME column and map its values (US-Prod, UK-Prod) to your Harness project’s Production environment and map the Stg values (US-Stg, UK-Stg) to your Harness project’s Staging environment.
This is recommended if a single metric source spans multiple environments.
Instead of selecting a column, set a fixed Harness FME environment for the entire Metric Source (e.g., always Production).
This is recommended if the source is scoped to one environment.
Configure your traffic types
Similar to environments, traffic types can be set up in two ways:
- Column Mapping
- Hardcoded Value
Select a traffic type column (e.g., ttid) and map its values to Harness FME traffic types (e.g., user, account, or anonymous).
This is recommended if the same Metric Source covers multiple population types.
Instead of selecting a column, set a fixed Harness FME traffic type for the entire Metric Source (e.g., always account).
This is recommended if the data set only represents one population type.
Configure events
Metric Sources allow flexibility in how event types are set up.
- Column Mapping
- Hardcoded Value
Select an event type column (e.g., EVENT_NAME) so the metric source can be reused across multiple metric definitions.
This is recommended for general-purpose event sources.
Instead of selecting a column, set a fixed event name for the entire metric source (e.g., always product_page_view).
This is recommended if the source is meant to be tightly scoped to a single event.
For each Metric Source, the following fields are required:
| Field | Description |
|---|---|
| Key | A unique identifier that matches the key in the associated Assignment Source. This allows Harness FME to join exposures and metrics correctly. |
| Timestamp | The timestamp when the event occurred. This is used to calculate metric values over time. |
If you plan to create Sum or Average metrics from this metric source, map an Event Value column that contains numeric data (for example, page_load_time_seconds or order_value). The Event Value field is optional and is only used for sum or average metrics.
Add custom fields
Click + Add new custom field to map any additional columns from your metric source that are not included in the required fields section.
- Select a column from the dropdown menu.
- Enter a label for the field.
- Click Save.
Custom fields can later be used to filter data when creating metrics. Common examples include PLAN_TIER, DEVICE, and COUNTRY.
Manage metric sources
In order to maintaining general-purpose reusable sources while also creating custom sources for sensitive or high-volume use cases, you can adopt a hybrid approach:
- Reusable, standardized sources are recommended if you want one source to power many metric definitions (e.g., a general events table with filtering by event type).
- Custom sources are useful if you want to tightly scope data for privacy, relevancy, or performance.
Once you've set up the metric sources that best fit your workflow, you can manage them directly in Harness FME.
- Edit: Update the query, mappings, or field configuration to align with schema changes in your warehouse. Changes may disrupt metrics or experiments relying on this source.
- Delete: Remove unused or invalid sources to prevent accidental use. Before deletion, confirm no metric definitions depend on the source.
Troubleshooting
If you encounter issues when configuring a Metric Source:
Test Connection or Run Query Fails
- Ensure the table or query is valid and accessible with your warehouse connection credentials.
- Verify that schemas and table names are spelled correctly.
No Data Appears in Preview
- Confirm the query/table returns rows for the event(s) you expect.
- If you are using event filtering in SQL, test the query directly in your warehouse.
Missing Columns
Verify that the required fields exist and are returned by your query.
Timestamp Format Issues
Event timestamps can be provided in epoch milliseconds (numeric) or in a timestamp/datetime format.
If you are using epoch values (e.g., EVENT_TIMESTAMP_MS), there is no need to convert them in your SQL query; Harness FME can handle both formats directly.
Incorrect Environment/Traffic Type Mapping
- Check that each warehouse value is mapped to the intended Harness environment or traffic type.
- Use hardcoded values if everything should map to a single option.
Unable to Delete Source
Check which metric definitions are currently using it. Delete or reassign those metrics before removing the source.