> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/utilities/json-and-xml-functors.md). # JSON and XML functors Use functors to select information from JSON and XML sources in pipeline steps This topic describes how to use JSON and XML functors to select contents from JSON and XML sources. These functors reduce the amount of shell scripting needed to pull JSON and XML information into your Harness pipeline steps. ## What you will learn from this topic * How to use the [JSON functor](#json-functor) methods `select()`, `object()`, `list()`, and `format()` * How to [fetch and mask attributes from JSON secrets](#fetch-and-mask-attributes-from-json-secrets) with `secretJson` * How to use the [XML functor](#xml-functor) `xml.select()` method * How to run a [YAML pipeline example](#yaml-pipeline-example) that demonstrates the functors with HTTP and Shell Script steps ## Before you begin * **HTTP step**: Most examples use an HTTP step and `httpResponseBody`. Go to [HTTP step](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/utilities/http-step.md) to configure the step. * **Shell Script step**: Examples echo output variables in a Shell Script step. Go to [Shell Script step](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/utilities/shell-script-step.md) to configure the step. * **Expressions**: Functors are used inside Harness expressions such as `<+json.select(...)>`. *** ## JSON functor The JSON functor uses the JSON library [JsonPath](https://github.com/json-path/JsonPath). The JSON functor methods are described below. ### select() The following reference covers the `select()` method. * **Syntax:** `select(string, string)` * **Description:** Select attribute values using a path. * **Parameters:** literal string, string (typically, the second string is `httpResponseBody`). This is a path to identify the desired JSON attribute value from a JSON response. **Example:** Here is the JSON array to select a value from: ```json { "data": { "attributes": { "name": "new-construction-api", "version_pins": { "mvn-service://new-construction-api": "0.0.253-feature_NC-6595.8d268cd~nc1.6312a66" } } } } ``` You can find this example at . To select the value `0.0.253-feature_NC-6595.8d268cd~nc1.6312a66`, you would use `select()` to specify the path to the value, like this: ```json <+json.select("data.attributes.version_pins.mvn-service://new-construction-api", httpResponseBody)> ``` The `httpResponseBody` argument indicates that you select the path within the HTTP response body. `httpResponseBody` is propagated from the HTTP request. A common use of `select()` is in an HTTP step. For example, the following HTTP step uses a variable named **book** and the `select()` method in **Value** to obtain the value `0.0.253-feature_NC-6595.8d268cd~nc1.6312a66` from the HTTP response payload at the URL specified in **URL**. ![HTTP step configured with json.select output variable](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-f42843292248d0014f0dac66783382eef725faa6%2Fjson-and-xml-functors-06.png?alt=media) When this HTTP step is executed, in its **Output** tab, you can see the HTTP response in **HTTP Response Body** and the selection in the **Output Variables**: ![HTTP step output showing selected JSON value](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-4cb17c84c5c077c64779de6cce9a9547376e0052%2Fjson-and-xml-functors-07.png?alt=media) You can also use a Shell Script step to echo the book output like this: ```bash echo <+pipeline.stages.Functors.spec.execution.steps.jsonselect.output.outputVariables.book> ``` When the pipeline is executed, the value of **book** is output: ![Shell Script step echoing the book output variable](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-411d1097305287ab2519653cd141570aaabecfa3%2Fjson-and-xml-functors-08.png?alt=media) ### object() The following reference covers the `object()` method. * **Syntax:** `object(string)` * **Description:** Selects objects from a JSON collection. * **Parameters:** string. This is a JSON key used to identify the desired JSON attribute value from a JSON response. Typically, `httpResponseBody`. **Example:** Here is the JSON to query: ```json {"item":"value1","level1":{"level2":"value2"}} ``` You can find this example at . Here is the query using the `object()` method to select `value1`: ```bash <+json.object(httpResponseBody).item> ``` You can add the `object()` method to an HTTP step and output it: ![HTTP step configured with json.object output variable](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-fd38d1b4e9757c8c552e864ebd8ca646921c1280%2Fjson-and-xml-functors-09.png?alt=media) When this HTTP step is executed, in its **Output** tab, you can see the HTTP response in **HTTP Response Body** and the object in the **Output Variables**: ![HTTP step output showing object selection](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-6adb6591fabd65778ac2f11a4f16fbd9cc27a90a%2Fjson-and-xml-functors-10.png?alt=media) ### list() The following reference covers the `list()` method. * **Syntax:** `list(string, string)` * **Description:** Returns list object. * **Parameters:** literal string, string (typically, `httpResponseBody`). Using the `list().get()` method returns items from the list. **Example:** Here is the JSON to query: ```json { "books": [{ "isbn": "9781593275846", "title": "Eloquent JavaScript, Second Edition", "subtitle": "A Modern Introduction to Programming", "author": "Marijn Haverbeke", "published": "2014-12-14T00:00:00.000Z", "publisher": "No Starch Press", "pages": "472", "description": "JavaScript lies at the heart of almost every modern web application, from social apps to the newest browser-based games. Though simple for beginners to pick up and play with, JavaScript is a flexible, complex language that you can use to build full-scale applications." }, { "isbn": "9781449331818", "title": "Learning JavaScript Design Patterns", "subtitle": "A JavaScript and jQuery Developer's Guide", "author": "Addy Osmani", "published": "2012-07-01T00:00:00.000Z", "publisher": "O'Reilly Media", "pages": "254", "description": "With Learning JavaScript Design Patterns, you'll learn how to write beautiful, structured, and maintainable JavaScript by applying classical and modern design patterns to the language. If you want to keep your code efficient, more manageable, and up-to-date with the latest best practices, this book is for you." }, { "isbn": "9781449365035", "title": "Speaking JavaScript", "subtitle": "An In-Depth Guide for Programmers", "author": "Axel Rauschmayer", "published": "2014-02-01T00:00:00.000Z", "publisher": "O'Reilly Media", "pages": "460", "description": "Like it or not, JavaScript is everywhere these days-from browser to server to mobile-and now you, too, need to learn the language or dive deeper than you have. This concise book guides you into and through JavaScript, written by a veteran programmer who once found himself in the same position." }, { "isbn": "9781491950296", "title": "Programming JavaScript Applications", "subtitle": "Robust Web Architecture with Node, HTML5, and Modern JS Libraries", "author": "Eric Elliott", "published": "2014-07-01T00:00:00.000Z", "publisher": "O'Reilly Media", "pages": "254", "description": "Take advantage of JavaScript's power to build robust web-scale or enterprise applications that are easy to extend and maintain. By applying the design patterns outlined in this practical book, experienced JavaScript developers will learn how to write flexible and resilient code that's easier-yes, easier-to work with as your code base grows." } ] } ``` You can find this example at . Here is the query using the `list()` method to select `pages` from the 3rd book: ```json <+json.list("books", httpResponseBody).get(2).pages> ``` Since the JSON array starts at 0, `get(2)` returns `pages` from the third list item (`"pages": "460"`). You can add the `list()` method to an HTTP step and output it using the variable **list**: ![HTTP step configured with json.list output variable](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-57ac5104d5c1635722570f34e90b96def3979f39%2Fjson-and-xml-functors-11.png?alt=media) When this HTTP step is executed, in its **Output** tab, you can see the HTTP response in **HTTP Response Body** and the list item in the **Output Variables**: ![HTTP step output showing list item selection](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-8dee71251506e4c19b0286d229797d2736e47f08%2Fjson-and-xml-functors-12.png?alt=media) ### format() The following reference covers the `format()` method. * **Syntax:** `format(object)` * **Description:** Format the array passed as the string value in JSON format. * **Parameters:** object. Typically, this is the response from the HTTP response body (`httpResponseBody`). The `httpResponseBody` argument indicates that you select the path within the HTTP response body. `httpResponseBody` is propagated from the HTTP request. **Example:** Add an HTTP step to obtain the `httpResponseBody`, then reference that value in a subsequent Shell Script step. This example uses . If you render the `httpResponseBody`, you get: ```json {data:{attributes:name:new-construction-api}} {data:{attributes:version_pins:{mvn-service://new-construction-api:0.0.253-feature_NC-6595.8d268cd~nc1.6312a66}}} ``` If you render it using `<+json.format(<+pipeline.stages.Functors.spec.execution.steps.jsonformat1.output.httpResponseBody>)>`, you get a JSON formatted string: ```json {\n "data": {\n "attributes": {\n "name": "new-construction-api",\n "version_pins": {\n "mvn-service://new-construction-api": "0.0.253-feature_NC-6595.8d268cd~nc1.6312a66"\n }\n }\n }\n}\n ``` {% hint style="info" %} **IMPORTANT** 1. JSON accepts the control sequence `\n` as strings. To format JSON, use `jq` to prettify the JSON. 2. Conditional expressions within double quotes are considered strings. So, `"<+json.select("fields.status.name", httpResponseBody)>"=="In Progress"` is treated as string comparison and will not work. Use `<+json.select("fields.status.name", httpResponseBody)>=="In Progress"` instead. The keyword null, too, should not be enclosed in quotes during comparison. Here is an example of a null comparison: `<+json.object(httpResponseBody).fields.parent>!=null` {% endhint %} ### Fetch and mask attributes from JSON secrets You can fetch attributes from JSON files marked as secrets and have them masked in the pipeline execution page outputs and execution logs by setting the output variable type to `secret`. This feature uses a functor called `secretJson`, along with support for the following functors: `select()`, `list()`, and `object()`. *** ## XML functor The [XPath](https://developer.mozilla.org/en-US/docs/Web/XPath) functor has one method: `xml.select()`. ### select() The following reference covers the `xml.select()` method. * **Syntax:** `xml.select(string, string)` * **Description:** Returns XML file. * **Parameters:** String using an XPath expression and XML file, and a string for `httpResponseBody`. **Example:** Here are the contents of the XML file to query: ```xml Everyday Italian Giada De Laurentiis 2005 30.00 Harry Potter J K. Rowling 2005 29.99 XQuery Kick Start James McGovern Per Bothner Kurt Cagle James Linn Vaidyanathan Nagarajan 2003 49.99 Learning XML Erik T. Ray 2003 39.95 ``` This example uses . Here is the query using the `xml.select()` method to select the title from the first book: ```bash <+xml.select("/bookstore/book[1]/title", httpResponseBody)> ``` You can add the `xml.select()` method to an HTTP step and output it using **Response Mapping**: ![HTTP step configured with xml.select output variable](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-ea3af742b9c728607f64ee4d49cdbc4bbb6b8f6b%2Fjson-and-xml-functors-13.png?alt=media) Next, reference the output variable **select** in a Shell Script step: ```bash echo <+pipeline.stages.Functors.spec.execution.steps.XML_select.output.outputVariables.select> ``` When the pipeline is executed, the result is: ```bash Executing command ... Everyday Italian Command completed with ExitCode (0) ``` You can also see the entire XML file in the deployment **Details** section: ![Deployment details showing XML response body](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-d2c138d763b58004fabbb64d7a8ede0b1d8c676e%2Fjson-and-xml-functors-14.png?alt=media) *** ## YAML pipeline example The following YAML is a pipeline that demonstrates all of the functors.
Functors pipeline YAML ```yaml pipeline: name: Functors identifier: Functors projectIdentifier: CD_Quickstart orgIdentifier: default tags: {} stages: - stage: name: Functors identifier: Functors description: "" type: Deployment spec: serviceConfig: serviceDefinition: type: Kubernetes spec: manifestOverrideSets: [] manifests: [] artifacts: sidecars: [] serviceRef: nginx infrastructure: environmentRef: quickstart infrastructureDefinition: type: KubernetesDirect spec: connectorRef: account.k8s_cluster namespace: default releaseName: docs allowSimultaneousDeployments: false infrastructureKey: "" execution: steps: - step: type: Http name: json.select identifier: jsonselect timeout: 40s spec: url: https://raw.githubusercontent.com/wings-software/harness-docs/main/functors/select.json method: GET headers: [] outputVariables: - name: book value: <+json.select("data.attributes.version_pins.mvn-service://new-construction-api", httpResponseBody)> type: String - step: type: ShellScript name: json select export identifier: json_select_export timeout: 40s spec: shell: Bash onDelegate: true source: type: Inline spec: script: echo <+pipeline.stages.Functors.spec.execution.steps.jsonselect.output.outputVariables.book> environmentVariables: [] outputVariables: [] executionTarget: {} - step: type: Http name: json.object identifier: jsonobject timeout: 40s spec: url: https://raw.githubusercontent.com/wings-software/harness-docs/main/functors/object.json method: GET headers: [] outputVariables: - name: item value: <+json.object(httpResponseBody).item> type: String - step: type: Http name: json.list identifier: jsonlist timeout: 40s spec: url: https://raw.githubusercontent.com/wings-software/harness-docs/main/functors/books.json method: GET headers: [] outputVariables: - name: list value: <+json.list("books", httpResponseBody).get(2).pages> type: String - step: type: Http name: jsonformat1 identifier: jsonformat1 timeout: 10s spec: url: https://raw.githubusercontent.com/wings-software/harness-docs/main/functors/select.json method: GET headers: [] outputVariables: [] - step: type: ShellScript name: jsonformatecho identifier: jsonformatecho timeout: 10m spec: shell: Bash onDelegate: true source: type: Inline spec: script: echo <+json.format(<+pipeline.stages.Functors.spec.execution.steps.jsonformat1.output.httpResponseBody>)> environmentVariables: [] outputVariables: [] executionTarget: {} - step: type: Http name: XML select identifier: XML_select timeout: 10s spec: url: https://raw.githubusercontent.com/wings-software/harness-docs/main/functors/books.xml method: GET headers: [] outputVariables: - name: select value: <+xml.select("/bookstore/book[1]/title", httpResponseBody)> type: String - step: type: ShellScript name: echo XML select identifier: echo_XML_select timeout: 10m spec: shell: Bash onDelegate: true source: type: Inline spec: script: echo <+pipeline.stages.Functors.spec.execution.steps.XML_select.output.outputVariables.select> environmentVariables: [] outputVariables: [] executionTarget: {} rollbackSteps: [] tags: {} failureStrategies: - onFailure: errors: - AnyOther action: type: StageRollback when: pipelineStatus: Success ```
*** ## Next steps * [HTTP step](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/utilities/http-step.md): Call URLs and map JSON or XML responses to output variables. * [Shell Script step](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/utilities/shell-script-step.md): Echo and transform functor output variables. * [Variables and expressions](https://developer.harness.io/docs/platform/variables-and-expressions/harness-variables): Review Harness expression syntax used with functors.