> For the complete documentation index, see [llms.txt](https://developer.harness.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.harness.io/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/aws/aws-deployment-faqs.md). # AWS deployment FAQs Answers to frequently asked questions about AWS deployments in Harness, covering SAM, ECS, Lambda, ASG, and AWS connector configuration. This article addresses some frequently asked questions about AWS deployments in Harness.
Where does Harness publish its images for AWS SAM Build and Deploy? Harness publishes the SAM Build and Deploy images in the Harness DockerHub registry for users to reference in their SAM Build or SAM Deploy steps: * SAM Build: * SAM Deploy: In your Harness steps, you can update the Container Configuration's Image section with a newer version of the image.
How do we debug the SAM Build or SAM Deploy steps? Under the AWS SAM Build Command Options, we recommend passing the `--debug` flag. This helps print more verbose errors when troubleshooting failures. ```yaml - step: type: AwsSamBuild name: SAM Build identifier: SAM_Build spec: connectorRef: account.harnessImage image: harnessdev/sam-build:1.82.0-latest imagePullPolicy: Always buildCommandOptions: - "--use-container" - "--debug" ## This field needs to be added. - "--build-image harnessdev/testing:5.6.4" ## This field will need to be added envVariables: PLUGIN_SAM_TEMPLATE_FILE_PATH: <+env.name>_template.yaml runAsUser: root samBuildDockerRegistryConnectorRef: account.harnessImage when: stageStatus: Success timeout: 10m ```
What versions of the SAM CLI are supported by Harness? * We support SAM CLI version 1.84.0. * We are working to stay up to date with the latest versions of SAM CLI and making sure it is compatible with newer versions of our steps.
Can we fetch build images from a public repo? Yes, Harness can fetch build images from a public repo. For more information, see [AWS SAM build image options](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-image-repositories.html) for the public ECR repo. In the AWS SAM Build Command Options, you can pass in an argument like so: ```sh --build-image public.ecr.aws/sam/build-nodejs18.x:1.100.0-20231031004056 ``` The following is the step YAML: ```yaml - step: type: AwsSamBuild name: SAM Build identifier: SAM_Build spec: connectorRef: account.harnessImage image: harnessdev/sam-build:1.82.0-latest imagePullPolicy: Always buildCommandOptions: - "--use-container" - "--debug" - "--build-image harnessdev/testing:5.6.4" ## This field will need to be added envVariables: PLUGIN_SAM_TEMPLATE_FILE_PATH: <+env.name>_template.yaml runAsUser: root samBuildDockerRegistryConnectorRef: account.harnessImage when: stageStatus: Success timeout: 10m ```
Can we fetch build images from a private repo? Yes, Harness can fetch your build images from a private repo. This requires the user to have the proper permissions configured and the correct access to the container repo. In the SAM Build step, under the SAM Build Docker Container Registry section, you can specify the image repository you want to fetch your SAM Build image from.
How do I access the SAM Build step outputs? * SAM build creates a `.aws-sam` directory in the same directory where your template.yml exists. * By default, the Download Manifests step downloads your repo to the `/harness/MANIFEST_IDENTIFIER/` path, which is `/harness/dev/` in this case. * If you customize the step and the SAM template.yaml exists at the root level, AWS SAM outputs will not be available in the root path. Make sure `.aws-sam` is present in the `/harness/dev/` path.
How do I pass in the SAM Template path as an environment variable? * Under the Environment Variables section of the step, you can specify a key and value. * For the key, you can provide `PLUGIN_SAM_TEMPLATE_FILE_PATH`, and the value can be a `fixed` or `expression` input. ```yaml - step: type: AwsSamBuild name: SAM Build identifier: SAM_Build spec: connectorRef: account.harnessImage image: harnessdev/sam-build:1.82.0-latest imagePullPolicy: Always buildCommandOptions: - "--use-container" - "--debug" - "--build-image harnessdev/testing:5.6.4" envVariables: PLUGIN_SAM_TEMPLATE_FILE_PATH: <+env.name>_template.yaml ## This field will need to be added runAsUser: root samBuildDockerRegistryConnectorRef: account.harnessImage when: stageStatus: Success timeout: 10m ```
How are the number of service instances calculated for Lambda deployments? The number of service instances for Lambda deployments is determined by the number of concurrent executions of the Lambda function. Harness automatically scales the number of service instances based on the traffic to the function.
How do ECS blue green deployments work? ECS Blue/Green deployments use old and new versions of your service running behind the load balancer. Your ELB uses two listeners, Prod and Stage, each forwarding to a different target group where ECS services are run. Blue/Green deployments are achieved by swapping listeners between the target groups, always pointing the Prod listener to the target group running the latest version. In Harness, you identify which listeners are the Prod and Stage listeners. When Harness deploys, it uses the target group for the Stage listener (for example, target1) to test the deployment, verifies the deployment, and then swaps the Prod listener to use that target group. Next, the Stage listener uses the old target group (target2). When a new version of the service is deployed, the Stage listener and its target group (target2) are first used, then, after verification, the swap happens and the Prod listener forwards to target2 and the Stage listener forwards to target1.
How does ECS rollback work? When an ECS deployment fails, the service tasks it was deploying are scaled down to 0. The service is scaled to its pre-setup number of tasks. If ECS Service Auto Scaling is configured, it is attached to the last production service at the end of the rollback. For multi-phase deployments (Canary), not all phases are rolled back at once. Phases are rolled back in the standard, reverse order. For example, in a 2 phase deployment with 2 rollback phases, the order is P1 → P2 → R2 → R1. Phases are rolled back in this order regardless of whether ECS Service Auto Scaling is used.
Can I run individual tasks separately? Yes. In addition to deploying tasks as part of your standard ECS deployment, you can use the **ECS Run Task** step to run individual tasks separately as a step in your ECS stage. The **ECS Run Task** step is available in all ECS strategy types. An example of when you run a task separately is a one-time or periodic batch job that does not need to keep running or restart when it finishes.
Can I use AWS Service Discovery? Yes. Harness ECS deployments support deploying ECS services with AWS Service Discovery. AWS Service Discovery is a cloud service provided by AWS that lets microservices applications discover and communicate with each other. It enables you to manage and discover the addresses of the services within your microservices application without hard-coded IP addresses or hostnames. You can use AWS Service Discovery as part of the ECS service definition in Harness.
Can I use ECS circuit breaker configurations? Yes. Harness ECS rolling and canary deployments support AWS ECS circuit breaker configurations. Circuit breaker configuration is implemented in the Harness ECS service **Service Definition**. Harness deploys the tasks in the ECS service definition containing the circuit breaker configuration. Once deployed, the circuit breaker is activated. During failure scenarios, ECS circuit breaker performs a rollback automatically based on the threshold configuration.
What is a Lambda deployment? In Harness, you specify the location of the function definition, artifact, and AWS account. Harness deploys the Lambda function and automatically routes the traffic from the old version of the Lambda function to the new version on each deployment. **Looking for Serverless.com Framework Lambda?** Harness also supports Serverless.com Framework Lambda deployments. For more information, see [AWS Lambda deployments (Serverless.com Framework)](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/serverless/serverless-lambda-cd-quickstart.md).
What are the Lambda deployment limitations? * Harness support only deploys and updates Lambda functions. Harness does not update auxiliary event source triggers like the API Gateway. * Lambda functions can be packaged as ZIP files in S3 buckets or as containers in AWS ECR. * If Harness were to support another repository, like Nexus, when the container is fetched by the API, AWS spins up AWS resources (S3, ECR) anyway, so Harness has limited support to S3 and ECR. * The containers must exist in ECR. Containers are not supported in other repositories.
What are the function definition requirements? Harness uses the AWS Lambda [Create Function API](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) to create a new Lambda function in the specified AWS account and region. The API takes a JSON object as input that defines the configuration settings for the Lambda function, such as the function name, runtime environment, handler function, memory allocation, and IAM role. This allows you to use the function definition as a configuration file in your Harness pipelines. In Harness, you use a JSON configuration file to define the AWS Lambda you want to deploy. This configuration lets you define all the function settings supported by the Create Function API. The minimal requirements for an AWS Lambda function definition are: * Function Name (`FunctionName`): A unique name for your Lambda function. * Runtime (`Runtime`): The programming language and version that your function code is written in. Lambda supports multiple programming languages, including Node.js, Python, Java, C#, and Go. * Handler (`Handler`): The name of the function within your code that Lambda should call when the function is invoked. * AWS IAM role (`Role`): The IAM role that the function should use to access other AWS services or resources. You can create an IAM role specifically for the Lambda function, or you can reuse an existing IAM role if it has the necessary permissions. For a full list of supported fields, see [AWS Lambda Create Function Request](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html). Harness supports all of the popular Git platforms for storing your function definition files.
Can we deploy multiple stacks (multiple SAM build and deployments in a single pipeline) using a single service? No. We support only one stack in the SAM Deploy step.
There is an option to specify multiple manifest files on the service definition. Can we have both the stacks' manifests defined in the same service? This is not supported. We support deploying to one stack only in the SAM Deploy step. For this scenario, we suggest that you have one pipeline with two Deploy stages with two different services and stacks. Alternatively, you can have one pipeline with one Deploy stage and specify two different services in the **Service** tab, and in the **SAM Deploy** step > **Stack Name** field, select runtime input. ![](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-b1b5e2227a630e0390464fdb57788277ef082700%2Faws-sam-faq.png?alt=media) ![](https://3694223630-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2Fy1JhZ4oKIppwY7d5AhPj%2Fuploads%2Fgit-blob-8467565d1030cc827fc4fe1cf915586b8c2875ab%2Faws-sam-runtime.png?alt=media)
What is a Spot deployment in Harness? You can deploy AWS ASGs using Harness and the Spot Elastigroup management platform.
What are the Spot deployment limitations? * Harness integrates with Spot for AWS deployments by using Elastigroups. * Spot Elastigroup deployments support AMI artifacts only. * Spot Elastigroup deployments support only basic, canary, and blue green execution types.
How do I use the token for an OCI repository in AWS ECR, as the token expires every 12 hours by default? 1. Set up the AWS Secret Manager connector and save the ECR auth token into it. 2. Set up automatic token rotation (for example, at 10 hour intervals) within AWS Secret Manager. 3. Have the Harness connector link to that AWS Secret Manager secret to pull a fresh token every time.
Can I encrypt the token or secret passed in the INIT_SCRIPT? It cannot be encrypted directly, but this can be achieved by creating the Kubernetes secret for the credentials and referring to them in the init script. For example: `aws_access_key=kubectl get secrets/pl-credentials --template={{.data.aws_access_key}} | base64 -d` `aws_secret_key= kubectl get secrets/pl-credentials --template={{.data.aws_secret_key}} | base64 -d` Another approach is saving the value in the Harness secret manager, or any other secret manager, and referencing it in the script. For more information, see [Add and use text secrets](https://developer.harness.io/harness-platform/use-harness-platform/secrets/add-use-text-secrets).
Can you provide an example of deploying the delegate in a task definition for Amazon Elastic Container Service (Amazon ECS)? Yes. You can find a step-by-step guide on how to deploy the delegate in a task definition for ECS on our [official documentation page](https://developer.harness.io/harness-platform/use-harness-platform/delegates/delegate/install-delegates/docker-delegate-to-ecs-fargate). Additionally, we have a GitHub repository with a Terraform module that demonstrates the process of [deploying the delegate in ECS Fargate](https://github.com/harness-community/terraform-aws-harness-delegate-ecs-fargate/tree/main). This resource can further assist you in implementing the delegate deployment.
Is there a method to simulate CloudFormation changes without actually applying them? Yes, you can achieve this by using the Change Set feature. First, create a change set to preview the changes that will be made. Once you are satisfied with the preview, you can execute the change set using the command [aws cloudformation execute-change-set](https://docs.aws.amazon.com/cli/latest/reference/cloudformation/execute-change-set.html). This allows you to assess the impact of the changes before applying them.
I am trying to use a specific KMS key to encrypt my AWS Lambda environment variables, but the Lambda seems to default to the AWS managed key. What could be the issue? By default, AWS Lambda uses an AWS managed key for environment variable encryption. If you are specifying a KMS key using the `kmsKeyArn` parameter in the Lambda function definition YAML but still seeing the AWS managed key being used, it might be due to how the `kmsKeyArn` is defined in your YAML.
How can I ensure that my specified KMS key is used to encrypt my Lambda environment variables? To make sure that your specified KMS key (`kmsKeyArn`) is used to encrypt your Lambda environment variables, you need to ensure that the YAML key is written in camel case format, which is `kmsKeyArn`. Additionally, make sure that the KMS key ARN is accurate and accessible in your AWS account.
Can you provide an example YAML snippet with the correct usage of kmsKeyArn for Lambda environment variables? ```yaml functionName: "ff2" handler: handler.hello role: "arn:aws:iam::01447erole2" runtime: nodejs14.x kmsKeyArn: "arn:aws:kms:ue78fb6117cfd" # Make sure the ARN is accurate environment: variables: key: "val" ```
How do I fetch a PEM certificate from AWS Secrets Manager without losing its formatting? If you have multi-line secrets, redirect the output to a temporary file and use that file for base64 encoding.
Is there any way to increase the task count of an ECS service without an ECS service deployment? The task count of ECS cannot be changed without a deployment in Harness, but you can make changes to the ECS deployment directly on AWS.
How do I solve the following error: Invalid request: "Profile definition must end with ']"? Harness Delegates do not control AWS profiles. This is likely configured manually through the delegate by the user and should be reviewed by the author.
Can we skip manually creating the kubeconfig when using the native EKS deployment method in AWS, since we provide connection details in the AWS connector? Yes, you do not need to create the kubeconfig file manually. You only need to have the `aws-iam-authenticator` binary installed on the delegate. For more information, see [AWS connector settings](https://developer.harness.io/harness-platform/use-harness-platform/connectors/cloud-providers/add-aws-connector/aws-connector-settings-reference#connect-to-elastic-kubernetes-service-eks).
What is the time parameter for the AWS back-off strategy? For the AWS back-off strategy, time parameters are in milliseconds. For more information, see [AWS connector settings](https://developer.harness.io/harness-platform/use-harness-platform/connectors/cloud-providers/add-aws-connector/aws-connector-settings-reference#aws-backoff-strategy/).
How do I create an AWS connector using aws-iam-authenticator on an EKS cluster with webIdentityToken? For more information about setting up the AWS connector on an EKS cluster, see [AWS connector settings](https://developer.harness.io/harness-platform/use-harness-platform/connectors/cloud-providers/add-aws-connector/aws-connector-settings-reference#connect-to-elastic-kubernetes-service-eks). Also, to align with AWS changes, it is important to note that the previous method of accomplishing this task was deprecated in `kubectl version 1.21`. To address this, when using the `Iam-authenticator plugin`, you need to create a `Fargate profile` as part of the updated procedure.
Is there a way to update the ECS scaling policy without redeploying the ECS service? Many users use a shell script approach: a configuration file is set as an environment variable, and a shell script runs the relevant AWS CLI commands to update scaling policies for a deployed service. This approach supports customization and automation across various scenarios and configurations.
When working with SAM templates, how can I specify the branch instead of the default master when providing a commit ID as a tag during the download manifests step, as it attempts to pull from the non-existent master branch? In the Download Manifests step, you can specify the branch name by using the expression `<+pipeline.stages.STAGE_NAME.spec.serviceConfig.serviceDefinition.spec.manifests.MANIFEST_IDENTIFIER.spec.store.spec.branch>`. Replace `STAGE_NAME` with the name of your stage and `MANIFEST_IDENTIFIER` with the identifier of your manifest. You can then use this expression in the Branch/Commit Id field to specify the branch you want to pull from. For more information, see [AWS SAM deployments](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/aws/aws-sam-deployments.md).
How do I trigger a Harness pipeline that creates a Jira issue from a Python script on AWS Lambda, and pass the Jira issue description into it? You can use the [Pipeline Execute API](https://apidocs.harness.io/tag/Pipeline-Execute#operation/postPipelineExecuteWithInputSetList) to run the pipeline. In the created pipeline, you can add a Jira update step with the required details to update the issue. For more information, see [Update Jira issues in CD stages](/continuous-delivery/use-continuous-delivery/cd-building-blocks/cd-steps/ticketing-systems/update-jira-issues-in-cd-stages.md).
While creating an MSK connector role, we encountered the error "User: arn:aws:sts::44ddsnfjdnfs-deploy-cicd-role/AWSCloudFormation is not authorized to perform: iam:CreateRole..." with a Status Code 403. How can we resolve this issue? The error indicates a permission issue related to IAM (Identity and Access Management) policies. Follow these steps to resolve the issue: 1. Review the IAM policy. 2. Review the IAM permissions boundary. 3. Review the service control policies (SCPs). 4. Review the policy trust relationships.
How do we know if a deployment is failing due to a service control policy implemented in our AWS account? If there is any restriction from an SCP, you get an error with an explicit deny in a service control policy. User: arn:aws:iam::7799:user/JohnDoe is not authorized to perform: codecommit:ListRepositories because no service control policy allows the codecommit:ListRespositories action
How does Harness provide enhanced control to users in the deployment sequence for applying autoscaling policies in ECS Blue-Green deployment? Harness exposes further control to users for when, in the deployment sequence, they want to apply the autoscaling policies. This ensures that the existing containers are not impacted during release, and after release it scales down based on traffic demand. For more information, see [Enable auto scaling in swap step](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/aws/ecs/ecs-deployment-tutorial.md#skip-application-auto-scaling).
How does Harness conduct pre-deployment validations and manage ECS Blue and Green services by updating tags based on target groups before initiating the deployment? Harness performs some validations before the deployment. Before the deployment, Harness identifies ECS Blue and Green services based on the target group and updates tags accordingly. It then starts the deployment. You can enable the feature flag `CDS_ECS_BG_VALIDATION` to use the feature on your account. For more information, see [ECS blue green service validation](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/aws/ecs/ecs-deployment-tutorial.md#ecs-blue-green-deployments).
What does this error in the Lambda function deployment mean? `aws/lambda/testLambda-dev already exists in stack arn:aws:cloudformation:us-east-2:xxxxxxxx...` This error occurs when there was a previous failed deployment but the logs still exist in the CloudWatch logs. Remove the log and try the deployment again.
Is it required to have the delegate installed on an ECS cluster? The delegate can be installed anywhere as long as it has access to the target ECS cluster. For more information, see [Install and register the Harness Delegate](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/aws/ecs/ecs-deployment-tutorial.md#install-and-register-the-harness-delegate).
Does AWS Secret Manager support IRSA? We support RSA for secret manager authentication. We can use the Assume Role STS option and configure the delegate for IRSA.
Does Harness support SSH deployments using the GCP connector like AWS and Azure? No, this feature is not yet supported. Use an SSH key, or a username and password, for the data center connection as an alternative.
If I manually increase or decrease the task count in AWS directly, will it impact the next pipeline deployment? It should not impact the deployment, except that if the subsequent deployment has a different task count, it overrides the existing one.
How can one handle credentials rotation policies for AWS connectors that require access key ID and secret access key without using a delegate? Prioritize using a functioning, stable delegate. If you prefer connecting through the platform, and you have an external secrets manager with a streamlined process for automatic key updates during rotations, you can consider that integration. **Note**: Continuous Delivery cannot use platform-based authentication for all connectors, because deployment jobs run on the delegate. GitHub-type connectors can use it, but AWS, GCP, and Azure cannot, because the credential exchange happens on the delegate.
How can I reduce the number of AWS calls during deployment to mitigate issues? You can reduce the number of AWS calls during deployment. For more information about AWS backoff strategies, see [AWS connector settings](https://developer.harness.io/harness-platform/use-harness-platform/connectors/cloud-providers/add-aws-connector/aws-connector-settings-reference#aws-backoff-strategy). These strategies can help manage AWS calls and improve the execution of your deployment scripts.
How can one use AWS CodeDeploy template support in Harness? The AWS CodeDeploy Deployment Template lets you set the infrastructure and fetch the instances deployed via AWS CodeDeploy. For more information, see [AWS CodeDeploy deployment template sample](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/custom/custom-deployment-tutorial.md#aws-codedeploy-deployment-template-sample).
Does Harness provide AWS SAM and Serverless.com manifests from the Harness File Store? Yes, you can download your AWS SAM and Serverless.com manifests from the Harness File Store and AWS S3. For more information, see [Manifest sources for Serverless Lambda](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/serverless/manifest-sources-for-serverless-lambda.md).
Is there a way to switch AWS accounts while using the native Terraform step? Yes, Harness supports an AWS connector that lets the Terraform Plan and Apply steps assume a role to perform the provisioning of infrastructure. For more information, see [Connector credentials](/continuous-delivery/use-continuous-delivery/provision-infrastructure/terraform-infra/run-a-terraform-plan-with-the-terraform-plan-step.md#connector-credentials).
Can we modify the audience in the generated JWT token to use it for purposes beyond AWS authentication? Modifying the audience (`aud`) claim in a JWT means changing who the token is intended for. In AWS authentication, it typically specifies AWS services. However, you can adjust it for other systems or services as long as they understand and accept this change. Make sure the system you are sending the token to knows how to handle the modified audience claim according to its own requirements and security rules.
How does the AWS connector handle OIDC authentication? Users can configure OIDC authentication through the AWS connector. This only leverages the account ID parameter when Harness sends the token payload to the AWS STS service. For more information, see [Harness AWS connector settings](https://developer.harness.io/harness-platform/use-harness-platform/connectors/cloud-providers/add-aws-connector/aws-connector-settings-reference#harness-aws-connector-settings).
How do I efficiently extract specific tag values from JSON output, like querying an AWS ECR repo via the AWS CLI, using the json.select feature with shell script variables or outputs? The output variable cannot be used in the same step in which it is created. Output variables can be used within: * the stage * the pipeline * a step group For more information, see [Output variable for PowerShell script](/continuous-delivery/troubleshooting-and-resources/knowledge-base-article/output-variable-for-powershell-script.md).
How do I troubleshoot an "Invalid request: Invalid Load balancer configuration in Service." error? This can happen in the following cases: * No target groups attached to the load balancer. * Multiple services attached to target groups. * A service is attached to both target groups. For more information about ECS blue-green steps, see [ECS deployment tutorial](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/aws/ecs/ecs-deployment-tutorial.md#visual-summary).
Are there guidelines or recommendations for scaling up VM build infrastructures, such as those hosted on EC2, for individuals managing and configuring their own infrastructure? Yes, you can scale up your AWS VM build infrastructure effectively by considering the following: **Scaling up**: Increase capacity by adding more templates, build nodes, and VMs to accommodate growing demands. **Cost optimization**: Implement cost-saving measures, such as configuring the runner to hibernate AWS Linux and Windows VMs during idle periods, to reduce infrastructure costs. For detailed setup and configuration instructions, see [Set up an AWS VM build infrastructure](https://developer.harness.io/continuous-integration/use-harness-ci/use-harness-ci/set-up-build-infrastructure/vm-build-infrastructure/set-up-an-aws-vm-build-infrastructure).
Does Harness support the use of OpenID Connect (OIDC) for connecting to various systems such as Amazon Web Services (AWS) and Google Cloud Platform (GCP)? Yes, Harness supports OIDC integration for [Google Cloud Platform (GCP)](https://developer.harness.io/harness-platform/use-harness-platform/connectors/cloud-providers/connect-to-google-cloud-platform-gcp/gcs-connector-settings-reference#use-openid-connect-oidc) and [Amazon Web Services (AWS)](https://developer.harness.io/harness-platform/use-harness-platform/connectors/cloud-providers/add-aws-connector/aws-connector-settings-reference#credentials). Harness plans to add additional support, including facilitating authentication, short-lived token acquisition based on Harness context, and various operational tasks like deployment, builds, or secret retrieval within the respective cloud provider environments.
What values should I enter for provider URL and audience for the OIDC AWS connector? Use the following values: ``` Provider URL - https://app.harness.io/ng/api/oidc/account/ Audience: sts.amazonaws.com ```
What authentication flows (grant types) are supported for AWS and GCP OIDC? For the Kubernetes connector, we go through client credentials. For AWS and GCP, we have implemented it slightly differently. For AWS, OIDC is also implicit: use OIDC to connect to AWS with OIDC. To do this, you need to create an OIDC identity provider in AWS and add it in a trust relationship with an IAM role you create that Harness uses to operate in AWS.
What is the default duration for assuming the role in the AWS Java SDK? The default duration for assuming the role in the aws-java-sdk is 15 minutes.
What ECR permissions policy does Harness create as part of the AWS Lambda service deployment? Harness creates the policy that is essential for performing various ECS-related actions, like creating and updating services, tasks, and container instances.
How do I avoid Harness creating a permissions policy and applying it to an AWS ECR repository that we are specifying as an artifact location for our AWS Lambda deployment configuration, since these permissions are creating Terraform state drift on the ECR repository? To prevent Terraform state drift, we recommend that you create the ECR repository with the required permissions beforehand. This can be achieved by crafting an IAM policy that grants the necessary permissions and attaching it to the IAM role used by the ECS cluster. Alternatively, you can prevent Harness from altering IAM policies by removing the relevant permissions from the Harness AWS connector. However, this could affect the functionality of your deployment pipeline.
What is an ASG deployment? Here is a summary of how Harness deploys new ASG versions: 1. First deployment: 1. Harness takes the launch template and ASG configuration files you provide and creates a new ASG and its instances in your AWS account and region. 2. Subsequent deployments: 1. Harness creates a new version of the launch template. 3. Harness uses the new version of the launch template to update the ASG. For example, if you increased the desired capacity (`desiredCapacity`) for the ASG in your ASG configuration file, Harness creates a new version of the ASG with the new desired capacity. 1. Instance refresh is triggered (a rolling replacement of all or some instances in the ASG).
What do I need for an ASG deployment? 1. For the Harness ASG service: 1. Launch template. 2. ASG configuration. 3. Scaling policies (optional). 4. Scheduled update group action (optional). 5. The AMI image to use for the ASG as an artifact. 2. Harness ASG environment: 1. AWS region where you want to deploy. 3. Harness ASG pipeline execution: 1. Deployment strategy (rolling, canary, blue green). Harness automatically creates the steps to deploy the new ASG.
What are the ASG deployment limitations? AWS has the following limitations to keep in mind: * You are limited to creating 5000 launch templates per region and 10000 versions per launch template. For more information, see [Launch an instance from a launch template](https://docs.aws.amazon.com) from AWS. * ASGs per region: 500. * Launch configurations per region: 200.
What is a Harness ECS deployment? You can deploy images to your Amazon Elastic Container Service (ECS) cluster using rolling, canary, and blue green deployment strategies. Harness supports standard ECS specifications: * [RegisterTaskDefinition](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RegisterTaskDefinition.html) * [CreateService](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_CreateService.html) * [RegisterScalableTarget](https://docs.aws.amazon.com/autoscaling/application/APIReference/API_RegisterScalableTarget.html) * [PutScalingPolicy](https://docs.aws.amazon.com/autoscaling/application/APIReference/API_PutScalingPolicy.html) * [RunTask](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RunTask.html)
Can I perform auto scaling? Yes. The ECS service(s) you deploy with Harness can be configured to use AWS Service Auto Scaling to adjust its desired ECS service count up or down in response to CloudWatch alarms. For more information on using Auto Scaling with ECS, see [Target Tracking Scaling Policies](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service-autoscaling-targettracking.html) from AWS. Before you set up Auto Scaling for the ECS service in Harness, you need to obtain the JSON for the Scalable Target and Scalable Policy resources from AWS. The JSON format used in the **Auto Scaler Configurations** settings should match the AWS standards as described in [ScalableTarget](https://docs.aws.amazon.com/autoscaling/application/APIReference/API_ScalableTarget.html) and [ScalablePolicy](https://docs.aws.amazon.com/autoscaling/application/APIReference/API_ScalingPolicy.html).
Can I deploy the same number of service instances as the previous deployments? Yes. The **Same as already running instances** setting in **ECS Rolling Deploy** sets the number of desired ECS service instances for this stage to the same number as the already running instances. Essentially, it ignores the desired count in the Service Definition of the new deployment.
Can I force a deployment? Yes. By default, deployments are not forced. You can use the **Force new deployment** setting in **ECS Rolling Deploy** to start a new deployment with no service definition changes. For example, you can update a service's tasks to use a newer Docker image with the same image/tag combination (`my_image:latest`) or to roll Fargate tasks onto a newer platform version.
What is the function definition syntax? * Harness expects camel case in the function definition. * You can provide YAML or JSON in the function definition. ```json "vpcConfig": { "securityGroupIds": ["string"], "subnetIds": ["string"] } ```
Does Harness support viewing the rendered launch template for ASG deployments after all the Harness expressions have been evaluated? No, Harness does not store the rendered launch template as an output variable, nor does it display the rendered launch templates in the console logs. We only indicate that the ASG configuration is created. For more details, see [Harness ASG services](/continuous-delivery/use-continuous-delivery/deploy-services-on-different-platforms/aws/asg/asg-tutorial.md#create-a-harness-asg-service).
Why does IRSA Kubernetes deployment fail with the error "java.lang.IllegalStateException: Not a JSON Object: null"? For EKS cluster deployments through an AWS connector that uses IRSA-based authentication, if the above error is seen during the initialization of the Kubernetes Rollout step, log in to the delegate to check if the `aws-iam-authenticator` being used is compatible with the CPU architecture of the node where the delegate is running.
How do I validate if the aws-iam-authenticator installed on the delegate is correct and works as expected? The `aws-iam-authenticator` generates a token that `kubectl` uses to authenticate with the target cluster. The Harness delegate runs the following command to generate a token in JSON format: ```sh aws-iam-authenticator token -i --role arn:aws:iam::12345678:role/CrossAccountEKSRole ```
When configuring the EKS infrastructure, the clusters are listed. However, when trying to deploy a service to the cluster, the Kubernetes deployment fails. Why does this happen? When configuring the infrastructure, the AWS connector uses the AWS APIs to query the cluster. However, during the Kubernetes deployment, the deployment uses `kubectl` to communicate with the Kubernetes API server's endpoint to deploy the Kubernetes manifest. For the deployment to be successful, there must be network connectivity from the delegate to the Kubernetes API server endpoint. This connectivity might not exist if the cluster is private, so the delegate must reside in the same VPC as the cluster. For more details, see [AWS documentation](https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html#modify-endpoint-access).
How is the kubeconfig file generated for Kubernetes deployments with cross-account STS IRSA-based connectors? During the Kubernetes Rollout stage, the Harness Delegate generates the `kubeconfig` file using details from the infrastructure configured on the stage. The authentication token is generated using the `aws-iam-authenticator` command available on the delegate by passing the cross-account role and the cluster name.
Will the kubeconfig file continue to exist on the delegate pods? The `kubeconfig` generation process during deployment happens on the fly. The file is kept in a temporary location for the duration of the Kubernetes Rollout step, and is cleaned up once the step completes or fails.
Does Harness support using IRSA with ECS for deployment? No, IRSA is not used with ECS. Instead, ECS tasks get their own task roles, which are set up similar to IRSA. A delegate runs in ECS with a base role, and STS is used to assume a secondary role.
Does Harness support using the Serverless CLI on a delegate? Yes, using the Serverless CLI on the delegate ensures that the delegate has the necessary permissions and environment setup. This includes adding node, Serverless, or AWS CLI to the delegate INIT script and managing STS/trust configurations appropriately for each environment and service role.
Does Harness support conditional input variables based on previous selections? No, Harness does not support dynamically showing different input variables based on previous selections during pipeline execution. For more details, see [Stage and step conditional execution settings](/continuous-delivery/use-continuous-delivery/cd-building-blocks/executions/step-and-stage-conditional-execution-settings.md#and-execute-this-stage-only-if-the-following-jexl-condition-evaluates-to-true).
Does Harness support displaying inputs only related to selected values in a pipeline? No, Harness does not support conditionally displaying input variables based on previous selections.
Does Harness support using allowed values for dynamic input options? Yes, you can use allowed values to specify different input options for a variable, though it does not fully cover dynamic input based on previous selections.
Why does the run step fail with a NullPointerException error even though the correct roles are attached to the delegate? ``` Exception getAmazonEcrAuthToken java.lang.NullPointerException: You must specify a value for roleArn and roleSessionName ``` Check to see if the AWS connector connectivity test passes. Also, check the service account attached to the delegate and see if it has the correct roles assigned to access the resource(s).
How can we add mean duration for prod and non-prod pipelines in dashboards? To get the mean duration for prod and non-prod pipelines, first create a measure to calculate the mean duration for the non-prod environment. Similarly, create a second measure for the prod environment. Finally, perform a table calculation on both custom measures and add them.
We need to extract the project tags based on the project name. Is there an API to do it? You can use the [Get Project](https://apidocs.harness.io/tag/Project#operation/getProject) API to get project details, including tags.
What happens when my ECS service tasks repeatedly fail to start? When ECS service tasks repeatedly fail to start, they transition from PENDING to STOPPED without reaching the RUNNING state. The ECS service scheduler then throttles the restart attempts, incrementally increasing the time between restarts up to a maximum of 27 minutes. This helps manage the impact on your ECS cluster resources or Fargate infrastructure costs.
Will the ECS service ever stop retrying to start failing tasks? No, Amazon ECS does not stop a failing service from retrying. The service continues to attempt restarts indefinitely, but the time between attempts increases incrementally if the tasks keep failing.
How can we stop my ECS service from being throttled? To stop the throttling, you need to update your service to use a new task definition. This action resets your service to a normal, non-throttled state immediately.
What should I do if my service tasks are being throttled? You should investigate and resolve the underlying issues causing the task failures. Common causes include insufficient resources, issues with pulling the Docker image, and insufficient disk space. Addressing these issues can help your tasks start successfully and avoid throttling.
Do tasks that stop after reaching the RUNNING state trigger throttling? No, tasks that are stopped after reaching the RUNNING state do not trigger the throttle logic or the associated service event message. For example, tasks stopped due to failed Elastic Load Balancing health checks, or tasks that exit with a non-zero exit code after moving to the RUNNING state, do not cause throttling.
What can cause insufficient resources for ECS tasks? Insufficient resources can be due to a lack of available ports, memory, or CPU units in your cluster. You see an "insufficient resource" service event message if this is the problem. Ensure your cluster has adequate resources to accommodate your tasks.
How does Amazon ECS notify me about service throttling? If your service initiates the throttle logic, you receive a service event message stating that your service is unable to consistently start tasks successfully. This helps you identify when throttling is occurring so you can take corrective action.
What are common causes of ECS service tasks failing to start? Common causes include: * Lack of resources (ports, memory, or CPU units) in your cluster. * Issues with pulling the Docker image due to incorrect image names, tags, or registry authentication problems. * Insufficient disk space on the container instance.
Why am I getting an "Invalid Request: No manifests found" error? ``` Invalid request: No manifests found in stage Deploy_AMI. AsgRollingDeploy step requires a manifest defined in stage service definition ``` This means that your Launch Template and ASG Configuration are not set up correctly in your service under AWS ASG Configurations.
What is the purpose of the ECS Scheduled Task Deployment Pipeline? This pipeline automates building, pushing, scheduling, and running ECS tasks using Harness. It helps dynamically update ECS task definitions and schedule tasks with AWS EventBridge.
What are the main stages in the ECS Scheduled Task Deployment Pipeline? The pipeline has three main stages: 1. ecr push: Builds and pushes a Docker image to Amazon ECR. 2. schedule task: Schedules the ECS task using AWS EventBridge. 3. deploy ecs: Updates the ECS task definition and optionally runs the ECS task.
How does the ecr_push stage work? The ecr\_push stage builds the Docker image, tags it, and pushes it to Amazon ECR. You can create an output variable to dynamically pass the updated image tag to the subsequent stage, for example, the deploy stage.
What does the schedule_task stage accomplish? The schedule\_task stage creates or updates an AWS EventBridge rule and sets the ECS task as its target. It uses AWS CLI commands to define scheduling parameters and associates the updated ECS task definition with the rule. You need to run these commands using a Shell Script step, since Harness does not natively support scheduling tasks in ECS.
How is the deploy_ecs stage used? The deploy\_ecs stage updates the ECS task definition with the new Docker image tag and uses the EcsRunTask step to optionally run the ECS task. Rollback options like ECS Canary Delete and Rolling Rollback are included to handle deployment failures.
What are the IAM role requirements for the ECS Scheduled Task Deployment Pipeline? You need an IAM role (`ecsEventBridgeRole`) with the following permissions: * Trust Policy: Allows EventBridge to assume the role. * Permissions: `ecs:RunTask`, `ecs:DescribeTaskDefinition`, `events:PutRule`, `events:PutTargets`.
How is the Docker image tag updated dynamically in the ECS Scheduled Task Deployment Pipeline? You can use output variables in your build stage where you are building your Docker image and reference that output variable in your ECS service step where you are defining your artifacts. This makes sure that the image tag is updated dynamically. For more information, see [Output variables in Harness](https://developer.harness.io/harness-platform/use-harness-platform/variables-and-expressions/harness-variables#input-and-output-variables).
How can I schedule the ECS task using the AWS CLI? The schedule\_task stage runs the following AWS CLI commands: 1. Create or update an EventBridge rule: ```bash aws events put-rule --schedule-expression "rate(5 minutes)" --name my-scheduled-task ``` 2. Set the ECS task as the rule's target using updated task definitions.
How does the rollback mechanism work? The pipeline includes rollback steps like ECS Canary Delete and ECS Rolling Rollback. These steps revert deployments if failures occur during task scheduling or execution.
Can I run the ECS task immediately after updating the task definition? Yes, the deploy\_ecs stage includes the EcsRunTask step, which allows running the ECS task immediately using the updated ECS task definition file.
Why am I getting an error stating that aws-iam-authenticator is required for EKS deployments? The error "aws-iam-authenticator is required" for EKS deployments indicates that the `aws-iam-authenticator` is not installed on the delegate. The `aws-iam-authenticator` is required for using AWS EKS Infrastructure for Kubernetes deployments using the AWS connector. Alternatively, a regular Kubernetes connector can be used.
Why is the AWS Connector using the default EC2 instance role instead of the role defined on the Delegate's Service Account? The AWS Connector uses the default EC2 instance role instead of the role defined on the Delegate's Service Account because the connector configuration is set to `InheritFromDelegate` instead of explicitly using `IRSA`. For roles obtained via IRSA, the `type: Irsa` setting must be explicitly specified in the connector configuration.
How do I use the API to create a trigger for ECS artifacts? Configure the `artifactRef` parameter correctly when creating a trigger via the API. Example API request structures can be found in the [Harness API documentation](https://apidocs.harness.io/tag/Triggers#operation/createTrigger).
How can we deploy ECS Scheduled Tasks without using Harness scheduling? Harness does not natively support ECS Scheduled Tasks, but you can: * Build and push the Docker image to ECR. * Update the ECS Task Definition. * Use AWS CLI commands to schedule the task execution.
How can we create a custom script to access a Bitbucket repository and deploy using the AWS CLI? You can: * Use the Bitbucket API to retrieve necessary files. * Implement an AWS CLI-based deployment script. * Automate the process via a Harness pipeline step.
{% @harness-feedback/feedback %}