Missing file in rosco image got Exception (Create Bake)
Issue
Armory support case management via Harness
Information on support for Spinnaker/Armory
400 Errors from Jenkins, even though JOBS can be seen in Spinnaker
Issue
403 and Permission Errors when Enabling New Services
Issue
403 Errors around GitHub access and Rate Limit Issues
Issue
2.32.1 upgrade path
The most important change introduced in Armory version 2.32 is artifact storage. This moves artifacts out of Orca database into the s3 bucket. This change will potentially reduce Orca database load and the execution context coming from the database.
Access denied to application on Deploy Stage without application WRITE permissions
Issue
Accessing Armory Scale Agent Endpoints to help in Troubleshooting
Introduction
Accessing Clouddriver Endpoints to help in Troubleshooting Armory Scale Agent issues
Introduction
Accessing the Spinnaker REST API using HTTPie and jq
Introduction
Accounts with liveManifestCalls Set to True Have Incorrect Dynamic Lookup Results
Issue
Adding a group with many members to application times out in GCP
Issue
Adding a Lambda Account Overwrites Existing AWS Accounts
Issue
Adding Additional Policies and Access to an EKS Cluster
Introduction
Adding Target Groups to the Deployment Pipeline in AWS by Using CloudFormation
Introduction
Adjust Clean Up Timing for Armory Agent
Introduction
Agent Deployment Error- exceptions.OperationTimedOutException- Timeout exceeded for operation
Issue
An Explanation of MySQl/PostGres Tables Involved in Armory Agent for Kubernetes
Armory Agent for Kubernetes requires an SQL-based database (MySQL/Postgres).
App Engine Deployments
To deploy to App Engine using different repositories for code and `app.yaml` file you need to configure the Git Repository Url pointing to your code repository.
Applying Custom Sizes to Services
Note: This guide assumes you’re deploying Spinnaker on Kubernetes using the Distributed deployment type with Halyard.
Armory Agent - Deploy Stage Issues - 404 (Not Found)- unknown kind ApiService in APIService ...
Issue
Armory Agent - Error msg= "agent is unreachable" when Connecting to External Accounts in Infrastructure Mode
Issue
Armory Agent Error receiving from ops from server- rpc error- code = Unavailable desc = transport is closing
Issue
Armory Agent Testing, Investigation, and Checks
Introduction
Armory Continous Deployment Managed High Availability and Disaster Recovery SLA Information
Support SLA's
Armory Continuous Deployment as a Service (CDaaS) Support
Welcome to the initial Armory Continuous Deployment as a Service (CD-as-a-Service) Support Page! For those starting your journey with Armory CD-as-a-Service, please refer to this page for some initial guidance on how Armory can assist you and your team.
Armory Diagnostics
This page has been recategorized and moved to our Documents Library:
Armory Enterprise (Spinnaker) Sizing and Scaling Guide
Introduction
Armory Halyard Overview
This page has been recategorized and moved to our Documents Library//docs.armory.io/overview/overview-of-armory-halyard/
Armory Managed Services
Focus on building apps and not managing Spinnaker. Armory experts will take over Spinnaker operations, upgrades and break-fix efforts in your environment.
Armory Priority Zero (P0) Case Handling Procedures
Armory Customers with a Support Service agreement may be required to open a case with the Armory Support team to track production outages within their production environment. Depending on the severity of the issue, customers are encouraged to open a Priority Zero (P0) level case with our Support team for assistance
Armory Product Demo Videos
Thank you for taking an interest in Armory SpinnakerWe have a series of product demo and videos outlining Armory-specific functionality across the entire SDLC before kicking the tires.For more information, please visit the link belowhttps://www.armory.io/blog/armory-product-demos/
Armory Release Definitions
Armory Release Definitions have been moved to our docs site as a part of our re-organization.You can find the information here:
Armory Support Information (Hours of Operation/SLAs/Procedures)
Armory offers a variety of support options depending on your needs. With the resources below, customers should be able to find answers to any questions regarding their support contract and what kind of service-level agreements to be expected as a customer of Armory. There are many ways to contact support, but our first suggestion is always to check our documentation at docs.armory.io or our kb.armory.io as we constantly seek to improve our customers’ self-service experience.
Aurora RDS Blue/Green Deployment in Spinnaker Services
Introduction
Authenticate Against Jenkins
Introduction
Authentication Timeout Results in 401 Error
Issue
Authorization Configuration with File Based Role Definitions
Introduction
Auto cleanup pipeline executions of specific applications in Spinnaker
Introduction
Autogenerated Applications Cannot be Deleted w/ onlySpinnakerManaged set to false
Issue
AWS Billing - Billing, Budget Alerts and Cost Explorer
The following is quick information about work that Technical Support Engineering have done in Budgets and Cost Explorer, and how to use both to review billing in AWS
AWS image caching issues for certain accounts, when multiple accounts are defined
Issue
AWS Lambda & Custom Webhook Stages
Introduction
AWS Lambda Plugin stage does not populate the existing Functions
Issue
AWS RDS Certificate Update
Introduction
Bake or Deployment Stage timeouts caused by external factors
Issue
Bake stage fails with Rosco error... Command not allowed when used memory > 'maxmemory'
Issue
Best Practices around Monitoring and Logging the Spinnaker Environment
Introduction
Best practices for deploying and scaling Clouddriver HA services
When Spinnaker turns on Clouddriver HA mode, Clouddriver will split and get deployed as four different services, each only performing a subset of the base Clouddriver’s operations:
BitBucket Cloud vs BitBucket on Premises
BitBucket Cloud and BitBucket on Premises APIs are divergent from each other up until certain versions.
BitBucket Webhook Fails with Unknown Bitbucket Event Type in Dinghy
Issue
Calculating Successful Pipeline Execution
Introduction
Can Users' Level of Access in the Armory Console be Retricted using RBAC
Introduction
Canary Config Metric Template displays Invalid MetricStat JSON- should NOT have additional properties (Example- AWS Cloudwatch)
Issue
Cancel a Stuck Pipeline
Introduction
Cannot add custom Kind to Delete Manifest Stage
Issue
Cannot Define EBS Volumes When Deploying Baked AMI
Issue
Cannot Enable Dinghy to Proceed with GitHub Pull Request Validations
Introduction
Cannot overwrite images in a Knowledge Articles (Update them or Adding an Image with Same Name)
Issue
Cannot See Pipeline History for Changes Made
Issue
Capacity Provider Concepts
Capacity provider concepts and setup
Capture Spinnaker configurations with the Armory Support Bundle
Introduction
Capturing Spinnaker configurations with Armory's Support Bundle (w/ Automated Script)
For a full detailed of the manual process this script is automating, please refer to KB0010398
Capturing Spinnaker configurations with Armory's Support Bundle (w/ Automated Script)
For a full detailed of the manual process this script is automating, please refer to KB0010398
Cases and Change Requests
Customers may find themselves needing to submit either a Case or a Change Request within our portal. The following article should assist customers with understanding when one type of submission should be used over the other.
Changes to a ConfigMap not reflected in new deployment
Issue
Changing Timeouts for Spinnaker Services Using okHTTP (Java Services)
Issue
Clean Up Schedule for Armory Scale Agent Operations Older than a Specific Time Period
Introduction
Cloud provider account disappearing from Spinnaker UI
Issue
Clouddriver agent names too long for the columns in Database
Issue
clouddriver-caching errors | ClusteredAgentScheduler - Unable to run agents
Issue
Clouddriver fails to start up Armory Agent during initial install
Issue
Clouddriver is not caching subnets or server groups
Issue
Clouddriver Logs Show Slow SQL Warnings
Issue
Clouddriver shows "Connection is not available, request timed out after 10000ms."
Issue
Clouddriver unable to download artifacts from private github instance due to SSL errors
Issue
Clouddriver will not Connect to MySQL Causing a Stoppage, DatabaseChangeLock Error
Issue
Combined version and service (module) name is too long when deploying with App Engine
Issue
Common Vulnerabilities and Exposures (CVE) Reports - Armory Continuous Delivery
Please note that all CVE Reports are classified as Armory Confidential and Proprietary Information, and should not be shared outside of a customer's organization. The documents fall under the customer's MSA and NDA agreements.
Common Vulnerabilities and Exposures (CVE) Reports - Armory Plugins
Please note that all CVE Reports are classified as Armory Confidential and Proprietary Information, and should not be shared outside of a customer's organization. The documents fall under the customer's MSA and NDA agreements.
Common Vulnerabilities and Exposures (CVE) Reports - Armory Continuous Delivery
Please note that all CVE Reports are classified as Armory Confidential and Proprietary Information, and should not be shared outside of a customer's organization. The documents fall under the customer's MSA and NDA agreements.
Common Vulnerabilities and Exposures (CVE) Reports - Armory Plugins
Please note that all CVE Reports are classified as Armory Confidential and Proprietary Information, and should not be shared outside of a customer's organization. The documents fall under the customer's MSA and NDA agreements.
Compliance and Auditing
Question:
Concurrent Pipelines running with Bakes Fail (Packer behavior in Rosco)
Issue
Configure ElastiCache with Spinnaker
Introduction
Configure persistent storage to capture heapdumps for Spinnaker Services
Introduction
Configure spinnaker to use dynamic Github authentication tokens
Introduction
Configure Spinnakers External Account Configuration with Vault
Introduction
Configuring AWS account using EAP - error running CreateServerGroupTask for pipeline Credentials not found*"
Issue
Configuring Azure Named profiles with Terraformer
Introduction
Configuring Spinnaker services to connect to Redis over mTLS
Introduction
Configuring Spinnaker to Use Internal Certificate Authorities
Introduction
Configuring VIM editor to work with YAML
Introduction
Confirming Environment Logs and Environment Debugging ID with Armory Support
Introduction
Container Image cannot be selected
Issue
Contributing to OSS Spinnaker (Plugins and Donations of Code)
Table of Contents
Controlling the Number of Caching Agents in Clouddriver
Introduction
Create AWS IAM Roles from a Terraform Script
Introduction
Create Kubernetes Configmaps and Secrets from Artifact Files
Inject Artifact File into Pipeline Context
Creating a Change Request in ServiceNow
At Armory, we aim to provide customers with a secure method to request changes to their environments. Authorized users can be added and managed by customers' self-designated ServiceNow Service Portal Customer Administrators.
Creating Cases and Case Management in ServiceNow
At Armory, we aim to provide world-class support by giving our customers choices for different support options. To optimize our customer's support experience, please read our guidelines at our Armory Support explanation page.
Creating Custom Deployment Strategies
Introduction
Critical Technical Notifications
As a part of being an Armory customer, our crew would like to make sure our customers are well informed of any issues discovered about Armory's products and services.
Custom RunJobs intermittently failing (Cannot Access Properties of Application)
Issue
Custom runjobs intermittently failing with "Access denied to application" error
Issue
Customer User does not have access to create Change Request
A Managed Customer user will need to have a different group assigned to them in order to have the functionality to file a change request.
Customers Unable to Access Resources due to GitHub sunseting Deprecated Teams API Endpoints
Issue
CVE-2021-43832 - FavExploit- Spinnaker RCE Vulnerability in Gate
Issue
CVE-2021-44228 - log4shell / log4j Vulnerability Analysis
Issue
CVE-2022-22965 - Spring Framework RCE w/ JDK 9+
Issue
CVE-2022-23506- Rosco/Packer - Insertion of Sensitive Information into Log File (OSS only)
Issue
CVE-2022-3786 / CVE-2022-3602- OpenSSL 3.x Vulnerability Analysis
Issue
CVE-2022-42889 - Apache Commons Text Vulnerability
Issue
Database recommendations for running spinnaker
Services within spinnaker such as Clouddriver, Front50 and Orca can be configured to use databases such as MySQL, PostgreSQL or Redis as their datastore. The databases can be configured by following the steps mentioned on the below docs
debug.armory.io is Down, Crashes Environments w/ Debugging Enabled into a crash loop restart
Issue
Debugging Deployment Errors
Issue
Debugging Slow Orca Performance
Issue
Deck will not connect to Gate. Receives 403 Error.
Issue
Define Application attributes using Dinghy
Introduction
Delayed Image Push Pipeline Trigger
Issue
Delete Pipeline Executions in Redis
Introduction
Deploy manifest performs SpEL substitution even if Skip SpEL Evaluation is checked
Issue
Deploy (Manifest) stage does not remove previous Helm2 deployment resources from Kubernetes
Issue
Deploy Stage Missing From Pipeline
Issue
Deploying Helm Charts with Armory Spinnaker and Managing Secrets in Helm
Introduction
Deploying through Armory Agent for Kubernetes with CRD results in "Error occurred when List customResourceDefinition/null"
Issue
Deployment error Exception ( Resolve Deploy Source Manifest )
Issue
Deployment fails with "Load balancer service <service-name> does not exist" error
Issue
Deployment Strategy Red/Black error- Load balancer service <service-name> does not exist
Issue
Deployments Cluster Tab missing "Undo Rollback" feature / Clusters Tab Missing Previously Available Options
Issue
Deployments Exceeding Progress Deadline
Issue
Deployments on PCF fail with invalid size for application memory.
Issue
Determine or Check the Environment's Spinnaker Version
Question:
Difference in the deployment phase takes and the service boot time
Issue
Differences Between GitHub and GitRepo Artifacts and Choosing the Right Artifact Type
Introduction
Dinghy 'roles' entry is not getting passed to actual Spinnaker Pipeline JSON
Issue
Dinghy cannot self-reference Pipelines created in the same Dinghyfile using PipelineID
Issue
Dinghy Crashing for Customers using 2.24+ when customers are using a Custom Endpoint
Issue
Dinghy Crashloop When Using Redis (Loss of connectivity/Loss of relationship)
Issue
Dinghy Fails to Create Pipelines in Armory 2.19.8 and Prior
Issue
Dinghy file does not update pipelines in Spinnaker
Issue
Dinghy github status is always failed when using YAML parser
Issue
Dinghy in 2.26 tries to validate README.md as a module
Issue
Dinghy Multi-Repo Strategy with RepoConfig
Introduction
Dinghy rendering failures - Forbidden- Access denied to application (Access Rights Verified)
Issue
Dinghy skips updating pipeline when local module is changed
Issue
Dinghy Slack Notifications not working - java.lang.NoSuchMethodError- void org.jsoup
Issue
Dinghy webhook shows an HTML response, rather than a JSON response
Issue
Dinghy will fail to start after configuration
Issue
Dinghyfile updates do not trigger Slack notifications
Issue
Dinghyfiles with defined Application Permissions fails to create/update pipelines (Forbidden Error)
Issue
Disable default repositories for Plugins on air-gapped environments and Specify the Repo
Introduction
Disable diagnostic settings results in echo pod unable to start (Armory 2.21.2)
Issue
Disabling TLS 1.1 in Spinnaker and Specifying the Protocols to be used
Introduction
Docker binding to matching repositories breaking pipelines
Issue
During Migration to IAM for Service Accounts, CloudDriver in HA returns error- Access Denied Service- Amazon S3; Status Code 403
Issue
Dynamics Accounts with GitHub Cannot Use Token
Issue
Echo Sends Duplicate Triggers
Issue
ECS- Supporting EphemeralStorage in Fargate
Issue
ECS Deployment errors with "All server group names for cluster....are taken"
Issue
ECS Deployment fails when using EXTERNAL launch type Task
Issue
ECS Deployment Failures
Issue
Editing application permissions yields an error "Cannot read property 'label' of undefined"
Issue
Emoji in REDIS Database error when Migrating from REDIS to MySQL
Issue
Enable Armory Agent for Kubernetes to only discover objects that are discovered by Spinnaker (or) Armory Agent
Introduction
Enable AWS CloudFormation
Introduction
Enable Basic Form Authentication for Spinnaker via Halyard
If you want to enable Simple Auth for Spinnaker using Halyard
Enable Detailed Logging / Debugging Mode in Spinnaker Environment Service
Introduction
Enable external Spring Cloud Config Server
Issue
Enable two pipelines to run as Mutually Exclusive
Introduction
Enabling Caching on Git Repos and Clearing Clouddriver Caching to Resolve Caching Issues
Introduction
Enabling the Managed Pipeline Templates UI
Introduction
Error - Credential not found (AWS)
Issue
Error- Algorithm HmacPBESHA256 not available when Inserting TLS Certificates into Spinnaker
Issue
Error- You cannot delete this application because it has server groups
Issue
Error calling module- error rendering imported module 'pipeline/....module.json'- invalid character
Issue
Error displayed on the UI- Exception (Resolve Target Manifest)
Issue
Error when deploying manifest kind CSIDriver with Armory Agent
Issue
Errors don't propagate from child to parent pipelines if they occur in stages configured to "ignore the failure"
Issue
Errors in deploying Dinghy while using AWS elastic cache with encryption in transit
Issue
Errors Initializing and Starting Up Policy Engine (Configuration Hints, Orca/CloudDriver/Front50)
Issue
Errors when moving account configuration to external S3 bucket
Issue
Escalating a Case
Case escalations can be a powerful tool to advise our Support Team of a change to the issue that was raised. The purpose of this ability is to allow customers to advise Support about certain changes
Exception Starting Plugin in Orca
Issue
Execution parameters shared when Run Job stages are running in parallel
Issue
ExecutionID & StageID are not unique in the webhooks sent by Spinnaker CustomWebhook stages
Issue
Executions after Migrating to Terraformer are delayed or take a lot more time to execute
Issue
Exposing Gate API With x509 Certificates Using A Separate Deployment
Introduction
Exposing Spinnaker
This page has been recategorized and moved to our Documents Library//docs.armory.io/spinnaker/exposingspinnaker/
External Accounts Plugin will not recover after a failure.
Issue
Facing 400 Error while Saving Canary Configuration
Issue
Facing Throttling Issues when Deploying Cloud Services
Issue
Failed to read manifest- "Exception (Wait for Manifest to Stabilize. Unexpected Task Failure)"
Issue
Fiat Ignores x509 cert and Check Role Provider for Permissions
Issue
FIAT unable to start in a CloudFoundry Environment
Issue
Find AMI stage finding images in unrelated account
Issue
FLUSHALL on Spinnaker's Redis Cache
Question:
Flushing Gate Sessions
Introduction
Front50 pods failing to start after upgrading Spinnaker version from 2.20.1 to 2.23.5
Issue
Gate Metrics not Available to the Monitoring-Daemon
Issue
GCP- Exception ( Wait For Server Group Disabled ) - Error while querying target pool instance health
Issue
GCP error- code = PermissionDenied desc = Forbidden- HTTP status code 403; transport-
Issue
General Spinnaker Optimization Suggestions
Introduction
Getting the IP of an EC2 Instance With Pipeline Expressions
Question
GitHub Changed (default) Base Branch from "master" to "main"
Issue
GitHub DinghyFiles referring to Mixed Case Applications Fail
Issue
GitHub issues with Certificates triggered Failure to be able to execute pipelines, due to Plugins (missing/unavailable)
Issue
Github Notifications Do Not Update Github Commit Status
Issue
Github trigger matches multiple artifacts on trigger (.tf files)
Issue
Google App Engine (GAE) Deploy Stage Not Showing Artifacts That Have Been Added and Can't Edit
Issue
Google Cloud Provider - Private GKE
Introduction
groovy.lang.MissingMethodException when referencing github config artifact for Google App Engine
Issue
Hardening and Security Suggestions for Spinnaker
Customers may find that there is a need to secure their instances depending on their company best practices. This will vary from customer to customer depending on their overall infrastructure design and the exposure for their Spinnaker environment.
Hardening Spinnaker to Protect Against SPEL Misuse with AWS Metadata
Issue
Hashicorp Terraform GPG Rotation (CodeCov vulnerability)
Issue
High number of GitHub emails after every Dinghyfile update
Issue
Hitting annotation hard limit when deploying a Configmap/Secret (Too long)
Issue
Hitting Igor's Caching Thresholds
Issue
HTTP/HTTPS Redirects with Authentication
Troubleshooting HTTP/HTTPS Redirects
IAM auth on Pods via IRSA
Introduction
Igor pod not running or starting up when configuring pipeline triggers (500 Internal Server Error)
Issue
Impersonate a User for ServiceNow Troubleshooting (ServiceNow Admins)
To help assist customers or Armory Users and ensure that they have roles set properly, it may be necessary to impersonate a user. This provides access to see the portal via the account's access.
Incorrect OAuth2.0 Redirects
Issue
Increase the Agent operation wait timeouts on Agent Clouddriver Plugin
Issue
Increase timeouts on Agent Plugin and Agent to avoid network latency issues
Introduction
Infinite Authentication Loop - User Initially Authenticate WIthout Issue, but Loops upon Timeout
Issue
Infinite UI Loop "Save Changes" stuck attempting to Save a JSON Canary Config in the Spinnaker UI
Issue
Infrastructure Tab not Showing Correct Deployment Details
Issue
Ingesting AWS CloudFormation Templates Larger than 51200 Bytes as an Artifact
Issue
Ingesting Metrics from Observability Plugin to Datadog
Introduction
Initiated Login Does Not Work With SAML 2.0
If you’ve set up SAML 2.0 authentication for your Spinnaker cluster and are able to login when your Identity Provider (iDP, ADFS/okta/etc.), but aren’t able to login when the Service Provider (SP, Spinnaker) initiates the login, try the following:y
Instance is not healthy but the deployment shows as succeeded
Issue
Instance registration teardown
Introduction
Integration with Nexus
Introduction
Internal error, failed calling webhook when deploying Armory Continuous Deployment after upgrading the Armory Operator
Issue
Invalid Token with Named Profiles Assume Role
Issue
Is it Possible to Restrict User Access to Certain Envrionments with RBAC
Introduction
Jason McIntosh's handy information about Setting Up Monitoring
Introduction
Jenkins CI driver can start but Spinnaker fails to monitor the Jenkins job (Tomcat Server)
Issue
Jenkins Timeout Issues
Issue
Jobs run by a service account in a different namespace generates generic errors (Index -1 out of bounds for length 0)
Issue
JSONPath Update Allows Constraints on Additional Webhooks Payload Values (Directory Specific)
Introduction
K8s v1.21 causing Valut intergation outages (with kuberenetes auth method)
Issue
Kayenta 404s in Armory Spinnaker
If you’ve enabled Kayenta in Armory Spinnaker 2.0, and are running into any of these issues:
Killing Processes where Parent pipeline triggered a large/infinite loop via webhook, keeps triggering child pipelines
Issue
Kubernetes and Docker Accounts are not visible in Spinnaker Console UI
Issue
Kubernetes Namespaces still appearing after deletion in Spinnaker UI
Issue
Kubernetes Service Name Change Creates A New Service (old service not deleted)
Issue
Kubernetes V1 Provider Removed
Issue
Large Fiat headers can cause permission errors due to timeouts and 400 errors.
Issue
Leverage Spinnaker API to Pull a List of Running Pipelines
Introduction
Load Balancer Service svc-spinnaker-demo Does Not Exist
Issue
Locked Pipelines Cannot be Unlocked Except by Admin Access Users When Using "Run As"
Issue
Long Force Cache Refresh (Kubernetes)
Issue
Manual Installation of Project Aurora for Spinnaker™ (No Helm)
Introduction
Maximum Call Stack Size Exceeded Error
Issue
Metric data does not flow into Splunk despite setting up HEC Connection
Issue
Migrate Clouddriver's Redis To Its Own Instance
Take a snapshot of your existing Redis cluster Create a new cluster with the snapshot from the previous step. This will include all the keys from the previous cluster that you can clear out later.Configure Clouddriver to use the new cluster. In your `/opt/spinnaker/config/clouddriver-local.yml` add the following:
Migrating from one cluster to another
Migrating pipelines:
Migrating to the Policy Engine Plugin from the Policy Engine Extension
Introduction
Missing file in rosco image got Exception (Create Bake)
Issue
Missing Run As User field for triggers in Spinnaker UI
Issue
Monitored Deploy Rollout
Introduction
MySQL communication failure when using TLS1.1
Issue
MySQL Table Name Change Error When Rolling Back Spinnaker (Undo Renamed Values)
Issue
Network latency between Clouddriver and Agent causing Timeout issues
Issue
New Relic Canary Integration
First you will need a Query Key and your account number.Follow https://docs.newrelic.com/docs/insights/insights-api/get-data/query-insights-event-data-api
Nexus- Could not parse error response Unexpected character ('<' (code 60))- expected a valid value
Issue
No "Expected Artifacts" Field
Issue
No ingress is supported when using API version "networking.k8s.io/v1"
Issue
No plugins error on service startup
Issue
nodeSelector/toleration settings do not get propagated while using Operator
Issue
NoUniqueBeanDefinitionException error with Armory Cloud enabled
Issue
Oauth Setup for Github
This page has been recategorized and moved to our Documents Library//docs.armory.io/spinnaker-install-admin-guides/authn-github/
Objects disappearing under the Cluster tab and a slow Deck UI Cluster page
Issue
Obscuring Last Names in Public Articles and Cases - Internal Users
To help with providing privacy options for Armory Employees, Support has added an option to obscure last names and how they are displayed. This is to give options to employees so that their full names are not available publicly to be scraped from the site, in public articles.
OPA Policies not working after Spinnaker Upgrade
Issue
Opa Policy Deployment Error- Failed to shutdown server gracefully
Issue
OpenShift- How to supply the complete certificate chain
Issue
Operator cannot connect to Halyard due to a TCP timeout
Issue
Operator error when performing a Spinnaker upgrade in an air-gapped environment
Issue
Orca mishandling SpEL Expressions
Issue
Orca Operations and Fine Tuning Orca Performance
The following KB article explains how a modern Orca operates in Spinnaker and provides guidance towards fine-tuning Orca performance in more modern Spinnaker environments.
Orca Stuck in CrashLoopBackoff (DB- mySQL)
Issue
Orca with SQL high memory usage / Tables information dive
WhatThis error can be because the data in SQL of Orca executions is too big and Orca can't load the data.Can happen because 2 factors, but basically too much data.1) One of Orca rows has reached over the maxallowedpacket size you have defined or over the max (1Gb)2) A pipeline execution in sum has reached over this maximum.How to fix this?
OSS Commits and their Relation to Armory Enterprise Releases
Introduction
Overriding Terraform Available Versions via init containers
Issue
Packet for query is too large - MySQL (max_allowed_packet)
Issue
PagerDuty - Setting up for New Users
Administration to Setup User in the Pager Duty Queues
PagerDuty Notifications Integration with Spinnaker
Introduction
Parameters with Kubernetes Manifest & Using SpEL
Question:
Passing Rosco/Packer Information to Next Stages using SPeL
Introduction
Password Resets
Password Resets can be accomplished by self-resetting their password
Pausing in-flight Kubernetes deployments in Spinnaker
Introduction
Perform clean up after pipeline fails/ Trigger pipeline when another is canceled/successful/failed
Introduction
Pipeline Execution Error "Service selector must have no label keys in common with target workload"
Issue
Pipeline Randomly Cancelled
Issue
Pipeline stuck on "Wait for Manifest to Stabilize" Task
Issue
Pipelines are triggered by anonymous without sufficient permission
Issue
Pipelines as Code (Dinghy) starts with multipleBranchesEnabled- true even though it has been explicitly set to false
Issue
Pipelines being triggered with Stale Artifacts
Issue
Pipelines with GAE gitHttpsUsername/gitHttpsPassword fail with 'Not authorized' Exception
Issue
Pod details do not show for some pods
Issue
Pods Going to Unhealthy State due to Zombie Processes
Issue
Pods Running Jobs or Scripts Remain After Completed State
Issue
Policy Engine - No Policy Decision Detected, missing spinnaker.persistence.pipelines
Issue
Policy Engine - OPA for STYRA DAS Deployments
Issue
Policy Engine- PluginRuntimeException- Failed to write file 'plugins/Armory.PolicyEngine-policy-engine-vX.X.X' to plugins folder
Issue
Policy Engine Error (TypeError- e.takeUntil is not a function) when Accessing Project Tab
Issue
Policy Engine RegoScript Examples
Introduction
Potential outages when Vault integration is on (w/ K8s auth method) and updating to K8s 1.21
Issue
Primary and Secondary Contacts and Customer Service Portal Administrators
Primary and Secondary Contacts are the users that are listed within a customer account (ServiceNow Admin access required). These are the contacts outlined by the company as their main administrators, and points of contact for Armory. We will use these contacts for any communications such as for NPS surveys and any other issues that we must contact them about (e.g. security items/alerts)
Producing Artifacts in the Webhook Stage
Introduction
Prometheus Canary Integration
In this example we are using the prometheus K8s operator
Provide Logs from Pods to Armory Support for Troubleshooting Purposes
Introduction
Pull Request Validation Error - File not Found (Module)
Issue
Quiet the Echo Service
Introduction
Redis to SQL Migration- Can't see old history
Issue
Reduce kubeconfig Size
Issue
Reducing AWS EKS get-token Calls Made by Kubectl
Introduction
Remove Ability to Manually Trigger Pipelines
Introduction
Renaming a ConfigMap creates a new copy and does not delete the original
Issue
Renaming An Application in Spinnaker
Question:
Render and Validate Dinghy Pipelines with Armory CLI
Introduction
ReplicaSet Versioning is erratic, sometimes will work, sometimes will not
Issue
Resolve cluster-wide DockerHub Rate Limit Warning
Issue
Resource requests on custom stages does not work- Error- got "map", expected "string"
Issue
Resources are duplicated in the UI when Migrating to Armory Agent
Issue
Restarting Spinnaker Unexpectedly Triggers Many Jenkins Jobs when using Redis
Issue
Restrict Application Creation
Introduction
Restricting the user into specific namespace in EKS cluster
Issue
Rosco's bakesCompleted_seconds_count metric unavailable in Prometheus
Issue
Rosco cannot set environment variables in Packer bakes
Issue
Rosco will crash causing a failed pipeline erratically
Issue
Run a Generic Shell Script with Spinnaker
Introduction
Run Job - Producing Artifacts
Introduction
Run Job (Manifest) Configuration has a non-zero backoff does not show results of multi-failure
Issue
Run Pipeline Stage picks up and uses Pipeline Artifacts from previous stages
Issue
RunJob Does not Execute Due to Write Permissions in FIAT
Issue
S3 buckets for Front50, revision history
Question:
Salesforce to ServiceNow Service Portal Migration
In the end of February, we will be migrating all of our Support Issues, Knowledge Base, and Customers from Salesforce to ServiceNow. We will be re-routing our URLs (e.g. https//portal.armory.io, https://registration.armory.io) to the ServiceNow portal on our go-live date, March 1st, 2021
Searching Applications folder with Execution ID times out
Issue
Secrets can cumulatively build up in Spinnaker Services and cause Cluster/Control Plane Crash
Issue
Securing Dinghy Deployments Occurring via GitHub
Introduction
Securing Webhook Endpoints
Introduction
Serverless Support with Spinnaker
Question:
ServiceNow - Add New Version of Spinnaker
Adding Versions to ServiceNow and Jira need to be done anytime a new major version of Spinnaker is created.
ServiceNow - All About Change Requests, Navigation and Anatomy
Change requests have a numbering scheme that starts with `ACHG`
ServiceNow - All About Support Cases, Navigation and Anatomy
Support cases begin with `CS` and are located in the Cases section that can be searched using the Sidebar Navigator
ServiceNow - Allowing Customers to See Tickets Across Organization Divisions
Customers (e.g. Apple) may want their team to be able to see all tickets that are opened across the entire Apple Organization when they are logged in and looking at their (ServiceNow) Company's tickets.
ServiceNow - Attaching KB Articles
Attaching KB Articles to a Case (newly created or old)
ServiceNow - Create/Edit Knowledge Base Articles
Overview
ServiceNow - Creating Cases and Change Requests on Behalf of Customers
Note https://support.armory.io/support?id=kbarticleview&sysparmarticle=KB0010266
ServiceNow - Creating Users
Note: Do not click on the New button at the `Users` table to create a new user. This will create a user under the `Users` table, but the user needs to be created under the `Contact` table instead.
ServiceNow - Customer Account/Company Creation
To create an account, you will need to add the organization account both in ServiceNow and in Jira. This is so that case, when created, will refer to the correct organization when synchronizing across both systems.
ServiceNow - Escalation Response for Support Engineers
The following outlines what happens after an escalation is sent from a customer.
ServiceNow - Finding a Resource (Case, Change Request, etc...)
If you are looking to find Cases or Change requests, there are several ways to go about doing so.
ServiceNow - Navigation of the Backend Site
Sites in ServiceNow
ServiceNow - Notifications on Case Comments
Armory Technical Support Engineers receive notifications on case updates based on a workflow designed to send DMs to the Support Engineers. The following explains how to manage the notifications and what the icons mean
ServiceNow - Sales Engineers - Opening and Managing Support Cases
For further information about Sales Engineers opening Support Cases, please check out the slab doc
ServiceNow - Using Lists and Views
Using Lists and Views
ServiceNow Service Portal Customer Administrators
Customer Administrators are privileged customer users who have been granted additional access and management rights to the ServiceNow Portal. Customers should inform Support or the designated TAM about any changes that need to happen with regards to their assigned administrator. Because the administrator has additional privileges, please be aware of the following information when assigning administrators.
Set Execution History Lookback
If you have a pipeline that hasn’t run in a long time, you might only ever see one pipeline execution in its history – and as soon as you run the pipeline again, it disappears, leaving only the latest.
Setting a Github Status Check on PR from a Pipeline Status
A common situation Administrators can hit is creating a pull request on Github with some Terraform changes that are run through pipelines. Administrators want to verify the Terraform changes cleanly and “plan” before applying them.
Setting an SQL Cleanup Job for Clouddriver
Introduction
Setting up and Querying FIAT backend in MySQL and Viewing Users that have Logged in to Spinnaker Using FIAT
Introduction
Setting up Logging without Tokens
Introduction
Settings for SysDig Monitoring of Spinnaker
Introduction
Settings to integrate with Application LoadBalancer (ALB)
Introduction
Shared Configuration Repository
Question:
${ in shell script introduce massive warning in Spinnaker UI (Failed to evaluate ... not found)
Issue
sourceAMI Must be Defined in Two Areas if a template JSON is used in AWS bakes
Issue
Specify a Subnet During Bake Stage
There may be occasions where it’s useful to specify the subnet used in your Bake stage. Targeting a specific subnet can ensure your Packer Builders (which Spinnaker relies on for baking images) are assigned a subnet which meets the requirements of the network (e.g. a subnet which does not auto-assign a public IP).
SpEL expression failure interpreting dollar sign bracket with Terraform Templates
Issue
SpEL expression issue- Must be of type string- "integer"
Issue
Spinnaker Access denied error with Armory Agent deployed and FIAT enabled (Exception (Monitor Pipeline))
Issue
Spinnaker and Helm
Which one works best? Helm or Spinnaker?The following blog post aims to answer this question about how both work together//www.armory.io/blog/deploy-helm-charts-with-spinnaker/
Spinnaker Cloud Provider Limit
Question:
Spinnaker does not properly clean up old artifacts and resources when redeploying
Issue
Spinnaker Fails to Deploy to Google AppEngine Flex Due to Name Being Too Long
Issue
Spinnaker Improperly Errors Even When SPEL Expressions Are Commented Out
Issue
Spinnaker in an Istio Mesh
Introduction
Spinnaker is Down with the Error of "OOM command not allowed when used memory > 'maxmemory" (Redis)
Issue
Spinnaker Kubernetes V2 FAQs
How to Create a Pipeline in Kubernetes V2 with Spinnaker?
Spinnaker not removing the old Auto Scaling Group after a new deployment
Issue
Spinnaker only supports unique account names across both AWS and ECS
Issue
Spinnaker pipelines not displaying in Infrastructure tab
Issue
Spinnaker supports KUSTOMIZE up to v3.8.5 as the Render Engine in Bake (Manifest) Configuration, but not v4.x (Artifact not found)
Issue
Spinnaker Timesout After 60000 milliseconds when Updating to Spinnaker 2.19.x from 2.18.x or lower
Issue
Splunk AMI Issues - IO WAIT Warning, Connection Lost/Splunk Crashed with Splunk Service
Issue
Spring Expression Language (SpEL) samples
Introduction
Spring Expression Language Tricks
Introduction
SSH Keys in Terraformer
Introduction
Stacktrace Error appears on the Functions Display
Issue
Stage Returns Error that Fargate Only Supports Network Mode awsvpc Even Though it is Configured
Issue
Stage Times Out During Implementation, Need to Extend Stage Timeout
Issue
Storing application secrets in vault for use in Spinnaker pipeline.
Application secrets should not be passed through Spinnaker or any other deployment tool as this is not safe from a security standpoint. If the tool is breached you now have all applications secrets that were passed through the pipeline exposed. The only things that should ever be passed through a deployment tool are location and/or references to the secret.The best practice for application secrets is for the application to fetch the secret itself during application startup. For VMs this is during the VM bootstrap or application startup process. For Kubernetes you would usually use do this using an init-container, sidecar, or both.For Vault here are some resources on how to get this working for Kubernetes//www.hashicorp.com/blog/injecting-vault-secrets-into-kubernetes-pods-via-a-sidecar/https//itnext.io/dynamic-vault-secrets-agent-sidecar-on-kubernetes-cc0ce3e54a94For AWS Secrets Manager and Vault see the following//www.godaddy.com/engineering/2019/04/16/kubernetes-external-secrets/https://github.com/godaddy/kubernetes-external-secrets
Streaming Spinnaker Events to External Resources with Echo Event Filtering Plugin (PagerDuty)
Introduction
Stuck Pipeline - Orca with MySQL Editing the Database
Issue
Suggestions and Recommendations for configuring and optimizing Armory Spinnaker
As Spinnaker supports variety of integrations, administrators may run into scalability issues when getting Spinnaker up and running. Following are the list of articles that provide recommendations on managing the environment to overcome scalability issues.
Suggestions on how to do Performance and Stress Testing in a Spinnaker Environment
This document talks about the steps taken to test the performance and the behavior of Orca and Spinnaker UI and arrives at the size and the number of stages the pipeline can have. It provides an example of testing that can be used to determine the limits of the environment.
Suggestions to Review before Performing Spinnaker Updates and Upgrades
The following is a list of articles and suggestions to review when exploring an upgrade path to a Spinnaker environment. These are general directions to help admins make their upgrade process as painless as possible, but it is by no means meant to be a final checklist of an upgrade process. It is intended for guidance for our customers to help mitigate some of the risks associated when seeking to upgrade the environment.
Support Portal User Registration
In order to submit cases to Armory, review cases for their team or view customer-only Knowledge Base Articles, users will need to register their information on our ServiceNow Portal.
SwaggerUI commands returns 4xx errors
Issue
System recommendations for Kubernetes Deployments
As the number of applications and the Kubernetes accounts scale, the resource requirements for the Spinnaker services would change. Below is the table containing the resource requirements for base deployment of Spinnaker targets organizations with
Task definition from a Github artifact does not replace task IAM role
Issue
Technical Support Case Statuses
Overview:
Technical Support Engineers - Creating Internal Cases to Track Project Work
To properly ascertain and track internal project work, Armory Technical Support Engineers are encouraged to open cases to register their work with pre-sales customers and any internal or cross-departmental collaboration projects that may occur.
Terraform 'Invalid character' Error
Issue
Terraform Apply Stage Doesn't Output the Full List of Changes
Issue
Terraform Deployment Using Armory Spinnaker
Introduction
Terraform Maximum File Size Error
Issue
Terraform Named Profiles are Unable to use Environment Variables to Store Secrets
Issue
Terraformer does not clone properly from Bitbucket
Issue
Terraformer Error exec- "terraform"- executable file not found in $PATH
Issue
Terraformer Named Profiles with GCP
Introduction
Terraformer Only Takes Input from a Single File from Github
Issue
Terraformer Stages Stop Running Jobs with a 'timeout' error (Logging Enabled)
Issue
terraformer Version doesn't render in pipeline JSON by default
Issue
Testing X509 Certificates Subject Filtering with Agent
Issue
The Role of HELM and Spinnaker
Question:
The timeout of Bake Stage does not respect the timeout in Bake Stage configuration
Issue
Troubleshoot Baking Helm 3 Charts
Issue
Troubleshooting AWS Role Assumptions in same AWS account or cross-accounts
Issue
Troubleshooting Gate/Deck CORS Issues
Introduction
Trying to turn on Vault results in a Missing Client Token Error
Issue
Unable to add ECS cluster to Spinnaker (not authorized)
Issue
Unable to Add Kubernetes Cloud Provider with a Secret Engine
Issue
Unable to configure permissions on a Spinnaker application with a PostgreSQL database
Issue
Unable to Deploy After Moving AWS Account into Profiles
Issue
Unable to deploy Spinnaker after changing Operator and Halyard versions
Issue
Unable to Deploy to a New Kubernetes Cluster to Spinnaker
Issue
Unable to evaluate SpEL variables in an email notification
Issue
Unable to list kubernetes Endpoints in namespace when running clouddriver with Agent
Issue
Unable to pick up the CredentialsTypeProperties bean and map it to Customer ArtifactCredentials
Introduction
Unable to retrigger a an existing pipeline from Spinnaker UI
Issue
Unable to see the accounts listed when using the App Engine plugin in a pipeline
Issue
Uninstalling Armory Spinnaker
Introduction
Unlock the DATABASECHANGELOGLOCK Table
Introduction
Updates of Previously Cached gitrepo Fails on Artifact Fetches
Issue
Updating Spinnaker Stops Dinghy From Updating Existing Pipelines
Issue
Upgrade from OSS to Armory Spinnaker
Introduction
Upgrade Spinnaker Using Armory Halyard
This page has been recategorized and moved to our Documents Library//spinnaker.io/docs/setup/install/upgrades/
Upgrade to Spinnaker (1.20.x+/2.20.x+) Causes Errors as Pipelines Deploy to Unavailable Namespace
Issue
Upgrade using Armory Operator is not working- got halyard response status 405, response- Request method 'POST' not supported
Issue
Upgrade Your Kustomize Version
Introduction
Upgrading an EKS Cluster
Introduction
Upgrading from 2.19.x to 2.20.x Error Authenticating Against EKS Clusters (heptio-authenticator-aws)
Issue
Use case for AWS Sticky Sessions
Issue
Use Custom Images with Halyard (Public and Private Docker Registry)
Introduction
Username read error when executing Terraform stages with BitBucket hosted artifacts
Issue
Users are Unable to Unlock a Spinnaker Pipeline through the Console - Unlock via UI unchecked
Issue
Users unable to execute or create pipelines due to large headers on API calls to Orca, front50
Issue
Using `Find Artifacts From Resource (Manifest)` overwrites rendered artifacts
Issue
Using Custom Resource Status Plugin and Scale Agent leads to An attempt was made to call a method that does not exist Error
Issue
Using external artifacts as pipeline parameters
Introduction
Using Openshift with Spinnaker
Question:
Using Pipelines as Code to Enable AWS Auto Scaling Group Metrics
Introduction
Using S3 as a Backend for Front50 (Settings, Suggestions, and Considerations)
Introduction
Using S3 as a Backend for Front50 (Settings, Suggestions, and Considerations)
Introduction
Using the Kubernetes V2 Patch Manifest Stage
In this video, Ethan Rogers demonstrates the Kubernetes V2 Patch Manifest stage using a practical example. By using Spinnaker to empower engineers to perform operational tasks, like putting applications into maintenance mode, operations teams allow engineering teams to move faster.
Using Travis CI with Spinnaker
Question:
Utilizing Kustomize of remote base in a different Git Repo
Introduction
Validator *validate.halValidator detected a fatal error
Issue
Viewing Pipelines as JSON and Developing Dinghy JSON Code
Introduction
VPCs and subnets not populating in the drop down menu in the UI
Issue
Vulnerability Management Policy
This document was derived directly from the official Armory Vulnerability Management Policy and Vulnerability Standard Operating procedure. A more recent version may exist. Contact Armory for the latest official version.
WaitForClusterShrinkTask of stage shrinkCluster times out after 30 minutes
Issue
What Spinnaker Services can be Affected Apply OPA Policy
Introduction
What to Consider When Using Terraformer with the Provisioner, Local-Exec
Introduction
When and why to enable the High Availability (HA) mode
Introduction
When deploying ECS, Subnets and VPC will not populate in the UI
Issue
When using aws-iam-authenticator, fail to read namespaces and failing to deploy to k8s clusters
Issue
Where Does Spinnaker Begin and Jenkins End
Question:
While Disabling Policy Engine, Orca Continually Advises that it Cannot Communicate with OPA Server
Issue
While using Armory ARM-CLI, Cannot Refer to PipelineID
Issue
Working with Dinghy Templates
Working with Dinghy TemplatesDinghy is powerful tool, but to enable its full potential we at Armory created a series of modules and Dinghy files for you to ramp up on Dinghy.This are hosted on Github and are public, any comments, pr with fixes, issues against it is welcomed.https//github.com/armory/dinghyTemplatesTLDR;Fork the repos.Create a Webhook for both repositories pointing to