Category: ADO

Azure Key Vault Integration with Azure Pipelines

This document assumes:

Azure CLI is installed (https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux)

You already have an agent pool with online agent in a deployment pool

And, finally, that you have a pipeline deployment that uses a static keystore. We will be replacing that static keystore file with one obtained from the Azure Key Vault.

First, ensure the Azure DevOps service connection used by the pipeline has access to LJRVenafiTestKeyVault with at least:

  • Secrets: Get, List

From the Azure command line, e.g.

az role assignment create –assignee-object-id 107d2d9a-4d1b-4d8b-9cd6-0f95587eb9ae –assignee-principal-type ServicePrincipal –role “Key Vault Secrets User” –scope “/subscriptions/dede429d-a340-4e90-8f76-05aa5280a1f5/resourceGroups/ljr-keyvault-demo/providers/Microsoft.KeyVault/vaults/LJRVenafiTestKeyVault”

If you do not know which service connection is being used, update and run the pipeline. It will fail with a permission error, but the service connection’s usage history will reflect the release pipeline’s use:

Update your pipeline to retrieve the certificate from the Azure KeyVault. Add an Azure CLI task using an inline script

set -euo pipefail

PFX_FILE=”$AGENT_TEMPDIRECTORY/VenafiDeployedCertificate.pfx”

az keyvault secret download \

–vault-name LJRVenafiTestKeyVault \

–name VenafiDeployedCertificate \

–file “$PFX_FILE” \

–encoding base64

echo “Downloaded PFX to $PFX_FILE”

echo “##vso[task.setvariable variable=PFX_PATH]$PFX_FILE”

If you need a JKS file, add an additional bash task with an inline script

set -euo pipefail

JKS_FILE=”$AGENT_TEMPDIRECTORY/VenafiDeployedCertificate.jks”

# Verify keytool exists

command -v keytool >/dev/null 2>&1 || { echo “keytool not found on agent”; exit 1; }

keytool -importkeystore \

-srckeystore “$(PFX_PATH)” \

-srcstoretype PKCS12 \

-srcstorepass “” \

-destkeystore “$JKS_FILE” \

-deststoretype JKS \

-deststorepass “$(JksPassword)” \

-destkeypass “$(JksPassword)” \

-noprompt

echo “Created JKS at $JKS_FILE”

echo “##vso[task.setvariable variable=JKS_PATH]$JKS_FILE”

Add a pipeline variable for the JKS Password – make sure to click the lock icon to protect the password

And, finally, add a bash task task to copy the JKS or PFX file to the proper place on the server

set -euo pipefail

# Copy JKS to location on server used in app config

TARGET_DIR=”/opt/credential-injection/certs”

TARGET_JKS=”$TARGET_DIR/VenafiDeployedCertificate.jks”

cp “$(JKS_PATH)” “$TARGET_JKS”

chmod 600 “$TARGET_JKS”

echo “JKS copied to $TARGET_JKS”

# Or copy pfx to location on server used in app config

TARGET_PFX=”$TARGET_DIR/VenafiDeployedCertificate.pfx”

cp “$(PFX_PATH)” “$TARGET_PFX”

chmod 600 “$TARGET_PFX”

Create a release to run the pipeline. Looking at the logs, you should see a confirmation that the pfx file was created

And, if you are creating a JKS file, a confirmation that it was created as well

You should also see the certificate file(s) on the server:

 

Chocolate Chip Cookies with Dark Cherries and Almond Flour

Anya made me birthday cookies! I wanted to save the recipe because they turned out really well.

 
Ingredients:
  • 1 3/4 cups all-purpose flour
  • 1/2 cup almond flour
  • 1 teaspoon baking soda
  • 1/2 teaspoon salt
  • 1 cup unsalted butter, softened
  • 1 cup maple syrup
  • 1/2 cup plain Greek yogurt
  • 2 large eggs
  • 2 cups semi-sweet chocolate chips
  • 1 cup frozen dark cherries, roughly chopped
Instructions:
  1. Preheat your oven to 375°F. Line baking sheets with parchment paper.
  2. In a medium bowl, whisk together the all-purpose flour, almond flour, baking soda, and salt. Set aside.
  3. In a large mixing bowl, beat the softened butter with the maple syrup until well mixed. The mixture will be looser than a typical creamed butter-sugar mixture due to the syrup.
  4. Mix in the Greek yogurt until smooth. Beat in the eggs one at a time, mixing well after each addition.
  5. Gradually add the dry ingredients to the wet ingredients, mixing until just combined. The dough will be a bit softer due to the syrup and yogurt.
  6. Gently fold in the chocolate chips and frozen dark cherries until evenly distributed throughout the dough.
  7. Drop rounded tablespoons of dough onto the prepared baking sheets, leaving about 2 inches between each cookie to allow for spreading.
  8. Bake the cookies in the preheated oven for 10-12 minutes, or until the edges are golden brown and the centers are set but still soft. If you made really big cookies, this may be more like 20 minutes!
  9. Allow the cookies to cool on the baking sheet for about 5 minutes before transferring them to a wire rack to cool completely.

Azure DevOps Pipeline Error – Veracode Scan Fails

Pipeline Error:

Build Failed: Error: Exiting Veracode Upload and Scan Task: App not in state where new builds are allowed.

Resolution: There’s a scan in Veracode that never completed. Log into the web UI and delete it!

View the scans in the sandbox. Select the one that says “Request Incomplete”

Use the ellipsis button to select “Delete Request”

Confirm deletion.

Voila – now you can re-run the pipeline and the scan will proceed.

Using Templates in Azure Build Pipelines

I inherited a Java application that is actually five applications — and the build pipeline had a lot of repetition. Tell maven to use this POM file, now use that one, and now the other one. It wasn’t great, but it got even more cumbersome when I needed to split the production and development builds to use a different pool (network rule: prod and dev servers may not communicate … so the dev agent talks to the dev image repo which is used by the dev deployment. The prod agent talks to the prod image repo which is used by the prod deployment). Instead of having five “hey, maven, do this build” blocks, I now have ten.

So I created a template for the build step — jdk-path and maven-path are pipeline variables. The rest is the Maven build task with parameters to supply the step display name, pom file to use, and environment flag.

Maven Build Template:

# maven-build-step.yml
parameters:
  - name: pomFile
    type: string
  - name: dockerEnv
    type: string
  - name: displayName
    type: string

steps:
  - task: Maven@3
    displayName: '${{ parameters.displayName }}'
    inputs:
      mavenPomFile: '${{ parameters.pomFile }}'
      mavenOptions: '-Xmx3072m'
      javaHomeOption: 'Path'
      jdkDirectory: $(jdk-path)
      mavenVersionOption: 'Path'
      mavenDirectory: $(maven-path)
      mavenSetM2Home: true
      jdkArchitectureOption: 'x64'
      publishJUnitResults: true
      testResultsFiles: '**/surefire-reports/TEST-*.xml'
      goals: 'package -Denv=${{ parameters.dockerEnv }} jib:build'

Then my build pipeline uses the template and supplies a few parameters

Pipeline:

# azure-pipelines.yml
trigger: none

variables:
  appName: 'NPM'

stages:
  - stage: Build
    jobs:
      - job: NonProdBuild
        condition: ne(variables['Build.SourceBranchName'], 'production')
        displayName: 'Build non-production branch'
        variables:
          DockerFlag: 'docker_dev'
        pool:
          name: 'Engineering NPM'
        steps:
          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/KafkaStreamsApp/npm/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Kafka Streams App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/DataSync/npmInfo/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Data Sync App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/GroupingRules/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Grouping Rules App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/Errorhandler/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Error Handler App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/Events/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Events App'

      - job: ProdBuild
        condition: eq(variables['Build.SourceBranchName'], 'production')
        displayName: 'Build production branch'
        variables:
          DockerFlag: 'docker_prod'
        pool:
          name: 'Engineering NPM Prod'
        steps:
          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/KafkaStreamsApp/npm/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Kafka Streams App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/DataSync/npmInfo/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Data Sync App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/GroupingRules/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Grouping Rules App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/Errorhandler/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Error Handler App'

          - template: maven-build-template.yml
            parameters:
              pomFile: 'JAVA/Events/pom.xml'
              dockerEnv: $(DockerFlag)
              displayName: 'Building Events App'

I think this could be made more concise … but it will do for now!

Azure DevOps Maven Feed — Deleted Package

Someone deleted one of the packages from the Azure DevOps Maven feed … figured it would be easy enough to just re-publish the package. And they got an error:

409 Conflict – The version 1.2.3 of package.example.com has been deleted. It cannot be restored or pushed. (DevOps Activity ID: E7E4DEB1551D) -> [Help 1]

There’s some not-outlandish logic behind it because they don’t want half of the people to have this version 1.2.3 and the other half to get that version 1.2.3 … if it’s your code, just make it version 1.2.4. Unfortunately, this logic doesn’t hold up well when you’re publishing someone else’s package. Not like I can say “oops, we’ll use 23.13 now”. But you can restore deleted packages — from the feed, go into the recycle bin

Check off the packages that were deleted in error & restore them

 

Creating an Azure DevOps Work Item From a Teams Message

You can use Power Automate to create an ADO work item (bug, user story, etc) when a user posts into specific Teams channel.

Log into Power Automate and create a new workflow. Find a Teams trigger that suits your need – in my case, I wanted to use a key word (you could even use different key words to create work items in different projects or with different content). Note that automation cycles accrue based on execution — so if you elect to link up to a busy Teams channel and filter for keywords to indicate you want an ADO item created, you may be “wasting” workflow cycles. In our case, I have a “user group” Teams space and set up a special channel where users can submit bug and feature requests. This means workflow cycles are accrued when someone specifically wishes to create an ADO item not when messages are posted into the user group’s general chat channels.

You can source messages from channels or group chats in the “Message type” selection. You cannot use hash-tags as key words! The workflow execution reports a gateway error. Select the Team and channel(s) that you want the workflow to monitor.

Add a new step to “create a work item” from Azure DevOps

Configure the project into which you want to create the work item – the organization and project name, the type of work item, and content of the work item.

If you want to set priority, add an assignment, etc – click on “Show advanced options”. I added a few fields to provide a clue as to where the bug report came from.

Save the workflow and post a message in your channel with the key word. Go into the ADO project work items; your Teams-initiated bug should be there.

 

Azure DevOps – Changing Work Item Type

I had to reorganize a lot of my work items in a way that required items not to be what they were. Fortunately, there’s a mechanism to change work item type. Within the work item, click on the ellipsis to access a menu of options. Select “Change type …”

Select the item type you want – I record the reason I needed the new type for posterity – then click “OK”. Save the work item and re-open it.

The one thing I’ve noticed is that fields that don’t exist on an item type (e.g. “effort” on “feature” items) are still present on the new item type even when that field does not normally display (e.g. “effort” on “user story” items).

 

Azure DevOps – Features, User Stories, and Story Points

I had wanted to classify my ADO work items as “features” (i.e. something someone asked to be added to an application), “bugs” (i.e. some intended functionality that was not working as designed). Bugs have a story point field, but features do not appear to have their own story point field. They, instead, are a roll-up of the story points of their subordinate user stories. Which makes sense except that I’ve now got to have two work items for every feature. Rolling up larger requests into sprint-sized work units is how we use epics. So I’ve instead found myself with user stories tagged with “features” that fall into epics (or don’t in the case of a small feature request).

 

Baked French Toast Casserole with Maple Syrup

Ingredients

  • 1 loaf French bread (~16 ounces)
  • 8 large eggs
  • 1.5 cups heavy whipping cream
  • 1.5 cups milk
  • 2 tablespoons maple syrup
  • 1 teaspoon vanilla extract
  • 1/4 teaspoon ground cinnamon
  • 1/4 teaspoon ground nutmeg
  • Dash salt

Directions

Slice bread into 20 slices, 1-inch each. Arrange slices in a generously buttered 9 by 13-inch baking dish in 2 rows, overlapping the slices. In a large bowl, combine the eggs, half-and-half, milk, sugar, vanilla, cinnamon, nutmeg and salt and beat with a rotary beater or whisk until blended but not too bubbly. Pour mixture over the bread slices, making sure all are covered evenly with the milk-egg mixture. Spoon some of the mixture in between the slices. Cover with foil and refrigerate overnight.

The next day, preheat oven to 350 degrees F.

Bake for 40 minutes, until puffed and lightly golden. Serve with maple syrup, fresh fruit, and fresh whipped cream.