Category: System Administration

Using polkit to allow non-priv user to restart service

As I work through automating certificate installation, most applications have a “service account” user that has write access to the SSL certificate files. However, that user does not generally have permission to restart the application service.

We could get the ID added to sudoers with specific rights to manage the service … but it seemed more straightforward to use Polkit for very granular control permitting the service account to run specific verbs with systemctl.

The following rule allows the “tomcatadmin” user to run systemctl start, stop, or restart with the apache-tomcat.service unit.

cat > /etc/polkit-1/rules.d/60-apache-tomcat-tomcatadmin.rules <<'EOF'
polkit.addRule(function(action, subject) {
    if (action.id == "org.freedesktop.systemd1.manage-units") {
        var unit = action.lookup("unit");
        var verb = action.lookup("verb");

        if (subject.user == "tomcatadmin" &&
            unit == "apache-tomcat.service" &&
            (verb == "start" || verb == "stop" || verb == "restart")) {
            return polkit.Result.YES;
        }
    }
});
EOF

Vexing RDPSign Issue

With recent Windows updates, users now get a big message saying “Caution: Unknown remote connection” when launching RDP sessions from our CyberArk server. Easy enough – I have an internal CA, I can generate a code signing certificate, so I can sign these RDP files.

Except, in testing, I continually got an error indicating rdpsign cannot find the certificate. It’s there. I have a private key. It’s a code signing certificate. An hour or so later, I realize the “sha256” value is actually the SHA-1 thumbprint. Which … not my first guess and really more of a “out of reasonable options, start trying silly things” guess.

“$env:SystemRoot\System32\rdpsign.exe” /v /sha256 $hash256 $rdp

Voila, “All rdp file(s) have been successfully signed.”

Sigh — and, after all this work? I go from the red “unknown publisher” error to a yellow “yeah, you should think about this” banner.

Signing PowerShell Scripts

A quick PowerShell script to report on its own signature data:

$scriptPath = $PSCommandPath

if (-not $scriptPath) {
    throw 'This script must be run from a .ps1 file so $PSCommandPath is available.'
}

$sig = Get-AuthenticodeSignature -FilePath $scriptPath

Write-Host "Script path: $scriptPath`n" -ForegroundColor Cyan

[PSCustomObject]@{
    Status                  = $sig.Status
    StatusMessage           = $sig.StatusMessage
    SignatureType           = $sig.SignatureType
    IsOSBinary              = $sig.IsOSBinary
    SignerSubject           = $sig.SignerCertificate.Subject
    SignerThumbprint        = $sig.SignerCertificate.Thumbprint
    SignerNotBefore         = $sig.SignerCertificate.NotBefore
    SignerNotAfter          = $sig.SignerCertificate.NotAfter
    TimeStamperSubject      = $sig.TimeStamperCertificate.Subject
    TimeStamperThumbprint   = $sig.TimeStamperCertificate.Thumbprint
} | Format-List

To sign the script:

$thumb = '87E4C1F40D1DB8486F1E9093A76626AB1DFDEA30'
$scriptPath = "$env:USERPROFILE\git\CyberSecurity\misc\CheckPSSignature.ps1"

$cert = Get-ChildItem Cert:\CurrentUser\My, Cert:\LocalMachine\My |
    Where-Object {
        $_.Thumbprint -eq $thumb -and
        $_.HasPrivateKey -and
        ($_.EnhancedKeyUsageList | Where-Object {
            $_.ObjectId -eq '1.3.6.1.5.5.7.3.3' -or $_.FriendlyName -eq 'Code Signing'
        })
    } |
    Select-Object -First 1

if (-not $cert) {
    throw "Code signing certificate $thumb not found."
}

Set-AuthenticodeSignature -FilePath $scriptPath -Certificate $cert
Get-AuthenticodeSignature -FilePath $scriptPath | Format-List *

And now the script is signed:

Venafi Trust Protect and Azure Key Vault Integration

Entra App Registration

Add a new Entra App registration for Venafi

There is no redirect URI needed for this registration

In this example, my App ID is 05151153-f5d5-4ce8-94cb-9086d70d3c05

On app, go to certificates & secrets. Upload PUBLIC key of a Digital Signature certificate.

Confirm the public key has been added

Key Vault Configuration

In the Azure Portal, navigate to the Key Vaults

Confirm you have an appropriate key vault, or create one. In this example, I am creating a new key vault.

Select the subscription and resource group to be used, provide a name for the vault.

In this example, my key vault is LJRVenafiTestKeyVault

This vault uses RBAC access. Click on “Access control (IAM)” to add rights for the Entra app to use this key vault

Select “Add role assignment”

Select the “Key Vault Certificates Officer” role

Add the application name to this role:

Venafi Configuration

Now, in Venafi, we can add an Azure Key Vault installation to a certificate.

First, we need to create a new certificate type credential to hold the private key for the certificate used in the app registration

Upload the certificate pfx file and supply the pfx password

Navigate to the certificate you want published into the Azure Key Vault. From the “Actions” menu, select “Add Installation”

Select “Track, validate, and automate installation of this certificate”

Select a device and chose the “Azure Key Vault” installation type:

The first half of the form does not need to be changed, although you can add a description explaining what the deployment is.

Select the device credential for the host. The “Application ID” is the Azure App ID from the registered application. The Certificate Credential is the Digital Signature private key uploaded for application authentication.

The Azure Key Vault Name is the name of the key vault created in Azure, and Certificate Name is the “friendly” name to be used in the certificate file deployed to the server. This often needs to be included in the application configuration (use this keystore file and use this certificate from the keystore). Because I am using this key in a release pipeline, I do not want to bind the certificate to a web service

The cert will be queued for installation into the Azure Key Vault

Once the installation has completed, return to the Azure Portal to confirm that the certificate is now present in the key vault.

Using the Key in a Pipeline

This document assumes:

Azure CLI is installed (https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux)

You already have an agent pool with online agent in a deployment pool

And, finally, that you have a pipeline deployment that uses a static keystore. We will be replacing that static keystore file with one obtained from the Azure Key Vault.

First, ensure the Azure DevOps service connection used by the pipeline has access to LJRVenafiTestKeyVault with at least:

  • Secrets: Get, List

From the Azure command line, e.g.

az role assignment create –assignee-object-id 107d2d9a-4d1b-4d8b-9cd6-0f95587eb9ae –assignee-principal-type ServicePrincipal –role “Key Vault Secrets User” –scope “/subscriptions/dede429d-a340-4e90-8f76-05aa5280a1f5/resourceGroups/ljr-keyvault-demo/providers/Microsoft.KeyVault/vaults/LJRVenafiTestKeyVault”

If you do not know which service connection is being used, update and run the pipeline. It will fail with a permission error, but the service connection’s usage history will reflect the release pipeline’s use:

Update your pipeline to retrieve the certificate from the Azure KeyVault. Add an Azure CLI task using an inline script

set -euo pipefail

PFX_FILE=”$AGENT_TEMPDIRECTORY/VenafiDeployedCertificate.pfx”

az keyvault secret download \

–vault-name LJRVenafiTestKeyVault \

–name VenafiDeployedCertificate \

–file “$PFX_FILE” \

–encoding base64

echo “Downloaded PFX to $PFX_FILE”

echo “##vso[task.setvariable variable=PFX_PATH]$PFX_FILE”

If you need a JKS file, add an additional bash task with an inline script

set -euo pipefail

JKS_FILE=”$AGENT_TEMPDIRECTORY/VenafiDeployedCertificate.jks”

# Verify keytool exists

command -v keytool >/dev/null 2>&1 || { echo “keytool not found on agent”; exit 1; }

keytool -importkeystore \

-srckeystore “$(PFX_PATH)” \

-srcstoretype PKCS12 \

-srcstorepass “” \

-destkeystore “$JKS_FILE” \

-deststoretype JKS \

-deststorepass “$(JksPassword)” \

-destkeypass “$(JksPassword)” \

-noprompt

echo “Created JKS at $JKS_FILE”

echo “##vso[task.setvariable variable=JKS_PATH]$JKS_FILE”

Add a pipeline variable for the JKS Password – make sure to click the lock icon to protect the password

And, finally, add a bash task task to copy the JKS or PFX file to the proper place on the server

set -euo pipefail

# Copy JKS to location on server used in app config

TARGET_DIR=”/opt/credential-injection/certs”

TARGET_JKS=”$TARGET_DIR/VenafiDeployedCertificate.jks”

cp “$(JKS_PATH)” “$TARGET_JKS”

chmod 600 “$TARGET_JKS”

echo “JKS copied to $TARGET_JKS”

# Or copy pfx to location on server used in app config

TARGET_PFX=”$TARGET_DIR/VenafiDeployedCertificate.pfx”

cp “$(PFX_PATH)” “$TARGET_PFX”

chmod 600 “$TARGET_PFX”

Create a release to run the pipeline. Looking at the logs, you should see a confirmation that the pfx file was created

And, if you are creating a JKS file, a confirmation that it was created as well

You should also see the certificate file(s) on the server:

 

Venafi TPP Installation Driver Ports

While Venafi documentation for each individual installation driver includes the port requirements, there was no single table view of the installation options and what port(s) are needed for that installation driver to work. Ended up putting together a table for our firewall rule discussion.

Installation TypeDescriptionDefault PortNotes
Adaptable Installation DriverPowershell22, 5985, 5986This is running a custom script
Amazon Web Services443To internet destination
Apache HTTPUses ssh22
Azure Key Vault443To internet destination
Citrix NetScalerNITRO API443
F5 BigIPiControl REST API443
Google Cloud Load Balancer443To internet destination
IBM DataPowerDataPower Gateway5550, 5554
IBM Keystore
JKSUses ssh22Tomcat, Jboss, WebLogic
Oracle iPlanet Web ServerUses ssh22
PEMUses ssh22
PKCS#12Uses ssh22
Windows CAPI & IISUses WinRM5985, 5986

Setting Windows Dynamic Port Range

In case anyone else ever needs to set a windows dynamic port range for magic RPC “stuff” — there’s a minimum range size of 255. If you make the range to small, you get an incredibly vague and not-useful “the parameter is incorrect” error. Increase num to at least the min value, and you don’t be going in circles trying to figure out what in your command doesn’t match the parameters in the documentation!

 

Venafi Cert Issuance Fails after Windows 2022 Upgrade

Certificate Issuance Fails

After requesting a certificate, the request immediately fails with the error:

Failed to post CSR with error: Unknown certificate profile type.

I think it is just a coincidence, but wanted to document the scenario in case it comes up again. The application makes web calls to a vendor API to issue certs. The API calls, after the upgrade, were failing.

In this scenario, a call was being made to {base_url}/api/ssl/v1/types, the connection failed. Since the list of valid certificate profiles could not be retrieved, the request failed saying the certificate profile was unknown.

GET https://hard.cert-manager.com/api/ssl/v1/types?organizationId=####

Looking at a debug trace, the following flow was observed:

  • Authentication headers sent: login=<REDACTED>, password=<REDACTED>, customerUri=<REDACTED>
  • Transport-level failure (no HTTP status returned on the failing attempt)
    • Symptoms: “Decrypt failed with error 0X90317” followed by “The underlying connection was closed: The connection was closed unexpectedly.”
    • Context: Revocation checks reported “revocation server was offline,” then the client proceeded; long idle/keep-alive reuse likely contributed to the close.

 

Connection reuse vs server keep-alive: Apache is advertising Keep-Alive: timeout=3. The .NET client is reusing long-idle TLS connections via the proxy; by the time it sends application data, the server/proxy has already closed the session, leading to “underlying connection was closed” errors.

Revocation checks through the proxy: The .NET trace shows “revocation server was offline” before proceeding. That extra handshake work plus proxy blocking CRL/OCSP can increase latency and contribute to idle reuse issues.

.NET SChannel quirks: Older HttpWebRequest/ServicePoint behaviors (Expect100-Continue, connection pooling) can interact poorly with short keep-alive servers/proxies.

Luckily, this is a .NET application, and you can create custom configuration files for .NET apps. In the file with the binary, look for a text file named BinaryName.exe.config

If none exists, create one. The following disables the proxy:

<?xml version=”1.0″ encoding=”utf-8″?>
<configuration>
<system.net>
<!– Turn off use of the system proxy for this app –>
<defaultProxy enabled=”true”>
<proxy usesystemdefault=”false” />
</defaultProxy>
</system.net>
</configuration>

 

Client Connections to HTTPS IIS Site Fail After Upgrade to Windows Server 2022

Client connections to the HTTPS IIS site failed with the following error:

Secure Connection Failed

An error occurred during a connection to webhost.example.com.

PR_CONNECT_RESET_ERROR

Error code: PR_CONNECT_RESET_ERROR

The page you are trying to view cannot be shown because the authenticity of the received data could not be verified. Please contact the website owners to inform them of this problem.

 

The IIS site was set to “accept” client certificates.

  • Client Certificates = Accept means IIS/HTTP.sys will try to retrieve a client certificate only if the app touches Request.ClientCertificate (or a module that maps/validates client certs). That retrieval is done via TLS renegotiation in TLS 1.2.
  • On Server 2022, browsers prefer TLS 1.3. TLS 1.3 does not support the old renegotiation used to fetch a client cert mid‑request. When your app/module at “/” accesses the client cert, IIS attempts renegotiation, fails, and the connection is reset.

Setting Client Certificates to “Ignore” in the site’s “SSL Settings” prevents IIS from attempting to renegotiate, so the site loads. This obviously isn’t a solution if you want to use client certificates to authenticate … but we’re authenticating through Ping, so don’t actually need the client certs.

Apache OIDC Authentication to PingFederate (or PingID) Using OIDC

This is kind of a silly update to my attempt to document using mod_auth_openidc in Apache. At the time, I didn’t know who set up the PingFederate side of the connection, so I just used Google as the authentication provider. Five years later, I am one of the people setting up the connections and can finally finish the other side. So here is an update — now using PingFederate as the OIDC/OAUTH provider.

OAUTH Client Setup – Apache

First, make sure mod_auth_openidc is installed

In your Apache config, you can add authentication to the entire site or just specific paths under the site. In this example, we are creating an authenticated sub-directory at /authtest

In the virtual host, I am adding an alias for the protected path as /authtest, configuring the directory, and configuring the location to require valid-user using openid-connect. I am then configuring the OIDC connection.

The OIDCClientID and OIDCClientSecret will be provided to you after the connection is set up in PingID. Just put placeholders in until the real values are known.

The OIDCRedirectURI needed to be a path under the protected directory for me – the Apache module handles the callback. Provide this path on the OIDC connection request.

The OIDCCryptoPassphrase just needs to be a long pseudo-random string. It can include special characters.

# Serve /authtest from local filesystem
Alias /authtest "/var/www/vhtml/sandbox/authtest/"

<Directory "/var/www/vhtml/sandbox/authtest">
Options -Indexes +FollowSymLinks
AllowOverride None
Require all granted
</Directory>

# mod_auth_openidc configuration for Ping (PingFederate/PingID)
# The firewall will need to be configured to allow web server to communicate with this host
OIDCProviderMetadataURL https://authpoint.example.com/.well-known/openid-configuration

# The ID and secret will be provided to you
OIDCClientID d5d53555-7525-4555-a565-b525c59545d5
OIDCClientSecret p78…Q2kxB

# Redirect/callback URI – provide this in the request form for the callback URL
OIDCRedirectURI https://www.rushworth.us/authtest/callback

# Session/cookie settings – you make up the OIDCCryptoPassphrase
OIDCCryptoPassphrase "…T9y"
OIDCCookiePath /authtest
OIDCSessionInactivityTimeout 3600
OIDCSessionMaxDuration 28800

# Scopes and client auth
OIDCScope "openid profile email"
OIDCRemoteUserClaim preferred_username
OIDCProviderTokenEndpointAuth client_secret_basic

# If Ping's TLS cert at https://localhost:9031 isn't trusted by the OS CA store,
# install the proper CA chain, or temporarily disable validation (not recommended long-term):
# OIDCSSLValidateServer Off

# Protect the URL path with OIDC
<Location /authtest>
AuthType openid-connect
Require valid-user
OIDCUnAuthAction auth
</Location>

Sample web code for the “protected” page if you want to use the user’s ID. The user’s email is found at $_server[‘OIDC_CLAIM_email’]

[lisa@fedora conf.d]# cat /var/www/vhtml/sandbox/authtest/index.php
<?php
     if( isset($_SERVER['OIDC_CLAIM_iss']) && $_SERVER['OIDC_CLAIM_iss'] == "https://authpoint.example.com"){
          echo "I trust you are " . $_SERVER['OIDC_CLAIM_username'] . "\n";
     }

else{
     print "Not authenticated ... \n";
     print "<UL>\n";
     foreach($_SERVER as $key_name => $key_value) {
          print "<LI>" . $key_name . " = " . $key_value . "\n";
     }
     
     print "</UL>\n";
}
?>

Results on the web page – user will be directed to PingID to authenticate, and you will verify that the auth point has authenticated them as the OIDC_CLAIM_username value.

OAUTH Client Setup – PingID

Client auth, add redirect URLs