Category: Technology

Fedora 43 to 44 upgrade: dnf5 plugin load failure after upgrade

After completing an in-place upgrade from Fedora 43 to Fedora 44, dnf5 failed to run with this error:

[lisa@fedora05 ~]# dnf5 update
Cannot load dnf5 plugin: /usr/lib64/dnf5/plugins/automatic_cmd_plugin.so
Cannot load shared library “/usr/lib64/dnf5/plugins/automatic_cmd_plugin.so”: libdnf5-cli.so.2: cannot open shared object file: No such file or directory

What happened

The Fedora 44 upgrade completed, and the installed dnf5 packages were all current Fedora 44 versions. However, there was a leftover plugin file still sitting in /usr/lib64/dnf5/plugins/automatic_cmd_plugin.so.

That file was not owned by any RPM package and had been built against an older library, libdnf5-cli.so.2.

But Fedora 44 had /usr/lib64/libdnf5-cli.so.3.

dnf5 was trying to load a stale plugin from before the upgrade.

How I verified it

These commands showed the problem:

ls -l /usr/lib64/libdnf5-cli.so*
rpm -qf /usr/lib64/dnf5/plugins/automatic_cmd_plugin.so
ldd /usr/lib64/dnf5/plugins/automatic_cmd_plugin.so

Results:

  • libdnf5-cli.so.3 existed
  • automatic_cmd_plugin.so was not owned by any package
  • ldd showed it was looking for libdnf5-cli.so.2

Fix

Remove the orphaned plugin file:

rm /usr/lib64/dnf5/plugins/automatic_cmd_plugin.so
ldconfig
dnf5 update

After deleting the stale plugin, dnf5 worked normally again.

Root cause

This appears to be a leftover orphaned dnf5 plugin from before the major version upgrade. Even though the main dnf5 and libdnf5 packages were updated correctly, dnf5 still tried to load the old .so file it found in the plugins directory.

Kafbat – OAUTH With RBAC

I encountered a challenge with a Kafka management tool — it supports SSO, and I was able to get an OAUTH connection set up to control what users could see when logging in through the UI, but the API component didn’t extract information from the bearer token and there was nothing in the rbac mapping to allow the bearer-token client ID to access anything.

Updates to allow the /api components to be authenticated by simple bearer tokens and a client ID mapped into a role are at https://github.com/ljr55555/kafka-ui

AuthorizationController.java was updated to properly support non-browser, machine-principal auth.

Added/fixed:

  • avoids null failure when authentication.getName() is missing
  • resolves principal name from alternate attributes such as:
    • client_id
    • sub
    • username
  • updated displayed permissions logic so /api/authorization uses the same effective RBAC matching idea as the backend

Result

/api/authorization now works for bearer-token API callers and shows:

  • username = client ID
  • populated permissions list

AccessControlService.java

Added support for API bearer-token principals

Previously, getUser() only worked when the authenticated principal was a RbacUser, which covered the browser/user flow.

Now it can also derive an AuthenticatedUser from opaque-token authenticated principals by extracting:

  • principal name
  • group-like values from attributes/authorities if present

Updated role matching logic

Previously, role matching was only role name matches one of user.groups(). Now it also supports role name matches user.principal(). That enables RBAC binding directly to the API client ID.

Result

RBAC now works for:

  • normal browser users via groups
  • API bearer-token callers via client principal name

DynamicConfigMapper.java

Fixed a mapper bug.

Before

The method mapping resource server config created a populated OAuth2ResourceServerProperties result object but always returned null.

After

It now returns result.

Result

Dynamic/config mapping for resource-server settings no longer silently discards the mapped object.

Build/package note

To preserve the browser UI, the jar needs to be built with frontend included — which you know if you read the doc … or you take my route, start it all up, test the API successfully, and then get baffled that the user UI throws 


        bash./gradlew clean assemble -Pinclude-frontend=true

application.yml

server:
  port: 8443
  ssl:
    enabled: true
    key-store: file:/etc/kafkaui/certs/kafbat.rushworth.us.p12
    key-store-password: ${KEYSTORE_PASSWORD}
    key-store-type: PKCS12
    key-alias: kafbat

auth:
  type: OAUTH2
  oauth2:
    client:
      pingfed:
        client-id: ${OAUTH_CLIENT_ID}
        client-secret: ${OAUTH_CLIENT_SECRET}
        scope:
          - openid
          - profile
          - email
        client-name: oauthclient
        provider: oauthclient
        redirect-uri: https://kafbat.rushworth.us:8443/login/oauth2/code/oauthclient
        authorization-grant-type: authorization_code
        issuer-uri: https://login.example.com
        jwk-set-uri: https://login.example.com/pf/JWKS
        authorization-uri: https://login.example.com/as/authorization.oauth2
        token-uri: https://login.example.com/as/token.oauth2
        user-info-uri: https://login.example.com/idp/userinfo.openid
        user-name-attribute: username
        custom-params:
          type: oauth
          roles-field: memberOf

    resource-server:
      opaque-token:
        client-id: ${OAUTH_CLIENT_ID}
        client-secret: ${OAUTH_CLIENT_SECRET}
        introspection-uri: https://login.example.com/as/introspect.oauth2

kafka:
  clusters:
    - name: test
      bootstrapServers: ${KAFKA_BOOTSTRAP_SERVERS}

roles.yml

rbac:
  roles:
    - name: "admins"
      clusters:
        - test
      subjects:
        - provider: oauth
          type: role
          value: "CN=KafbatAdmins,OU=SecurityGroups,DC=example,DC=com"
      permissions:
        - resource: applicationconfig
          actions: all

        - resource: clusterconfig
          actions: all

        - resource: topic
          value: ".*"
          actions: all

        - resource: consumer
          value: ".*"
          actions: all

        - resource: schema
          value: ".*"
          actions: all

        - resource: connect
          value: ".*"
          actions: all

        - resource: ksql
          actions: all

        - resource: acl
          actions: [ view ]

    - name: "${OAUTH_CLIENT_ID}"
      clusters:
        - test
      subjects:
        - provider: oauth
          type: user
          value: "${OAUTH_CLIENT_ID}"
      permissions:
        - resource: applicationconfig
          actions: all

        - resource: clusterconfig
          actions: all

        - resource: topic
          value: ".*"
          actions: all

        - resource: consumer
          value: ".*"
          actions: all

        - resource: schema
          value: ".*"
 
        - resource: connect
          value: ".*"
          actions: all

        - resource: ksql
          actions: all

        - resource: acl
          actions: [ view ]

docker-compose.yml

services:
  redpanda:
    image: redpandadata/redpanda:v25.1.2
    container_name: redpanda
    command:
      - redpanda
      - start
      - --overprovisioned
      - --smp=1
      - --memory=1G
      - --reserve-memory=0M
      - --node-id=0
      - --check=false
      - --kafka-addr=PLAINTEXT://0.0.0.0:9092
      - --advertise-kafka-addr=PLAINTEXT://redpanda:9092
    ports:
      - "9092:9092"

  kafbat-ui:
    image: ghcr.io/kafbat/kafka-ui:latest
    container_name: kafbat-ui
    restart: unless-stopped
    depends_on:
      - redpanda
    ports:
      - "8443:8443"
    volumes:
      - ./config/application.yml:/etc/kafkaui/application.yml:ro
      - ./config/roles.yml:/etc/kafkaui/roles.yml:ro
      - ./certs:/etc/kafkaui/certs:ro
    environment:
      SPRING_CONFIG_LOCATION: file:/etc/kafkaui/application.yml
      SPRING_CONFIG_ADDITIONAL_LOCATION: file:/etc/kafkaui/roles.yml
      KEYSTORE_PASSWORD: REDACTED
      OAUTH_CLIENT_ID: REDACTED
      OAUTH_CLIENT_SECRET: REDACTED
      KAFKA_BOOTSTRAP_SERVERS: redpanda:9092

Docker for Java Builds

Instead of flipping back and forth between java versions for various builds, you can just use a docker container for the proper Java version to run the build. 

[lisa@docker kafka-ui]# docker run --rm -it   --user $(id -u):$(id -g)   -v "$PWD":/workspace   -w /workspace   eclipse-temurin:25   bash -lc './gradlew clean build'
Downloading https://services.gradle.org/distributions/gradle-9.2.0-bin.zip
............10%.............20%.............30%.............40%.............50%.............60%.............70%.............80%.............90%.............100%

Welcome to Gradle 9.2.0!

Here are the highlights of this release:
 - Windows ARM support
 - Improved publishing APIs
 - Better guidance for dependency verification failures

For more details see https://docs.gradle.org/9.2.0/release-notes.html

Starting a Gradle Daemon (subsequent builds will be faster)
<=============> 100% CONFIGURING [1m 46s]
> Resolve dependencies of :api:detachedConfiguration273

Using polkit to allow non-priv user to restart service

As I work through automating certificate installation, most applications have a “service account” user that has write access to the SSL certificate files. However, that user does not generally have permission to restart the application service.

We could get the ID added to sudoers with specific rights to manage the service … but it seemed more straightforward to use Polkit for very granular control permitting the service account to run specific verbs with systemctl.

The following rule allows the “tomcatadmin” user to run systemctl start, stop, or restart with the apache-tomcat.service unit. 

cat > /etc/polkit-1/rules.d/60-apache-tomcat-tomcatadmin.rules <<'EOF'
polkit.addRule(function(action, subject) {
    if (action.id == "org.freedesktop.systemd1.manage-units") {
        var unit = action.lookup("unit");
        var verb = action.lookup("verb");

        if (subject.user == "tomcatadmin" &&
            unit == "apache-tomcat.service" &&
            (verb == "start" || verb == "stop" || verb == "restart")) {
            return polkit.Result.YES;
        }
    }
});
EOF

Vexing RDPSign Issue

With recent Windows updates, users now get a big message saying “Caution: Unknown remote connection” when launching RDP sessions from our CyberArk server. Easy enough – I have an internal CA, I can generate a code signing certificate, so I can sign these RDP files.

Except, in testing, I continually got an error indicating rdpsign cannot find the certificate. It’s there. I have a private key. It’s a code signing certificate. An hour or so later, I realize the “sha256” value is actually the SHA-1 thumbprint. Which … not my first guess and really more of a “out of reasonable options, start trying silly things” guess.

“$env:SystemRoot\System32\rdpsign.exe” /v /sha256 $hash256 $rdp

Voila, “All rdp file(s) have been successfully signed.”

Sigh — and, after all this work? I go from the red “unknown publisher” error to a yellow “yeah, you should think about this” banner.

Blender Script: Distance Between Two Points

We were having a lot of trouble using the measure tool in Blender — after discovering that you can hold the Ctrl key and “snap” your selection to a vertex, it actually worked in the way we wanted it to. But, before that discovery, my thought was to manually select two vertices and use a Python script to measure the distance between those points. That is “blender units” … and the metric or imperial units would be more meaningful. So I converted based on the scene configuration. 

import bpy
import bmesh


def format_metric(meters: float) -> str:
    a = abs(meters)

    if a >= 1.0:
        return f"{meters:.6f} m"
    elif a >= 0.01:
        return f"{meters * 100.0:.3f} cm"
    else:
        return f"{meters * 1000.0:.3f} mm"


def format_imperial(meters: float) -> str:
    total_inches = meters / 0.0254
    sign = "-" if total_inches < 0 else ""
    total_inches = abs(total_inches)

    feet = int(total_inches // 12)
    inches = total_inches - (feet * 12)

    if feet > 0:
        return f"{sign}{feet} ft {inches:.3f} in"
    else:
        return f"{sign}{total_inches:.3f} in"


def measure_selected_vertices():
    ctx = bpy.context
    obj = ctx.edit_object

    if obj is None or obj.type != 'MESH':
        raise RuntimeError("Go into Edit Mode on a mesh and select exactly 2 vertices.")

    bm = bmesh.from_edit_mesh(obj.data)
    selected_verts = [v for v in bm.verts if v.select]

    if len(selected_verts) != 2:
        raise RuntimeError(f"Expected exactly 2 selected vertices, found {len(selected_verts)}.")

    v1, v2 = selected_verts

    # Convert local vertex coordinates to world-space coordinates
    p1 = obj.matrix_world @ v1.co
    p2 = obj.matrix_world @ v2.co

    # Raw world-space distance in Blender units
    raw_distance = (p2 - p1).length

    scene = ctx.scene
    units = scene.unit_settings
    scale_length = units.scale_length if units.scale_length != 0 else 1.0

    # Convert Blender units to meters according to the scene unit scale
    distance_meters = raw_distance * scale_length

    print("\n----- Vertex Distance -----")
    print(f"Object: {obj.name}")
    print(f"Raw distance (Blender units): {raw_distance:.6f}")
    print(f"Scene unit system: {units.system}")
    print(f"Scene unit scale: {scale_length}")

    if units.system == 'METRIC':
        print(f"Formatted distance: {format_metric(distance_meters)}")

    elif units.system == 'IMPERIAL':
        print(f"Formatted distance: {format_imperial(distance_meters)}")

    else:
        print("Formatted distance: Scene unit system is 'NONE'")
        print(f"Interpreted using current unit scale: {format_metric(distance_meters)}")


measure_selected_vertices()

Signing PowerShell Scripts

A quick PowerShell script to report on its own signature data:

$scriptPath = $PSCommandPath

if (-not $scriptPath) {
    throw 'This script must be run from a .ps1 file so $PSCommandPath is available.'
}

$sig = Get-AuthenticodeSignature -FilePath $scriptPath

Write-Host "Script path: $scriptPath`n" -ForegroundColor Cyan

[PSCustomObject]@{
    Status                  = $sig.Status
    StatusMessage           = $sig.StatusMessage
    SignatureType           = $sig.SignatureType
    IsOSBinary              = $sig.IsOSBinary
    SignerSubject           = $sig.SignerCertificate.Subject
    SignerThumbprint        = $sig.SignerCertificate.Thumbprint
    SignerNotBefore         = $sig.SignerCertificate.NotBefore
    SignerNotAfter          = $sig.SignerCertificate.NotAfter
    TimeStamperSubject      = $sig.TimeStamperCertificate.Subject
    TimeStamperThumbprint   = $sig.TimeStamperCertificate.Thumbprint
} | Format-List

To sign the script:

$thumb = '87E4C1F40D1DB8486F1E9093A76626AB1DFDEA30'
$scriptPath = "$env:USERPROFILE\git\CyberSecurity\misc\CheckPSSignature.ps1"

$cert = Get-ChildItem Cert:\CurrentUser\My, Cert:\LocalMachine\My |
    Where-Object {
        $_.Thumbprint -eq $thumb -and
        $_.HasPrivateKey -and
        ($_.EnhancedKeyUsageList | Where-Object {
            $_.ObjectId -eq '1.3.6.1.5.5.7.3.3' -or $_.FriendlyName -eq 'Code Signing'
        })
    } |
    Select-Object -First 1

if (-not $cert) {
    throw "Code signing certificate $thumb not found."
}

Set-AuthenticodeSignature -FilePath $scriptPath -Certificate $cert
Get-AuthenticodeSignature -FilePath $scriptPath | Format-List *

And now the script is signed:

Azure Key Vault Integration with Azure Pipelines

This document assumes:

Azure CLI is installed (https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-linux)

You already have an agent pool with online agent in a deployment pool

And, finally, that you have a pipeline deployment that uses a static keystore. We will be replacing that static keystore file with one obtained from the Azure Key Vault.

First, ensure the Azure DevOps service connection used by the pipeline has access to LJRVenafiTestKeyVault with at least:

  • Secrets: Get, List

From the Azure command line, e.g.

az role assignment create –assignee-object-id 107d2d9a-4d1b-4d8b-9cd6-0f95587eb9ae –assignee-principal-type ServicePrincipal –role “Key Vault Secrets User” –scope “/subscriptions/dede429d-a340-4e90-8f76-05aa5280a1f5/resourceGroups/ljr-keyvault-demo/providers/Microsoft.KeyVault/vaults/LJRVenafiTestKeyVault”

If you do not know which service connection is being used, update and run the pipeline. It will fail with a permission error, but the service connection’s usage history will reflect the release pipeline’s use:

Update your pipeline to retrieve the certificate from the Azure KeyVault. Add an Azure CLI task using an inline script

set -euo pipefail

PFX_FILE=”$AGENT_TEMPDIRECTORY/VenafiDeployedCertificate.pfx”

az keyvault secret download \

–vault-name LJRVenafiTestKeyVault \

–name VenafiDeployedCertificate \

–file “$PFX_FILE” \

–encoding base64

echo “Downloaded PFX to $PFX_FILE”

echo “##vso[task.setvariable variable=PFX_PATH]$PFX_FILE”

If you need a JKS file, add an additional bash task with an inline script

set -euo pipefail

JKS_FILE=”$AGENT_TEMPDIRECTORY/VenafiDeployedCertificate.jks”

# Verify keytool exists

command -v keytool >/dev/null 2>&1 || { echo “keytool not found on agent”; exit 1; }

keytool -importkeystore \

-srckeystore “$(PFX_PATH)” \

-srcstoretype PKCS12 \

-srcstorepass “” \

-destkeystore “$JKS_FILE” \

-deststoretype JKS \

-deststorepass “$(JksPassword)” \

-destkeypass “$(JksPassword)” \

-noprompt

echo “Created JKS at $JKS_FILE”

echo “##vso[task.setvariable variable=JKS_PATH]$JKS_FILE”

Add a pipeline variable for the JKS Password – make sure to click the lock icon to protect the password

And, finally, add a bash task task to copy the JKS or PFX file to the proper place on the server

set -euo pipefail

# Copy JKS to location on server used in app config

TARGET_DIR=”/opt/credential-injection/certs”

TARGET_JKS=”$TARGET_DIR/VenafiDeployedCertificate.jks”

cp “$(JKS_PATH)” “$TARGET_JKS”

chmod 600 “$TARGET_JKS”

echo “JKS copied to $TARGET_JKS”

# Or copy pfx to location on server used in app config

TARGET_PFX=”$TARGET_DIR/VenafiDeployedCertificate.pfx”

cp “$(PFX_PATH)” “$TARGET_PFX”

chmod 600 “$TARGET_PFX”

Create a release to run the pipeline. Looking at the logs, you should see a confirmation that the pfx file was created

And, if you are creating a JKS file, a confirmation that it was created as well

You should also see the certificate file(s) on the server:

 

Time to Move Away from JKS Keystores

For many Java-based applications, Java KeyStore (JKS) has been the default for years. It’s familiar, widely used, and still supported. But “still supported” is not the same as “still the best choice.” If your application or platform supports it, now is a good time to move away from JKS and standardize on PKCS#12.

Why move away from JKS?

1. JKS is a proprietary format

JKS is Java-specific and tied to older Java conventions. By contrast, PKCS#12 is a standards-based format supported across platforms, tools, and vendors.

That difference matters operationally. Certificate and key material increasingly needs to work across Java applications, web servers, load balancers, cloud services, and automation tooling. PKCS#12 is far better suited to that multi-platform reality.

2. JKS has legacy security characteristics

JKS should not be considered a modern format for protecting private keys.

It uses non-standard, legacy protection mechanisms and has historically relied on weaker constructions than modern PKCS#12 implementations. As with any password-protected container, security depends heavily on password strength—but JKS offers less margin for error compared to more modern formats.

This becomes especially relevant if a keystore file is exposed. Offline password cracking is a realistic risk, and widely available tools can target JKS files—particularly when organizations use weak or reused passwords.

This does not mean existing JKS files are inherently compromised. It does mean JKS should no longer be the default when stronger, more widely supported alternatives exist.

3. Java itself moved on

The Java platform has already made this transition. Starting with Java 9, PKCS#12 became the default keystore type. JKS remains supported, but PKCS#12 is now the preferred standard in modern Java environments.

4. Many applications already support PKCS#12

In many environments, JKS persists simply because it’s what teams have always used—not because it’s required. Most modern Java frameworks, application servers, and tools support PKCS#12. For example, Tomcat has supported PKCS#12 since version 5.0, and current Java tooling handles it natively.

In practice, many applications can switch with little to no functional impact.

Why PKCS#12 is the better choice

PKCS#12 offers several clear advantages:

  • Broad interoperability across platforms and vendors
  • Better alignment with modern tooling and certificate automation
  • Reduced reliance on Java-specific legacy formats
  • Default support in current Java versions

What to do

If you manage Java applications or infrastructure, this is a good opportunity to review current keystore usage.

  • Identify where JKS keystores are currently in use
  • Verify whether those applications support PKCS#12
  • Review vendor documentation for any requirements or constraints
  • Update deployment standards to prefer PKCS#12 for new systems
  • Gradually migrate existing JKS-based deployments where practical

For many use cases, converting is straightforward. For example:

keytool -importkeystore \
  -srckeystore keystore.jks \
  -destkeystore keystore.p12 \
  -deststoretype PKCS12

Check vendor guidance

Before making changes, confirm support with the relevant application or platform vendor.

Key questions:

  • Does the application support PKCS#12 for keys and certificates?
  • Are there version-specific considerations?
  • Are configuration changes required?
  • Does the vendor recommend PKCS#12 for current deployments?

In most cases, the answer will be yes—but it’s still worth validating before making production changes.

Bottom line

JKS is not deprecated, but it is no longer the format organizations should be choosing by default.

It is a legacy, Java-specific keystore format with limited interoperability and older security characteristics. Meanwhile, PKCS#12 is standards-based, broadly supported, and the default in modern Java. If your application supports PKCS#12, prefer it. If you’re unsure, check—because in many cases, you can make the switch with minimal effort.

Choosing a modern keystore format is a small change that can meaningfully improve both security posture and operational flexibility.