Category: System Administration

April 8, 2018

Running Sendmail In A CHROOT Jail

My employer’s OS-support model restricts root access to members of the Unix support team. Applications are normally installed into a package directory and run under a service ID. While this model works well for most applications, sendmail is tightly integrated into the OS and is not readily built into an application directory. We attempted to run sendmail as a non-root user with modified permissions on application directories such as /var/spool/mqueue – this worked, until OS patches were applied and permissions reset. We needed a way to run sendmail as a non-root user and allow the OS support team to patch servers without impacting the sendmail application.

Chroot is a mechanism that uses a supplied directory path as the environment’s root directory. The jailed process, and its children, should not be able to access any part of the file hierarchy outside of the new root. As a security mechanism, the approach has several flaws – abridged version of the story is that it’s not terribly difficult to break out of jail here; and there are far more effective security approaches (e.g. SELinux). However, chroot jails have their own copies of system owned directories (such as /var/spool/mqueue), binaries, and libraries. Using a chroot jail will allow us to maintain a sendmail application in the package directory that is not impacted by OS updates.

This approach works on relaying mail servers (i.e. those that queue mail to /var/spool/mqueue and send it on its merry way). If sendmail is hosting mailboxes, there are additional challenges to designing a chroot configuration that actually drops messages into mailbox files that users can access.

Preliminaries: To copy/paste, view the single article. Create a service account under which sendmail will run. The installation directory should be owned by the service account user.

Set up the chroot jail location in the installation directory. In this example, that directory is /smt00p20.

mkdir /smt00p20/sendmail
mkdir /smt00p20/sendmail/dev
mkdir /smt00p20/opendkim

We need a null and random in the sendmail jail. On a command line, run:

# Create sendmail jail /dev/null
mknod /smt00p20/sendmail/dev/null c 1 3
# Create sendmail jail /dev/random
mknod /smt00p20/sendmail/dev/random c 1 8

We need an rsyslog socket added under each jail. In /etc/rsyslog.conf, add the following:

# additional log sockets for chroot'ed jail
# Idea from http://www.ispcolohost.com/2014/03/14/how-to-get-syslog-records-of-chrooted-ssh-sftp-server-activity/
$AddUnixListenSocket /smt00p20/sendmail/dev/log
$AddUnixListenSocket /smt00p20/opendkim/dev/log

Additionally, these instructions assume both sendmail and sendmail-cf have been installed on the server. If they have not, you can download the RPMs, unpack them, and copy the files to the appropriate relative jail locations.

Chrooting Sendmail

Logged in with the sendmail ID, ensure you have a .bash_profile that loads .bashrc

-bash-4.2$ cat ~/.bash_profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

Edit ~/.bashrc and add the following, where smt00p20 is the appropriate installation directory, to allow copy/paste

export SENDMAILJAIL=/smt00p20/sendmail
export OPENDKIMJAIL=/smt00p20/opendkim

Log out of the service account and back in (or just source in the .bashrc file). Verify SENDMAILJAIL and OPENDKIMJAIL are set.

Copy a whole heap of ‘stuff’ into the jail – this includes some utilities used to troubleshoot issues within the jail which aren’t strictly needed. I’ve also unpacked the strace RPM to the respective directories within the jail.

mkdir $SENDMAILJAIL/bin
mkdir $SENDMAILJAIL/etc
mkdir $SENDMAILJAIL/etc/alternatives
mkdir $SENDMAILJAIL/etc/mail
mkdir $SENDMAILJAIL/etc/smrsh
mkdir $SENDMAILJAIL/lib64
mkdir $SENDMAILJAIL/lib
mkdir $SENDMAILJAIL/lib/tls
mkdir $SENDMAILJAIL/tmp
mkdir $SENDMAILJAIL/usr
mkdir $SENDMAILJAIL/usr/bin
mkdir $SENDMAILJAIL/usr/sbin
mkdir $SENDMAILJAIL/usr/lib
mkdir $SENDMAILJAIL/usr/lib/sasl2
mkdir $SENDMAILJAIL/var
mkdir $SENDMAILJAIL/var/log
mkdir $SENDMAILJAIL/var/log/mail
mkdir $SENDMAILJAIL/var/run
mkdir $SENDMAILJAIL/var/spool
mkdir $SENDMAILJAIL/var/spool/mqueue
mkdir $SENDMAILJAIL/var/spool/clientmqueue
 
cp /etc/aliases $SENDMAILJAIL/etc/
cp /etc/aliases.db $SENDMAILJAIL/etc/
cp /etc/passwd $SENDMAILJAIL/etc/
cp /etc/group $SENDMAILJAIL/etc/
cp /etc/resolv.conf $SENDMAILJAIL/etc/
cp /etc/host.conf $SENDMAILJAIL/etc/
cp /etc/nsswitch.conf $SENDMAILJAIL/etc/
cp /etc/services $SENDMAILJAIL/etc/
cp /etc/hosts $SENDMAILJAIL/etc/
cp /etc/localtime $SENDMAILJAIL/etc/
 

# If cloning an existing server, scp /etc/mail/* from source to /smt00p20/sendmail/etc/mail

# Verify the sendmail.mc has a RUNAS_USER set to the same service account you are using - the account on our servers is named 'sendmail'. Our old servers are not all set up with a runas user, and failing to have one will cause write failures to the jail /var/spool/mqueue

cp -r /etc/mail/ $SENDMAILJAIL/etc/etc/mail/
cp /usr/sbin/sendmail.sendmail $SENDMAILJAIL/usr/sbin/sendmail.sendmail

cd /smt00p20/sendmail/etc/alternatives
ln -s ../../usr/sbin/sendmail.sendmail ./mta

cd /smt00p20/sendmail/usr/sbin
ln -s ../../etc/alternatives/mta ./sendmail
ln -s ./sendmail ./newaliases
ln -s ./sendmail ./newaliases.sendmail

cd /smt00p20/sendmail/usr/bin
ln -s ../sbin/sendmail ./mailq
ln -s ../sbin/sendmail ./mailq.sendmail
ln -s ../sbin/sendmail.sendmail ./hoststat
ln -s ../sbin/sendmail.sendmail ./purgestat
ln -s ../sbin/makemap ./makemap
ln -s ./rmail.sendmail ./rmail
cp /usr/lib64/libssl.so.10 $SENDMAILJAIL/usr/lib64/libssl.so.10
cp /usr/lib64/libcrypto.so.10 $SENDMAILJAIL/usr/lib64/libcrypto.so.10
cp /usr/lib64/libnsl.so.1 $SENDMAILJAIL/usr/lib64/libnsl.so.1
cp /usr/lib64/libwrap.so.0 $SENDMAILJAIL/usr/lib64/libwrap.so.0
cp /usr/lib64/libhesiod.so.0 $SENDMAILJAIL/usr/lib64/libhesiod.so.0
cp /usr/lib64/libcrypt.so.1 $SENDMAILJAIL/usr/lib64/libcrypt.so.1
cp /usr/lib64/libdb-5.3.so $SENDMAILJAIL/usr/lib64/libdb-5.3.so
cp /usr/lib64/libresolv.so.2 $SENDMAILJAIL/usr/lib64/libresolv.so.2
cp /usr/lib64/libsasl2.so.3 $SENDMAILJAIL/usr/lib64/libsasl2.so.3
cp /usr/lib64/libldap-2.4.so.2 $SENDMAILJAIL/usr/lib64/libldap-2.4.so.2
cp /usr/lib64/liblber-2.4.so.2 $SENDMAILJAIL/usr/lib64/liblber-2.4.so.2
cp /usr/lib64/libc.so.6 $SENDMAILJAIL/usr/lib64/libc.so.6
cp /usr/lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/usr/lib64/libgssapi_krb5.so.2
cp /usr/lib64/libkrb5.so.3 $SENDMAILJAIL/usr/lib64/libkrb5.so.3
cp /usr/lib64/libcom_err.so.2 $SENDMAILJAIL/usr/lib64/libcom_err.so.2
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/libk5crypto.so.3
cp /usr/lib64/libdl.so.2 $SENDMAILJAIL/usr/lib64/libdl.so.2
cp /usr/lib64/libz.so.1 $SENDMAILJAIL/usr/lib64/libz.so.1
cp /usr/lib64/libidn.so.11 $SENDMAILJAIL/usr/lib64/libidn.so.11
cp /usr/lib64/libfreebl3.so $SENDMAILJAIL/usr/lib64/libfreebl3.so
cp /usr/lib64/libpthread.so.0 $SENDMAILJAIL/usr/lib64/libpthread.so.0
cp /usr/lib64/libssl3.so $SENDMAILJAIL/usr/lib64/libssl3.so
cp /usr/lib64/libsmime3.so $SENDMAILJAIL/usr/lib64/libsmime3.so
cp /usr/lib64/libnss3.so $SENDMAILJAIL/usr/lib64/libnss3.so
cp /usr/lib64/libnssutil3.so $SENDMAILJAIL/usr/lib64/libnssutil3.so
cp /usr/lib64/libplds4.so $SENDMAILJAIL/usr/lib64/libplds4.so
cp /usr/lib64/libplc4.so $SENDMAILJAIL/usr/lib64/libplc4.so
cp /usr/lib64/libnspr4.so $SENDMAILJAIL/usr/lib64/libnspr4.so
cp /usr/lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/usr/lib64/ld-linux-x86-64.so.2
cp /usr/lib64/libkrb5support.so.0 $SENDMAILJAIL/usr/lib64/libkrb5support.so.0
cp /usr/lib64/libkeyutils.so.1 $SENDMAILJAIL/usr/lib64/libkeyutils.so.1
cp /usr/lib64/librt.so.1 $SENDMAILJAIL/usr/lib64/librt.so.1
cp /usr/lib64/libselinux.so.1 $SENDMAILJAIL/usr/lib64/libselinux.so.1
cp /usr/lib64/libpcre.so.1 $SENDMAILJAIL/usr/lib64/libpcre.so.1
cp /usr/lib64/libnss_dns.so.2 $SENDMAILJAIL/usr/lib64/libnss_dns.so.2
cp /usr/lib64/libnss_files.so.2 $SENDMAILJAIL/usr/lib64/libnss_files.so.2

cd $SENDMAILJAIL/lib64
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib64/libnss_dns-2.17.so
ln -s ./libnss_dns-2.17.so ./libnss_dns.so.2

cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib64/libresolv-2.17.so
ln -s ./lib64/libresolv-2.17.so ./libresolv.so.2

cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib64/libnss_files-2.17.so
ln -s ./lib64/libnss_files-2.17.so ./libnss_files.so.2

cd $SENDMAILJAIL/lib 
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib/libnss_dns-2.17.so
ln -s ./lib/libnss_dns-2.17.so ./libnss_dns.so.2

cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib/libresolv-2.17.so
ln -s ./lib/libresolv-2.17.so ./libresolv.so.2

cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib/libnss_files-2.17.so
ln -s ./lib/libnss_files-2.17.so ./libnss_files.so.2

mkdir $SENDMAILJAIL/usr/lib64/sasl2
cp /usr/lib64/sasl2/* $SENDMAILJAIL/usr/lib64/sasl2/

mkdir $SENDMAILJAIL/lib64/sasl2/
cp /lib64/sasl2/* $SENDMAILJAIL/lib64/sasl2/
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/usr/lib64/sasl2/

mkdir $SENDMAILJAIL/etc/sasl2
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/etc/sasl2/


cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
ln -s ../sbin/makemap ./makemap
cp /usr/bin/rmail.sendmail $SENDMAILJAIL/usr/bin/rmail.sendmail
ln -s ./rmail.sendmail ./rmail

cp /usr/sbin/mailstats $SENDMAILJAIL/usr/sbin/mailstats
cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
cp /usr/sbin/praliases $SENDMAILJAIL/usr/sbin/praliases
cp /usr/sbin/smrsh $SENDMAILJAIL/usr/sbin/smrsh

cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libdb-5.3.so $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libfreebl3.so $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libhesiod.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3: $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/liblber-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libldap-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libnsl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libnspr4.so $SENDMAILJAIL/lib64/
cp /lib64/libnss3.so $SENDMAILJAIL/lib64/
cp /lib64/libnssutil3.so $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libplc4.so $SENDMAILJAIL/lib64/
cp /lib64/libplds4.so $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsasl2.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsmime3.so $SENDMAILJAIL/lib64/
cp /lib64/libssl.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libssl3.so $SENDMAILJAIL/lib64/
cp /lib64/libwrap.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/

cp /lib64/libdns.so.100 $SENDMAILJAIL/lib64/
cp /lib64/liblwres.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libbind9.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccfg.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccc.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisc.so.95 $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libGeoIP.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libxml2.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /bin/dig $SENDMAILJAIL/bin/

cp /lib64/libtinfo.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /bin/bash $SENDMAILJAIL/bin/

cp /bin/ls $SENDMAILJAIL/bin/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libacl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/

cp /bin/vi $SENDMAILJAIL/bin/
cp /usr/sbin/pidof $SENDMAILJAIL/usr/sbin/pidof
cp /lib64/libprocps.so.4 $SENDMAILJAIL/lib64/
cp /lib64/libsystemd.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libgcrypt.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libgpg-error.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdw.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libgcc_s.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libelf.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libbz2.so.1 $SENDMAILJAIL/lib64/
cp /bin/rm $SENDMAILJAIL/bin/

Under your ID, ensure the proper permissions are set on the chroot jail

sudo chown -R sendmail:mail /smt00p20/sendmail/
sudo chown sendmail /smt00p20/sendmail/var/spool/mqueue
sudo chmod 0700 /smt00p20/sendmail/var/spool/mqueue
sudo chmod -R go-w /smt00p20/sendmail
sudo chmod 0400 /smt00p20/sendmail/etc/mail/*.cf

Now verify it works – still under your ID as you have sudo permission to run chroot.

sudo /sbin/chroot /smt00p20/sendmail /bin/ls
# You should see a directory listing like this, not an error
bin  dev  etc  lib  lib64  tmp  usr  var

Assuming there are no problems, run sendmail:

sudo /sbin/chroot /smt00p20/sendmail /usr/sbin/sendmail -bd -q5m

Test sending mail through the server to verify proper functionality.

Unit Config: Edit the systemd unit file and add the “RootDirectory” directive

sudo vi /etc/systemd/system/multi-user.target.wants/sendmail.service

[Unit]
Description=Sendmail Mail Transport Agent
After=syslog.target network.target
Conflicts=postfix.service exim.service
Wants=sm-client.service

[Service]
RootDirectory=/smt00p20/sendmail
Type=forking
StartLimitInterval=0
# Known issue – pid causes service hang/timeout that bothers Unix guys
# https://bugzilla.redhat.com/show_bug.cgi?id=1253840
#PIDFile=/run/sendmail.pid
Environment=SENDMAIL_OPTS=-q15m
EnvironmentFile=-/smt00p20/sendmail/etc/sysconfig/sendmail
ExecStart=/usr/sbin/sendmail -bd $SENDMAIL_OPTS $SENDMAIL_OPTARG

[Install]
WantedBy=multi-user.target
Also=sm-client.service

Then run “systemctl daemon-reload” to ingest the changes.

You can now use systemctl to start and stop the sendmail service.

Chrooting opendkim

Create the chroot jail and lib64 directory, create the base directories, then add a few core Linux files so you have a bash shell:

mkdir $OPENDKIMJAIL
mkdir $OPENDKIMJAIL/lib64
mkdir $OPENDKIMJAIL/usr/lib64
mkdir $OPENDKIMJAIL/bin
mkdir $OPENDKIMJAIL/etc

cp /lib64/libtinfo.so.5 $OPENDKIMJAIL/lib64/
cp /lib64/libdl.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libc.so.6 $OPENDKIMJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $OPENDKIMJAIL/lib64/

cp /bin/bash $OPENDKIMJAIL/bin/
cp /lib64/libstdc++.so.6* $OPENDKIMJAIL/lib64
cp /lib64/libm.so.6 $OPENDKIMJAIL/lib64
cp /lib64/libgcc_s.so.1 $OPENDKIMJAIL/lib64
cp /lib64/libnss_files* $OPENDKIMJAIL/lib64/

Unpack the following RPMs:

rpm2cpio opendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio libopendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio sendmail-milter-8.14.7-5.el7.x86_64.rpm | cpio -idmv
rpm2cpio opendbx-1.4.6-6.el7.x86_64.rpm | cpio -idmv
rpm2cpio libmemcached-1.0.16-5.el7.x86_64.rpm | cpio -idvm
rpm2cpio libbsd-0.6.0-3.el7.elrepo.x86_64.rpm | cpio -idvm

Then move the unpacked files into the corresponding location in the $OPENDKIMJAIL directory.

Copy host configuration ‘stuff’ from /etc

cp /etc/aliases $OPENDKIMJAIL/etc/
cp /etc/aliases.db $OPENDKIMJAIL/etc/
cp /etc/passwd $OPENDKIMJAIL/etc/
cp /etc/group $OPENDKIMJAIL/etc/
cp /etc/resolv.conf $OPENDKIMJAIL/etc/
cp /etc/host.conf $OPENDKIMJAIL/etc/
cp /etc/nsswitch.conf $OPENDKIMJAIL/etc/
cp /etc/services $OPENDKIMJAIL/etc/
cp /etc/hosts $OPENDKIMJAIL/etc/
cp /etc/localtime $OPENDKIMJAIL/etc/

Copy some more files:

cp /lib64/libcom_err.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libcrypt.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libcrypto.so.10 $OPENDKIMJAIL/lib64/
cp /lib64/libdb-5.3.so $OPENDKIMJAIL/lib64/
cp /lib64/libfreebl3.so $OPENDKIMJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libk5crypto.so.3 $OPENDKIMJAIL/lib64/
cp /lib64/libkeyutils.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libkrb5.so.3 $OPENDKIMJAIL/lib64/
cp /lib64/libkrb5support.so.0 $OPENDKIMJAIL/lib64/
cp /lib64/liblber-2.4.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libldap-2.4.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libnspr4.so $OPENDKIMJAIL/lib64/
cp /lib64/libnss3.so $OPENDKIMJAIL/lib64/
cp /lib64/libnssutil3.so $OPENDKIMJAIL/lib64/
cp /lib64/libpcre.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libplc4.so $OPENDKIMJAIL/lib64/
cp /lib64/libplds4.so $OPENDKIMJAIL/lib64/
cp /lib64/libpthread.so.0 $OPENDKIMJAIL/lib64/
cp /lib64/libresolv.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/librt.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libsasl2.so.3 $OPENDKIMJAIL/lib64/
cp /lib64/libselinux.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libsmime3.so $OPENDKIMJAIL/lib64/
cp /lib64/libssl.so.10 $OPENDKIMJAIL/lib64/
cp /lib64/libssl3.so $OPENDKIMJAIL/lib64/
cp /lib64/libz.so.1 $OPENDKIMJAIL/lib64/
cp /usr/lib64/libssl.so.10 $OPENDKIMJAIL/usr/lib64/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/usr/lib/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/usr/lib/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/lib64/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/lib64/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/usr/lib/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/usr/lib/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/lib64/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/lib64/

Configure OpenDKIM ($DKIMJAIL/etc/opendkim.conf) and populate keys (copy from server being replaced or generate new keys). Then, under your ID, run:

sudo /sbin/chroot /smt00p20/opendkim /usr/sbin/opendkim -u sendmail -v

The systemd unit file, /usr/lib/systemd/system/opendkim.service, needs to contain:

# If you are using OpenDKIM with SQL datasets it might be necessary to start OpenDKIM after the database servers.
# For example, if using both MariaDB and PostgreSQL, change "After=" in the "[Unit]" section to:
# After=network.target nss-lookup.target syslog.target mariadb.service postgresql.service

[Unit]
Description=DomainKeys Identified Mail (DKIM) Milter
Documentation=man:opendkim(8) man:opendkim.conf(5) man:opendkim-genkey(8) man:opendkim-genzone(8) man:opendkim-testadsp(8) man:opendkim-testkey http://www.opendkim.org/docs.html
After=network.target nss-lookup.target syslog.target

[Service]
RootDirectory=/smt00p20/opendkim
Type=forking
PIDFile=/smt00p20/opendkim/var/run/opendkim/opendkim.pid
EnvironmentFile=-/etc/sysconfig/opendkim
ExecStart=/usr/sbin/opendkim -u sendmail -v $OPTIONS
ExecReload=/bin/kill -USR1 $MAINPID
User=sendmail
Group=mail

[Install]
WantedBy=multi-user.target

Upgrading Sendmail – After Unix Applies Patches

This process grabs a new copy of sendmail, associated diagnostic utilities, and their dependencies from the OS installation. If you want to apply patches prior to Unix support doing so, you can stage a sendmail build (everything up to ‘make install’) and copy the files out or, if an updated RPM is in the repo but just not installed, download the RPMs, unpack them, and copy the files in. I would do that in addition to (and after) this process to ensure library updates are reflected in our jailed sendmail installation (i.e. if there’s an update to the crypto libraries, we get those updates).

cp /usr/sbin/sendmail.sendmail $SENDMAILJAIL/usr/sbin/sendmail.sendmail
cp /usr/lib64/libssl.so.10 $SENDMAILJAIL/usr/lib64/libssl.so.10
cp /usr/lib64/libcrypto.so.10 $SENDMAILJAIL/usr/lib64/libcrypto.so.10
cp /usr/lib64/libnsl.so.1 $SENDMAILJAIL/usr/lib64/libnsl.so.1
cp /usr/lib64/libwrap.so.0 $SENDMAILJAIL/usr/lib64/libwrap.so.0
cp /usr/lib64/libhesiod.so.0 $SENDMAILJAIL/usr/lib64/libhesiod.so.0
cp /usr/lib64/libcrypt.so.1 $SENDMAILJAIL/usr/lib64/libcrypt.so.1
cp /usr/lib64/libdb-5.3.so $SENDMAILJAIL/usr/lib64/libdb-5.3.so
cp /usr/lib64/libresolv.so.2 $SENDMAILJAIL/usr/lib64/libresolv.so.2
cp /usr/lib64/libsasl2.so.3 $SENDMAILJAIL/usr/lib64/libsasl2.so.3
cp /usr/lib64/libldap-2.4.so.2 $SENDMAILJAIL/usr/lib64/libldap-2.4.so.2
cp /usr/lib64/liblber-2.4.so.2 $SENDMAILJAIL/usr/lib64/liblber-2.4.so.2
cp /usr/lib64/libc.so.6 $SENDMAILJAIL/usr/lib64/libc.so.6
cp /usr/lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/usr/lib64/libgssapi_krb5.so.2
cp /usr/lib64/libkrb5.so.3 $SENDMAILJAIL/usr/lib64/libkrb5.so.3
cp /usr/lib64/libcom_err.so.2 $SENDMAILJAIL/usr/lib64/libcom_err.so.2
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/libk5crypto.so.3
cp /usr/lib64/libdl.so.2 $SENDMAILJAIL/usr/lib64/libdl.so.2
cp /usr/lib64/libz.so.1 $SENDMAILJAIL/usr/lib64/libz.so.1
cp /usr/lib64/libidn.so.11 $SENDMAILJAIL/usr/lib64/libidn.so.11
cp /usr/lib64/libfreebl3.so $SENDMAILJAIL/usr/lib64/libfreebl3.so
cp /usr/lib64/libpthread.so.0 $SENDMAILJAIL/usr/lib64/libpthread.so.0
cp /usr/lib64/libssl3.so $SENDMAILJAIL/usr/lib64/libssl3.so
cp /usr/lib64/libsmime3.so $SENDMAILJAIL/usr/lib64/libsmime3.so
cp /usr/lib64/libnss3.so $SENDMAILJAIL/usr/lib64/libnss3.so
cp /usr/lib64/libnssutil3.so $SENDMAILJAIL/usr/lib64/libnssutil3.so
cp /usr/lib64/libplds4.so $SENDMAILJAIL/usr/lib64/libplds4.so
cp /usr/lib64/libplc4.so $SENDMAILJAIL/usr/lib64/libplc4.so
cp /usr/lib64/libnspr4.so $SENDMAILJAIL/usr/lib64/libnspr4.so
cp /usr/lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/usr/lib64/ld-linux-x86-64.so.2
cp /usr/lib64/libkrb5support.so.0 $SENDMAILJAIL/usr/lib64/libkrb5support.so.0
cp /usr/lib64/libkeyutils.so.1 $SENDMAILJAIL/usr/lib64/libkeyutils.so.1
cp /usr/lib64/librt.so.1 $SENDMAILJAIL/usr/lib64/librt.so.1
cp /usr/lib64/libselinux.so.1 $SENDMAILJAIL/usr/lib64/libselinux.so.1
cp /usr/lib64/libpcre.so.1 $SENDMAILJAIL/usr/lib64/libpcre.so.1
cp /usr/lib64/libnss_dns.so.2 $SENDMAILJAIL/usr/lib64/libnss_dns.so.2
cp /usr/lib64/libnss_files.so.2 $SENDMAILJAIL/usr/lib64/libnss_files.so.2
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib64/libnss_dns-2.17.so
cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib64/libresolv-2.17.so
cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib64/libnss_files-2.17.so
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib/libnss_dns-2.17.so
cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib/libresolv-2.17.so
cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib/libnss_files-2.17.so
cp /usr/lib64/sasl2/* $SENDMAILJAIL/usr/lib64/sasl2/
cp /lib64/sasl2/* $SENDMAILJAIL/lib64/sasl2/
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/usr/lib64/sasl2/
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/etc/sasl2/
cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
cp /usr/bin/rmail.sendmail $SENDMAILJAIL/usr/bin/rmail.sendmail
cp /usr/sbin/mailstats $SENDMAILJAIL/usr/sbin/mailstats
cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
cp /usr/sbin/praliases $SENDMAILJAIL/usr/sbin/praliases
cp /usr/sbin/smrsh $SENDMAILJAIL/usr/sbin/smrsh

cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libdb-5.3.so $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libfreebl3.so $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libhesiod.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3: $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/liblber-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libldap-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libnsl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libnspr4.so $SENDMAILJAIL/lib64/
cp /lib64/libnss3.so $SENDMAILJAIL/lib64/
cp /lib64/libnssutil3.so $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libplc4.so $SENDMAILJAIL/lib64/
cp /lib64/libplds4.so $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsasl2.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsmime3.so $SENDMAILJAIL/lib64/
cp /lib64/libssl.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libssl3.so $SENDMAILJAIL/lib64/
cp /lib64/libwrap.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/

cp /lib64/libdns.so.100 $SENDMAILJAIL/lib64/
cp /lib64/liblwres.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libbind9.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccfg.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccc.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisc.so.95 $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libGeoIP.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libxml2.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /bin/dig $SENDMAILJAIL/bin/

cp /lib64/libtinfo.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /bin/bash $SENDMAILJAIL/bin/

cp /bin/ls $SENDMAILJAIL/bin/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libacl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/

cp /bin/vi $SENDMAILJAIL/bin/
cp /usr/sbin/pidof $SENDMAILJAIL/usr/sbin/pidof
cp /lib64/libprocps.so.4 $SENDMAILJAIL/lib64/
cp /lib64/libsystemd.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libgcrypt.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libgpg-error.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdw.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libgcc_s.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libelf.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libbz2.so.1 $SENDMAILJAIL/lib64/

cp /bin/rm $SENDMAILJAIL/bin/

Under your ID, ensure the proper permissions are set on the chroot jail

sudo chown -R sendmail:mail /smt00p20/sendmail/
sudo chown sendmail /smt00p20/sendmail/var/spool/mqueue
sudo chmod 0700 /smt00p20/sendmail/var/spool/mqueue
sudo chmod -R go-w /smt00p20/sendmail
sudo chmod 0400 /smt00p20/sendmail/etc/mail/*.cf

Then start sendmail and verify functionality.

Updating OpenDKIM

cp /lib64/libtinfo.so.5 $OPENDKIMJAIL/lib64/
cp /lib64/libdl.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libc.so.6 $OPENDKIMJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $OPENDKIMJAIL/lib64/
cp /bin/bash $OPENDKIMJAIL/bin/
cp /lib64/libstdc++.so.6* $OPENDKIMJAIL/lib64
cp /lib64/libm.so.6 $OPENDKIMJAIL/lib64
cp /lib64/libgcc_s.so.1 $OPENDKIMJAIL/lib64
cp /lib64/libnss_files* $OPENDKIMJAIL/lib64/

If there is an update to the opendkim packages, unpack the updated RPM files and move the new files into the corresponding jail locations.

rpm2cpio opendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio libopendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio sendmail-milter-8.14.7-5.el7.x86_64.rpm | cpio -idmv
rpm2cpio opendbx-1.4.6-6.el7.x86_64.rpm | cpio -idmv
rpm2cpio libmemcached-1.0.16-5.el7.x86_64.rpm | cpio -idvm
rpm2cpio libbsd-0.6.0-3.el7.elrepo.x86_64.rpm | cpio -idvm

March 30, 2018

Creating An OpenHAB 2.3.0 Snapshot Docker Container

We found quick instructions for creating a Docker container for the OpenHAB 2.3.0 snapshot. These instructions evidently presuppose some basic knowledge of building Docker containers, so I thought I’d write the “I don’t know what I am doing” version of the instructions. Beyond the obvious download & install Docker, then make sure it’s functional (service starts).

The linked Dockerfile is not the only thing you need. Go up a level — you need both the Dockerfile and entrypoint.sh files. Create a directory somewhere and grab these two files. Then build the container using

docker build -t oh2imagename .

I used a short, alpha-numeric only name for my image. When I used slashes as in the example, the container would not start. Then make the folders you want to map into OpenHAB2:

mkdir /some/path/to/openhab/addons
mkdir /some/path/to/openhab/conf
mkdir /some/path/to/openhab/userdata

The instructions conflate local users/groups with in-container users/groups. You do not need to create a local user. You do need to indicate the uidNumber and gidNumber for the openhab user and group. Even if you do create the local user and group, then change the /some/path/to/openhab permissions to provide full access to the user … you may well not be able to access the files. That is SELinux, not a file permission issue. The quick/dirty solution is to start the container with the privileged flag:

--privileged=true

Alternately, consult the Universal Archive of All IT Knowledge and figure out how to allow the docker service to write files where you want them. And how to access USB devices if you are trying to use something like a ZWave dongle. We went with the privileged route 🙂 The –name option is just the container name. The –net uses the host network for container communications instead of the bridge network. Saves mapping ports, although you could easily use the bridge network and map out the handful of OpenHab specific ports. The -d runs the container in detached mode. The -e sets some environment flags (used by the user/group creation script that runs upon container startup). The –tty (or -t) attaches a console. Not really used here.

docker run --privileged --name oh2containername --net=host --tty -d -e USER_ID=5555 \
 -e GROUP_ID=5555 oh2imagename

Ideally, your OpenHAB2 instance will be running. Use “docker ps” to list out the running containers. If you don’t see a container with the name supplied above … then something went wrong. You can use “docker history oh2containername” to view a quick history, but “docker logs oh2containername” will probably provide more useful information. We encountered file permission issues (as noted above, due to SELinux) which prevented the initial container setup from running. Once that was sorted, the container showed up in the running container list.

You’re ready to use it — you can access the web console using your computer’s IP address (assuming you set this container up in the host network and not the bridge — if you used the bridge, you can use “docker inspect oh2containername” and look for IPAddress under NetworkSettings) on the default port. You can ssh into the Karaf console with the default user/password on the default port. Or you can shell into the container.

docker exec -it oh2containername /bin/bash

This is a bash shell running on the OH2 container — you’ll find a lot of ‘stuff’ hasn’t been installed, and your normal command aliases won’t be present. But it’s a shell on the server and can be used to start/stop OH2.

March 23, 2018

Enable OUD Changelog Without Replication Partner

Since the Sun Directory Server Enterprise Edition went the way of, well, Sun, we’ve migrated to the Oracle Unified Directory 11g platform for the company’s pure LDAP directory. There is an Oracle Identity Management application that reads the LDAP changelog to ingest user lockout events. In production, our servers replicate with a couple of partners to provide capacity and high availability. In development / sandbox, environments … not so much. But the IDM platform still needs to read the changelog.

Oracle’s documentation tells me to enable replication … which, great, I’ve got to bring up a second, off-port, directory instance and monitor for replication failures just to get a changelog. The site does say “By using this method, you can conceivably set up replication on a standalone server, which will enable you to have access to an ECL on a standalone server.” … conceivably, but it would be nice if they’d mention how. Since all of their documentation for using the dsreplication binary includes a partner server and valid credentials over yonder … that’s a bit of a bust.

But I’ve finally worked out a technique for enabling replication just enough to get the changelog created without having to provide valid credentials on a foreign host with which replication will be established.

./dsconfig -h localhost -p 4444 -D "cn=directory manager" -j ~/pwd.txt -X -n create-replication-server --provider-name 'Multimaster Synchronization' --set replication-port:8989 --set replication-server-id:1 --type generic
 
 
./dsconfig -h localhost -p 4444 -D "cn=directory manager" -j ~/pwd.txt -X -n create-replication-domain --provider-name 'Multimaster Synchronization' --set base-dn:o=windstream --set replication-server:localhost:8989 --set server-id:1 --type generic --domain-name o=windstream

Volia, make changes and we’ve got stuff under cn=changelog.

March 15, 2018

Fedora 26 => 27 & PHP

Since I like to discover major changes by upgrading my server and then realizing something doesn’t work (well “like” might be too strong a word … but I certainly do it) … I randomly upgraded to Fedora 27 without reading any documentation on changes. Aaaand we have PHP! Evidently mod_php has gone away and I’m going to have to figure out how to use FastCGI (php-fpm). Luckily there’s a quick way to switch back to mod_php in the interim:

/etc/httpd/conf.modules.d/00-mpm.conf

# Uncomment this line to use mod_php
LoadModule mpm_prefork_module modules/mod_mpm_prefork.so

# worker MPM: Multi-Processing Module implementing a hybrid
# multi-threaded multi-process web server
# See: http://httpd.apache.org/docs/2.4/mod/worker.html
#
#LoadModule mpm_worker_module modules/mod_mpm_worker.so

# event MPM: A variant of the worker MPM with the goal of consuming
# threads only for connections with active processing
# See: http://httpd.apache.org/docs/2.4/mod/event.html
#
# Commend out this line to use mod_php
#LoadModule mpm_event_module modules/mod_mpm_event.so

/etc/httpd/conf.modules.d/15-php.conf

<IfModule !mod_php5.c>
<IfModule prefork.c>
LoadModule php7_module modules/libphp7.so
</IfModule>
</IfModule>

<IfModule !mod_php5.c>
<IfModule !prefork.c>
# ZTS module is not supported, so FPM is preferred
LoadModule php7_module modules/libphp7-zts.so
</IfModule>
</IfModule>

March 8, 2018

Unable to ‘send as’ from Outlook With Exchange

I’ve had a confounding problem — we have sendmail magicing up millions of e-mail addresses for us, but occasionally we need to be able to send from one of these addresses too. I’ve got a web form that allows text-based messages (html or plain text), but I don’t want to figure out how to upload and attach images via a web form. Until I get around to updating the web form, I just set up a new Exchange mailbox and grant myself full access (which includes send as permission)

Except ever since we got always updating Office 2016, I’ve gotten nondelivery reports when I subsequently try to send from this new mailbox that claim I don’t have permission to send as the user in question. And I’ve verified my access three times. Even added explicit send-as in addition to full mailbox access.

I’ve finally discovered why I get this false error. The ‘from’ in Outlook allows free-form text which then may or may not resolve against an Exchange mailbox. And based on the permissions of the mailbox (or the lack of permissions of the non-resolved mailbox), it may or may not work. So I don’t have permission to send from newmailbox@ourdomain.ccTLD, I do have permission to send from the Exchange mailbox that happens to have that as its primary SMTP address. Sigh!.

When you use offline mode / cached Exchange mode, and an offline address list, the SMTP address doesn’t resolve out to that mailbox. And Exchange quite properly reports an error. To get the whole thing to work (assuming “wait until tomorrow” isn’t a good answer):

First, the offline address needs to be updated (either wait or hit the powershell management console on the server)

Update-OfflineAddressBook -Identity "Default Offline Address Book"

Secondly, Outlook needs to retrieve the updated address book. Within the Outlook client, use send/receive to update the address lists. Then you can send as the mailbox to which you have perfectly configured access.

March 6, 2018

WordPress Pages With Custom Info (SEO-type stuff)

I happened across a business who wanted to create several hundred unique WordPress pages so a long list of cities would have a “customized” page offering the service in that area. Makes sense, especially as an SEO endeavor since I search for ‘service city state’ fairly often when it is something I specifically want to obtain locally. Thing is, they were looking to pay someone to duplicate the post & manually edit each duplicate to use the individual locations. There’s a much easier way – the wp_insert_post function. Now it requires that you be able to execute PHP code either from the server’s command line (i.e. it’s your OS) or upload custom code (can be a WordPress plug-in, but it is easier if the code can be installed next to WordPress and called from its URL).

You need a variable for the template text – I wanted to include the location information in both the page title and page content, so I have a title variable as well. Include in the text some string that would never appear in your template (here, I used VARCSZ for variable containing city, state, and zip). Iterate through an array of locations and use str_replace to insert the individual locations into the title and content. Then create the page. Voila, 350 pages posted in a few minutes.

To create a single-column page (although different types of pages can be created by altering the $strContent variable):

<?php
require('/path/to/your/wordpress/html/wp-load.php');

$strTitle = 'Service Offered In VARCSZ';
$strContent = '<section id="builder-section-text_11" class="builder-section-first builder-section builder-section-text builder-section-last builder-text-columns-1" style="background-size: cover; background-repeat: no-repeat;background-position: center center;">
     <div class="builder-section-content">
     <div class="builder-text-row">
     <div class="builder-text-column builder-text-column-1" id="builder-section-text_11-column-1">
     <div class="builder-text-content">
     <p><b>Service Offered In VARCSZ</b></p>
     <p>And here is where we provide some information about the service we are offering, why you want this service, and what we do that is super awesome. </P>
     <p><b>More about our service in VARCSZ</b></p>
     <p>Info about our company and the service we provide in VARCSZ</p>
     <p><b>Call NOW for our service in VARCSZ</b></p>
     <p>For this service in VARCSZ, call us.</p>
     <p><b>Call 800-555-1212</b></p>
     </div>
     </div>
     </div>
     </div>
     </section>';

$strArrayOfLocations = array('Abington, PA 19001', 'Ambler, PA 19002', 'Ardmore, PA 19003', 'Bala Cynwyd, PA 19004', 'Huntingdon Valley, PA 19006', 'Bristol, PA 19007', 'Broomall, PA 19008', 'Bryn Athyn, PA 19009', 'Bryn Mawr, PA 19010', 'Cheltenham, PA 19012', 'Chester, PA 19013', 'Aston, PA 19014', 'Brookhaven, PA 19015', 'Chester, PA 19016', 'Chester Heights, PA 19017', 'Clifton Heights, PA 19018', 'Philadelphia, PA 19019', 'Bensalem, PA 19020', 'Croydon, PA 19021', 'Crum Lynne, PA 19022', 'Darby, PA 19023', 'Dresher, PA 19025', 'Drexel Hill, PA 19026', 'Elkins Park, PA 19027', 'Edgemont, PA 19028', 'Essington, PA 19029', 'Fairless Hills, PA 19030', 'Flourtown, PA 19031', 'Folcroft, PA 19032', 'Folsom, PA 19033', 'Fort Washington, PA 19034', 'Gladwyne, PA 19035', 'Glenolden, PA 19036', 'Glen Riddle, PA 19037', 'Glenside, PA 19038', 'Gradyville, PA 19039', 'Hatboro, PA 19040', 'Haverford, PA 19041', 'Holmes, PA 19043', 'Horsham, PA 19044', 'Jenkintown, PA 19046', 'Langhorne, PA 19047', 'Fort Washington, PA 19048', 'Fort Washington, PA 19049', 'Lansdowne, PA 19050', 'Lenni, PA 19052', 'Feasterville, PA 19053', 'Levittown, PA 19054', 'Levittown, PA 19055', 'Levittown, PA 19056', 'Levittown, PA 19057', 'Levittown, PA 19058', 'Garnet Valley, PA 19060', 'Marcus Hook, PA 19061', 'Media, PA 19063', 'Springfield, PA 19064', 'Media, PA 19065', 'Merion Station, PA 19066', 'Morrisville, PA 19067', 'Morton, PA 19070', 'Narberth, PA 19072', 'Newtown Square, PA 19073', 'Norwood, PA 19074', 'Oreland, PA 19075', 'Prospect Park, PA 19076', 'Ridley Park, PA 19078', 'Sharon Hill, PA 19079', 'Wayne, PA 19080', 'Swarthmore, PA 19081', 'Upper Darby, PA 19082', 'Havertown, PA 19083', 'Villanova, PA 19085', 'Wallingford, PA 19086', 'Wayne, PA 19087', 'Radnor, PA 19088', 'Radnor, PA 19089', 'Willow Grove, PA 19090', 'Media , PA 19091', 'Philadelphia , PA 19092', 'Philadelphia , PA 19093', 'Woodlyn, PA 19094', 'Wyncote, PA 19095', 'Wynnewood, PA 19096', 'Holmes , PA 19098', 'Philadelphia , PA 19099', 'Philadelphia, PA 19101', 'Philadelphia, PA 19102', 'Philadelphia, PA 19103', 'Philadelphia, PA 19104', 'Philadelphia, PA 19105', 'Philadelphia, PA 19106', 'Philadelphia, PA 19107', 'Philadelphia, PA 19108', 'Philadelphia, PA 19109', 'Philadelphia, PA 19110', 'Philadelphia, PA 19111', 'Philadelphia, PA 19112', 'Philadelphia, PA 19113', 'Philadelphia, PA 19114', 'Philadelphia, PA 19115', 'Philadelphia, PA 19116', 'Philadelphia, PA 19118', 'Philadelphia, PA 19119', 'Philadelphia, PA 19120', 'Philadelphia, PA 19121', 'Philadelphia, PA 19122', 'Philadelphia, PA 19123', 'Philadelphia, PA 19124', 'Philadelphia, PA 19125', 'Philadelphia, PA 19126', 'Philadelphia, PA 19127', 'Philadelphia, PA 19128', 'Philadelphia, PA 19129', 'Philadelphia, PA 19130', 'Philadelphia, PA 19131', 'Philadelphia, PA 19132', 'Philadelphia, PA 19133', 'Philadelphia, PA 19134', 'Philadelphia, PA 19135', 'Philadelphia, PA 19136', 'Philadelphia, PA 19137', 'Philadelphia, PA 19138', 'Philadelphia, PA 19139', 'Philadelphia, PA 19140', 'Philadelphia, PA 19141', 'Philadelphia, PA 19142', 'Philadelphia, PA 19143', 'Philadelphia, PA 19144', 'Philadelphia, PA 19145', 'Philadelphia, PA 19146', 'Philadelphia, PA 19147', 'Philadelphia, PA 19148', 'Philadelphia, PA 19149', 'Philadelphia, PA 19150', 'Philadelphia, PA 19151', 'Philadelphia, PA 19152', 'Philadelphia, PA 19153', 'Philadelphia, PA 19154', 'Philadelphia, PA 19155', 'Philadelphia, PA 19160', 'Philadelphia, PA 19161', 'Philadelphia, PA 19162', 'Philadelphia, PA 19170', 'Philadelphia, PA 19171', 'Philadelphia, PA 19172', 'Philadelphia, PA 19173', 'Philadelphia, PA 19175', 'Philadelphia, PA 19176', 'Philadelphia, PA 19177', 'Philadelphia, PA 19178', 'Philadelphia, PA 19179', 'Philadelphia, PA 19181', 'Philadelphia, PA 19182', 'Philadelphia, PA 19183', 'Philadelphia, PA 19184', 'Philadelphia, PA 19185', 'Philadelphia, PA 19187', 'Philadelphia, PA 19188', 'Philadelphia, PA 19190', 'Philadelphia, PA 19191', 'Philadelphia, PA 19192', 'Philadelphia, PA 19193', 'Philadelphia , PA 19194', 'Philadelphia , PA 19195', 'Philadelphia, PA 19196', 'Philadelphia, PA 19197', 'Philadelphia , PA 19244', 'Philadelphia , PA 19255', 'Paoli, PA 19301', 'Atglen, PA 19310', 'Avondale, PA 19311', 'Berwyn, PA 19312', 'Brandamore, PA 19316', 'Chadds Ford, PA 19317', 'Chatham, PA 19318', 'Cheyney, PA 19319', 'Coatesville, PA 19320', 'Cochranville, PA 19330', 'Concordville, PA 19331', 'Devon, PA 19333', 'Downingtown, PA 19335', 'Concordville , PA 19339', 'Concordville , PA 19340', 'Exton, PA 19341', 'Glen Mills, PA 19342', 'Glenmoore, PA 19343', 'Honey Brook, PA 19344', 'Immaculata, PA 19345', 'Kelton, PA 19346', 'Kemblesville, PA 19347', 'Kennett Square, PA 19348', 'Landenberg, PA 19350', 'Lewisville, PA 19351', 'Lincoln University, PA 19352', 'Lionville, PA 19353', 'Lyndell, PA 19354', 'Malvern, PA 19355', 'Mendenhall, PA 19357', 'Modena, PA 19358', 'New London, PA 19360', 'Nottingham, PA 19362', 'Oxford, PA 19363', 'Parkesburg, PA 19365', 'Pocopson, PA 19366', 'Pomeroy, PA 19367', 'Sadsburyville, PA 19369', 'Suplee, PA 19371', 'Thorndale, PA 19372', 'Thornton, PA 19373', 'Toughkenamon, PA 19374', 'Unionville, PA 19375', 'Wagontown, PA 19376', 'West Chester, PA 19380', 'West Chester, PA 19381', 'West Chester, PA 19382', 'West Chester, PA 19383', 'West Chester, PA 19388', 'West Grove, PA 19390', 'Westtown, PA 19395', 'Southeastern, PA 19397', 'Southeastern, PA 19398', 'Southeastern, PA 19399', 'Norristown, PA 19401', 'Norristown, PA 19403', 'Norristown, PA 19404', 'Bridgeport, PA 19405', 'King Of Prussia, PA 19406', 'Audubon, PA 19407', 'Eagleville, PA 19408', 'Fairview Village, PA 19409', 'Eagleville , PA 19415', 'Arcola, PA 19420', 'Birchrunville, PA 19421', 'Blue Bell, PA 19422', 'Cedars, PA 19423', 'Blue Bell , PA 19424', 'Chester Springs, PA 19425', 'Collegeville, PA 19426', 'Conshohocken, PA 19428', 'Conshohocken , PA 19429', 'Creamery, PA 19430', 'Devault, PA 19432', 'Frederick, PA 19435', 'Gwynedd, PA 19436', 'Gwynedd Valley, PA 19437', 'Harleysville, PA 19438', 'Hatfield, PA 19440', 'Harleysville , PA 19441', 'Kimberton, PA 19442', 'Kulpsville, PA 19443', 'Lafayette Hill, PA 19444', 'Lansdale, PA 19446', 'Lederach, PA 19450', 'Mainland, PA 19451', 'Mont Clare, PA 19453', 'North Wales, PA 19454', 'North Wales , PA 19455', 'Oaks, PA 19456', 'Parker Ford, PA 19457', 'Phoenixville, PA 19460', 'Plymouth Meeting, PA 19462', 'Pottstown, PA 19464', 'Pottstown, PA 19465', 'Royersford, PA 19468', 'Saint Peters, PA 19470', 'Sassamansville, PA 19472', 'Schwenksville, PA 19473', 'Skippack, PA 19474', 'Spring City, PA 19475', 'Spring House, PA 19477', 'Spring Mount, PA 19478', 'Uwchland, PA 19480', 'Valley Forge, PA 19481', 'Valley Forge, PA 19482', 'Valley Forge , PA 19483', 'Valley Forge, PA 19484', 'Valley Forge, PA 19485', 'West Point, PA 19486', 'King Of Prussia, PA 19487', 'Norristown, PA 19488', 'Norristown, PA 19489', 'Worcester, PA 19490', 'Zieglerville, PA 19492', 'Valley Forge , PA 19493', 'Valley Forge , PA 19494', 'Valley Forge , PA 19495', 'Valley Forge , PA 19496', 'Adamstown, PA 19501', 'Bally, PA 19503', 'Barto, PA 19504', 'Bechtelsville, PA 19505', 'Bernville, PA 19506', 'Bethel, PA 19507', 'Birdsboro, PA 19508', 'Blandon, PA 19510', 'Bowers, PA 19511', 'Boyertown, PA 19512', 'Centerport, PA 19516', 'Douglassville, PA 19518', 'Earlville, PA 19519', 'Elverson, PA 19520', 'Fleetwood, PA 19522', 'Geigertown, PA 19523', 'Gilbertsville, PA 19525', 'Hamburg, PA 19526', 'Kempton, PA 19529', 'Kutztown, PA 19530', 'Leesport, PA 19533', 'Lenhartsville, PA 19534', 'Limekiln, PA 19535', 'Lyon Station, PA 19536', 'Maxatawny, PA 19538', 'Mertztown, PA 19539', 'Mohnton, PA 19540', 'Mohrsville, PA 19541', 'Monocacy Station, PA 19542', 'Morgantown, PA 19543', 'Mount Aetna, PA 19544', 'New Berlinville, PA 19545', 'Oley, PA 19547', 'Pine Forge, PA 19548', 'Port Clinton, PA 19549', 'Rehrersburg, PA 19550', 'Robesonia, PA 19551', 'Shartlesville, PA 19554', 'Shoemakersville, PA 19555', 'Strausstown, PA 19559', 'Temple, PA 19560', 'Topton, PA 19562', 'Virginville, PA 19564', 'Wernersville, PA 19565', 'Womelsdorf, PA 19567', 'Reading, PA 19601', 'Reading, PA 19602', 'Reading, PA 19603', 'Reading, PA 19604', 'Reading, PA 19605', 'Reading, PA 19606', 'Reading, PA 19607', 'Reading, PA 19608', 'Reading, PA 19609', 'Reading, PA 19610', 'Reading, PA 19611', 'Reading, PA 19612', 'Reading, PA 19640');

echo "<ul>\n";
foreach($strArrayOfLocations as $strLocation){
     $strSEOTitle = str_replace(VARCSZ,$strLocation,$strTitle);
     $strSEOContent = str_replace(VARCSZ,$strLocation,$strContent);

     $postObject = array();
     $postObject['post_title'] = $strSEOTitle;
     $postObject['post_content'] = $strSEOContent;
     $postObject['post_status'] = 'publish';
     $postObject['post_author'] = 1;
     $postObject['post_type'] = 'page';
     $postObject['post_category'] = array(0);
     
     $iPostID = wp_insert_post( $postObject);
     echo "<li>$iPostID created for $strLocation</li>\n";
}
echo "</ul>\n";
?>

Now if you wanted to get really fancy … add some code to list all of the city/state/zip combos for the country (or subset there-of). And for the other attributes you can set on a post, see https://developer.wordpress.org/reference/functions/wp_insert_post/.

March 5, 2018

Extracting RPM Packages

I’ve encountered a few scenarios of late where I couldn’t install an RPM package but needed its content. One is the security config at work where I have sudo access for cp but not install rights. Sigh! But more recently, I needed to compare a library from an updated package to the currently installed one. Listing package content confirms it is the same file name and path.

[root@fedora02 tmp]# rpm -q --filesbypkg -p ./mariadb-libs-10.2.13-2.fc27.i686.rpm
mariadb-libs              /etc/my.cnf.d/client.cnf
mariadb-libs              /usr/lib/.build-id
mariadb-libs              /usr/lib/.build-id/7c
mariadb-libs              /usr/lib/.build-id/7c/c8e65deafbdcc28b3089da60f295a6f757cf4f
mariadb-libs              /usr/lib/libmariadb.so.3

Extracting the rpm allowed me to actually compare the files, swap back and forth to see which worked, etc.

[lisa@fedora tmp]# rpm2cpio mariadb-libs-10.2.13-2.fc27.x86_64.rpm | cpio -idmv

February 19, 2018

WebLogic LDAP Authentication

Configuring an LDAP Authentication provider in WebLogic (version 11g used in this documentation)

In configuring LDAP authentication, I add a new authentication provider but continue to use the local provider for the system account under which WebLogic is launched. Partially because I don’t really use WebLogic (there’s an Oracle app with its own management site that runs within WebLogic – very small number of users, so our configuration is in no way optimized), but partially because using a network-sourced system account can prevent your WebLogic instance from launching. If your config isn’t right, or if the network is down, or a firewall gets in the way, or the LDAP server is down …. Your WebLogic fails to launch because its system ID is not validated.

WebLogic Configuration

Lock & Edit the site so we can make changes. On the left-hand pane, scroll down & find Security Realms

Go into your realm, select the “providers” tab. Supply a name for the provider (I included “LDAP” in the name to ensure it was clear which provider this was – may even want to specify something like “CompanyXLDAPAuthProvider”)

Select type “LDAPAuthenticator” for generic LDAP (I was using Sun DSEE, and moved to Oracle OUD without changing the authenticator type). Click OK to create.

Change the control flag on your default authenticator. Click the hyperlink for the default provider. On the “Common” tab, change the “Control Flag” to “SUFFICIENT” and save.

Click the hyperlink for the newly created provider. On the “Common” tab, change the “Control Flag” to “SUFFICIENT” and save.

Select the “Provider specific” tab.

Connection

Host: <your LDAP server>

Port: 636

Principal: <Your system account, provided when you request access to the LDAP directory>

Credentials: <Your system account password>

Confirm Credentials: <same as credentials>

SSLEnabled: Check this box (for testing purposes, i.e. if you are unable to connect with these instructions as provided, you can set the port to 389 and not check this box to help with troubleshooting the problem. But production authentication needs to be done over SSL)

Users

User Base DN: <get this from your LDAP admin. Ours is “ou=people,o=CompanyX”)

All User Filter: (&(objectClass=inetOrgPerson))

For applications with a single group restricting valid users, you can use the filter: (&(objectClass=inetOrgPerson)(isMemberOf=cn=GroupNameHere,ou=groups,o=CompanyX))

Users from name filter: (&(uid=%u)(objectClass=inetOrgPerson))

User Search Type: subtree (onelevel may be fine, but verify with your LDAP administrator)

User Name Attribute: uid

User Object Class: inetOrgPerson

Use Retrieved User Name as Principal – I didn’t select this, don’t really know what it does

Groups

Group Base DN: <another one to get from your LDAP admin. Ours is “ou=groups,o=CompanyX”>

All Groups Filter: (&(objectClass=groupOfUniqueNames))

If your group names all have the same prefix, you could limit “all” groups to just your groups with a filter like (&(objectClass=groupOfUniqueNames)(cn=MyApp*))

Group from name filter: (&(cn=%g)(objectclass=groupofuniquenames))

Group search scope: subtree (again, onelevel may be fine)

Group membership searching: <We select ‘limited’ because there are no nested groups in the LDAP directories. If you need to resolve nested group memberships, this and the next value will be different>

Max group membership search level: 0

Ignore duplicate membership: Doesn’t really matter as we don’t have duplicates. I left this unchecked.

Static groups

Static group Attribute name: cn

Static group Object Class: groupOfUniqueNames

Static Member DN Attribute: uniqueMember

Static Group DNs from Member filter: (&(uniquemember=%M)(objectclass=groupofuniquenames))

Dynamic Groups this section is left blank/defaults as we don’t use dynamic groups

General

Connection Pool Size: Ideal value dependent on your anticipated application load – default of 6 is a good place to start.

Connect timeout: Default is 0. I don’t know if this is something particular to WebLogic, but I generally use a 15 or 30 second timeout. If the server hasn’t responded in that period, it is not going to respond and there’s no need to hang the thread waiting.

Connection Retry Limit: Default is 1, this should be sufficient but if you see a lot of connection errors, either increase the connect timeout or increase this retry limit

Parallel Connect Delay: 0 (default) is fine

Result time limit: 0 (default) is OK. On my the LDAP server, there is no time limit for searches. Since WebLogic is making very simple searches, you could put a limit in here to retry any search that takes abnormally long

Keep Alive Enabled: Please do not enable keep alive unless you have a specific need for it. Bringing up a new session uses slightly more time/resources on your app server than re-using an existing connection but that keep alive is a LOT of extra “hey, I’m still here” pings against the LDAP servers

Follow Referrals: Un-check this box unless your LDAP admin tells you referrals are in use and should be followed.

Bind Anonymously on referrals: Leave unchecked if you are not following referrals. If referrals are used and followed – ask the LDAP admin how to bind

Propagate cause for logon exception: I check this box because I *want* the ugly LDAP error code that explains why the logon failed (49 == bad user/password pair; 19 == account locked out). But no *need* to check the box

Cache Related Settings: This is something that would require more knowledge of WebLogic than I have ?

If you enable caching, you may not see changes for whatever delta-time is the cache duration. So, the defaults of enabling cache & retaining it for 60 seconds wouldn’t really create a problem. If you set the cache duration to one day (a silly setting to make the problem cache can create clear) …. If I logged into your application at 2PM, did a whole bunch of work, went home, came back the next morning & saw my “your password is about to expire” warning … so go out to the password portal and change my password. Reboot, get logged back into my computer …. and try to access your application, I will get told my password is invalid. I could try again, even type what I *know* is my password into notepad & paste it into your app … still not able to log on. My old password, were I to try it, would work … but otherwise I’d have to wait until after 2PM before my new password would work.

Group membership changes could be a problem too – with the same 24 hour cache, if I am a valid user of your application who signs in at 2PM today, but my job function changes tomorrow morning & my access is revoked … I will still have application access until the cache expires. I am not sure if WebLogic does negative caching – basically if I am *not* a user, try to sign in and cannot because I lack the group membership & get an access request approved *really quickly* to become a group member, I may still be unable to access the application until the “Lisa is not a member of group XYZ” cache expires. If WebLogic does not do negative caching, then this scenario is not an issue.

So you might be able to lower utilization on your app server & my LDAP server by enabling cache (if your app, for instance, re-auths the object **each time the user changes pages** or something, then caching would be good). If you are just checking authentication and authorization on logon … probably not going to do much to lower utilization. But certainly keep the cache TTL low (like minutes, not days).

GUID Attribute: nsUniqueID

Establishing The SSL Trust

For encryption to be negotiated with the LDAP servers, you need to have a keystore that includes the public keys from the CA used to sign the LDAP server cert. Obtain the base 64 encoded public keys either from the PKI admin or the LDAP admin. Place these file(s) on your server – I use the /tmp/ directory since they are no longer needed after import.

From the domain structure section, select: Environment=>Servers and select your server. On the “Configuration” tab, click the keystores sub-tab. If you are not already using a custom trust, you need to change they keystore type to use a custom trust (and specify a filename in a path to which the WebLogic account has access – keystore type is JKS and the password is whatever you are going to make the keystore password). If you *are* already using a custom trust, just record the file name of the custom trust keystore.

Use keytool to import the CA keys to the file specified in the custom trust. The following examples use a root and signing CA from my company, the CA chain which signs our LDAP SSL certs.

./keytool -import -v -trustcacerts -alias WIN-ROOT -file /tmp/WIN-ROOT-CA.b64 -keystore /path/to/the/TrustFile.jks -keypass YourKeystorePassword -storepass YourKeystorePassword

./keytool -import -v -trustcacerts -alias WIN-WEB -file /tmp/WIN-WEB-CA.b64 -keystore /path/to/the/TrustFile.jks -keypass YourKeystorePassword -storepass YourKeystorePassword

*** Under advanced, I had to check off “Use JSSE SSL” for SSL to work. Without that checked off, I got the following error in the log:

####<Feb 23, 2018 10:11:36 AM EST> <Notice> <Security> <server115.CompanyX.com> <AdminServer> <[ACTIVE] ExecuteThread: ’12’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<WLS Kernel>> <> <58b1979606d98df5:292a2ff6:161c336d0ba:-8000-0000000000000007> <1519398696289> <BEA-090898> <Ignoring the trusted CA certificate “CN=WIN-WEB-CA,DC=CompanyX,DC=com”. The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

####<Feb 23, 2018 10:11:36 AM EST> <Notice> <Security> <server115.CompanyX.com> <AdminServer> <[ACTIVE] ExecuteThread: ’12’ for queue: ‘weblogic.kernel.Default (self-tuning)’> <<WLS Kernel>> <> <58b1979606d98df5:292a2ff6:161c336d0ba:-8000-0000000000000007> <1519398696289> <BEA-090898> <Ignoring the trusted CA certificate “CN=WIN-Root-CA”. The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

An alternate solution would be to update your WebLogic instance – there are supposedly patches, but not sure which rev and it wasn’t worth trial-and-erroring WebLogic patches for my one WebLogic instance with a dozen users.

Whew, now save those changes. Activate changes & you will probably need to restart your WebLogic service to have the changes go into effect. You can go into the roles & add LDAP groups as — specifically, I added our LDAP group’s CN to the administrators WebLogic role.

January 30, 2018

Active Directory Federation Services (ADFS) Relying Party Trust Cert Expiry

At work, we received a critical ticket for an application that was unable to authenticate to ADFS. Nothing globally wrong – other applications are authenticating. A long call later, we discovered that the app’s certificate has expired. Why would the application not monitor their certificate expiry dates?? That’s an excellent question, but not one over which I have any control.

I can monitor their certs on our side. So I wrote a quick powershell script to grab certificates from the relying party trusts and alerts us if any certs will be expiring in the next 30 days. It has to run on the ADFS server – I’d love to get it moved to the automation server in the future. I expect get-adfsrelyingpartytrust returns disabled agreements. I want to filter out disabled agreements.

January 3, 2018

Spectre & Meltdown

The academic whitepapers for both of these vulnerabilities can be found at https://spectreattack.com/ — or El Reg’s article and their other article provide a good summary for those not included to slog through technical nuances. There’s a lot of talk about chip manufacturer’s stock drops and vendor patches … but I don’t see anyone asking how bad this is on hosted platforms. Can I sign up for a free Azure trial and start accessing data on your instance? Even if they isolate free trial accounts (and accounts given to students through University relationships), is a potential trove of data worth a few hundred bucks to a hacker? Companies run web storefronts that process credit card info, so there’s potentially profit to be made. Hell, is the data worth a few million to some state-sponsored entity or someone getting into industrial espionage? I’m really curious if MS uses the same Azure farms for their hosted Exchange and SharePoint services.

While Meltdown has patches (not such a big deal if you’re use cases are GPU intensive games, but does a company want a 30% performance hit on business process servers, automated build and testing machines, data mining servers?), Spectre patches turn IT security into TSA regulations. We can make a patch to mitigate the last exploit that occurred. Great for everyone else, but doesn’t help anyone who experienced that last exploit. Or the people about to get hit with the next exploit.

I wonder if Azure and AWS are going to give customers a 5-30% discount after they apply the performance reducing patch? If I agreed to pay x$ for y processing capacity, now they’re supplying 0.87y … why wouldn’t I pay 0.87x$?