Category: Technology

May 4, 2018

Load Balance and Failover Sendmail Mailertable Relays

A coworker asked me today how to get the mailertable relays to load balance instead of fail over. Trick is to think beyond sendmail. The square brackets around hosts tell sendmail not to check for an MX record (you’re generally using an A record, so this saves a tiny little bit of time … not to mention *if* there is an MX record there, it creates a whole heap-o confusion). *But* the MX lookup is right useful when setting up load balanced or failover relay targets.

Single host relay in the mailertable looks like this:
yourdomain.gTLD relay:[somehost.mydomain.gTLD]

If you want to fail over between relays (that is try #1, if it is unavailable try #2, and so on), you can stay within the mailertable and use:
yourdomain.gTLD relay:[somehost.mydomain.gTLD]:[someotherhost.mydomain.gTLD]

Or even try direct delivery and fail back to a smart host:
yourdomain.gTLD relay:%1:smart-host

But none of this evenly distributes traffic across multiple servers. The trick to load balancing within the mailertable is to create equal weight MX records in your domain to be used as the relay.

In ISC Bind, this looks like:
yourdomainmailrouting.mydomain.gTLD IN MX 10 somehost.mydomain.gTLD.
yourdomainmailrouting.mydomain.gTLD IN MX 10 somehost.mydomain.gTLD.

Once you have created the DNS records, simply use the MX record hostname in your mailertable:

yourdomain.gTLD relay:yourdomainmailrouting.mydomain.gTLD

By leaving out the square brackets, sendmail will resolve an MX record for ‘yourdomainmailrouting.mydomian.gTLD’, find the equal weight MX records, and do the normal sendmail thing to use both.

April 20, 2018

Data Privacy

Facebook is getting a lot of attention for the information it gathers and how well it secures personal data you provide. We should look just as intently at other companies. Some provide services to individuals in exchange for advertising data, and some provide advertising targeting services without offering anything to the individuals being tracked.

LinkedIn — Maybe because “professional” information about oneself does not feel as private as that which is shared on Facebook, LinkedIn gets overlooked a bit. The companies I’ve worked for and titles I’ve held almost seem like public records. You can download a copy of “your data” (like Facebook, this is not apt to contain meta-data they’ve gathered regarding you – just data you have submitted to the site). In your settings, use the privacy tab and scroll down to “How LinkedIn uses your data” – the first selection is to download your data.

Nothing stunning – a list of contacts, my various employers and titles. But LinkedIn is trying to slurp in my entire contact list, maintain a web of people who know people, and allow advertisers to target users. There’s a whole tab apart from your privacy settings to control how your data is used for advertising purposes. “Advertisers” seem to be corporate hiring agents and recruiters, so this marketing is not always mentally classified as “advertising”.

LinkedIn also has a setting which allows you to opt-out (mine was on, and I’ve never opted in so I assume it is an opt-out deal) of having some of your data made available to third parties for policy and academic research.

And remember that Facebook Pixel? LinkedIn wants to track information about “websites you’ve visited” and “information you’ve shared with businesses” to show you more relevant jobs and ads.

Beyond the data feeling less private, having high-paying jobs that need my exact skill set and tend to hire people with my browsing history … well, that feels like a score compared to Facebook’s ad trying to coerce me once again to buy a pair of roller skates I already decided wouldn’t work for my daughter. Even if you’re not actively interested in changing jobs, it is nice to feel wanted. But that’s a nice veneer to data hording, analysis, and target marketing. They’ve even got a peculiar setting under the “Communications” tab that wants to use algorithms to analyze your messages to formulate suggested replies. This too seems to be an opt-out setting.

Google — no one uses Google+ (pity, that) but Google amasses information from searches, e-mails, Hangouts, Android phones. You can request an archive of your data through https://takeout.google.com — it takes a long time for the archive to be built, and it was an incredible amount of data. A few +1s from mis-clicks that there is no immediately obvious way to delete. “Bookmarks” that all appear to be map locations. A calendar that apparently was syncing with my home server back in 2009 since that’s the create date on all of the items. A whole folder for Chrome with 75 meg of browsing history and another meg of bookmarks (a meg of text is a *lot* of data, but I *love* that my bookmarks sync between devices). A handful of contacts that I assume my husband created in our shared account. The totality of every conversation I’ve ever had in Hangouts. Some Google Keep notes that I also assume are my husband’s from our shared account. My entire GMail mailbox, which is an obvious data source. The very tiny set of profile data I actually shared with Google.

Hell, Google has years worth of location data that I guess comes from my phone (it’s got fairly accurate lat/long coordinates, so GPS is the likely source). Following Google’s directions to delete the data didn’t work either (on the map, hit the hamburger menu then scroll ALL THE WAY DOWN to the ‘history’ selection”. Google both claims to have no history data for me and has 423 places on my timeline. Sooo, yeah, that would be history data. I finally managed to delete the stuff through my phone. There is a “Google Settings” app. Select “Location” from it, then “Google Location History”. There is a “Manage Activities” selection (use Google Maps to open it). Confirm you don’t want to use location history because, of course, it asks you to turn it on. Then use the hamburger menu button and select “Settings”. Waaay down at the bottom, there’s an option to delete all history or a date range of history. A couple of warnings later, the timeline map shows no data.

Then there are the photos. Gig after gig of photos. I had an Android phone that went into a reboot loop. I spent a few days wiping and reloading my phone, then failed back to an old phone. One of those iterations, evidently, slurped up all of the photos on my SD card because companies *want* your data. So the initial phone setup pushes you to backup your data, sync up your media, and generally upload ‘stuff’. One erroneous click and they’ve got metadata they’ll be able to keep forever. And there’s no readily apparent way to delete everything at once either. I’ve spent days on the web site deleting a couple hundred photos at a time. Not fun. Click the first picture, scroll down a bit, hold shift and click another picture. If you’re lucky, you didn’t select more than whatever the limit is (guessing 500) and you’ll get “389 Selected” in the upper left hand corner. At which point, you can click the delete and remove that chunk of photos. If you are not lucky, you get “2 Selected” and have to try again.

Ceasing data collection is much easier than removing data they’ve already grabbed. From your account settings, elect to “Manage your Google activity”. Then go into “Go To Activity Controls” and turn off (well, pause) whatever you want to turn off.

And I assume any bucket into which they’ve placed you based on previously gathered information will be retained even if you’ve deleted the underlying data.

April 17, 2018

Sendmail In CHROOT Jail

Running our sendmail mail relay in a chroot jail, ‘make’ does not update sendmail config files with changes. While I’m certain there’s a way to sort that, it’s a lot easier to go back to the old-school way of updating sendmail.cf and sendmail’s hash files.

Modifying Sendmail Configuration (sendmail.mc) on Servers with CHROOT Jailed Sendmail

SSH to server using your ID
Change to the sendmail service account (e.g. sudo /bin/su – sendmail)
Change directory to the jailed sendmail /etc/mail locatio (e.g. cd /smt00p20/sendmail/etc/mail)
vi sendmail.mc
Make requisite changes and save file
m4 sendmail.mc > sendmail.cf
Under your ID, restart sendmail using “sudo systemctl stop sendmail stop;sudo systemctl start sendmail”
Validate changes

Modifying Sendmail Data Files on Servers with CHROOT Jailed Sendmail

SSH to server using your ID
Change to the sendmail service account (e.g. sudo /bin/su – sendmail)
Change directory to the jailed sendmail /etc/mail locatio (e.g. cd /smt00p20/sendmail/etc/mail)
vi filetoedit
Make requisite changes and save file
makemap hash ./filetoedit.db < ./filetoedit
Under your ID, restart sendmail using “sudo systemctl stop sendmail stop;sudo systemctl start sendmail”
Validate changes

Where filetoedit is the name of the data file. For example, run “makemap hash ./access.db < ./access” to update the changes to the access file into access.db

April 12, 2018

Corporate Privacy

We had the Senate & House Facebook thing playing Tue/Wed – kind of background noise because anyone who didn’t realize a billion dollar corporation offering a “free” service was making money somehow on the back-end … well, didn’t bother thinking about it. But there were a few interesting tidbits (not the least of which being how many things one can claim, before a Congressional panel, to be ignorant of in spite of the topic being germane to the core operation of one’s company). The thing that stood out most to me through two days of testimony is that no one questioned the validity of the underlying service – consumerism is good, hence serving ads more likely to convince a person to buy the product is good too. I’ve got friends exclaiming that they’ve found products they’d never have known existed without targeted ads — which to me sounds like you’ve spent money on “stuff” that you didn’t need enough to go out and research something to fill that gap. Not a bad thing per se, but certainly not the laudable endeavor they make personalized advertising out to be. The flip side to presenting me ads that are more likely to convince me to buy something (assuming this is true, which dunno … sounds good on the face of it, but I tend to be put off by it and less likely to buy something) is, well, me buying more ‘stuff’ which is not always to my economic benefit.

But when they got onto the topic of Facebook Pixels (which work around people who block third party cookies), it got me thinking about the lack of control we all have over metadata. A lot of companies serve a menagerie the third party cookies from their site, and then execute a couple of third party JS trackers too. Because, as a company, it provides those third parties with data that potentially help drive sales. In theory. But do those marketing companies have some kind of non-compete clauses included in the contract they write with WIN? Can FB, Adobe, Google, etc have code embedded in a telco’s site, take the info they gather from my telco’s embedded JS code, and use it to promote non-telecom services? Cable TV even though it competes with a component of our business? An alternate telecom even though it’s a major line of our business? Is there a meta-category of “people who looked at my site but also looked at two competitors sites” v/s “people who have only looked at my site”? At least that’s governed by contract and might be tightly controlled — although I doubt an org like Facebook tracks the provenance of each bit of metadata it collects to isolate its usage, that’s based on a feeling rather than any knowledge of their internal algorithms.

Employees visiting various sites — what data to we leak and how can that be used? It’s not like my company has any sort of agreement in place to control how CompanyX uses data gathered as our employees use CompanyY’s web site. My super paranoid brain goes to the potential for abuse — a competitor using our information against us. Not the marketing company directly – like FB doesn’t sell my name and data (that’s what they make their money on after all, using my data to throw me into advertising buckets) … but the company gathering the data can get acquired. Quite a few companies use Triblio – some niche B2B tracking thing as well as Google Analytics. Now Google isn’t a big acquisition target, but some small B2B marketing company? VZ bought Yahoo, so it’s not like the only thing they’re buying is towers and fiber. VZ buys Triblio and we’re in the beginning stages of forming some new product line through some company that uses Triblio. VZ doesn’t exactly know what we’re planning to sell in six months … but they’ve got a good idea. Or even industrial espionage — it’s getting to the point it makes a lot more sense to target one of these data brokers than to target a specific company.

I get that’s a little far-fetched and more than a little paranoid. Is targeted marketing effective for companies too – are company-targeted ads convincing the company’s employees to buy more stuff on the company’s behalf?

As a company are we benefiting, harmed, or indifferent to information being gathered from our employees as they navigate the web. Employees are going to show up from an assigned netblock most of the time (i.e. from the office or VPN), so it isn’t like it’s a super-hard-to-ascertain where the individual works. Is there benefit to blocking the tracking ‘stuff’ on a corporate level (and maintaining a default browser config that blocks third party cookies)? Is there harm in blocking the trackers? The parade of horrors approach would say with Facebook/Google specifically, widespread blocking would necessitate some other revenue stream for the company (i.e. we’d end up buying 1$ hundred search passes or something). Dedicated targeted advertising companies – beyond putting a company out of business (e.g. Triblio which seems to be a dedicated marketing data company) or reducing revenue (e.g. Adobe since they’ve got other profitable lines of business), not much direct impact. A vividly imagined parade would be worldwide recession as psychologically engineered spending prompts disappear and consequently consumer spending retracts. Worst thing I can come up with is being perceived as a bunch of hypocrites who track everything customers do on their site but specifically took efforts to prevent employees from being tracked around the web.

April 11, 2018

House Facebook Hearings

Day two didn’t change my opinion from day one, but it does introduce a few new nuances. If you consider “my” information to be content (text, video, images, likes) that I’ve personally submitted to Facebook … sure I have some control over ‘my’ data. Not the granular level of control I would prefer, not always readily usable control, and like all things on the Internet (including user data downloaded by a third party), I don’t have control over what people who have access to my data can subsequently do with it. But Facebook has a whole other realm of my data — metadata from images or videos, geo-location information (maybe IP-based with low accuracy, maybe GPS with high accuracy), how long I spent looking at what content, what time of day I log on … and that’s just information gathered directly from my usage of the web site.

Block third party cookies in your web browser (seriously, do it) and see how often adobetm.com, disqus.com, doubleclick.net, facebook.com, google.com, twitter.com, and youtube.com show up in the blocked cookie list.

Particular interesting tidbit from the House proceedings was the “Facebook Pixel” – so named because of the single transparent pixel served from a Facebook site if the actual script-based tracking is blocked by the browser. It’s a little code snippet with a function that allows the site owner to track specific actions within the site (i.e. there’s a difference between “someone who visited my site two months ago and has not been back”, “someone who visits my site every other day”, and “someone who spent 100 bucks at my site”) using the standard events (currently nine) and a custom catch-all event. Advertisers then have target audiences created for their custom site data — this means the advertiser cannot see that I visited their site twice a week or spent over ten bucks in the past quarter but they can elect to spend money on ads delivered to people who have visited their site twice in the past week or not deliver ads to people who purchased merchandise in the last month.

Looking through the developer documentation, that is a LOT of really personal information about me that I am not consenting to provide Facebook (in fact, they’re getting that information for people who aren’t even account holders – just their “match pixel to user” algorithm falls out and creates some phantom profile to track the individual instead of landing on a known user’s account). And it’s a lot of really personal information over which I have no control. There’s a difference between opting out of interest based advertising and opting out of tracking. And how exactly can I go about

In the particular case of the Facebook pixel, the script function is housed on a Facebook server. You can pretty easily prevent this bit of tracking. Add a line in your hosts file (/etc/hosts, c:\windows\system32\drivers\etc\hosts) to map the hosting server to your loopback address:

127.0.0.1 connect.facebook.net

Voila, fbq is no longer a valid function. I haven’t noticed any adverse impact to actual Facebook use (although I assume were a significant number of people to block their script host … they’d move it over to a URI that impacted site usage).

Facebook’s debugging tool, meant for advertisers and their developers, confirms the code failed to execute. Browser specific if the <noscript> content is loaded or not – it’s not in my case.

The same approach can be used to block a number of tracking services – script content served from dedicated servers don’t impact general web usability.

127.0.0.1 connect.facebook.net
127.0.0.1 www.google-analytics.com
127.0.0.1 disqus.com
127.0.0.1 cse.google.com
127.0.0.1 bat.bing.com
127.0.0.1 www.googleadservices.com
127.0.0.1 sjs.bizographics.com
127.0.0.1 www.googletagmanager.com
127.0.0.1 chimpstatic.com
127.0.0.1 cdnjs.cloudflare.com
127.0.0.1 api.cartstack.com
127.0.0.1 js-agent.newrelic.com
127.0.0.1 se.monetate.net
127.0.0.1 assets.adobetm.com
127.0.0.1 tribl.io

April 10, 2018

Senate Facebook Hearings

The hearing today reminds me of digital discovery pre-Zubulake – bunch of folks who I suspect might be investigating edgy technologies to ditch cuneiform script making rulings regarding how search and seizure case-law applies to electronic data. Not terribly encouraging that they intend to draft legislation controlling … what? Digital privacy in general? Social media platforms? Here’s hoping a good number of Congresspersons take Scheindlin’s initiative to educate themselves about that on which they seek to rule.

Something that stands out to me is how much of the platform’s operations, litigation, and regulation about which Zuckerberg claims not to know anything. I get not wanting to provide an answer that looks bad for your company, not wanting to provide inaccurate information in a Congressional hearing … but I expected they would have come up with a more reasonable boilerplate fob off answer than, essentially, “I don’t know about that stuff”

The anti-trust thread is an interesting path to go down, although I doubt Graham will follow that path. Shame, too. I had great hopes for Google+ — backed by a company with enough money to compete, enhancing Google’s current ad platform, and the idea of circles to provide granular control of who can see what. An idea which would have vastly limited the impact here. In Google+, I could avoid sharing a lot of personal information with vague acquaintances and distant family members. Heck, close family too if they’re the types who are always downloading rubbish and infecting their computer.

Consumerism and advertising is a priori accepted as a good thing. Not shocking, considering the way of American society, but it really stood out to me throughout the testimony that no one questions the benefit of having stuff more effectively marketed, to having ads that are more apt to result in a sale. They’ve spent enormous sums of money, dedicated incredible human capital to delivering an ad that is more likely to show a shirt I like. Why is that a good thing? I have clothes. If I needed more, I would either go to a store or search online. I understand why a business wants to sell me a shirt … but how is more effectively separating me from my earnings a personal boon??

And the American public is having a good self-education week. There’s interest in taint teams from Cohen yesterday, and today we’re understanding the actual business model of large tech companies — the nuance between “selling my data” and “using my data to form advertising profiles and sell my services in presenting advertising based on those advertising profiles”. Back when the ISPs wanted to be able to commoditize web history, I encountered a lot of uproar about literally selling someone’s browsing history. Which – and no offense meant – your browsing history? Not a thrilling read. Taking your browsing history and turning it into profiles, then using those profiles to sell services presenting ads to customers. Objecting to “selling my data” provides a strawman for the companies to tear down (as Zuckerberg did several times with “we don’t do that”).

Hopefully people are gaining a more complete understanding of what information is available through the “Facebook Platform” … and that you are trusting not just Facebook but the other company to act in good faith regarding your privacy. When the ToS says they may sell data or analytics to a third party … well, they may well do that. What does that third party do with the data? How much control can you, Facebook, or the app developer exert over the data sold to the third party? Not a whole lot.

Finally – the bigger question that doesn’t get asked … how can Americans insulate themselves from having personal information used to foment discontent? How can we get better and analyzing “news” and identifying real fake news. Not Trump-style FAKE NEWS which basically means “something I don’t like hearing” but actual disinformation.

April 8, 2018

Running Sendmail In A CHROOT Jail

My employer’s OS-support model restricts root access to members of the Unix support team. Applications are normally installed into a package directory and run under a service ID. While this model works well for most applications, sendmail is tightly integrated into the OS and is not readily built into an application directory. We attempted to run sendmail as a non-root user with modified permissions on application directories such as /var/spool/mqueue – this worked, until OS patches were applied and permissions reset. We needed a way to run sendmail as a non-root user and allow the OS support team to patch servers without impacting the sendmail application.

Chroot is a mechanism that uses a supplied directory path as the environment’s root directory. The jailed process, and its children, should not be able to access any part of the file hierarchy outside of the new root. As a security mechanism, the approach has several flaws – abridged version of the story is that it’s not terribly difficult to break out of jail here; and there are far more effective security approaches (e.g. SELinux). However, chroot jails have their own copies of system owned directories (such as /var/spool/mqueue), binaries, and libraries. Using a chroot jail will allow us to maintain a sendmail application in the package directory that is not impacted by OS updates.

This approach works on relaying mail servers (i.e. those that queue mail to /var/spool/mqueue and send it on its merry way). If sendmail is hosting mailboxes, there are additional challenges to designing a chroot configuration that actually drops messages into mailbox files that users can access.

Preliminaries: To copy/paste, view the single article. Create a service account under which sendmail will run. The installation directory should be owned by the service account user.

Set up the chroot jail location in the installation directory. In this example, that directory is /smt00p20.

mkdir /smt00p20/sendmail
mkdir /smt00p20/sendmail/dev
mkdir /smt00p20/opendkim

We need a null and random in the sendmail jail. On a command line, run:

# Create sendmail jail /dev/null
mknod /smt00p20/sendmail/dev/null c 1 3
# Create sendmail jail /dev/random
mknod /smt00p20/sendmail/dev/random c 1 8

We need an rsyslog socket added under each jail. In /etc/rsyslog.conf, add the following:

# additional log sockets for chroot'ed jail
# Idea from http://www.ispcolohost.com/2014/03/14/how-to-get-syslog-records-of-chrooted-ssh-sftp-server-activity/
$AddUnixListenSocket /smt00p20/sendmail/dev/log
$AddUnixListenSocket /smt00p20/opendkim/dev/log

Additionally, these instructions assume both sendmail and sendmail-cf have been installed on the server. If they have not, you can download the RPMs, unpack them, and copy the files to the appropriate relative jail locations.

Chrooting Sendmail

Logged in with the sendmail ID, ensure you have a .bash_profile that loads .bashrc

-bash-4.2$ cat ~/.bash_profile
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

Edit ~/.bashrc and add the following, where smt00p20 is the appropriate installation directory, to allow copy/paste

export SENDMAILJAIL=/smt00p20/sendmail
export OPENDKIMJAIL=/smt00p20/opendkim

Log out of the service account and back in (or just source in the .bashrc file). Verify SENDMAILJAIL and OPENDKIMJAIL are set.

Copy a whole heap of ‘stuff’ into the jail – this includes some utilities used to troubleshoot issues within the jail which aren’t strictly needed. I’ve also unpacked the strace RPM to the respective directories within the jail.

mkdir $SENDMAILJAIL/bin
mkdir $SENDMAILJAIL/etc
mkdir $SENDMAILJAIL/etc/alternatives
mkdir $SENDMAILJAIL/etc/mail
mkdir $SENDMAILJAIL/etc/smrsh
mkdir $SENDMAILJAIL/lib64
mkdir $SENDMAILJAIL/lib
mkdir $SENDMAILJAIL/lib/tls
mkdir $SENDMAILJAIL/tmp
mkdir $SENDMAILJAIL/usr
mkdir $SENDMAILJAIL/usr/bin
mkdir $SENDMAILJAIL/usr/sbin
mkdir $SENDMAILJAIL/usr/lib
mkdir $SENDMAILJAIL/usr/lib/sasl2
mkdir $SENDMAILJAIL/var
mkdir $SENDMAILJAIL/var/log
mkdir $SENDMAILJAIL/var/log/mail
mkdir $SENDMAILJAIL/var/run
mkdir $SENDMAILJAIL/var/spool
mkdir $SENDMAILJAIL/var/spool/mqueue
mkdir $SENDMAILJAIL/var/spool/clientmqueue
 
cp /etc/aliases $SENDMAILJAIL/etc/
cp /etc/aliases.db $SENDMAILJAIL/etc/
cp /etc/passwd $SENDMAILJAIL/etc/
cp /etc/group $SENDMAILJAIL/etc/
cp /etc/resolv.conf $SENDMAILJAIL/etc/
cp /etc/host.conf $SENDMAILJAIL/etc/
cp /etc/nsswitch.conf $SENDMAILJAIL/etc/
cp /etc/services $SENDMAILJAIL/etc/
cp /etc/hosts $SENDMAILJAIL/etc/
cp /etc/localtime $SENDMAILJAIL/etc/
 

# If cloning an existing server, scp /etc/mail/* from source to /smt00p20/sendmail/etc/mail

# Verify the sendmail.mc has a RUNAS_USER set to the same service account you are using - the account on our servers is named 'sendmail'. Our old servers are not all set up with a runas user, and failing to have one will cause write failures to the jail /var/spool/mqueue

cp -r /etc/mail/ $SENDMAILJAIL/etc/etc/mail/
cp /usr/sbin/sendmail.sendmail $SENDMAILJAIL/usr/sbin/sendmail.sendmail

cd /smt00p20/sendmail/etc/alternatives
ln -s ../../usr/sbin/sendmail.sendmail ./mta

cd /smt00p20/sendmail/usr/sbin
ln -s ../../etc/alternatives/mta ./sendmail
ln -s ./sendmail ./newaliases
ln -s ./sendmail ./newaliases.sendmail

cd /smt00p20/sendmail/usr/bin
ln -s ../sbin/sendmail ./mailq
ln -s ../sbin/sendmail ./mailq.sendmail
ln -s ../sbin/sendmail.sendmail ./hoststat
ln -s ../sbin/sendmail.sendmail ./purgestat
ln -s ../sbin/makemap ./makemap
ln -s ./rmail.sendmail ./rmail
cp /usr/lib64/libssl.so.10 $SENDMAILJAIL/usr/lib64/libssl.so.10
cp /usr/lib64/libcrypto.so.10 $SENDMAILJAIL/usr/lib64/libcrypto.so.10
cp /usr/lib64/libnsl.so.1 $SENDMAILJAIL/usr/lib64/libnsl.so.1
cp /usr/lib64/libwrap.so.0 $SENDMAILJAIL/usr/lib64/libwrap.so.0
cp /usr/lib64/libhesiod.so.0 $SENDMAILJAIL/usr/lib64/libhesiod.so.0
cp /usr/lib64/libcrypt.so.1 $SENDMAILJAIL/usr/lib64/libcrypt.so.1
cp /usr/lib64/libdb-5.3.so $SENDMAILJAIL/usr/lib64/libdb-5.3.so
cp /usr/lib64/libresolv.so.2 $SENDMAILJAIL/usr/lib64/libresolv.so.2
cp /usr/lib64/libsasl2.so.3 $SENDMAILJAIL/usr/lib64/libsasl2.so.3
cp /usr/lib64/libldap-2.4.so.2 $SENDMAILJAIL/usr/lib64/libldap-2.4.so.2
cp /usr/lib64/liblber-2.4.so.2 $SENDMAILJAIL/usr/lib64/liblber-2.4.so.2
cp /usr/lib64/libc.so.6 $SENDMAILJAIL/usr/lib64/libc.so.6
cp /usr/lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/usr/lib64/libgssapi_krb5.so.2
cp /usr/lib64/libkrb5.so.3 $SENDMAILJAIL/usr/lib64/libkrb5.so.3
cp /usr/lib64/libcom_err.so.2 $SENDMAILJAIL/usr/lib64/libcom_err.so.2
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/libk5crypto.so.3
cp /usr/lib64/libdl.so.2 $SENDMAILJAIL/usr/lib64/libdl.so.2
cp /usr/lib64/libz.so.1 $SENDMAILJAIL/usr/lib64/libz.so.1
cp /usr/lib64/libidn.so.11 $SENDMAILJAIL/usr/lib64/libidn.so.11
cp /usr/lib64/libfreebl3.so $SENDMAILJAIL/usr/lib64/libfreebl3.so
cp /usr/lib64/libpthread.so.0 $SENDMAILJAIL/usr/lib64/libpthread.so.0
cp /usr/lib64/libssl3.so $SENDMAILJAIL/usr/lib64/libssl3.so
cp /usr/lib64/libsmime3.so $SENDMAILJAIL/usr/lib64/libsmime3.so
cp /usr/lib64/libnss3.so $SENDMAILJAIL/usr/lib64/libnss3.so
cp /usr/lib64/libnssutil3.so $SENDMAILJAIL/usr/lib64/libnssutil3.so
cp /usr/lib64/libplds4.so $SENDMAILJAIL/usr/lib64/libplds4.so
cp /usr/lib64/libplc4.so $SENDMAILJAIL/usr/lib64/libplc4.so
cp /usr/lib64/libnspr4.so $SENDMAILJAIL/usr/lib64/libnspr4.so
cp /usr/lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/usr/lib64/ld-linux-x86-64.so.2
cp /usr/lib64/libkrb5support.so.0 $SENDMAILJAIL/usr/lib64/libkrb5support.so.0
cp /usr/lib64/libkeyutils.so.1 $SENDMAILJAIL/usr/lib64/libkeyutils.so.1
cp /usr/lib64/librt.so.1 $SENDMAILJAIL/usr/lib64/librt.so.1
cp /usr/lib64/libselinux.so.1 $SENDMAILJAIL/usr/lib64/libselinux.so.1
cp /usr/lib64/libpcre.so.1 $SENDMAILJAIL/usr/lib64/libpcre.so.1
cp /usr/lib64/libnss_dns.so.2 $SENDMAILJAIL/usr/lib64/libnss_dns.so.2
cp /usr/lib64/libnss_files.so.2 $SENDMAILJAIL/usr/lib64/libnss_files.so.2

cd $SENDMAILJAIL/lib64
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib64/libnss_dns-2.17.so
ln -s ./libnss_dns-2.17.so ./libnss_dns.so.2

cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib64/libresolv-2.17.so
ln -s ./lib64/libresolv-2.17.so ./libresolv.so.2

cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib64/libnss_files-2.17.so
ln -s ./lib64/libnss_files-2.17.so ./libnss_files.so.2

cd $SENDMAILJAIL/lib 
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib/libnss_dns-2.17.so
ln -s ./lib/libnss_dns-2.17.so ./libnss_dns.so.2

cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib/libresolv-2.17.so
ln -s ./lib/libresolv-2.17.so ./libresolv.so.2

cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib/libnss_files-2.17.so
ln -s ./lib/libnss_files-2.17.so ./libnss_files.so.2

mkdir $SENDMAILJAIL/usr/lib64/sasl2
cp /usr/lib64/sasl2/* $SENDMAILJAIL/usr/lib64/sasl2/

mkdir $SENDMAILJAIL/lib64/sasl2/
cp /lib64/sasl2/* $SENDMAILJAIL/lib64/sasl2/
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/usr/lib64/sasl2/

mkdir $SENDMAILJAIL/etc/sasl2
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/etc/sasl2/


cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
ln -s ../sbin/makemap ./makemap
cp /usr/bin/rmail.sendmail $SENDMAILJAIL/usr/bin/rmail.sendmail
ln -s ./rmail.sendmail ./rmail

cp /usr/sbin/mailstats $SENDMAILJAIL/usr/sbin/mailstats
cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
cp /usr/sbin/praliases $SENDMAILJAIL/usr/sbin/praliases
cp /usr/sbin/smrsh $SENDMAILJAIL/usr/sbin/smrsh

cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libdb-5.3.so $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libfreebl3.so $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libhesiod.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3: $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/liblber-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libldap-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libnsl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libnspr4.so $SENDMAILJAIL/lib64/
cp /lib64/libnss3.so $SENDMAILJAIL/lib64/
cp /lib64/libnssutil3.so $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libplc4.so $SENDMAILJAIL/lib64/
cp /lib64/libplds4.so $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsasl2.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsmime3.so $SENDMAILJAIL/lib64/
cp /lib64/libssl.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libssl3.so $SENDMAILJAIL/lib64/
cp /lib64/libwrap.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/

cp /lib64/libdns.so.100 $SENDMAILJAIL/lib64/
cp /lib64/liblwres.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libbind9.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccfg.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccc.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisc.so.95 $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libGeoIP.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libxml2.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /bin/dig $SENDMAILJAIL/bin/

cp /lib64/libtinfo.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /bin/bash $SENDMAILJAIL/bin/

cp /bin/ls $SENDMAILJAIL/bin/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libacl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/

cp /bin/vi $SENDMAILJAIL/bin/
cp /usr/sbin/pidof $SENDMAILJAIL/usr/sbin/pidof
cp /lib64/libprocps.so.4 $SENDMAILJAIL/lib64/
cp /lib64/libsystemd.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libgcrypt.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libgpg-error.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdw.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libgcc_s.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libelf.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libbz2.so.1 $SENDMAILJAIL/lib64/
cp /bin/rm $SENDMAILJAIL/bin/

Under your ID, ensure the proper permissions are set on the chroot jail

sudo chown -R sendmail:mail /smt00p20/sendmail/
sudo chown sendmail /smt00p20/sendmail/var/spool/mqueue
sudo chmod 0700 /smt00p20/sendmail/var/spool/mqueue
sudo chmod -R go-w /smt00p20/sendmail
sudo chmod 0400 /smt00p20/sendmail/etc/mail/*.cf

Now verify it works – still under your ID as you have sudo permission to run chroot.

sudo /sbin/chroot /smt00p20/sendmail /bin/ls
# You should see a directory listing like this, not an error
bin  dev  etc  lib  lib64  tmp  usr  var

Assuming there are no problems, run sendmail:

sudo /sbin/chroot /smt00p20/sendmail /usr/sbin/sendmail -bd -q5m

Test sending mail through the server to verify proper functionality.

Unit Config: Edit the systemd unit file and add the “RootDirectory” directive

sudo vi /etc/systemd/system/multi-user.target.wants/sendmail.service

[Unit]
Description=Sendmail Mail Transport Agent
After=syslog.target network.target
Conflicts=postfix.service exim.service
Wants=sm-client.service

[Service]
RootDirectory=/smt00p20/sendmail
Type=forking
StartLimitInterval=0
# Known issue – pid causes service hang/timeout that bothers Unix guys
# https://bugzilla.redhat.com/show_bug.cgi?id=1253840
#PIDFile=/run/sendmail.pid
Environment=SENDMAIL_OPTS=-q15m
EnvironmentFile=-/smt00p20/sendmail/etc/sysconfig/sendmail
ExecStart=/usr/sbin/sendmail -bd $SENDMAIL_OPTS $SENDMAIL_OPTARG

[Install]
WantedBy=multi-user.target
Also=sm-client.service

Then run “systemctl daemon-reload” to ingest the changes.

You can now use systemctl to start and stop the sendmail service.

Chrooting opendkim

Create the chroot jail and lib64 directory, create the base directories, then add a few core Linux files so you have a bash shell:

mkdir $OPENDKIMJAIL
mkdir $OPENDKIMJAIL/lib64
mkdir $OPENDKIMJAIL/usr/lib64
mkdir $OPENDKIMJAIL/bin
mkdir $OPENDKIMJAIL/etc

cp /lib64/libtinfo.so.5 $OPENDKIMJAIL/lib64/
cp /lib64/libdl.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libc.so.6 $OPENDKIMJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $OPENDKIMJAIL/lib64/

cp /bin/bash $OPENDKIMJAIL/bin/
cp /lib64/libstdc++.so.6* $OPENDKIMJAIL/lib64
cp /lib64/libm.so.6 $OPENDKIMJAIL/lib64
cp /lib64/libgcc_s.so.1 $OPENDKIMJAIL/lib64
cp /lib64/libnss_files* $OPENDKIMJAIL/lib64/

Unpack the following RPMs:

rpm2cpio opendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio libopendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio sendmail-milter-8.14.7-5.el7.x86_64.rpm | cpio -idmv
rpm2cpio opendbx-1.4.6-6.el7.x86_64.rpm | cpio -idmv
rpm2cpio libmemcached-1.0.16-5.el7.x86_64.rpm | cpio -idvm
rpm2cpio libbsd-0.6.0-3.el7.elrepo.x86_64.rpm | cpio -idvm

Then move the unpacked files into the corresponding location in the $OPENDKIMJAIL directory.

Copy host configuration ‘stuff’ from /etc

cp /etc/aliases $OPENDKIMJAIL/etc/
cp /etc/aliases.db $OPENDKIMJAIL/etc/
cp /etc/passwd $OPENDKIMJAIL/etc/
cp /etc/group $OPENDKIMJAIL/etc/
cp /etc/resolv.conf $OPENDKIMJAIL/etc/
cp /etc/host.conf $OPENDKIMJAIL/etc/
cp /etc/nsswitch.conf $OPENDKIMJAIL/etc/
cp /etc/services $OPENDKIMJAIL/etc/
cp /etc/hosts $OPENDKIMJAIL/etc/
cp /etc/localtime $OPENDKIMJAIL/etc/

Copy some more files:

cp /lib64/libcom_err.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libcrypt.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libcrypto.so.10 $OPENDKIMJAIL/lib64/
cp /lib64/libdb-5.3.so $OPENDKIMJAIL/lib64/
cp /lib64/libfreebl3.so $OPENDKIMJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libk5crypto.so.3 $OPENDKIMJAIL/lib64/
cp /lib64/libkeyutils.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libkrb5.so.3 $OPENDKIMJAIL/lib64/
cp /lib64/libkrb5support.so.0 $OPENDKIMJAIL/lib64/
cp /lib64/liblber-2.4.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libldap-2.4.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libnspr4.so $OPENDKIMJAIL/lib64/
cp /lib64/libnss3.so $OPENDKIMJAIL/lib64/
cp /lib64/libnssutil3.so $OPENDKIMJAIL/lib64/
cp /lib64/libpcre.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libplc4.so $OPENDKIMJAIL/lib64/
cp /lib64/libplds4.so $OPENDKIMJAIL/lib64/
cp /lib64/libpthread.so.0 $OPENDKIMJAIL/lib64/
cp /lib64/libresolv.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/librt.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libsasl2.so.3 $OPENDKIMJAIL/lib64/
cp /lib64/libselinux.so.1 $OPENDKIMJAIL/lib64/
cp /lib64/libsmime3.so $OPENDKIMJAIL/lib64/
cp /lib64/libssl.so.10 $OPENDKIMJAIL/lib64/
cp /lib64/libssl3.so $OPENDKIMJAIL/lib64/
cp /lib64/libz.so.1 $OPENDKIMJAIL/lib64/
cp /usr/lib64/libssl.so.10 $OPENDKIMJAIL/usr/lib64/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/usr/lib/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/usr/lib/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/lib64/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/lib64/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/usr/lib/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/usr/lib/

cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0 $OPENDKIMJAIL/lib64/
cp $OPENDKIMJAIL/usr/lib64/libmilter.so.1.0.1 $OPENDKIMJAIL/lib64/

Configure OpenDKIM ($DKIMJAIL/etc/opendkim.conf) and populate keys (copy from server being replaced or generate new keys). Then, under your ID, run:

sudo /sbin/chroot /smt00p20/opendkim /usr/sbin/opendkim -u sendmail -v

The systemd unit file, /usr/lib/systemd/system/opendkim.service, needs to contain:

# If you are using OpenDKIM with SQL datasets it might be necessary to start OpenDKIM after the database servers.
# For example, if using both MariaDB and PostgreSQL, change "After=" in the "[Unit]" section to:
# After=network.target nss-lookup.target syslog.target mariadb.service postgresql.service

[Unit]
Description=DomainKeys Identified Mail (DKIM) Milter
Documentation=man:opendkim(8) man:opendkim.conf(5) man:opendkim-genkey(8) man:opendkim-genzone(8) man:opendkim-testadsp(8) man:opendkim-testkey http://www.opendkim.org/docs.html
After=network.target nss-lookup.target syslog.target

[Service]
RootDirectory=/smt00p20/opendkim
Type=forking
PIDFile=/smt00p20/opendkim/var/run/opendkim/opendkim.pid
EnvironmentFile=-/etc/sysconfig/opendkim
ExecStart=/usr/sbin/opendkim -u sendmail -v $OPTIONS
ExecReload=/bin/kill -USR1 $MAINPID
User=sendmail
Group=mail

[Install]
WantedBy=multi-user.target

Upgrading Sendmail – After Unix Applies Patches

This process grabs a new copy of sendmail, associated diagnostic utilities, and their dependencies from the OS installation. If you want to apply patches prior to Unix support doing so, you can stage a sendmail build (everything up to ‘make install’) and copy the files out or, if an updated RPM is in the repo but just not installed, download the RPMs, unpack them, and copy the files in. I would do that in addition to (and after) this process to ensure library updates are reflected in our jailed sendmail installation (i.e. if there’s an update to the crypto libraries, we get those updates).

cp /usr/sbin/sendmail.sendmail $SENDMAILJAIL/usr/sbin/sendmail.sendmail
cp /usr/lib64/libssl.so.10 $SENDMAILJAIL/usr/lib64/libssl.so.10
cp /usr/lib64/libcrypto.so.10 $SENDMAILJAIL/usr/lib64/libcrypto.so.10
cp /usr/lib64/libnsl.so.1 $SENDMAILJAIL/usr/lib64/libnsl.so.1
cp /usr/lib64/libwrap.so.0 $SENDMAILJAIL/usr/lib64/libwrap.so.0
cp /usr/lib64/libhesiod.so.0 $SENDMAILJAIL/usr/lib64/libhesiod.so.0
cp /usr/lib64/libcrypt.so.1 $SENDMAILJAIL/usr/lib64/libcrypt.so.1
cp /usr/lib64/libdb-5.3.so $SENDMAILJAIL/usr/lib64/libdb-5.3.so
cp /usr/lib64/libresolv.so.2 $SENDMAILJAIL/usr/lib64/libresolv.so.2
cp /usr/lib64/libsasl2.so.3 $SENDMAILJAIL/usr/lib64/libsasl2.so.3
cp /usr/lib64/libldap-2.4.so.2 $SENDMAILJAIL/usr/lib64/libldap-2.4.so.2
cp /usr/lib64/liblber-2.4.so.2 $SENDMAILJAIL/usr/lib64/liblber-2.4.so.2
cp /usr/lib64/libc.so.6 $SENDMAILJAIL/usr/lib64/libc.so.6
cp /usr/lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/usr/lib64/libgssapi_krb5.so.2
cp /usr/lib64/libkrb5.so.3 $SENDMAILJAIL/usr/lib64/libkrb5.so.3
cp /usr/lib64/libcom_err.so.2 $SENDMAILJAIL/usr/lib64/libcom_err.so.2
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/libk5crypto.so.3
cp /usr/lib64/libdl.so.2 $SENDMAILJAIL/usr/lib64/libdl.so.2
cp /usr/lib64/libz.so.1 $SENDMAILJAIL/usr/lib64/libz.so.1
cp /usr/lib64/libidn.so.11 $SENDMAILJAIL/usr/lib64/libidn.so.11
cp /usr/lib64/libfreebl3.so $SENDMAILJAIL/usr/lib64/libfreebl3.so
cp /usr/lib64/libpthread.so.0 $SENDMAILJAIL/usr/lib64/libpthread.so.0
cp /usr/lib64/libssl3.so $SENDMAILJAIL/usr/lib64/libssl3.so
cp /usr/lib64/libsmime3.so $SENDMAILJAIL/usr/lib64/libsmime3.so
cp /usr/lib64/libnss3.so $SENDMAILJAIL/usr/lib64/libnss3.so
cp /usr/lib64/libnssutil3.so $SENDMAILJAIL/usr/lib64/libnssutil3.so
cp /usr/lib64/libplds4.so $SENDMAILJAIL/usr/lib64/libplds4.so
cp /usr/lib64/libplc4.so $SENDMAILJAIL/usr/lib64/libplc4.so
cp /usr/lib64/libnspr4.so $SENDMAILJAIL/usr/lib64/libnspr4.so
cp /usr/lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/usr/lib64/ld-linux-x86-64.so.2
cp /usr/lib64/libkrb5support.so.0 $SENDMAILJAIL/usr/lib64/libkrb5support.so.0
cp /usr/lib64/libkeyutils.so.1 $SENDMAILJAIL/usr/lib64/libkeyutils.so.1
cp /usr/lib64/librt.so.1 $SENDMAILJAIL/usr/lib64/librt.so.1
cp /usr/lib64/libselinux.so.1 $SENDMAILJAIL/usr/lib64/libselinux.so.1
cp /usr/lib64/libpcre.so.1 $SENDMAILJAIL/usr/lib64/libpcre.so.1
cp /usr/lib64/libnss_dns.so.2 $SENDMAILJAIL/usr/lib64/libnss_dns.so.2
cp /usr/lib64/libnss_files.so.2 $SENDMAILJAIL/usr/lib64/libnss_files.so.2
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib64/libnss_dns-2.17.so
cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib64/libresolv-2.17.so
cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib64/libnss_files-2.17.so
cp /lib64/libnss_dns-2.17.so $SENDMAILJAIL/lib/libnss_dns-2.17.so
cp /lib64/libresolv-2.17.so $SENDMAILJAIL/lib/libresolv-2.17.so
cp /lib64/libnss_files-2.17.so $SENDMAILJAIL/lib/libnss_files-2.17.so
cp /usr/lib64/sasl2/* $SENDMAILJAIL/usr/lib64/sasl2/
cp /lib64/sasl2/* $SENDMAILJAIL/lib64/sasl2/
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/usr/lib64/sasl2/
cp /etc/sasl2/Sendmail.conf $SENDMAILJAIL/etc/sasl2/
cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
cp /usr/bin/rmail.sendmail $SENDMAILJAIL/usr/bin/rmail.sendmail
cp /usr/sbin/mailstats $SENDMAILJAIL/usr/sbin/mailstats
cp /usr/sbin/makemap $SENDMAILJAIL/usr/sbin/makemap
cp /usr/sbin/praliases $SENDMAILJAIL/usr/sbin/praliases
cp /usr/sbin/smrsh $SENDMAILJAIL/usr/sbin/smrsh

cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libdb-5.3.so $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libfreebl3.so $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libhesiod.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3: $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/liblber-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libldap-2.4.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libnsl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libnspr4.so $SENDMAILJAIL/lib64/
cp /lib64/libnss3.so $SENDMAILJAIL/lib64/
cp /lib64/libnssutil3.so $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libplc4.so $SENDMAILJAIL/lib64/
cp /lib64/libplds4.so $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsasl2.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libsmime3.so $SENDMAILJAIL/lib64/
cp /lib64/libssl.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libssl3.so $SENDMAILJAIL/lib64/
cp /lib64/libwrap.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /usr/lib64/libk5crypto.so.3 $SENDMAILJAIL/usr/lib64/

cp /lib64/libdns.so.100 $SENDMAILJAIL/lib64/
cp /lib64/liblwres.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libbind9.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccfg.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisccc.so.90 $SENDMAILJAIL/lib64/
cp /lib64/libisc.so.95 $SENDMAILJAIL/lib64/
cp /lib64/libgssapi_krb5.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libk5crypto.so.3 $SENDMAILJAIL/lib64/
cp /lib64/libcom_err.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libcrypto.so.10 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libGeoIP.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libxml2.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libidn.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libkrb5support.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libkeyutils.so.1 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /bin/dig $SENDMAILJAIL/bin/

cp /lib64/libtinfo.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /bin/bash $SENDMAILJAIL/bin/

cp /bin/ls $SENDMAILJAIL/bin/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libacl.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/

cp /bin/vi $SENDMAILJAIL/bin/
cp /usr/sbin/pidof $SENDMAILJAIL/usr/sbin/pidof
cp /lib64/libprocps.so.4 $SENDMAILJAIL/lib64/
cp /lib64/libsystemd.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdl.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libc.so.6 $SENDMAILJAIL/lib64/
cp /lib64/libcap.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libm.so.6 $SENDMAILJAIL/lib64/
cp /lib64/librt.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libselinux.so.1 $SENDMAILJAIL/lib64/
cp /lib64/liblzma.so.5 $SENDMAILJAIL/lib64/
cp /lib64/libgcrypt.so.11 $SENDMAILJAIL/lib64/
cp /lib64/libgpg-error.so.0 $SENDMAILJAIL/lib64/
cp /lib64/libdw.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libgcc_s.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpthread.so.0 $SENDMAILJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $SENDMAILJAIL/lib64/
cp /lib64/libattr.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libpcre.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libelf.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libz.so.1 $SENDMAILJAIL/lib64/
cp /lib64/libbz2.so.1 $SENDMAILJAIL/lib64/

cp /bin/rm $SENDMAILJAIL/bin/

Under your ID, ensure the proper permissions are set on the chroot jail

sudo chown -R sendmail:mail /smt00p20/sendmail/
sudo chown sendmail /smt00p20/sendmail/var/spool/mqueue
sudo chmod 0700 /smt00p20/sendmail/var/spool/mqueue
sudo chmod -R go-w /smt00p20/sendmail
sudo chmod 0400 /smt00p20/sendmail/etc/mail/*.cf

Then start sendmail and verify functionality.

Updating OpenDKIM

cp /lib64/libtinfo.so.5 $OPENDKIMJAIL/lib64/
cp /lib64/libdl.so.2 $OPENDKIMJAIL/lib64/
cp /lib64/libc.so.6 $OPENDKIMJAIL/lib64/
cp /lib64/ld-linux-x86-64.so.2 $OPENDKIMJAIL/lib64/
cp /bin/bash $OPENDKIMJAIL/bin/
cp /lib64/libstdc++.so.6* $OPENDKIMJAIL/lib64
cp /lib64/libm.so.6 $OPENDKIMJAIL/lib64
cp /lib64/libgcc_s.so.1 $OPENDKIMJAIL/lib64
cp /lib64/libnss_files* $OPENDKIMJAIL/lib64/

If there is an update to the opendkim packages, unpack the updated RPM files and move the new files into the corresponding jail locations.

rpm2cpio opendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio libopendkim-2.11.0-0.1.el7.x86_64.rpm | cpio -idmv
rpm2cpio sendmail-milter-8.14.7-5.el7.x86_64.rpm | cpio -idmv
rpm2cpio opendbx-1.4.6-6.el7.x86_64.rpm | cpio -idmv
rpm2cpio libmemcached-1.0.16-5.el7.x86_64.rpm | cpio -idvm
rpm2cpio libbsd-0.6.0-3.el7.elrepo.x86_64.rpm | cpio -idvm

March 30, 2018

Creating An OpenHAB 2.3.0 Snapshot Docker Container

We found quick instructions for creating a Docker container for the OpenHAB 2.3.0 snapshot. These instructions evidently presuppose some basic knowledge of building Docker containers, so I thought I’d write the “I don’t know what I am doing” version of the instructions. Beyond the obvious download & install Docker, then make sure it’s functional (service starts).

The linked Dockerfile is not the only thing you need. Go up a level — you need both the Dockerfile and entrypoint.sh files. Create a directory somewhere and grab these two files. Then build the container using

docker build -t oh2imagename .

I used a short, alpha-numeric only name for my image. When I used slashes as in the example, the container would not start. Then make the folders you want to map into OpenHAB2:

mkdir /some/path/to/openhab/addons
mkdir /some/path/to/openhab/conf
mkdir /some/path/to/openhab/userdata

The instructions conflate local users/groups with in-container users/groups. You do not need to create a local user. You do need to indicate the uidNumber and gidNumber for the openhab user and group. Even if you do create the local user and group, then change the /some/path/to/openhab permissions to provide full access to the user … you may well not be able to access the files. That is SELinux, not a file permission issue. The quick/dirty solution is to start the container with the privileged flag:

--privileged=true

Alternately, consult the Universal Archive of All IT Knowledge and figure out how to allow the docker service to write files where you want them. And how to access USB devices if you are trying to use something like a ZWave dongle. We went with the privileged route 🙂 The –name option is just the container name. The –net uses the host network for container communications instead of the bridge network. Saves mapping ports, although you could easily use the bridge network and map out the handful of OpenHab specific ports. The -d runs the container in detached mode. The -e sets some environment flags (used by the user/group creation script that runs upon container startup). The –tty (or -t) attaches a console. Not really used here.

docker run --privileged --name oh2containername --net=host --tty -d -e USER_ID=5555 \
 -e GROUP_ID=5555 oh2imagename

Ideally, your OpenHAB2 instance will be running. Use “docker ps” to list out the running containers. If you don’t see a container with the name supplied above … then something went wrong. You can use “docker history oh2containername” to view a quick history, but “docker logs oh2containername” will probably provide more useful information. We encountered file permission issues (as noted above, due to SELinux) which prevented the initial container setup from running. Once that was sorted, the container showed up in the running container list.

You’re ready to use it — you can access the web console using your computer’s IP address (assuming you set this container up in the host network and not the bridge — if you used the bridge, you can use “docker inspect oh2containername” and look for IPAddress under NetworkSettings) on the default port. You can ssh into the Karaf console with the default user/password on the default port. Or you can shell into the container.

docker exec -it oh2containername /bin/bash

This is a bash shell running on the OH2 container — you’ll find a lot of ‘stuff’ hasn’t been installed, and your normal command aliases won’t be present. But it’s a shell on the server and can be used to start/stop OH2.

March 28, 2018

Gathering Android Log Files

A friend has an Android application that isn’t functioning properly. On Windows or Unix, I know how to stack-trace and debug apps … so she figured I’d be all set up to do the same thing on Android too. Except I’ve never encountered a problem that required debugging Android apps. Consult the universal archive of all IT knowledge (a.k.a. Google).

I don’t have the Android developer kit installed, nor do I need it, so I elected to use “Minimal ADB and Fastboot” to grab the log data. My device did not show up without a Windows driver for the debug bridge. Once the driver was installed and the adb server restarted, my device appeared.

C:\Users\lisa>"c:\program files (x86)\Minimal ADB and Fastboot\adb.exe" kill-server

C:\Users\lisa>"c:\program files (x86)\Minimal ADB and Fastboot\adb.exe" start-server
* daemon not running; starting now at tcp:5037
* daemon started successfully

C:\Users\lisa>"c:\program files (x86)\Minimal ADB and Fastboot\adb.exe" devices
List of devices attached
g142ef0c        device

Something interesting about the adb log data – it can include an hour or more of history. Which is awesome if your app crashed a few minutes ago and you want to capture historic data, but for a reproducible error … well, there’s no need to slog through thousands of lines to find where the problem actually started. Clear the log buffers first then start capturing the adb log data:

"c:\program files (x86)\Minimal ADB and Fastboot\adb.exe" logcat -b main -b system ^
-b radio -b events -c&&"c:\program files (x86)\Minimal ADB and Fastboot\adb.exe" logcat ^
-v threadtime -f "C:\temp\SessionLog.txt"

March 23, 2018

Enable OUD Changelog Without Replication Partner

Since the Sun Directory Server Enterprise Edition went the way of, well, Sun, we’ve migrated to the Oracle Unified Directory 11g platform for the company’s pure LDAP directory. There is an Oracle Identity Management application that reads the LDAP changelog to ingest user lockout events. In production, our servers replicate with a couple of partners to provide capacity and high availability. In development / sandbox, environments … not so much. But the IDM platform still needs to read the changelog.

Oracle’s documentation tells me to enable replication … which, great, I’ve got to bring up a second, off-port, directory instance and monitor for replication failures just to get a changelog. The site does say “By using this method, you can conceivably set up replication on a standalone server, which will enable you to have access to an ECL on a standalone server.” … conceivably, but it would be nice if they’d mention how. Since all of their documentation for using the dsreplication binary includes a partner server and valid credentials over yonder … that’s a bit of a bust.

But I’ve finally worked out a technique for enabling replication just enough to get the changelog created without having to provide valid credentials on a foreign host with which replication will be established.

./dsconfig -h localhost -p 4444 -D "cn=directory manager" -j ~/pwd.txt -X -n create-replication-server --provider-name 'Multimaster Synchronization' --set replication-port:8989 --set replication-server-id:1 --type generic
 
 
./dsconfig -h localhost -p 4444 -D "cn=directory manager" -j ~/pwd.txt -X -n create-replication-domain --provider-name 'Multimaster Synchronization' --set base-dn:o=windstream --set replication-server:localhost:8989 --set server-id:1 --type generic --domain-name o=windstream

Volia, make changes and we’ve got stuff under cn=changelog.