Category: Technology

Building a Docker Image Without Internet Access (kinda)

We’ve been moving a bunch of servers into magic cloudy land. A move which comes with a whole lot of additional security restrictions (and accompanying marketing that the cloud is so much more secure … uhh, no … any host to which you used a keylogged jump server to access & it had absolutely no access to the internal or external network without a specific request and firewall rule would be equally secure. You just haven’t bothered with all of those controls before!). As such, we cannot just pull an image from Docker Hub. We also cannot just use apt-get to install/update packages.

So … how can you build a Docker image with updated software for use on these locked down boxes? Bit of a trick question — you cannot. You can, however, build such an image elsewhere and then export/import the image.

Build the image using your Dockerfile

docker build -t my_app_base .

Then export the image you created

docker save my_api_base | gzip > myapp_image.tar.gz

Download the TGZ file and transfer it to your restricted-access host. Then import the image

docker load < myapp_image.tar.gz

Verify the image loaded successfully

[user@server ~]$ docker images
REPOSITORY              TAG         IMAGE ID      CREATED            SIZE
localhost/my_app_base  latest      5z8e35z99d5z  19 minutes ago     995 MB
<none>                  <none>     5zdbcfz6393z  About an hour ago  1.01 GB

Now use docker run with your my_app_base:latest image to create a running container based on the image.

Apache HTTPD and DER Encoded Certificate

We are in the process of updating one of the web servers at work to a newer OS – along with a newer Apache HTTPD and PHP iteration. Ran into a snag just setting up the SSL web site – we couldn’t get HTTPD started with our Venafi certificate.

[Fri Jan 28 14:35:05.092086 2022] [ssl:emerg] [pid 57739:tid 139948816931136] AH02561: Failed to configure certificate hostname.example.com:443:0, check /path/to/certs/production/server.crt

[Fri Jan 28 14:35:05.092103 2022] [ssl:emerg] [pid 57739:tid 139948816931136] SSL Library Error: error:0909006C:PEM routines:get_name:no start line (Expecting: CERTIFICATE) — Bad file contents or format – or even just a forgotten SSLCertificateKeyFile?

[Fri Jan 28 14:35:05.092115 2022] [ssl:emerg] [pid 57739:tid 139948816931136] SSL Library Error: error:140AD009:SSL routines:SSL_CTX_use_certificate_file:PEM lib

The certificate was DER encoded – that’s not what I use, but it was working on the old server.

I think there might be something between httpd-2.4.6-97 and httpd-2.4.37-43 that stopped DER encoded certificates from working. Rather than figure out some way to coerce HTTPD to use this DER file that I don’t really care if I’ve got … I just used a quick command to export the B64 version of the certificate, copied the header/footer/stuff in between, and made a base-64 encoded certificate file.

openssl x509 -inform DER -in server.crt | openssl x509 -text

And, voila, we’ve got a web server.

 

Azure DevOps – Changing Work Item Type

I had to reorganize a lot of my work items in a way that required items not to be what they were. Fortunately, there’s a mechanism to change work item type. Within the work item, click on the ellipsis to access a menu of options. Select “Change type …”

Select the item type you want – I record the reason I needed the new type for posterity – then click “OK”. Save the work item and re-open it.

The one thing I’ve noticed is that fields that don’t exist on an item type (e.g. “effort” on “feature” items) are still present on the new item type even when that field does not normally display (e.g. “effort” on “user story” items).

 

Azure DevOps – Features, User Stories, and Story Points

I had wanted to classify my ADO work items as “features” (i.e. something someone asked to be added to an application), “bugs” (i.e. some intended functionality that was not working as designed). Bugs have a story point field, but features do not appear to have their own story point field. They, instead, are a roll-up of the story points of their subordinate user stories. Which makes sense except that I’ve now got to have two work items for every feature. Rolling up larger requests into sprint-sized work units is how we use epics. So I’ve instead found myself with user stories tagged with “features” that fall into epics (or don’t in the case of a small feature request).

 

ElasticSearch 7.7.0 Log4J Vulnerability Remediation

We are a little stuck with our ElasticSearch implementation — we need the OpenDistro authentication, so either need to buy newer ElasticSearch or move to OpenSearch. That’s an ongoing project, but it won’t be accomplished in a timely fashion to address these log4j issues.

To address the existing Log4J issues in ElasticSearch as well as, evidently, another challenge that meant a new log4j build over the weekend, I am manually replacing the Log4j jar files for ElasticSearch 7.7.0, OpenDistro Security, and the S3 backup plugin. The 2.11.1 version that was bundled with the distribution can be replaced with the 2.17.0 release from Dec 18th.

The new JARs can be downloaded from Maven Repo at:

https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core/
https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-api/
https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-slf4j-impl/
https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-1.2-api/

To upgrade just log4j, the following script is run … well first allocation is disabled:

# Stuff to stop/start routing in Kibana
POST /_flush/synced

PUT /_cluster/settings 
{ "transient" : { "cluster.routing.allocation.enable": "none" } }

Then the script is run:

export l4jver=2.17.0
systemctl stop elasticsearch 

# Remove old jars
rm  --interactive=never /opt/elk/elasticsearch/lib/log4j-api-*.jar
rm  --interactive=never /opt/elk/elasticsearch/lib/log4j-core-*.jar
rm  --interactive=never /opt/elk/elasticsearch/modules/x-pack-identity-provider/log4j-slf4j-impl-*.jar
rm  --interactive=never /opt/elk/elasticsearch/modules/x-pack-security/log4j-slf4j-impl-*.jar
rm  --interactive=never /opt/elk/elasticsearch/plugins/opendistro_security/log4j-slf4j-impl-*.jar
rm  --interactive=never /opt/elk/elasticsearch/modules/x-pack-core/log4j-1.2-api-*.jar 
rm  --interactive=never /opt/elk/elasticsearch/plugins/repository-s3/log4j-1.2-api-*.jar 

# Copy in upgraded jars
cp /tmp/log4j-api-$l4jver*.jar /opt/elk/elasticsearch/lib/
cp /tmp/log4j-core-$l4jver*.jar /opt/elk/elasticsearch/lib/
cp /tmp/log4j-slf4j-impl-$l4jver*.jar /opt/elk/elasticsearch/modules/x-pack-identity-provider/
cp /tmp/log4j-slf4j-impl-$l4jver*.jar /opt/elk/elasticsearch/modules/x-pack-security/
cp /tmp/log4j-slf4j-impl-$l4jver*.jar /opt/elk/elasticsearch/plugins/opendistro_security/
cp /tmp/log4j-1.2-api-$l4jver*.jar /opt/elk/elasticsearch/modules/x-pack-core/
cp /tmp/log4j-1.2-api-$l4jver*.jar /opt/elk/elasticsearch/plugins/repository-s3/

# Fix permissions
chown elkadmin:elkadmin /opt/elk/elasticsearch/lib/log4j*
chown elkadmin:elkadmin /opt/elk/elasticsearch/modules/x-pack-core/log4j*
chown elkadmin:elkadmin /opt/elk/elasticsearch/modules/x-pack-identity-provider/log4j*
chown elkadmin:elkadmin /opt/elk/elasticsearch/modules/x-pack-security/log4j*
chown elkadmin:elkadmin /opt/elk/elasticsearch/plugins/repository-s3/log4j*
chown elkadmin:elkadmin /opt/elk/elasticsearch/plugins/opendistro_security/log4j*

systemctl start elasticsearch 

# Clean up temp files
rm /tmp/log4j*

And finally routing is re-enabled:

PUT /_cluster/settings 
{ "transient" : { "cluster.routing.allocation.enable" : "all" } }

Home Automation and Gardening

Something like 20 years ago, I tried to grow a plumeria flower in my apartment. I had a broad-spectrum light, plenty of heat, and plenty of humidity. But getting the light turned on and off at the right times wasn’t easy (especially if I was at work all day!).

This seems like a really good use for home automation — our home automation system tracks the sunrise and sunset times for our zip code. It’s possible to essentially cron “stuff” off of these times — e.g. get the birds ten minutes before sunset. I could easily track sunrise and sunset in Honolulu then have my light turn on at sunrise (or first light) and off at sunset (or last light). Voila — “sunlight” that runs for the proper duration every day.

Squid Custom Error

We’ve been having a challenge with Anya getting her school work completed. Part of the problem is the school’s own fault — they provide a site where kids are encouraged to read, but don’t provide any way to ensure this reading is done after classwork has been completed. But, even if that site didn’t exist … the Internet has all sorts of fun ways you can find to waste some time.

So her computer now routes through my proxy server. I’d set up a squid server so *I* could use the Internet unfettered whilst VPN’d in to work. It’s really annoying to get told you’re a naughty hacker every time you want to see some code example on StackOverflow!

While I didn’t really care about the default messages for my use (nor did I actually block anything for it to matter), I want Anya to be able to differentiate between “technical problem” the site didn’t load and “you are not allowed to be using this site now” the site didn’t load. So I customized the Squid error message for access denied. This can quickly be done by editing /usr/share/squid/errors/en-us/ERR_ACCESS_DENIED (you’ll need to make a backup of your version & may need to replace the file when upgrading squid in the future).

 

Python List Joining

I’ve seen lists joined with a delimiter like this before:

strDelimiter = “\n”
strDelimiter.join(listOfStuff)

But it seems silly to allocate memory just for the delimiter … not a big deal from a resource perspective, and probably using the delimiter variable is more comprehensible in the future … but I’ve always wondered if you couldn’t just use a static string with the join method. Turns out you can —

msgContent.attach(MIMEText(“\n”.join(listOfStuff), ‘plain’))