Category: System Administration

Two Approaches to Using PIP Through an Integrated Authenticated Proxy

The proxy at work uses integrated authentication. While BASIC auth prompts happily let you use a proxy of http://uid:pass@proxy:port, ours does not. There are two ways I’ve managed to use pip to install packages.

Proxied Proxy

The easiest approach is to use something that can handle the authenticated proxy, like Fiddler, as an intermediary. I do the same thing with perl’s PPM, docker pull … basically anywhere I’ve got to use a proxy that wants to pass through a proxy username.

Select Tools => Options to open the configuration dialog. Make sure you are handling SSL traffic — if not, check the box to “Capture HTTPS CONNECTs, “Decrypt HTTPS traffic”, and “Ignore server certificate errors” (it’s only unsafe if you don’t understand what you’re doing … don’t log into your bank account bouncing traffic through this config!)

On the “Connections” tab, check the port on which Fiddler is listening. If you cannot install Fiddler on the same box where you want to use pip, you’ll need to check off “Allow remote computers to connect” (and you won’t use localhost as the proxy hostname). Click OK, start capturing traffic (F12), and we’re ready to go.

Use the PIP command line to install the package but proxy the request through your Fiddler instance. In this example, Fiddler is installed on the local box and uses port 8888.

pip –trusted-host pypi.org –trusted-host files.pythonhosted.org –proxy http://localhost:8888 install SomePackage

This is nice because pip will automatically resolve dependencies. Not great if you’re not allowed to install your own software and cannot get Fiddler installed.

Dependency Nighmare

Back in the early days of Linux (think waaaay before package managers working against online repositories), we called this “dependency hell” — navigating dependency chains to get all of the required “stuff” on the box so you can install the thing you actually wanted.

Make a folder for all these wheels we’re going to end up downloading so it’s easy to clean up once we’re done. Search PyPi for the package you want. On the package page, select ‘Download Files’ and then download the whl

Use “pip install something.whl” to attempt installing it. If you’re lucky, you’ve got all the dependencies and the package will install. If you don’t have all of the dependencies, you’ll get an error telling you the first missing one. Go back to the pypi website & get that. Use “pip install somethingelse.whl” to install it and maybe get a dependency error here too. Once you get a dependency installed, try the package you want again. Got another error? Start downloading and installing again. Eventually, though, you’ll get all of the dependencies satisfied and be able to install the package you actually want.

Adding Python to Fedora Alternatives

Gimp installed Python 2.7). Which, of course, took over my system so nothing was using Python 3 anymore. We’ve used ‘alternatives’ to manage the Java installation, and I thought that might be a good solution in case I ever need to use Python 2

Add both Python versions to alternatives:

[lisa@fedora ~]# sudo alternatives –install /usr/bin/python python /usr/bin/python3.7 1
[lisa@fedora ~]# sudo alternatives –install /usr/bin/python python /usr/bin/python2.7 2

Select which one you want to use:

[lisa@fedora ~]# sudo alternatives –config python

There are 2 programs which provide ‘python’.

Selection Command
———————————————–
1 /usr/bin/python3.7
+ 2 /usr/bin/python2.7

Enter to keep the current selection[+], or type selection number: 1

And, of course, repeat the process for PIP:

[lisa@fedora ~]# sudo alternatives –install /usr/bin/pip pip /usr/bin/pip2.7 2
[lisa@fedora ~]# sudo alternatives –install /usr/bin/pip pip /usr/bin/pip3.7 1

[lisa@fedora ~]# sudo alternatives –config pip

Updating ll alias

Fedora and Red Hat Enterprise Linux have an alias “ll” which uses the long listing format. It’s a quick little tweak that I love, but I generally want to list hidden files too. Seemed easy enough to tweak the alias … but I never had any luck overriding the system setting or finding the source of the alias. Typing “ll -a” gave me what I wanted, although that’s not appreciably easier than typing “ls -al” …

The ll alias is defined in /etc/profile.d/colorls.sh (or colorls.csh if you use the C shell). Add the ‘a’ and “ll” produces a long list format of all files.

lisa@fedora123 ~]# grep “alias ll” /etc/profile.d/colorls.sh
alias ll=’ls -la’ 2>/dev/null
alias ll=’ls -la –color=auto’ 2>/dev/null

Ransomware Insurance

Another city paying hackers in hope of decrypting their data. Interesting note that the payoff is being covered by insurance. I wonder if business insurers are going to go down the PCI-DSS route as part of the risk/pricing calculation. Running unpatched XP workstations and W2K servers? Your insurance is going to get expensive. Show proof of quarterly full DR validation restoring everything from last night & magic cloudy mail/file storage with versioning, that’ll bring your insurance cost down.

 

Quicker Way To Set Up Key-Based Authentication

I’ve always added my public key to a remote host’s authorized_keys file manually, but happened across the “ssh-copy-id” command which does that for you.

[lisa@workstation-fedora .ssh]$ ssh-copy-id -o PreferredAuthentications=password -o PubkeyAuthentication=no lisa@fedora123.example.com
The authenticity of host ‘fedora123.example.com (10.1.2.3)’ can’t be established.
ECDSA key fingerprint is SHA256:5EuKd5LNRnx5sHgQNFb6HO6W/p0hQk4pEmShTgj3zyU.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed — if you are prompted now it is to install the new keys
lisa@fedora123.example.com’s password:

Number of key(s) added: 1

Now try logging into the machine, with: “ssh -o ‘PreferredAuthentications=password’ -o ‘PubkeyAuthentication=no’ ‘lisa@fedora123.example.com'”
and check to make sure that only the key(s) you wanted were added.

Omit the -o options when attempting to log in over the key-based authentication. This, of course, presupposes that you have a public/private key pair. To create one, use ssh-keygen -t rsa -b 2048

Linux Mounts Windows Volumes As Read Only

Since I’ve got a larger hard drive installed, I have both Fedora and Windows in a dual boot configuration. I have a shared NTFS partition for data, but it’s mounted as read-only under Fedora. Turns out that Fedora sees the file system as not cleanly shut down when Windows Fast Boot is enabled. I disabled fast boot in power management, and the shared data volume is mounted rw as expected.

Finding Block ID

I upgraded my Fedora kernel to 5.1 and the secondary disk mounted to /var disappeared. I use the old-school device notation in fstab;  when a disk comes up with a different name, the partition fails to mount. I wanted to change fstab to use a UUID. But first I needed to find the UUID. Enter blkid

[root@linux123 ~]# blkid /dev/sdb1
/dev/sdb1: LABEL=”mnt-var” UUID=”50545e50-75c5-45q5-95b5-34f5456515d5″ TYPE=”ext4″ PARTUUID=”m50525d5-05″

The command output includes the device UUID which is used instead of the /dev/sdb# string.

#/dev/sdb1                                                                           /var ext4 nodev,nosuid 0 2
UUID=50545e50-75c5-45q5-95b5-34f5456515d5 /var ext4 nodev,nosuid 0 2

Rebooted and my partition mounted.

Using sed to insert lines into a file

I’ve used sed to replace file content — use a regex to replace the sendmail.cf line that routes mail directly with a smarthost directive

sed -i -e 's/^DS/DS\\\[mailTWB.example.com\\\]/' $strSendmailDirectory/etc/mail/sendmail.cf

But I’ve needed to prepend text to a file. Turns out sed acn do that. In fact, you can insert strings at any line number. Using “sed -i ‘5s;^;StringsToInsert\n;’ filename.xtn will insert “StringsToInsert\n” at line 5. To prepend text to a file, use “1s”

[lisa@fedora tmp]# cat test.txt;sed -i ‘5s;^;NewLine1\nNewLine2\n;’ test.txt;cat test.txt
Line 1
Line 2
Line 3
Line 4
Line 5
Line 6
**********
Line 1
Line 2
Line 3
Line 4
NewLine1
NewLine2
Line 5
Line 6
**********

 

I’ve also come across an oddity in the Win32 sed — the method I usually use to blow away everything after a newline for some reason blows away everything after the first line. Works fine on RHEL7 and Fedora29, so the quick solution is “run it from the Linux box”.

C:\temp>cat input.txt
line 1
line 2

line 3
line 4
line 5
C:\temp>sed -i ‘/^$/q’ input.txt&cat input.txt
line 1

Using ZoneMinder v1.32.3 With OpenHAB2

I documented a temporary fix to return ZM_PATH_ZMS and ZM_OPT_FRAME_SERVER through the ./api/configs/view/<KEYNAME>.json API so ZoneMinder 1.31.45 worked with the OpenHAB2 binding. Upon upgrading ZoneMinder to 1.32.3, the binding was no longer able to communicate with our ZoneMinder server.

In the OpenHAB2 log, errors indicated malformed JSON was received.

Caused by: com.google.gson.stream.MalformedJsonException: Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 7 path $
at com.google.gson.stream.JsonReader.syntaxError(JsonReader.java:1568) ~[?:?]
at com.google.gson.stream.JsonReader.checkLenient(JsonReader.java:1409) ~[?:?]
at com.google.gson.stream.JsonReader.doPeek(JsonReader.java:542) ~[?:?]
at com.google.gson.stream.JsonReader.peek(JsonReader.java:425) ~[?:?]
at com.google.gson.JsonParser.parse(JsonParser.java:60) ~[?:?]
at com.google.gson.JsonParser.parse(JsonParser.java:45) ~[?:?]
at name.eskildsen.zoneminder.jetty.JettyConnectionInfo.fetchDataAsJson(JettyConnectionInfo.java:352) ~[?:?]

Using a web browser to access <ZoneMinderURL>/zm/api/configs/view/ZM_PATH_ZMS.json, malformed JSON is returned.

Conf files are not updated when new packages are installed – an conf.rpmnew is created instead. The changes from the new config (zoneminder.conf.rpmnew) file need to be merged into the existing config file (zoneminder.conf). In our zm.conf file, I added:

ZM_DB_SSL_CA_CERT=
ZM_DB_SSL_CLIENT_KEY=
ZM_DB_SSL_CLIENT_CERT=

Reloading the page in my browser confirmed that the JSON response is valid.

When the ZoneMinder binding started, it successfully attached to our monitors and detected a motion alarm.

This does not negate the need for the original fix — config.php still needs to have the strcmp I added. When ZoneMinder is upgraded, /usr/bin/zmupdate.pl is run (I needed to run “/usr/bin/zmupdate.pl -f” to stop zmc from existing with return code 255), the values I added to the ZoneMinder Config table are removed — they need to be re-added.