Category: Technology

Docker and Windows — Unable to Allocate Port

On the most recent iteration of Windows (20H2 build 19042.1052) and Docker Desktop (20.10.7 built Wed Jun 2 11:54:58 2021), I found myself unable to launch my Oracle container. The error indicated that the binding was forbidden.

 

C:\WINDOWS\system32>docker start oracleDB
Error response from daemon: Ports are not available: listen tcp 0.0.0.0:1521: bind: An attempt was made to access a socket in a way forbidden by its access permissions.
Error: failed to start containers: oracleDB

Forbidden by whom?! Windows, it seems. Checking excluded ports using netsh:

netsh int ipv4 show excludedportrange protocol=tcp

Shows that there are all sorts of ports being forbidden — Hyper-V is grabbing a lot of ports when it starts. To avoid that, you’ve got to add a manual excluded port for the one you want to use.

To reserve the port for your own use, disable Hyper-V (reboot), add a port exclusion, and enable Hyper-V (reboot)

REM Disable Hyper-V
dism.exe /Online /Disable-Feature:Microsoft-Hyper-V 
REM REBOOT ... then add an exclusion for the Oracle DB Port
netsh int ipv4 add excludedportrange protocol=tcp startport=1521 numberofports=1 
dism.exe /Online /Enable-Feature:Microsoft-Hyper-V /All
REM REBOOT again

Now 1521 is reserved for Oracle

ADO – Migrating a Repository to Azure Repos (and keeping your commit history)

The most direct way to migrate a repo into Azure Repos is to create a new, blank repository. This may mean making a new project. From the organization’s main page, click “New project”

Or it may mean making a new repo in an existing project. From an existing repo, click the drown-down next to the repo name and select “New repository”

Name the repository but don’t add a README. We want a blank repo

Note the URL to the repository – in this case, it’s https://ado0255@dev.azure.com/ado0255/History%20Test/_git/Another%20History%20Test

Find the URL for your existing Git repo – if you cd into the project’s folder and run “git remote -v”, you will get a list of the repos. Make a new folder somewhere – this is a temporary staging area to move the data from your existing repo over to the new Azure Repo. Change directories into your new folder. Run git clone –mirror URLToOldRepo

You will see data being downloaded from your git server.

Change directories into the folder that just got downloaded. You won’t see your code like you normally do when you clone a git repo. You’re looking at the underlying git stuff that makes up the repo. You’re code is all in there, as are all of the branches and commit history.

Now add the new Azure Repo as a remote – in this case, I’m naming the remote “ado”. Then run “git push ado –all” to push everything up to the new Azure Repo.

Stuff will transfer – you may be prompted to log into your ADO repository first. Eventually, you’ll see new branches being created on the remote and the process will complete.

Refreshing the Azure Repo, you’ll now see the files.

Selecting “Commits” will display the commit history.

Anyone else using the repo will need to add the new remote. Use “git remote rm origin” to remove the existing origin, then use “git remote add origin url” to add the new Azure Repo as origin.

ADO – Cleaning up test repos and projects

I find the process to delete repositories and projects to be nonintuitive. Since I create a lot of projects and repos for testing and documentation, it’s nice to be able to clean them up when I’m done!

To delete an Azure Repo, navigate to a repo and select the drop-down next to the repo name. Select “Manage repositories”

With your mouse over a repository, there’s a hamburger menu at the right-hand side of the listing. Click it and select “Delete”

You’ll need to type the repository’s full name to activate the delete button.

To delete a project, go to the organization’s home page and select “Organization Settings” from the lower left-hand corner of the screen.

Select “Projects” from the left-hand navigation bar

With your mouse over the project listing, you’ll have a hamburger menu. Click it and select “Delete Project”

You’ll need to type the project name to activate the delete button.

 

Git – Removing Confidential Info From History

The first cut of code may contain … not best practice code. Sometimes this is just hard coding something you’ll want to compute / look up in the future. Hard coding user input isn’t a problem if my first cut is always searching for ABC123. Hard coding the system creds? Not good. You sort that before you actually deploy the code. But some old iteration of the file has MyP2s5w0rD sitting right there in plain text. That’s bad in a system that maintains file history! The quick/easy way to clean up passwords stashed within the history is to download the BFG JAR file.

For this test, I created a new repository in .\source then created three clones of the repo (.\clone1, .\clone2, and .\clone3). In each cloneX copy, I created a tX folder that has a file named ldapAuthTest.py — a file that contains a statically assigned password as

strSystemAccountPass = "MyP2s5w0rD"

The first thing I did was to redact the password in the files — this means anyone looking at HEAD won’t see the password. Source, clone1, and clone2 are all current. The clone3 copy has pulled all changes but has a local change committed but not merged.

To clean the password from the git history, first create a backup of your repo (just in case!). Then mirror the repo to work on it

mkdir mirror
cd mirror
git clone --mirror d:\git\testFilterBranch\source

 

Create file .\replacements.txt with the string to be redacted — in this case:

strSystemAccountPass = "MyP2s5w0rD"

Formatting notes for replacements.txt

MyP2s5w0rD # Replaces string with default ***REMOVED***
MyP2s4w0rD==>REDACTED # Replaces string using custom string REDACTED
MyP2s3w0rD==> # Replaces string with null -- i.e. removes the string
regex:strSystemAccountPass\s?=\s?"MyP2s2w0rD""==>REDACTED # Uses a regex match -- in this case we may or may not have a space around the equal sign

So, in my mirror folder, I have the replacement.txt file which defines which strings are replaced. I have a folder that contains the mirror of my repo.

lisa@FLEX3 /cygdrive/d/git/testFilterBranch/mirror
$ ls
replacements.txt source.git

To replace my “stuff”, run bfg using the –replace-text option. Because I only want to replace the text in files named ldapAuthTest.py, I also added the -fi option

java -jar ../bfg-1.14.0.jar --replace-text ..\replacements.txt -fi ldapAuthTest.py source.git

 

lisa@FLEX3 /cygdrive/d/git/testFilterBranch/mirror
$ java -jar ../bfg-1.14.0.jar --replace-text replacements.txt -fi ldapAuthTest.py source.git

Using repo : D:\git\testFilterBranch\mirror\source.git

Found 3 objects to protect
Found 2 commit-pointing refs : HEAD, refs/heads/master

Protected commits
-----------------
These are your protected commits, and so their contents will NOT be altered:
* commit 87f1b398 (protected by 'HEAD')

Cleaning
--------
Found 5 commits
Cleaning commits: 100% (5/5)
Cleaning commits completed in 613 ms.

Updating 1 Ref
--------------

Ref Before After
---------------------------------------
refs/heads/master | 87f1b398 | 919c8f0f

Updating references: 100% (1/1)
...Ref update completed in 151 ms.

Commit Tree-Dirt History
------------------------

Earliest Latest
| |
. D D D m

D = dirty commits (file tree fixed)
m = modified commits (commit message or parents changed)
. = clean commits (no changes to file tree)

Before After
-------------------------------------------
First modified commit | dc2cd935 | 8764f6f1
Last dirty commit | 9665c4e0 | ccdf0359

Changed files
-------------

Filename Before & After
-------------------------------------
ldapAuthTest.py | 25e79fa6 ? 4d12fdad

In total, 8 object ids were changed. Full details are logged here:
D:\git\testFilterBranch\mirror\source.git.bfg-report\2021-06-23\12-50-00

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive

Check to make sure nothing looks abjectly wrong. Assuming the repo is sound, we’re ready to clean up and push these changes.

cd source.git

git reflog expire --expire=now --all && git gc --prune=now --aggressive
git push

 

Pulling the update from my source repo, I have merge conflicts

These are readily resolved and the source repo can be merged into my local copy.

And the change I had committed but not pushed is still there.

Pushing that change produces no errors

Now … pushing the bfg changes may not work. In my case, the real repo has a bunch of branchs and I am getting “non fast-forward merges”. To get the history changed, I need to do a force push. Not so good for the other developers! In that case, everyone should get their changes committed and pushed. The servers should be checked to ensure they are up to date. Then the force push can be done and everyone can pull the new “good” data (which, really, shouldn’t differ from the old data … it’s just the history that is being tweaked).

DNF — What Provides This File?

“Dependency hell” used to be a big problem — you’d download one package, attempt to install it, and find out you needed three other packages. Download one of them, attempt to install it, and learn about five other packages. Fifty seven packages later, you forgot what you were trying to install in the first place and went home. Or, I suppose, you managed to install that first package and actually use it. The advent of repo-based deployments — where dependencies can be resolved and automatically downloaded — has mostly eliminated dependency hell. But, occasionally, you’ll have a manual install that says “oh, I cannot complete. I need libgdkglext-x11-1.0.so.0 or libminizip.so.1 … and, if there’s a package that’s named libgdkglext-x11 or libminizip … you’re good. There’s not. Fortunately, you can use “dnf provides” to search for a package that provides a specific file — thus learning that you need the gtkglext-libs and minizip-compat packages to resolve your dependencies.

Adding MariaDB/MySQL Users

Quick notes on creating a database user — MariaDB and MySQL use a combination of username and source host to determine access. This means ‘me’@’localhost’ and ‘me’@’remotehost’ can have different passwords and privilege sets. How do you know what the hostname is for your connection? I usually try to connect and read the host from the error message — it’ll say ‘someone’@’something’ cannot access the database.

# Create a user that is allowed to connect from a specific host
create user 'username'@'hostname' identified by 'S0m3P@s5w0rd';
GRANT ALL PRIVILEGES ON dbname.* TO 'username'@'hostname';

# Create a user that is allowed to connect from a specific IP
create user 'username1'@'10.5.1.2' identified by 'S0m3P@s5w0rd';
GRANT ALL PRIVILEGES ON dbname.* TO 'username1'@'10.5.1.2';

# Create a user that is allowed to connect from database server
create user 'username2'@'localhost' identified by 'S0m3P@s5w0rd';
GRANT ALL PRIVILEGES ON dbname.* TO 'username2'@'localhost';

# Create a user that is allowed to connect from any host
create user 'username3'@'%' identified by 'S0m3P@s5w0rd';
GRANT ALL PRIVILEGES ON dbname.* TO 'username3'@'%';

# Flush so new privileges are effective
flush privileges;

# View list of database users
SELECT User, Host FROM mysql.user;
+----------------+------------+
| User           | Host       |
+----------------+------------+
| username3      | %          |
| username2      | 10.5.1.2   |
| username       | hostname   |
| root           | 127.0.0.1  |
| root           | ::1        |
| root           | localhost  |
+----------------+------------+
6 rows in set (0.000 sec)

Bootleg Zoom Recordings

Our township has been holding meetings in Zoom, but the recordings are for the purpose of generating minutes only. It’s an end-run around document retention management (it’s not a record, so we don’t need to retain it) … but it also means residents cannot just catch the meeting whenever they’ve got time. In theory, it seemed easy enough to use a screen recording program (I use OBS Studio) to record the meetings. Problem is, though … I’m one of the people who isn’t generally available in the late afternoon / early evening when they schedule their meetings. Surely I could join the meeting early & just edit out the “dead air” within the recording …

(0) To configure Zoom client settings, you need to sign up for an account. Now that your client is signed in, there are settings.

(1) Audio isn’t automatically connected. In the settings window, under “Audio”, there is an option to automatically join computer audio. While you’re there, mute the mic when joining a meeting too.

(2) The window into which the main video is placed might be 2″ across. Under “General” is an option to use full screen when joining a meeting.

(3) Disable your video when joining the meeting — there’s no reason for anyone to stare at the wall behind my desk!

Hopefully these settings allow me to successfully record a Township meeting to watch later. In OBS Studio, I’ve got the audio input linked to my speakers and have the mic disabled as a recording source … this means someone talking near my computer doesn’t ruin my recording. There’s another meeting tomorrow … we’ll see if I’ve finally got it!

Update — this works unless the meeting organize throws up a “this meeting is being recorded” banner. In which case, you have a big dialog box in the middle of the meeting window. I give up, and we’ll just ensue someone’s at the computer to actually set up the recording.

Proof of Concept

Reading about the meat processing that’s been attacked by ransomware, and thinking about the petrol pipeline … this really seems like proof of concept stuff to me. I’m sure there’s some ‘making money’ and more than a little ego stroking involved. Before we purchase and implement some major system at work (or spend a lot of time developing code), we run a proof of concept test. A quick, slimmed down implementation that runs on some virtual system that lets people see how it’ll work without sinking the time and money into a full-scale implementation. If the thing seems useful, then we buy it and have a capital budget for implementation. If it wasn’t useful … well, we lost some time, but not much.

Attacking small players in various industries to see what kind of impact you have have … seems a lot like a proof of concept series of attacks. How well secured was the company? What kind of incident response were they able to mount? How much access did you manage? What came offline? What was the public impact?

Microsoft’s 1601 Time Base

Microsoft uses the number of 100-nanosecond intervals since 01 January 1601. Why? No idea. But I’ve had to deal with their funky large integer for a DateTime value as long as I’ve been working with AD. I’ve written functions to turn it int something useful, but that’s a lot of effort when I see a lockoutTime and need to know how recent that is. Enter w32tm which has an “ntte” switch — this allows me to readily tell that the lockout was at 3:01 today and something I need to be investigating.