Tag: git

Git: Listing Conflicting Files

To list the unmerged files — where you’ve got merge conflicts to resolve:

git diff --name-only --diff-filter=U

Filters are:

  • Added (A)
  • Copied (C)
  • Deleted (D)
  • Modified (M)
  • Renamed (R)
  • Type changed (T)
  • Unmerged (U)
  • Unknown (X)
  • pairing Broken (B)

(use lower case letters to exclude)

Renaming a Branch in Git

I finally had a situation where I needed to rename a branch in git. When I was the only one involved in a development effort (or even looking at it!), it didn’t really matter if I typo’d something. Exchange and Exchagne … I know what I meant. But working under a more formal development process, I started naming my branch after the issue ID. And managed to typo the first one. Sigh!

# Check out the incorrectly named branch

git checkout OSSA166

# Rename it with the correct name

git branch -m OSSA163

# See what you’ve got — the local one is right now, but the remote is still incorrectly named

git branch -a

* OSSA163
master
remotes/origin/HEAD -> origin/master
remotes/origin/OSSA166
remotes/origin/master
remotes/origin/uat

# Push a change to rename the remote one too

git push origin :OSSA166 OSSA163

Total 0 (delta 0), reused 0 (delta 0)

To ssh://git.example.com/path/to/my/repo.git

– [deleted]         OSSA166
* [new branch]      OSSA163 -> OSSA163

# And see what you’ve got again

git branch -a

* OSSA163
master
remotes/origin/HEAD -> origin/master
remotes/origin/OSSA163
remotes/origin/master
remotes/origin/uat

 

Sparse Checkout With Git

I’ve encountered a few repositories that are huge. Unwieldy huge, and stuffed with files that aren’t relevant to what I need. The straight-forward solution is to use multiple repositories — that’s what I do at work with my code samples. There’s a different repo for each language because the PHP developers really don’t care what the C# code looks like. The Java developers don’t need a copy of the Python code. But there are advantages to having a single repository that may preclude you from taking the simple solution. Git sub-modules are an interesting approach — combining multiple repositories into a single functional unit. But that’s a pretty big change to an existing repo. And, if you participate in open source projects, it may not be your decision anyway.

There’s another option for selectively cloning when you’re working with a large repo — an option that doesn’t require any changes to the repository. An end user can perform a sparse checkout — essentially use a filter like .gitignore to select or deselect certain files/folders from being pulled into the local working directory. The file is named sparse-checkout and is located in .git\info — unlike a .gitignore file which indicates what shouldn’t get included, sparse-checking controls what is included (if you want an entire repo except one folder, use !path/to/folder/**)

The sparse-checkout file used to get just the core components of Scott’s OpenHAB helper libraries plus the OpenWeatherMap community scripts is:

.github/**
Core/**
Community/OpenWeatherMap/**

To use sparse checkout, set the core.sparseCheckout config value to true. You can add sparse checkout to a repo you’ve already cloned and use

git read-tree -mu HEAD

to “clean up” unwanted files. Or you can set up sparse checkout before you clone the repo

D:\tmp>mkdir ljrtest

D:\tmp>cd ljrtest

D:\tmp\ljrtest>git init
Initialized empty Git repository in D:/tmp/ljrtest/.git/

D:\tmp\ljrtest>git remote add origin https://github.com/openhab-scripters/openhab-helper-libraries

D:\tmp\ljrtest>git config core.sparseCheckout true

D:\tmp\ljrtest>copy ..\sparse-checkout .git\info\
1 file(s) copied.

D:\tmp\ljrtest>git pull origin master
remote: Enumerating objects: 3591, done.
remote: Total 3591 (delta 0), reused 0 (delta 0), pack-reused 3591R ), 7.00 MiB | 6.95 MiB/s
Receiving objects: 100% (3591/3591), 9.26 MiB | 7.22 MiB/s, done.
Resolving deltas: 100% (1786/1786), done.
From https://github.com/openhab-scripters/openhab-helper-libraries
* branch master -> FETCH_HEAD
* [new branch] master -> origin/master

D:\tmp\ljrtest>dir
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> .github
07/03/2019 09:07 AM <DIR> Community
07/03/2019 09:07 AM <DIR> Core
0 File(s) 0 bytes
5 Dir(s) 386,515,042,304 bytes free

D:\tmp\ljrtest>dir .\Community
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest\Community

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> OpenWeatherMap
0 File(s) 0 bytes
3 Dir(s) 386,515,042,304 bytes free

Using sparse checkout, no one else has to do anything. Configure your client to get the files you want, and you’re set.

 

Git Commands

There are a few git commands that we use when working with the OpenHAB and helper library repositories. The OpenHAB Eclipse project sets uses a split push/pull repository where the *fetch* repo is the organization and the *push* repo is your personal repo. This is reasonable because you do not have permissions to write to the organizational repository. You can use the same split-repository setup for other projects. Clone the project either from the organization’s repo, and then change the push URL to your personal repository.

# Show list of remotes
[lisa@linux ~]# git remote -v
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/openhab-scripters/openhab-helper-libraries (push)

# Set push remote to PERSONAL repository
[lisa@linux ~]# git remote set-url –push origin https://github.com/ljr55555/openhab-helper-libraries

# Show list of remotes — verification step
[lisa@linux ~]# git remote -v
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/ljr55555/openhab-helper-libraries (push)

While the split repository setup prevents accidentally attempting to push changes to a repo to which you lack write access, I find it a little confusing. Instead, I add specific repos for ORG (the organizational repo) and my personal repo.
The drawback to this configuration is that you *can* attempt to push changes directly to the organization repo — which will either yield an error because you lack access or will inadvertently publish code in the org repo because you don’t lack access.

# Add ORG repo with organizational repo URL
[lisa@linux ~]# git remote add ORG https://github.com/openhab-scripters/openhab-helper-libraries
# Add LJR repo with personal fork URL
[lisa@linux ~]# git remote add LJR https://github.com/ljr55555/openhab-helper-libraries
[lisa@linux ~]# git remote -v
LJR https://github.com/ljr55555/openhab-helper-libraries (fetch)
LJR https://github.com/ljr55555/openhab-helper-libraries (push)
ORG https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
ORG https://github.com/openhab-scripters/openhab-helper-libraries (push)
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/ljr55555/openhab-helper-libraries (push)

# Scenario: Someone has updated ORG master branch
# I want to incorporate those changes into PERSONAL master branch and push them into my repo
[lisa@linux ~]# git checkout master # Switch to your local master branch
[lisa@linux ~]# git fetch ORG/master # Get changes from Organization master
[lisa@linux ~]# git rebase ORG/master # Apply those changes to local master
[lisa@linux ~]# git push –force LJR master # Overwrite personal repo master with updated info

# Scenario: Someone has updated ORG master branch.
# I want to incorporate those changes in PERSONAL lucid-migration branch
[lisa@linux ~]# git checkout master # Switch to your local master branch
[lisa@linux ~]# git fetch ORG/master # Get changes from Organization master
[lisa@linux ~]# git rebase ORG/master # Apply those changes to local master
[lisa@linux ~]# git checkout lucid-migration # Switch back to your local lucid-migration branch
[lisa@linux ~]# git rebase –preserve-merges master # Rebase your local lucid-migration (checked out branch) onto local master
[lisa@linux ~]# git push –force-with-lease LJR lucid-migration # Overwrite personal repo lucid-migration branch with updated info

 

And a few misc commands that I want to remember
# Check username and email
[lisa@linux ~]# git config –list

# Set username and e-mail address
[lisa@linux ~]# git config –global user.name “FIRST_NAME LAST_NAME”
[lisa@linux ~]# git config –global user.email “MY_NAME@example.com”

# merge gone bad, bail!
[lisa@linux ~]# git merge –abort

# Forgot to add sign-off on commit
[lisa@linux ~]# git commit –amend

Off Label Use: Git

I store a lot of code in Git-based systems, and I read through parts of my state’s Revised Code on occasion. But it never occurred to me to store the Revised Code in a Git-based system

Imagine being able to comment on a pull request and have your commentary considered during the public hearings for the proposed change. Sure, there’s elitism to assuming everyone’s got a computer and time to peruse proposed changes. But not more than assuming everyone’s got an hour to head over to Town Hall and attend the public hearing. 

And the storage system retains history for you — a pull request created by government officials due to newly enacted laws could reference the law itself. If you’re convinced the auxiliary building was totally in compliance of zoning regs when you built it … you could run through the history and identify exactly when the setback requirements changed. 

New Process Police

As an operational support group, we did not have a software development methodology. Doesn’t mean we didn’t develop software — one of the great things within operational support is the ability to automate day-to-day tasks to reduce workload. Why have someone check for application patches when a process can watch an RSS feed or file repository and notify us when there’s an update. Why have someone clickity-click provisioning users into groups when the user can make a web request, the group owner can approve the request, and an automated process can add the user into the group? The end result of our automation programming is, well, quite a bit of software.

And with a small number of people, informal application development worked. Wasn’t ideal, but it worked. If you want to write in Java while I use C# … not ideal, but the alternative is that one of us needs to learn a new programming language. Problem is the next guy we hired uses VBS, the next guy uses PowerShell … and I’ll use perl for simpler processes. Then someone starts tweaking my code and buggers it up … and we’ve got to figure out what happened and roll back based on some tape backup.

To get our internal software development processes organized, I developed a process. And ran a training session so everyone was familiar with both the process and the tools. Some of us have used the process well — don’t edit production code, clone the repo locally, make a branch for your edits, test it and have another group member sign off on the changes, merge your branch back into master, test more, then pull the code into production. The majority, it seems, have not followed the process at all. Changes are made to the code running in production, not incorporated into the Git repo. Six months after the new development process went in place, half of our code has improperly made changes!

To an extent, I consider this a management problem … if the department doesn’t want software development to be a free-for-all, then the department managers need to ensure their staff follows the process. If the department wants everyone to do their own thing — then get rid of the process and declare our methodology as “do whatever you want”. The challenge for managers, though, is that they don’t know that someone has edited code in production and failed to commit their changes into the repo. If only there were some way to watch for improperly edited code and alert us promptly.

Other scripts I’ve found to perform a similar function attempt to parse ‘git status’ to identify all sorts of issues — but that doesn’t address the specific problem that I’ve got. To facilitate identifying offenders, I wrote a quick Python script that searches a directory tree for git repositories and alerts us when changes have not been staged for commit. If you’ve staged the changes for commit, that won’t be identified. But the particular problem we encounter frequently … there are alerts for that.

New Git Server

I love GitLab, but it’s resource intensive and I wasn’t able to run it on the current hardware. Getting a new server has been a slow process, so I needed another solution. We use the Bonobo Git Server at work — it’s really simple and doesn’t provide all of the analysis and workflow functionality of GitLab (e.g. there are no pull requests!) … but it does keep revisions of code to which you can revert. And that’s better than trying to figure out what changed and broke a program. So I set one up at home.

Super simple — except they forget a few steps — like they don’t mean you need to install .NET 4.6. You need to go in the Add Roles & Features, under Application Development Features, and enable ASP.NET. 4.5 worked fine for me. You’ve also got to restart your IIS MMC if you left it running, and change the application pool over to one that actually uses ASP.NET.

Once that was installed, a few quick changes to web.config enabled AD-based authentication. And now we’ve got a git server at home that I can leave running 24×7.

Except … I tried using a new git server and got the following error:
schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) –
The revocation function was unable to check revocation for the certificate.

To resolve the issue, I needed to changed from Windows to OpenSSL certificate handling:
git config –global http.sslBackend openssl

Except that blew away my config line that defines the SSL CA CRT file. Once I restored my configuration … voila, I can use the git client with my new git server
git config –system http.sslcainfo “c:\Program Files\Git\mingw64\ssl\certs\ca-bundle.crt”

Git For Configuration Management

I am starting to use git to manage application server configurations — partially to ensure team members are familiarizing themselves with git and thinking about it when they update code (we’ve seen a LOT of tweaks that are not pushed to the git server), but also to reduce the administrative overhead of managing servers.

The best use case thus far has been our sendmail environment — seven servers with three configuration bases. By issuing certificates with SAN values for each host name and the VIP name, we are able to use the same cert and config file on each server in a functional group. Admins can make changes to the config offline (i.e. we’re not live-editing config files on the sendmail servers), there is history to who made the changes {and a quick means of reverting changes), and, using a cron’d pull, we can ensure changes are consistent across the environment.

Active Directory Federation Services (ADFS) Relying Party Trust Cert Expiry

At work, we received a critical ticket for an application that was unable to authenticate to ADFS. Nothing globally wrong – other applications are authenticating. A long call later, we discovered that the app’s certificate has expired. Why would the application not monitor their certificate expiry dates?? That’s an excellent question, but not one over which I have any control.

can monitor their certs on our side. So I wrote a quick powershell script to grab certificates from the relying party trusts and alerts us if any certs will be expiring in the next 30 days. It has to run on the ADFS server – I’d love to get it moved to the automation server in the future. I expect get-adfsrelyingpartytrust returns disabled agreements. I want to filter out disabled agreements.