Tag: git

Using File System As A Git Repository

I’ve used GitLab for quite some time, and as a full featured CI/CD platform that also provides git functionality … it’s awesome. But it’s also serious overkill for someone who wants to coordinate code with another developer or two. Or just keep history for their code. Or a backup. To accomplish this, all you need is drive space. If you’ve got a file server, any folder on the share can be a git repository.

On the server, create a git repository and add something to it.

Z:\Temp\test>git init
Initialized empty Git repository in Z:/Temp/test/.git/

Z:\Temp\test>notepad testfile.txt

Z:\Temp\test>git add testfile.txt

Z:\Temp\test>git commit -m "Initial file upload"
[master (root-commit) 9a3ebe7] Initial file upload
1 file changed, 1 insertion(+)
create mode 100644 testfile.txt

Then on your client, either clone the repo from the drive path

C:\Users\lisa>git clone file://z:/temp/test
Cloning into 'test'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

Or from the UNC path

C:\Users\lisa>git clone file://server01.example.com/data/temp/test
Cloning into 'test'...
remote: Enumerating objects: 3, done.
remote: Counting objects: 100% (3/3), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

I prefer to use the UNC paths – if my drive letter fails to map for some reason, the UNC path is still available.

If you’ve got pre-existing code, there’s a bit of a different process. On the server, create an empty folder and use “git init” to initialize the empty repo. On the client where the code exists, run:

git init
git add *
git commit -m “Initial code upload”
git remote add origin git clone file://server01.example.com/data/temp/test
git push origin master

 

Git: Listing Conflicting Files

To list the unmerged files — where you’ve got merge conflicts to resolve:

git diff --name-only --diff-filter=U

Filters are:

  • Added (A)
  • Copied (C)
  • Deleted (D)
  • Modified (M)
  • Renamed (R)
  • Type changed (T)
  • Unmerged (U)
  • Unknown (X)
  • pairing Broken (B)

(use lower case letters to exclude)

Renaming a Branch in Git

I finally had a situation where I needed to rename a branch in git. When I was the only one involved in a development effort (or even looking at it!), it didn’t really matter if I typo’d something. Exchange and Exchagne … I know what I meant. But working under a more formal development process, I started naming my branch after the issue ID. And managed to typo the first one. Sigh!

# Check out the incorrectly named branch

git checkout OSSA166

# Rename it with the correct name

git branch -m OSSA163

# See what you’ve got — the local one is right now, but the remote is still incorrectly named

git branch -a

* OSSA163
master
remotes/origin/HEAD -> origin/master
remotes/origin/OSSA166
remotes/origin/master
remotes/origin/uat

# Push a change to rename the remote one too

git push origin :OSSA166 OSSA163

Total 0 (delta 0), reused 0 (delta 0)

To ssh://git.example.com/path/to/my/repo.git

– [deleted]         OSSA166
* [new branch]      OSSA163 -> OSSA163

# And see what you’ve got again

git branch -a

* OSSA163
master
remotes/origin/HEAD -> origin/master
remotes/origin/OSSA163
remotes/origin/master
remotes/origin/uat

 

Sparse Checkout With Git

I’ve encountered a few repositories that are huge. Unwieldy huge, and stuffed with files that aren’t relevant to what I need. The straight-forward solution is to use multiple repositories — that’s what I do at work with my code samples. There’s a different repo for each language because the PHP developers really don’t care what the C# code looks like. The Java developers don’t need a copy of the Python code. But there are advantages to having a single repository that may preclude you from taking the simple solution. Git sub-modules are an interesting approach — combining multiple repositories into a single functional unit. But that’s a pretty big change to an existing repo. And, if you participate in open source projects, it may not be your decision anyway.

There’s another option for selectively cloning when you’re working with a large repo — an option that doesn’t require any changes to the repository. An end user can perform a sparse checkout — essentially use a filter like .gitignore to select or deselect certain files/folders from being pulled into the local working directory. The file is named sparse-checkout and is located in .git\info — unlike a .gitignore file which indicates what shouldn’t get included, sparse-checking controls what is included (if you want an entire repo except one folder, use !path/to/folder/**)

The sparse-checkout file used to get just the core components of Scott’s OpenHAB helper libraries plus the OpenWeatherMap community scripts is:

.github/**
Core/**
Community/OpenWeatherMap/**

To use sparse checkout, set the core.sparseCheckout config value to true. You can add sparse checkout to a repo you’ve already cloned and use

git read-tree -mu HEAD

to “clean up” unwanted files. Or you can set up sparse checkout before you clone the repo

D:\tmp>mkdir ljrtest

D:\tmp>cd ljrtest

D:\tmp\ljrtest>git init
Initialized empty Git repository in D:/tmp/ljrtest/.git/

D:\tmp\ljrtest>git remote add origin https://github.com/openhab-scripters/openhab-helper-libraries

D:\tmp\ljrtest>git config core.sparseCheckout true

D:\tmp\ljrtest>copy ..\sparse-checkout .git\info\
1 file(s) copied.

D:\tmp\ljrtest>git pull origin master
remote: Enumerating objects: 3591, done.
remote: Total 3591 (delta 0), reused 0 (delta 0), pack-reused 3591R ), 7.00 MiB | 6.95 MiB/s
Receiving objects: 100% (3591/3591), 9.26 MiB | 7.22 MiB/s, done.
Resolving deltas: 100% (1786/1786), done.
From https://github.com/openhab-scripters/openhab-helper-libraries
* branch master -> FETCH_HEAD
* [new branch] master -> origin/master

D:\tmp\ljrtest>dir
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> .github
07/03/2019 09:07 AM <DIR> Community
07/03/2019 09:07 AM <DIR> Core
0 File(s) 0 bytes
5 Dir(s) 386,515,042,304 bytes free

D:\tmp\ljrtest>dir .\Community
Volume in drive D is DATA
Volume Serial Number is D8E9-3B61

Directory of D:\tmp\ljrtest\Community

07/03/2019 09:07 AM <DIR> .
07/03/2019 09:07 AM <DIR> ..
07/03/2019 09:07 AM <DIR> OpenWeatherMap
0 File(s) 0 bytes
3 Dir(s) 386,515,042,304 bytes free

Using sparse checkout, no one else has to do anything. Configure your client to get the files you want, and you’re set.

 

Git Commands

There are a few git commands that we use when working with the OpenHAB and helper library repositories. The OpenHAB Eclipse project sets uses a split push/pull repository where the *fetch* repo is the organization and the *push* repo is your personal repo. This is reasonable because you do not have permissions to write to the organizational repository. You can use the same split-repository setup for other projects. Clone the project either from the organization’s repo, and then change the push URL to your personal repository.

# Show list of remotes
[lisa@linux ~]# git remote -v
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/openhab-scripters/openhab-helper-libraries (push)

# Set push remote to PERSONAL repository
[lisa@linux ~]# git remote set-url –push origin https://github.com/ljr55555/openhab-helper-libraries

# Show list of remotes — verification step
[lisa@linux ~]# git remote -v
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/ljr55555/openhab-helper-libraries (push)

While the split repository setup prevents accidentally attempting to push changes to a repo to which you lack write access, I find it a little confusing. Instead, I add specific repos for ORG (the organizational repo) and my personal repo.
The drawback to this configuration is that you *can* attempt to push changes directly to the organization repo — which will either yield an error because you lack access or will inadvertently publish code in the org repo because you don’t lack access.

# Add ORG repo with organizational repo URL
[lisa@linux ~]# git remote add ORG https://github.com/openhab-scripters/openhab-helper-libraries
# Add LJR repo with personal fork URL
[lisa@linux ~]# git remote add LJR https://github.com/ljr55555/openhab-helper-libraries
[lisa@linux ~]# git remote -v
LJR https://github.com/ljr55555/openhab-helper-libraries (fetch)
LJR https://github.com/ljr55555/openhab-helper-libraries (push)
ORG https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
ORG https://github.com/openhab-scripters/openhab-helper-libraries (push)
origin https://github.com/openhab-scripters/openhab-helper-libraries (fetch)
origin https://github.com/ljr55555/openhab-helper-libraries (push)

# Scenario: Someone has updated ORG master branch
# I want to incorporate those changes into PERSONAL master branch and push them into my repo
[lisa@linux ~]# git checkout master # Switch to your local master branch
[lisa@linux ~]# git fetch ORG/master # Get changes from Organization master
[lisa@linux ~]# git rebase ORG/master # Apply those changes to local master
[lisa@linux ~]# git push –force LJR master # Overwrite personal repo master with updated info

# Scenario: Someone has updated ORG master branch.
# I want to incorporate those changes in PERSONAL lucid-migration branch
[lisa@linux ~]# git checkout master # Switch to your local master branch
[lisa@linux ~]# git fetch ORG/master # Get changes from Organization master
[lisa@linux ~]# git rebase ORG/master # Apply those changes to local master
[lisa@linux ~]# git checkout lucid-migration # Switch back to your local lucid-migration branch
[lisa@linux ~]# git rebase –preserve-merges master # Rebase your local lucid-migration (checked out branch) onto local master
[lisa@linux ~]# git push –force-with-lease LJR lucid-migration # Overwrite personal repo lucid-migration branch with updated info

 

And a few misc commands that I want to remember
# Check username and email
[lisa@linux ~]# git config –list

# Set username and e-mail address
[lisa@linux ~]# git config –global user.name “FIRST_NAME LAST_NAME”
[lisa@linux ~]# git config –global user.email “MY_NAME@example.com”

# merge gone bad, bail!
[lisa@linux ~]# git merge –abort

# Forgot to add sign-off on commit
[lisa@linux ~]# git commit –amend

Off Label Use: Git

I store a lot of code in Git-based systems, and I read through parts of my state’s Revised Code on occasion. But it never occurred to me to store the Revised Code in a Git-based system

Imagine being able to comment on a pull request and have your commentary considered during the public hearings for the proposed change. Sure, there’s elitism to assuming everyone’s got a computer and time to peruse proposed changes. But not more than assuming everyone’s got an hour to head over to Town Hall and attend the public hearing. 

And the storage system retains history for you — a pull request created by government officials due to newly enacted laws could reference the law itself. If you’re convinced the auxiliary building was totally in compliance of zoning regs when you built it … you could run through the history and identify exactly when the setback requirements changed. 

New Process Police

As an operational support group, we did not have a software development methodology. Doesn’t mean we didn’t develop software — one of the great things within operational support is the ability to automate day-to-day tasks to reduce workload. Why have someone check for application patches when a process can watch an RSS feed or file repository and notify us when there’s an update. Why have someone clickity-click provisioning users into groups when the user can make a web request, the group owner can approve the request, and an automated process can add the user into the group? The end result of our automation programming is, well, quite a bit of software.

And with a small number of people, informal application development worked. Wasn’t ideal, but it worked. If you want to write in Java while I use C# … not ideal, but the alternative is that one of us needs to learn a new programming language. Problem is the next guy we hired uses VBS, the next guy uses PowerShell … and I’ll use perl for simpler processes. Then someone starts tweaking my code and buggers it up … and we’ve got to figure out what happened and roll back based on some tape backup.

To get our internal software development processes organized, I developed a process. And ran a training session so everyone was familiar with both the process and the tools. Some of us have used the process well — don’t edit production code, clone the repo locally, make a branch for your edits, test it and have another group member sign off on the changes, merge your branch back into master, test more, then pull the code into production. The majority, it seems, have not followed the process at all. Changes are made to the code running in production, not incorporated into the Git repo. Six months after the new development process went in place, half of our code has improperly made changes!

To an extent, I consider this a management problem … if the department doesn’t want software development to be a free-for-all, then the department managers need to ensure their staff follows the process. If the department wants everyone to do their own thing — then get rid of the process and declare our methodology as “do whatever you want”. The challenge for managers, though, is that they don’t know that someone has edited code in production and failed to commit their changes into the repo. If only there were some way to watch for improperly edited code and alert us promptly.

Other scripts I’ve found to perform a similar function attempt to parse ‘git status’ to identify all sorts of issues — but that doesn’t address the specific problem that I’ve got. To facilitate identifying offenders, I wrote a quick Python script that searches a directory tree for git repositories and alerts us when changes have not been staged for commit. If you’ve staged the changes for commit, that won’t be identified. But the particular problem we encounter frequently … there are alerts for that.

New Git Server

I love GitLab, but it’s resource intensive and I wasn’t able to run it on the current hardware. Getting a new server has been a slow process, so I needed another solution. We use the Bonobo Git Server at work — it’s really simple and doesn’t provide all of the analysis and workflow functionality of GitLab (e.g. there are no pull requests!) … but it does keep revisions of code to which you can revert. And that’s better than trying to figure out what changed and broke a program. So I set one up at home.

Super simple — except they forget a few steps — like they don’t mean you need to install .NET 4.6. You need to go in the Add Roles & Features, under Application Development Features, and enable ASP.NET. 4.5 worked fine for me. You’ve also got to restart your IIS MMC if you left it running, and change the application pool over to one that actually uses ASP.NET.

Once that was installed, a few quick changes to web.config enabled AD-based authentication. And now we’ve got a git server at home that I can leave running 24×7.

Except … I tried using a new git server and got the following error:
schannel: next InitializeSecurityContext failed: Unknown error (0x80092012) –
The revocation function was unable to check revocation for the certificate.

To resolve the issue, I needed to changed from Windows to OpenSSL certificate handling:
git config –global http.sslBackend openssl

Except that blew away my config line that defines the SSL CA CRT file. Once I restored my configuration … voila, I can use the git client with my new git server
git config –system http.sslcainfo “c:\Program Files\Git\mingw64\ssl\certs\ca-bundle.crt”