Git Hacks and Tricks

From OPeNDAP Documentation

1 Git resources

2 Setup a username and access token for GitHub

git config --global github.user <name>
git config --global github.token <token>
where the token is made using the instructions at https://help.github.com/articles/creating-an-access-token-for-command-line-use

3 Git configuration commands

simplify working with submodules (the BES repository uses submodules)
git config --global status.submoduleSummary true

4 How to trigger a Coverity Scan

The Coverity scan of the BES will also scan all of the modules, so if you want to see the scan results for changes to a module, just scan the BES. Assumption: The module's changes are on its master branch and the current BES master branch is building.

Follow these steps

git checkout coverity_scan
Switch to the Coverity scan branch. The .travis.yml file configures Travis to look for pushes to this branch and funnels the code into Coverity. The handler code is taken from their master branches.
git merge master
Merge changes onto the coverity_scan branch so that there is something to push...
git push
The push operation will trigger the remote build, which runs the code through Coverity when it happens on this 'special' branch.

Oops... What to do when the push fails with a message that "Updates were rejected because the tip of your current branch is behind its remote counterpart. Integrate the remote changes (e.g., 'git pull ...') before pushing again. See the 'Note about fast-forwards' in 'git push --help' for details."

git pull
Get those more recent edits
git log
Take a look at the log to see the commit hashes and then use "git diff <hash>" to see what's what. It's often an update in a handler's submodule hash. See the screen dumps below if you need more information about what this all looks like.
git push
Push the result of merging those 'edits' you pulled in the first step.
4.1 Screen dumps of a 'conflict' when pushing a recent merge of info to the coverity_scan branch
edamame:bes jimg$ git log
commit b0c240d02a9132093bfba535df7db15b38163498
Merge: 306e9bc 87c84df
Author: James Gallagher <jgallagher@opendap.org>
Date:   Mon Jan 11 12:09:05 2016 -0700

    Merge branch 'coverity_scan' of https://github.com/opendap/bes into coverity_scan

commit 306e9bc03d9fd5a0bc0972dbed55c678acd3ef31
Merge: 91f0dbe 77e726f
Author: James Gallagher <jgallagher@opendap.org>
Date:   Mon Jan 11 12:08:36 2016 -0700

    Merge branch 'master' into coverity_scan

...


edamame:bes jimg$ git diff 306e9bc
diff --git a/modules/hdf4_handler b/modules/hdf4_handler
index cc2cc0a..0387d23 160000
--- a/modules/hdf4_handler
+++ b/modules/hdf4_handler
@@ -1 +1 @@
-Subproject commit cc2cc0a84f108fc8ebfbca58ac33ae49688695ff
+Subproject commit 0387d233554a14c240f8423aacf249d8ea451bc1
diff --git a/modules/hdf5_handler b/modules/hdf5_handler
index 4063f71..214d064 160000
--- a/modules/hdf5_handler
+++ b/modules/hdf5_handler
@@ -1 +1 @@
-Subproject commit 4063f71829af1189a29be9908f455eb55e985eea
+Subproject commit 214d0647d708b77a9ed38a7ef8814efa7f55ce44
edamame:bes jimg$ 

5 Cheat sheet items

These are simple things that are not really obvious from the git book or other sources

I keep getting a 'Permission Denied (publickey)' error when I push!
Follow these steps to make and use a public/private key pair for github
cd ~/.ssh.
Within .ssh, there should be these two files: id_rsa and id_rsa.pub. If those two files are not there...
To create the SSH keys, type ssh-keygen -t rsa -C "your_email@example.com".
Open id_rsa.pub in a text editor, and copy the contents, exactly as it appears, of id_rsa.pub
and paste it into GitHub and/or BitBucket under the Account Menu (upper right corner) Settings > SSH Keys.
I just made a perfectly good commit to the wrong branch. How do I undo the last commit in my master branch and then take those same changes and get them into my upgrade branch?
If you haven't yet pushed your changes, you can also do a soft reset:
git reset --soft HEAD^
This will revert the commit, but put the committed changes back into your index. Assuming the branches are relatively up-to-date with regard to each other, git will let you do a checkout into the other branch, whereupon you can simply commit:
git checkout [-b] branch
git commit
The disadvantage is that you need to re-enter your commit message.
How to see a list of 'conflicted' files after a merge
git diff --name-only --diff-filter=U
How to see the difference between to commits
git diff <commit-hash-1> <commit-hash-2>, e.g., git diff 0da94be 59ff30c
...for a specific file: git diff <commit-hash-1> <commit-hash-2> -- <file>
...and don't forget the shorthand for the hashes: git diff HEAD^^..HEAD -- main.c where HEAD^ is the parent of HEAD. HEAD{n} is the Nth parent.
How to see the different remote branches
git remote show origin
Fetch all the branches on origin
git fetch origin
How do I list the remote branches (that have been fetched)?
git branch -a
How do I switch to a branch from a remote origin?
git checkout -b test origin/test
or, with newer versions of git: git checkout test
How do I see what would be pushed to a remote repo?
git push --dry-run
git diff origin/master # Assumes you have run git fetch, I think
git diff --stat origin/master # --stat just shows the file names stats, not the diffs
To get a specific file from a specific branch
git show dap4:./gdal_dds.cc > gdal_dds.dap4.cc You can use checkout instead of show and that will overwrite the file.
the general syntax is object (that's the 'dap4:./gdal_dds.cc' part) and it can use the ^ and ~n syntax to specify various commits on the given branch. A SHA can also be used.
How to change the 'origin' for a remote repo
git remote set-url origin git://new.url.here (https URLs work too...)
How to push a local branch to a remote repo
git push -u origin feature_branch_name
How to make and track a new (local) branch
How to cause Travis CI to skip a build
Add [ci skip] to the log text. See the about topic on amending a commit log, which can be handy
git checkout -b <branch name>
How to track a remote branch
git checkout --track origin/serverfix or git checkout -b sf origin/serverfix
How do I make an existing local branch track an existing remote branch?
git branch --set-upstream upstream/foo foo where upstream is probably actually origin.
Commited my code, then made a bunch of changes that just seem like a bad idea in retrospect. How do I go back to my previous commit for everything in a directory? I don't care if I loose all my changes since the last commit.
git reset HEAD --hard (Note that this is one of the very few git commands where you really cannot undo what you have done).
How to undo a commit (that has not been pushed)
git reset --soft HEAD~1. This leaves the files in their changed state in your working dir so that you can edit them and recommit. You can also change to a different branch and commit there, then change back.
In the above case, To reuse the old commit message
git commit -c ORIG_HEAD <-- This works because 'reset' copied the old head to .git/ORIG_HEAD. If you don't need to edit the old message, use -C instead of -c.
How to delete a remote brnach
git push origin --delete serverfix The data are kept for a little bit - before git runs garbage collection - so it may be possible to undo this.
How to delete a local branch
git branch -d the_local_branch and delete the remote branch you were tracking with the same name git push origin :the_remote_branch
How to I set up a git cloned repo on a remote machine so I don't have to type my password all the time?
This page shows how to make a PKI key-pair with a secure password, configure the machine to remember the password using ssh-agent and upload the public key to your github account so it'll use the key for authentication. https://help.github.com/articles/generating-ssh-keys/
How can I know which branches are already merged into the master branch?
git branch --merged master lists branches merged into master
git branch --merged lists branches merged into HEAD (i.e. tip of current branch)
git branch --no-merged lists branches that have not been merged
By default this applies to only the local branches. The -a flag will show both local and remote branches, and the -r flag shows only the remote branches.
Switching remote URLs from HTTPS to SSH
git remote -v
# origin https://github.com/USERNAME/REPOSITORY.git (fetch)
# origin https://github.com/USERNAME/REPOSITORY.git (push)
git remote set-url origin git@github.com:USERNAME/OTHERREPOSITORY.git
git remote -v
# Verify new remote URL
# origin git@github.com:USERNAME/OTHERREPOSITORY.git (fetch)
# origin git@github.com:USERNAME/OTHERREPOSITORY.git (push)
Amending the commit message
git commit --amend
git commit --amend -m "New commit message"
How do I revert a commit after if it has been pushed?
Given:
e512d38 Adding taunts to management.
bd89039 Adding kill switch in case I'm fired.
da8af4d Adding performance optimizations to master loop.
db0c012 Fixing bug in the doohickey
If you just want to revert the commits without modifying the history, you can do the following:
git revert e512d38
git revert bd89039
Alternatively, if you don’t want others to see that you added the kill switch and then removed it, you can roll back the repository using the following (however, this will cause problems for others who have already pulled your changes from the remote):
git reset --hard da8af4d
git push origin -f localBranch:remoteBranch
The gitlog-to-changelog script comes in handy to generate a GNU-style ChangeLog.
As shown by gitlog-to-changelog --help, you may select the commits used to generate a ChangeLog file using either the option --since:
gitlog-to-changelog --since=2008-01-01 > ChangeLog
or by passing additional arguments after --, which will be passed to git-log (called internally by gitlog-to-changelog):
gitlog-to-changelog -- -n 5 foo > last-5-commits-to-branch-foo
Amending the commit message
git commit --amend
Tagging stuff
git tag will list the existing tags
git tag -a <tag name> adds a new tag
git push origin <tag name> pushes that tag up to the server origin
git push origin --tags pushes all new tags up to origin
How to resolve conflicts in a submodule when you've just merged master down to a branch
Run git status - make a note of the submodule folder with conflicts
Reset the submodule to the version that was last committed in the current branch:
git reset HEAD path/to/submodule
At this point, you have a conflict-free version of your submodule which you can now update to the latest version in the submodule's repository:
cd path/to/submodule
git pull origin SUBMODULE-BRANCH-NAME
And now you can commit that and get back to work.
How to move a submodule into the main repo
If all you want is to put your submodule code into the main repository, you just need to remove the submodule and re-add the files into the main repo, follow the prescription below. If you want to see how to add the branches, history, etc. to the repo, see http://stackoverflow.com/questions/1759587/un-submodule-a-git-submodule:
git rm --cached submodule_path # delete reference to submodule HEAD (no trailing slash)
git rm .gitmodules # if you have more than one submodules, you need to edit this file instead of deleting!
rm -rf submodule_path/.git # make sure you have backup!!
git add submodule_path # will add files instead of commit reference
git commit -m "remove submodule"
Checking out a tag
You will not be able to checkout the tags if its not locally in your repo so first you have to fetch it all.
First make sure that the tag exists locally by doing
  1. --all will fetch all the remotes.
  2. --tags will fetch all tags as well
git fetch --all --tags --prune
Then check out the tag by running
git checkout tags/<tag_name> -b <branch_name>
Instead of origin use the tags/ prefix.
How to remove old/unused/deleted branches
git remote prune origin prunes tracking branches not on the remote.
git branch --merged lists branches that have been merged into the current branch )but maybe including master so be careful about the next part).
xargs git branch -d deletes branches listed on standard input.
Be careful deleting branches listed by git branch --merged. The list could include master or other branches you'd prefer not to delete.
How do I merge just one file?
A simple command already solved the problem for me if I assume that all changes are committed in both branches A and B
git checkout A
git checkout --patch B f
The first command switches into branch A, into where I want to merge B 's version of the file f. The second command patches the file f with f of HEAD of B. You may even accept/discard single parts of the patch. Instead of B you can specify any commit here, it does not have to be HEAD.

6 Continuous Integration builds involving submodules

There are two ways to handle getting a CI build to run when you've editing a submodule used by the BES. The **best** way is to use a branch of the BES to runt he build as part of a **pull request** and is decribed as option number one below. Another way is to use the master branch of the BES and is describe as the second choice.

6.1 Using a BES branch and a GitHub pull request

This is the better of the two ways. It requires a bit more work but does not introduce code to the master branch before that code has passed the CI build.

Once your work on the submodule is complete:

  • Commit and push the code in your submodule's branch
git commit
git push
  • Goto the top of the BES and checkout a new branch. Choose a name that is similar to the name of the branch used for the submodule's changes
cd bes
git checkout -b <name>
  • Commit the submodule's current commit hash to the .gitmodules file (easier than is sounds)
'git commit -a' or 'git add <path to submoule>' and then 'git commit'
  • Push this to GitHub
git push
  • Goto Github and issue a pull request for the BES <name> branch.

This will trigger a CI build of that branch. This does not change the BES master branch at all, which is the goal here - to build without affecting the master branch.

Once the build works, merge the submodule branch to the submodule's master. Then delete the BES <name> branch and make sure to update the BES master so that it references the new master branch for the submodule.

6.2 An alternative, that uses the BES master branch

Once your code is in committed and pushed on the master branch, go to the top of the bes project and run ‘git commit -a’. This will prompt you with a commit that shows a new HDF5 handler version. Add a commit message (e.g., “New HDF5 handler version”) and then push. This works because the new hash code for the commit that is on the master branch is recorded in a hidden file managed by git (named ‘.gitmodules’). Since that file is changed, the push results in changed files in bes and than triggers a new build. The new build will reference the new commit hash for your latest changes on master and that hash will be checked out for the build.

7 Merging in a branch where submodules have been removed and replaced with directories

Recently (02/03/17) we dropped most of the submodules in the bes (except for the hdf*_handler modules) and replaced them with regular code directories.

What to do if you have an existing branch the you need to update with the new master:

  1. Check in all of your changes.
  2. Push your branch to github.
  3. Start with a brand new clone for the bes repo from github:
    git clone https://github.com/opendap/bes
  4. DO NOT RUN submodule init
  5. In the new clone, checkout your branch.
  6. In the modules directory remove crucial submodules:
    rm -rf csv_handler dap-server debug_functions fileout_* fits_handler freeform_handler gateway_module gdal_handler ncml_module netcdf_handler ugrid_functions w10n_handler xml_data_handler
  7. Now, merge the master to your branch:
    git merge master
  8. Cleanup any trouble (there should not be much)
  9. Build and test
  10. Merge your branch to master when you're ready.