COMP0233: Research Software Engineering With Python


Working with multiple remotes

Distributed versus centralised

Older version control systems (cvs, svn) were "centralised"; the history was kept only on a server, and all commits required an internet.

Centralised Distributed
Server has history Every user has full history
Your computer has one snapshot Many local branches
To access history, need internet History always available
You commit to remote server Users synchronise histories
cvs, subversion(svn) git, mercurial (hg), bazaar (bzr)

With modern distributed systems, we can add a second remote. This might be a personal fork on github:

In [1]:
import os
top_dir = os.getcwd()
git_dir = os.path.join(top_dir, 'learning_git')
working_dir = os.path.join(git_dir, 'git_example')
In [2]:
git switch main
git remote add arc git@github.com:UCL-ARC-RSEing-with-Python/github-example.git
git remote -v
Switched to branch 'main'
Your branch is ahead of 'origin/main' by 1 commit.
  (use "git push" to publish your local commits)
arc	git@github.com:UCL-ARC-RSEing-with-Python/github-example.git (fetch)
arc	git@github.com:UCL-ARC-RSEing-with-Python/github-example.git (push)
origin	git@github.com:UCL/github-example.git (fetch)
origin	git@github.com:UCL/github-example.git (push)

We can push to a named remote:

In [3]:
%%writefile Pennines.md

Mountains In the Pennines

* Cross Fell
* Whernside
Overwriting Pennines.md
In [4]:
git commit -am "Add Whernside"
[main f953f9d] Add Whernside
 1 file changed, 1 insertion(+)
In [5]:
git push -uf arc main
To github.com:UCL-ARC-RSEing-with-Python/github-example.git
 + b6f1c44...f953f9d main -> main (forced update)
branch 'main' set up to track 'arc/main'.

Referencing remotes

You can always refer to commits on a remote like this:

In [6]:
git fetch
git log --oneline --left-right arc/main...origin/main
From github.com:UCL-ARC-RSEing-with-Python/github-example
 * [new branch]      gh-pages   -> arc/gh-pages
< f953f9d Add Whernside
< 142d60c Add github pages YAML frontmatter

To see the differences between remotes, for example.

To see what files you have changed that aren't updated on a particular remote, for example:

In [7]:
git diff --name-only origin/main

When you reference remotes like this, you're working with a cached copy of the last time you interacted with the remote. You can do git fetch to update local data with the remotes without actually pulling. You can also get useful information about whether tracking branches are ahead or behind the remote branches they track:

In [8]:
git branch -vv
  gh-pages 142d60c [origin/gh-pages] Add github pages YAML frontmatter
* main     f953f9d [arc/main] Add Whernside

Hosting Servers

Hosting a local server

  • Any repository can be a remote for pulls
  • Can pull/push over shared folders or ssh
  • Pushing to someone's working copy is dangerous
  • Use git init --bare to make a copy for pushing
  • You don't need to create a "server" as such, any 'bare' git repo will do.
In [9]:
bare_dir = os.path.join(git_dir, 'bare_repo')
In [10]:
mkdir -p bare_repo
cd bare_repo
git init --bare
Initialized empty Git repository in /home/runner/work/rsd-engineeringcourse/rsd-engineeringcourse/ch00git/learning_git/bare_repo/
In [11]:
In [12]:
git remote add local_bare ../bare_repo
git push -u local_bare main
To ../bare_repo
 * [new branch]      main -> main
branch 'main' set up to track 'local_bare/main'.
In [13]:
git remote -v
arc	git@github.com:UCL-ARC-RSEing-with-Python/github-example.git (fetch)
arc	git@github.com:UCL-ARC-RSEing-with-Python/github-example.git (push)
local_bare	../bare_repo (fetch)
local_bare	../bare_repo (push)
origin	git@github.com:UCL/github-example.git (fetch)
origin	git@github.com:UCL/github-example.git (push)

You can now work with this local repository, just as with any other git server. If you have a colleague on a shared file system, you can use this approach to collaborate through that file system.

Home-made SSH servers

Classroom exercise: Try creating a server for yourself using a machine you can SSH to:

ssh <mymachine>
mkdir mygitserver
cd mygitserver
git init --bare
git remote add <somename> ssh://user@host/mygitserver
git push -u <somename> main

SSH keys and GitHub

Classroom exercise: If you haven't already, you should set things up so that you don't have to keep typing in your password whenever you interact with GitHub via the command line.

You can do this with an "ssh keypair". You may have created a keypair in the Software Carpentry shell training. Go to the ssh settings page on GitHub and upload your public key by copying the content from your computer. (Probably at .ssh/id_rsa.pub)

If you have difficulties, the instructions for this are on the GitHub website.