Setup git mirror
From OMAPpedia
Contents |
[edit] Scope
The purpose of this wiki is to provide detailed instructions on how to mirror a GIT server on a local mirror. The main reasons to do so are:
- Reduce the load/bandwidth on the main public GIT server
- Improve download time on the client side
- Avoid messing up with firewall in case you are behind a corporate firewall
For example, with the instructions below you can mirror all the GIT trees on http://git.omapzoom.org or/and http://dev.omapzoom.org. Unlike http://git.kernel.org site which is mirrored all around the world, TI servers aren't. TI servers are located in US, so if you are in a far country it can easily take several hours to fetch a TI kernel tree. As such it is much better to setup a local mirror.
The setup described below is being used inside TI to replicate the main TI servers.
[edit] Git server setup
You don't actually need to setup a real git server on your end, a local hard disk is enough. If you need or want to setup a HTTP front-end (such as gitweb) there are many instructions on the Internet.
[edit] git-mirror scripts
The following scripts are available: http://github.com/ndechesne/git-mirror.
The script 'git-mirror' can be used to create a local mirror of an existing GIT server that is using gitweb interface, such as http://git.omapzoom.org
The script will first fetch the list of all projects on the server, and then it will update the local mirror:
- adding new created trees
- removing trees that were deleted on the main server
- updating existing trees
Note: for now this is only working if the remote GIT server is using the gitweb interface, since it relies on the gitweb project list download link.
This example will download all trees available on omapzoom git server.
./git-mirror -o /var/www/git/ -s http://git.omapzoom.org -g git://git.omapzoom.org
Once the initial mirror is done, running this command again will only fetch the changes since the last update. As such it is very convenient to run this command as a cron job daily.
Don't forget to use the '-n' option first to check what will be done by the script.
All projects from the remote Git server are mirrored in a sub directory called after the name of the server (e.g. /var/www/git/git.omapzoom.org in the previous example).
If you encounter any issue with the script, please report them using the project page on the github page mentioned above.
[edit] How to use the local mirror
Of course the first time you run the script it will take a while since it will download all the trees. Once the trees are locally mirrored, you need to consider them as read-only since any commit that would be pushed on the mirror would not be uploaded on the main server!
[edit] Git configuration to fetch from the local mirror
The easiest way to fetch from the local mirror, is to add the following configuration in your user's Git configuration file:
[url "/var/www/git/git.omapzoom.org/"]
insteadOf = git://git.omapzoom.org/
With this configuration, you can continue to clone or fetch using the regular link, and Git will automatically use the local mirror. For example, you just do this as usual:
git clone git://git.omapzoom.org/platform/omapmanifest.git
If you have setup a local Git server, you can use something like this:
[url "git://git.mycompany.com/git.omapzoom.org/"]
insteadOf = git://git.omapzoom.org/
[edit] How to push if you have commit rights
If you happen to mirror Git trees where you have commit rights, then you will need some extra configuration because you don't want to commit your changes to your local mirror.
If you clone or fetch using the SSH protocol, then it won't go through the mirror, but to the remote Git server. As such the recommended configuration is the following:
- clone using the local mirror as explained in the previous section. As such the origin remote will be your local mirror
- add a new remote for the actual mirrored public Git server using the SSH protocol, so that you can push there.
For example, you can run the following commands:
git clone git://git.omapzoom.org/platform/omapmanifest.git git remote add main :/platform/omapmanifest.git
In this case, you can then run the following commands to fetch and or push:
git fetch origin git push main
If you push changes to a tree on the main public Git server, the changes will be fetched in your local mirror during the next sync.
