Newcomer to GIT with large project

Hello,
I have been using VSS for many years to version control my products. It is obviously very long in the tooth now and I feel it is time for me to transition to GIT. VS 2013 has a GIT plugin so I hope that you folks will be able to advise me on making the transition.

The biggest issue I have is with two closely related products. These two products consist of multiple projects and the projects share many source files. VSS handles this all pretty well and when a file needs changing for one of the projects but not the other the change is done using conditional compilation. So my challenge is to move these two products to a GIT repository and I would appreciate some input as to how structure the repository.
Sid.
Sid PriceSoftware Systems Architect/DesignerAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

evilrixSenior Software Engineer (Avast)Commented:
Git is a very different beast to VSS. It will sound like a crazy and painful thing for me to suggest (but trust me when I say I've been there and learnt the hard way), the best thing you can do is break your project(s) up into small logical units of functionality and put each into its own Git repo. What you want is to get to a point where the projects are loosely coupled. Common code should go in one (or many several) repo(s). The core projects should each be in their own repo.

Will this be easy? Probably not? Will it be worth it? Oh yes. Not only will your repo management be easier, your ability to add/remove new functionality and/or modules will be greatly improved. You'll also be able to automate unit testing (using something like Jenkins) with a lot more ease. It also makes adopting a sensible branching workflow a lot simpler.

Don't forget that, unlike VSS, Git is a decentralised repo and so you will work better if you can isolate your projects into logical self contained smaller repos that you can take away and work on without too much worry that other's making changing will cause you pain when it comes to merge conflicts. Small logical repos minimise this.

I won't pretend it won't initially be painful the in the end it'll be far less painful than trying to use Git like VSS. If you're going to do that you may as well stick with VSS. Git isn't designed to be one huge monolithic repo. It works best when it's a collection of small repos that work together, each a separate mini project.
0
Sid PriceSoftware Systems Architect/DesignerAuthor Commented:
Many thanks for your response, I am sure that your approach is well worth the front-end effort. However I am still struggling to understand how to manage two interrelated products that share a significant part of the source code. There are also some source files unique to  each product.

The product is an application and a bunch of DLLs so I could split it into  a repo for the application and one for each DLL.

So, splitting the code up is probably understood, the issue remains of how to organize the repos for the two products. It almost sounds like running Product_1 and Product_2 master branches in parallel. Would that work? It sounds wrong somehow but I can't think of another way to do it.
Sid.
0
evilrixSenior Software Engineer (Avast)Commented:
>>  that share a significant part of the source code
This is the mistake, thinking in terms of sharing "source code". Your projects should not share source code directly; rather, common code should be built as a library that they can both use. Forget the old ways where projects all lived in one bit directory and you could access code from one project from another. It's a completely unsustainable way of working. Modularisation is the way to go. Create libraries from your common code. Each library should be a git repo. Your "point products" use the libraries, not the source code.

>> I could split it into  a repo for the application and one for each DLL.
That's almost certainly the approach I would take.

>> It almost sounds like running Product_1 and Product_2 master branches in parallel. Would that work?
Why do you need to keep branches in parallel? You see, in git a branch is really nothing more than a named reference. Unlike other VCS (such as SVN) where a branch is a completely copy of code, in git it's nothing more than a named reference. Work in branches, feature branches, hotfix branches, release branches and use tags to syncronise releases.

My way of working is that we always cut releases from master. Each developer has his own development branch and from that he can create as many feature branches as they want. They merge features back into dev branches and dev branches are used to build QA releases. Once a build passes QA it is merged back into master. The master branch is always beta quality and ready for a release to be cut. When we cut a release from related projects we tag the projects with a common reference so we know which git references were used.
0
Get your problem seen by more experts

Be seen. Boost your question’s priority for more expert views and faster solutions

Sid PriceSoftware Systems Architect/DesignerAuthor Commented:
Well the more you write the more I think that I am going to have to leave these products controlled the way they are with VSS, very sad. The level of work you are suggesting would consume a huge amount of time. I understand your approach and given a new product that can be structured the way you describe I would be driven to take your approach. However with mature products that were just not design that way I think I have to leave things alone.
Many thanks for your input and the great link to the Git article.
Sid.
0
evilrixSenior Software Engineer (Avast)Commented:
When my company too this on we took oboist a year to plan and migrate on very large SVN report to about 25 git repository. It was a collection of data services and dependent libraries. It was a lot if work but we continued to develop in parallel. We became 3 or 4 time more efficient, but I won't lie,  it was a lot of work.

You could just migrate to one big report but you'd be missing out on a big part of what makes git so useful.
0
evilrixSenior Software Engineer (Avast)Commented:
This is an interesting article that discusses someone else's approach to this.

http://accu.org/index.php/journals/1828
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Sid PriceSoftware Systems Architect/DesignerAuthor Commented:
What a great article, thank you for your help and persistence.

I think I had a thinking breakthrough this morning. With VSS one is trained (over may, many years of usage) to think of a whole project. It occurred to me that I needed to change that thinking and think only of a repo of files. What defines a product are the build files and not some kind of snapshot of the source repos.

I can't express how much you have helped. I will mark your multiple posts as "answers", but I hope that when I attempt this magic you see my cry for help and are available.
Sid
0
evilrixSenior Software Engineer (Avast)Commented:
Hi Sid,

I'm glad I was able to give you some ideas. It is very hard to change your thinking. I originally came from an SVN background (been using Git for about 6 years now and would never user anything else).

With other repos you are of the mindset that it's one big monolithic beast and you end up with inter-references all over the place. Of course, as software engineers we wouldn't write code this way and the same principle applies to the code files themselves. They should be loosely coupled and references should only be at an interface/library level and not at a source code level. Once you realise this (and go through the pain of setting it up) our life becomes so much easier.

If you have any further queries please don't hesitate to ping back here. I'll do what I can to help.

Best regards.
0
Sid PriceSoftware Systems Architect/DesignerAuthor Commented:
Thanks.  BTW: While I am in the USA now I once had a business in Luton!
Sid.
0
evilrixSenior Software Engineer (Avast)Commented:
Are you from the UK?
0
Sid PriceSoftware Systems Architect/DesignerAuthor Commented:
Many, many years ago yes.
0
evilrixSenior Software Engineer (Avast)Commented:
Which part? I'm originally from North London. Tottenham.
0
Sid PriceSoftware Systems Architect/DesignerAuthor Commented:
So I have a related question about GIT.

Assuming my product is built from several repos how does one handle a product release? The only way I can see is that each release in the master branch for each component must be tagged to indicate the product release it is a part of. This seems a little open to errors when needing to go back to a particular  release to branch to take care of a hot-fix requirement.

Is there a more automated way to branch all the project repos?
Sid.
0
evilrixSenior Software Engineer (Avast)Commented:
You want to look a Jenkins build pipelines.
http://www.infoq.com/articles/orch-pipelines-jenkins

Basically, a pipeline is a chain of builds, one that follows the other for all the components of a particular product. As part of the pipeline you can have your repos automatically tagged. The pipeline can include build jobs to not only compile and build the binaries to to also package those binaries and deliver them to a publishing server.

Yes, you should tag each reference that is part of the release (don't forget, you are tagging a reference NOT a branch).

>> This seems a little open to errors when needing to go back to a particular  release to branch to take care of a hot-fix requirement.
 Why? If you need to hotfix you create a branch from that tag, fix the problem, create a hotfix from that branch (tag the top of the branch with the hotfix release version) and don't forget to forward-port the fix as required.

>> Is there a more automated way to branch all the project repos?
Forget about branches. In git a branch is an arbitrary thing, it's just an ever moving reference. It is a user friendly name that will always point to the tip of the current divergence from the main line. Every time you commit the branch is moved to follow the tip. Once you merge the branch you can (if you so choose) delete the branch. It's just a name. It doesn't actually exist. It's not like, for example, SVN, where it's a whole other copy of the code.

Git works differently. Git (internally) uses SHA1 references. When you create a branch you are just creating a tag, it's just that it's a special tag that follows the HEAD. A normal tag stays put. It's really hard to get used to when you come from VSS and/or SVN where branches are these magical things that are full of awe and wonder and most people are scared to use because of the mess they can get into. With Git a branch is nothing more than a pointer. It's not a copy of code, it's nothing.

One way to think of the commits in git is like a spur in a directed graph. A branch will always be pointing to the head of the spur. A tag can point to any other node. When you add something new to the list the branch is automatically changed to point to the new head. Once you merge you can remove the branch, but all the nodes still exist, all you've done is remove the user friendly pointer - the branch name.
0
Sid PriceSoftware Systems Architect/DesignerAuthor Commented:
As always ... great input thanks,
Sid.
0
evilrixSenior Software Engineer (Avast)Commented:
Any time, Sid.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Version Control

From novice to tech pro — start learning today.