Linux on external hard disk

This works fine. The internal boot restriction is only for windows.

Use a boot media to boot Linux installer and have your external HDD plugged in; selecting it in the process.

Since it is a removable drive, make sure the installer writs the boot loader on that drive! Then use BIOS boot device select to boot from the external drive. If your BIOS does not have this menu, set the primary boot order to boot first from USB and internal HDD second.
I would recommend using UEFI boot for this setup, since it can dynamically detect bootable external disks; but I think Vista leaves you out of options here.

You might want to consider running the linux as a virtual machine.  I find it much more convenient since you can easily switch between your windows and the virtual machines and you don't have to have linux partitions on the system either.  The 'partition' for virtual machine is in a container file within the windows file system which can be put anywhere - so just put in on the external.  

Virtual box is free,
https://www.virtualbox.org/

The setup is simple,  you setup a 'machine'  then run it and install linux just like you would on a regular computer.

This may not work  well for everything as there is some performance lost since you are running both windows and linux at the same time.  So may depend on what you are going to be doing with linux.

I will be doing C++, Java, Scripting development on Linux with performance oriented applications. So what do you suggest?

Hm, i did not suggest this because of the limited space available ... but of a VM works fine in most cases.

If you do a lot of compiling, go for a physical machine. If performance is needed, this is essential.

ok. But can I use external hard disk to install VM so that all linux operations will happen there instead of my laptop hard drive? My current space already gets overflowed during page swapping etc when I run any high performance app like photoshop. Please advise.

When you say physical machine you mean "not to go for VM"? Yes. Lot of compiling will be there and with lot of test data. Do you think I should go for new machine or should replace my hard disk with bigger one rather than buying external hard disk?

If you can, replacing your internal Hard Drive preferably for an SSD would be the best option indeed. You will not need that much space with Linux since there are no huge applications; my 80GB Fedora partition serves me well.

With physical machine I mean a non virtual host, classic install. The performance you get with USB will not be great though. But still better then with virtual box or any VM.

Hint: You can clone over your old system with linux tools (or clonezilla, which is debian, for instance). Then, resize Windows to your needs and only after Windows runs install Linux. It is in the nature of Windows do wipe any other bootable option other than Windows when you install it.

Actually, I dont really want to boot up with linux that much. I more likely want to use linux like we use it in office thru putty. So environment should be Windows always but I want to open linux OS as a "terminal" thru something like "putty" and should be able to write/compile large c++/java/perl programs. Now I undetstand that in order to use Linux as a "terminal" I need to have it installed on some "server". But I want that "server" thing  to start (or boot) on automatically when I start my PC. And then its the same user experience that we get in office. Is this possible to achieve?

If you want that, running Linux in a vm would ideal of course. Boot it up on system start of your windows. You do not even need to emulate a graphic subsystem but a serial port would do. use putty for a ssh session.

Keep in mind, depending on how large your builds will be, the compile time  will be a multiple of a physical maschine; but this might still be ok for you. testing this setup will cost you nothing but time.

Didn't understand this - "You do not even need to emulate a graphic subsystem but a serial port would do. ".  Can you please elaborate? Just to add, my c++/java development will not involve any UI development. Its mostly non-UI high performance data processing apps.

Depending on the hypervisor you use, you can generally choose what hardware to emulate. Emulation is always a process witch might involve considerable overhead. With your Linux, you would not need emulated graphics; basically making it use less resources and a little bit faster.
Other then windows, Linux can run without a graphic card only with a serial com port attached.
To archive speed in virtualization, a common way is using the virtual machine's 'help' with the emulation; e.g with a special driver for some hardware features. This is called para virtualization. So, enable this when ever you come across it to get more performance.

ok. Thanks. This all buying new hardware and testing set up will take me some time. So please bear with me.

Why don't you use VMWorkstation kind of software and put your VM storage on new disk if space issue is there.

By this you will be using both Windows :( and Linux :)