Be aware: physical connection not only means the cable, it also includes the USB hubs between. And don't forget: inside your PC there is an internal hub, especially if you are using a docking station with your laptop.
If you have several Arduino boards you are free to disconnect one and connect another without rebooting your computer. Give your operating system some time to find the newly connected one.
Unfortunately, this is not true when a Serial Terminal connection between PC and Arduino is in operation. In this case, it is very likely that you loose your COM-port until you reboot your computer.
You might want to disconnect the USB cable to add or remove shields or other hardware connected to your Arduino board safely. To be on the safe side close any Terminal connection, unplug the USB and then handle the connected hardware.
If your Arduino resides on a breadboard or a standalone solution you might consider installing a power switch to temporarily disable your board and handle the components without any risk.
... added after a long while
As times goes by, you will come across alot of Arduino-compatible modules. And one fine day, there will be one which strictly refuses to operate as expected. If you are lucky, it's only because of a missing or wrong bootloader. This can be fixed easily if you got a second Arduino which is working. See "installing bootloader" to solve this type of problem.
The more wicked case is when the USB interfaces will not communicate with any possible driver you install.
The standard Arduino has the Atmel MEGA32U2 (or similar). But you also might read
"PL2303TA G14211E TD5W18300"
"FTDI 1308-C GO280601 FT232RL"
printed on that chip.
Wait: before you throw it away, take a look at this
It will take you some time to perform all the steps (you will have to press the STOP key repeatedly) and at the end you are done.