Comets are different from the other objects in our solar system; unlike meteors and moons, they are "dirty snowballs," made up of frozen water, frozen gas, and a slurry of other elements. Because they are made of different stuff, comets look different from other celestial objects, especially when they approach the sun. The unique nature and appearance of comets led people for a long time to think of them as supernatural omens, but now we know that they are "leftovers" from the formation of the Solar System, and that they come from regions called the "scattered disc" and the "Oort Cloud."
The scattered disc, also called the Kuiper Belt, isn't really a disc, and the Oort Cloud isn't really a cloud; both are regions in space populated by billions of icy objects like comets.
Both are unimaginably distant. Objects in the scattered disc are between 30 and 100 times further from the Sun than Earth--that's almost TEN BILLION miles. The Oort Cloud is even more distant--astronomers think that it could take light from the Sun up to a year to reach the most distant objects in the Cloud. The Oort Cloud is so dark and far away that astronomers can't directly observe it; they hypothesize that the Cloud exists based on the behavior of comets.
How did the Oort Cloud and the scattered disc get way out there? Scientists think that the light elements that make up the objects in the Oort Cloud and scattered disc (the same elements that are in comets) were sling-shotted out of the inner solar system by Neptune and other outer planets while the solar system was still being formed.
Scattered disc objects are still in unstable orbits, which can be changed by the passing of Neptune or other objects. Objects in the scattered disk are constantly being sent inward toward the sun, where they might eventually become comets, or outward to the Oort Cloud.
Comets that come from the scattered disc are "short-period," meaning that we can usually predict a rough date when the comet might return to the neighborhood of Earth and the Sun. Comets from the Oort Cloud are "long period," meaning that once an Oort Cloud comet has passed by, it isn't returning any time soon.
Orbits in the Oort Cloud, like those in the scattered disc, are subject to change, but since the Oort Cloud is so far away from the planets, it isn't planetary objects that perturb orbits; it is overall "tidal effects" coming from the gravitational pull of the Milky Way Galaxy, and from nearby stars. The effect of distant objects on the Oort Cloud is similar to the effect the moon has on ocean tides.
Read More about the Oort Cloud and the scattered disc:
Solar Views: Oort Cloud
Info from NASA
Wikipedia: Scattered Disc, (Featured Article)*
Wikipedia: Oort Cloud, (Featured Article)*
*Why are we linking to Wikipedia? Nature reports that, on science, it's about as accurate as Encyclopedia Britannica.