Tails on comets are formed when radiation from the sun causes the material in the nucleus of the comet to vaporize.
A tail on a comet is not like a tail on a dog. A tail on a dog always goes on the back end, but that usually isn't true of a comet. A comet's tail has two parts: the dust tail and the ion or gas tail. The dust tail is formed by particles of dust that are dislodged from the nucleus, and tend to stream out behind the comet in its orbital path. The gases that are dislodged are more strongly affected by magnetic forces than the dust that is dislodged, so it streams out in the direction of the solar wind, away from the sun.
Since comets never move directly toward the sun, it is rare for a single tail to be seen to stream out right behind a comet like a contrail from behind a jetliner. The observed directions of the twin tails of comets depends on the relative orientation of the comet's orbit, the sun, and the Earth.
To Learn More:
NASA Page on Comets
Comets: Creators and Destroyers, by David H. Levy (Amazon link.) (HOLLIS Link)