An ultrahigh-Q optical microcavity coupled with a tapered fiber is an ideal system for the cavity quantum electrodynamics (CQED). In particular realizing this system at cryogenic temperature is vitally important and has been recently explored for various CQED applications including solid-state atom-photon strong coupling, vibrational mode cooling, and photonic quantum gates. These cryogenic fiber-coupled microcavity systems, however, suffer from mechanical vibrations due to cooling systems and distortions caused by large temperature change. These factors may cause the degradation in polarization of probe light field in the system. Here we report the analysis of the polarization state in a tapered-fiber-coupled microsphere cavity at cryogenic temperatures. By scanning the wavelength of the probe light at around 637 nm, which can be used for the diamond nitrogen vacancy centers, the spectral analysis of the polarization state was performed at 8-30 K. We have found that the degree of polarization (DOP, classical analogue of purity) at cryogenic temperatures does not show significant change compared to that measured at room temperature. This fact indicates that the system can conserve the polarization at low temperature to the extent comparable to that at room temperature, which is enough for the evaluation of the quantum phase gate.