Cavitation behavior in poly(4-methyl-1-pentene) upon stretching below glass transition temperature was investigated by in situ ultra-small angle X-ray scattering technique. Strong stress-whitening was observed indicating an extensive occurrence of cavitation in the material during tensile deformation below Tg. The X-ray scattering patterns suggest oriented disc-shaped cavities with normal mostly parallel to the stretching direction occurred. Structural parameters of such cavities such as thickness, radius and tilting angle of the normal of the disc with respect to the stretching direction have been successfully calculated using a model fitting procedure. The results exhibited a two-step process of cavitation that small amount of large cavities appeared first and then small cavities were triggered extensively in the samples at larger strains. This two-step cavitation phenomenon can be weakened after the quenched sample was annealed or the sample was prepared by slow cooling. This peculiar two-step cavitation process can be understood as a result of high frozen in internal stress in quenched sample that led to local failure of the materials.