  We report on the results of calibrating and simulating the instrumental
polarization properties of the ESO VLT adaptive optics camera system
NAOS/CONICA (NACO) in the Ks-band. We use the Stokes/Mueller formalism for
metallic reflections to describe the instrumental polarization. The model is
compared to standard-star observations and time-resolved observations of bright
sources in the Galactic center. We find the instrumental polarization to be
highly dependent on the pointing position of the telescope and about 4% at
maximum. We report a polarization angle offset of 13.28{\deg} due to a position
angle offset of the half-wave plate that affects the calibration of NACO data
taken before autumn 2009. With the new model of the instrumental polarization
of NACO it is possible to measure the polarization with an accuracy of 1% in
polarization degree. The uncertainty of the polarization angle is < 5{\deg} for
polarization degrees > 4%. For highly sampled polarimetric time series we find
that the improved understanding of the polarization properties gives results
that are fully consistent with the previously used method to derive the
polarization. The small difference between the derived and the previously
employed polarization calibration is well within the statistical uncertainties
of the measurements, and for Sgr A* they do not affect the results from our
relativistic modeling of the accretion process.
