  A novel property of the quantum critical heavy fermion superconductor
$\beta$-YbAlB$_4$ is revealed theoretically. By analyzing the crystalline
electronic field (CEF) on the basis of the hybridization picture, odd parity
CEF is shown to exist because of sevenfold configuration of B atoms around Yb,
which breaks the local inversion symmetry. This allows onsite admixture of 4f
and 5d wavefunctions with a pure imaginary coefficient, giving rise to the
magnetic toroidal (MT) degree of freedom. By constructing a realistic minimal
model for $\beta$-YbAlB$_4$, we show that onsite 4f-5d Coulomb repulsion drives
charge transfer between the 4f and 5d states at Yb, which makes the MT
fluctuation as well as the electric dipole fluctuation diverge simultaneously
with the critical Yb-valence fluctuation at the quantum critical point of the
valence transition.
