  In the inert scalar doublet framework, we analyze what would be the effect of
a $B-L$ asymmetry that could have been produced in the Universe thermal bath at
high temperature. We show that, unless the "$\lambda_5$" scalar interaction is
tiny, this asymmetry is automatically reprocessed in part into an inert scalar
asymmetry that could be at the origin of dark matter today. Along this
scenario, the inert mass scale lies in the few-TeV range and direct detection
constraints require that the inert scalar particles decay into a lighter dark
matter particle which, as the inert doublet, is odd under a $Z_2$ symmetry.
