  We describe a comprehensive methodology for developing user-voice
personalized automatic speech recognition (ASR) models by effectively training
models on mobile phones, allowing user data and models to be stored and used
locally. To achieve this, we propose a resource-aware sub-model-based training
approach that considers the RAM, and battery capabilities of mobile phones. By
considering the evaluation metric and resource constraints of the mobile
phones, we are able to perform efficient training and halt the process
accordingly. To simulate real users, we use speakers with various accents. The
entire on-device training and evaluation framework was then tested on various
mobile phones across brands. We show that fine-tuning the models and selecting
the right hyperparameter values is a trade-off between the lowest achievable
performance metric, on-device training time, and memory consumption. Overall,
our methodology offers a comprehensive solution for developing personalized ASR
models while leveraging the capabilities of mobile phones, and balancing the
need for accuracy with resource constraints.
