  StreamVoice has recently pushed the boundaries of zero-shot voice conversion
(VC) in the streaming domain. It uses a streamable language model (LM) with a
context-aware approach to convert semantic features from automatic speech
recognition (ASR) into acoustic features with the desired speaker timbre.
Despite its innovations, StreamVoice faces challenges due to its dependency on
a streaming ASR within a cascaded framework, which complicates system
deployment and optimization, affects VC system's design and performance based
on the choice of ASR, and struggles with conversion stability when faced with
low-quality semantic inputs. To overcome these limitations, we introduce
StreamVoice+, an enhanced LM-based end-to-end streaming framework that operates
independently of streaming ASR. StreamVoice+ integrates a semantic encoder and
a connector with the original StreamVoice framework, now trained using a
non-streaming ASR. This model undergoes a two-stage training process:
initially, the StreamVoice backbone is pre-trained for voice conversion and the
semantic encoder for robust semantic extraction. Subsequently, the system is
fine-tuned end-to-end, incorporating a LoRA matrix to activate comprehensive
streaming functionality. Furthermore, StreamVoice+ mainly introduces two
strategic enhancements to boost conversion quality: a residual compensation
mechanism in the connector to ensure effective semantic transmission and a
self-refinement strategy that leverages pseudo-parallel speech pairs generated
by the conversion backbone to improve speech decoupling. Experiments
demonstrate that StreamVoice+ not only achieves higher naturalness and speaker
similarity in voice conversion than its predecessor but also provides versatile
support for both streaming and non-streaming conversion scenarios.
