  Existing event stream-based pattern recognition models usually represent the
event stream as the point cloud, voxel, image, etc., and design various deep
neural networks to learn their features. Although considerable results can be
achieved in simple cases, however, the model performance may be limited by
monotonous modality expressions, sub-optimal fusion, and readout mechanisms. In
this paper, we propose a novel dual-stream framework for event stream-based
pattern recognition via differentiated fusion, termed EFV++. It models two
common event representations simultaneously, i.e., event images and event
voxels. The spatial and three-dimensional stereo information can be learned
separately by utilizing Transformer and Graph Neural Network (GNN). We believe
the features of each representation still contain both efficient and redundant
features and a sub-optimal solution may be obtained if we directly fuse them
without differentiation. Thus, we divide each feature into three levels and
retain high-quality features, blend medium-quality features, and exchange
low-quality features. The enhanced dual features will be fed into the fusion
Transformer together with bottleneck features. In addition, we introduce a
novel hybrid interaction readout mechanism to enhance the diversity of features
as final representations. Extensive experiments demonstrate that our proposed
framework achieves state-of-the-art performance on multiple widely used event
stream-based classification datasets. Specifically, we achieve new
state-of-the-art performance on the Bullying10k dataset, i.e., $90.51\%$, which
exceeds the second place by $+2.21\%$. The source code of this paper has been
released on
\url{https://github.com/Event-AHU/EFV_event_classification/tree/EFVpp}.
