  Although Transformers have gained success in several speech processing tasks
like spoken language understanding (SLU) and speech translation (ST), achieving
online processing while keeping competitive performance is still essential for
real-world interaction. In this paper, we take the first step on streaming SLU
and simultaneous ST using a blockwise streaming Transformer, which is based on
contextual block processing and blockwise synchronous beam search. Furthermore,
we design an automatic speech recognition (ASR)-based intermediate loss
regularization for the streaming SLU task to improve the classification
performance further. As for the simultaneous ST task, we propose a
cross-lingual encoding method, which employs a CTC branch optimized with target
language translations. In addition, the CTC translation output is also used to
refine the search space with CTC prefix score, achieving joint CTC/attention
simultaneous translation for the first time. Experiments for SLU are conducted
on FSC and SLURP corpora, while the ST task is evaluated on Fisher-CallHome
Spanish and MuST-C En-De corpora. Experimental results show that the blockwise
streaming Transformer achieves competitive results compared to offline models,
especially with our proposed methods that further yield a 2.4% accuracy gain on
the SLU task and a 4.3 BLEU gain on the ST task over streaming baselines.
