  Although the semantic communication with joint semantic-channel coding design
has shown promising performance in transmitting data of different modalities
over physical layer channels, the synchronization and packet-level forward
error correction of multimodal semantics have not been well studied. Due to the
independent design of semantic encoders, synchronizing multimodal features in
both the semantic and time domains is a challenging problem. In this paper, we
take the facial video and speech transmission as an example and propose a
Synchronous Multimodal Semantic Communication System (SyncSC) with Packet-Level
Coding. To achieve semantic and time synchronization, 3D Morphable Mode (3DMM)
coefficients and text are transmitted as semantics, and we propose a semantic
codec that achieves similar quality of reconstruction and synchronization with
lower bandwidth, compared to traditional methods. To protect semantic packets
under the erasure channel, we propose a packet-Level Forward Error Correction
(FEC) method, called PacSC, that maintains a certain visual quality performance
even at high packet loss rates. Particularly, for text packets, a text packet
loss concealment module, called TextPC, based on Bidirectional Encoder
Representations from Transformers (BERT) is proposed, which significantly
improves the performance of traditional FEC methods. The simulation results
show that our proposed SyncSC reduce transmission overhead and achieve
high-quality synchronous transmission of video and speech over the packet loss
network.
