  Advancements in 6G wireless technology have elevated the importance of
beamforming, especially for attaining ultra-high data rates via millimeter-wave
(mmWave) frequency deployment. Although promising, mmWave bands require
substantial beam training to achieve precise beamforming. While initial deep
learning models that use RGB camera images demonstrated promise in reducing
beam training overhead, their performance suffers due to sensitivity to
lighting and environmental variations. Due to this sensitivity, Quality of
Service (QoS) fluctuates, eventually affecting the stability and dependability
of networks in dynamic environments. This emphasizes a critical need for robust
solutions. This paper proposes a robust beamforming technique to ensure
consistent QoS under varying environmental conditions. An optimization problem
has been formulated to maximize users' data rates. To solve the formulated
NP-hard optimization problem, we decompose it into two subproblems: the
semantic localization problem and the optimal beam selection problem. To solve
the semantic localization problem, we propose a novel method that leverages the
K-means clustering and YOLOv8 model. To solve the beam selection problem, we
propose a novel lightweight hybrid architecture that combines a lightweight
transformer with a CNN architecture through a weighted entropy mechanism. This
hybrid architecture utilizes multimodal data sources to dynamically predict the
optimal beams. A novel metric, Accuracy-Complexity Efficiency (ACE), has been
proposed to quantify this. Six testing scenarios have been developed to
evaluate the robustness of the proposed model. Finally, the simulation result
demonstrates that the proposed model outperforms several state-of-the-art
baselines regarding beam prediction accuracy, received power, and ACE in the
developed test scenarios.
