  Although advancements in machine learning have driven the development of
malicious URL detection technology, current techniques still face significant
challenges in their capacity to generalize and their resilience against
evolving threats. In this paper, we propose PyraTrans, a novel method that
integrates pretrained Transformers with pyramid feature learning to detect
malicious URL. PyraTrans utilizes a pretrained CharBERT as its foundation and
is augmented with three interconnected feature modules: 1) Encoder Feature
Extraction, extracting multi-order feature matrices from each CharBERT encoder
layer; 2) Multi-Scale Feature Learning, capturing local contextual insights at
various scales and aggregating information across encoder layers; and 3)
Spatial Pyramid Attention, focusing on regional-level attention to emphasize
areas rich in expressive information. The proposed approach addresses the
limitations of the Transformer in local feature learning and regional
relational awareness, which are vital for capturing URL-specific word patterns,
character combinations, or structural anomalies. In several challenging
experimental scenarios, the proposed method has shown significant improvements
in accuracy, generalization, and robustness in malicious URL detection. For
instance, it achieved a peak F1-score improvement of 40% in class-imbalanced
scenarios, and exceeded the best baseline result by 14.13% in accuracy in
adversarial attack scenarios. Additionally, we conduct a case study where our
method accurately identifies all 30 active malicious web pages, whereas two
pior SOTA methods miss 4 and 7 malicious web pages respectively. Codes and data
are available at:https://github.com/Alixyvtte/PyraTrans.
