  Recently, large language models (LLMs) have expanded into various domains.
However, there remains a need to evaluate how these models perform when
prompted with commonplace queries compared to domain-specific queries, which
may be useful for benchmarking prior to fine-tuning domain-specific downstream
tasks. This study evaluates LLMs, specifically Gemma-2B and Gemma-7B, across
diverse domains, including cybersecurity, medicine, and finance, compared to
common knowledge queries. This study employs a comprehensive methodology to
evaluate foundational models, encompassing problem formulation, data analysis,
and the development of novel outlier detection techniques. This methodological
rigor enhances the credibility of the presented evaluation frameworks. This
study focused on assessing inference time, response length, throughput,
quality, and resource utilization and investigated the correlations between
these factors. The results indicate that model size and types of prompts used
for inference significantly influenced response length and quality. In
addition, common prompts, which include various types of queries, generate
diverse and inconsistent responses at irregular intervals. In contrast,
domain-specific prompts consistently generate concise responses within a
reasonable time. Overall, this study underscores the need for comprehensive
evaluation frameworks to enhance the reliability of benchmarking procedures in
multidomain AI research.
