  Pre-trained large language models can perform natural language processing
downstream tasks by conditioning on human-designed prompts. However, a
prompt-based approach often requires "prompt engineering" to design different
prompts, primarily hand-crafted through laborious trial and error, requiring
human intervention and expertise. It is a challenging problem when constructing
a prompt-based keyphrase extraction method. Therefore, we investigate and study
the effectiveness of different prompts on the keyphrase extraction task to
verify the impact of the cherry-picked prompts on the performance of extracting
keyphrases. Extensive experimental results on six benchmark keyphrase
extraction datasets and different pre-trained large language models demonstrate
that (1) designing complex prompts may not necessarily be more effective than
designing simple prompts; (2) individual keyword changes in the designed
prompts can affect the overall performance; (3) designing complex prompts
achieve better performance than designing simple prompts when facing long
documents.
