  Unsupervised clustering of student responses to open-ended questions into
behavioral and cognitive profiles using pre-trained LLM embeddings is an
emerging technique, but little is known about how well this captures
pedagogically meaningful information. We investigate this in the context of
student responses to open-ended questions in biology, which were previously
analyzed and clustered by experts into theory-driven Knowledge Profiles (KPs).
Comparing these KPs to ones discovered by purely data-driven clustering
techniques, we report poor discoverability of most KPs, except for the ones
including the correct answers. We trace this "discoverability bias" to the
representations of KPs in the pre-trained LLM embeddings space.
