  Despite substantial advancements, Natural Language Processing (NLP) models
often require post-training adjustments to enforce business rules, rectify
undesired behavior, and align with user values. These adjustments involve
operationalizing "concepts"--dictating desired model responses to certain
inputs. However, it's difficult for a single entity to enumerate and define all
possible concepts, indicating a need for a multi-user, collaborative model
alignment framework. Moreover, the exhaustive delineation of a concept is
challenging, and an improper approach can create shortcuts or interfere with
original data or other concepts.
  To address these challenges, we introduce CoDev, a framework that enables
multi-user interaction with the model, thereby mitigating individual
limitations. CoDev aids users in operationalizing their concepts using Large
Language Models, and relying on the principle that NLP models exhibit simpler
behaviors in local regions. Our main insight is learning a \emph{local} model
for each concept, and a \emph{global} model to integrate the original data with
all concepts. We then steer a large language model to generate instances within
concept boundaries where local and global disagree. Our experiments show CoDev
is effective at helping multiple users operationalize concepts and avoid
interference for a variety of scenarios, tasks, and models.
