  Semantic role labeling (SRL) has multiple disjoint label sets, e.g., VerbNet
and PropBank. Creating these datasets is challenging, therefore a natural
question is how to use each one to help the other. Prior work has shown that
cross-task interaction helps, but only explored multitask learning so far. A
common issue with multi-task setup is that argument sequences are still
separately decoded, running the risk of generating structurally inconsistent
label sequences (as per lexicons like Semlink). In this paper, we eliminate
such issue with a framework that jointly models VerbNet and PropBank labels as
one sequence. In this setup, we show that enforcing Semlink constraints during
decoding constantly improves the overall F1. With special input constructions,
our joint model infers VerbNet arguments from given PropBank arguments with
over 99 F1. For learning, we propose a constrained marginal model that learns
with knowledge defined in Semlink to further benefit from the large amounts of
PropBank-only data. On the joint benchmark based on CoNLL05, our models achieve
state-of-the-art F1's, outperforming the prior best in-domain model by 3.5
(VerbNet) and 0.8 (PropBank). For out-of-domain generalization, our models
surpass the prior best by 3.4 (VerbNet) and 0.2 (PropBank).
