  We consider the problem of learning user preferences over robot trajectories
for environments rich in objects and humans. This is challenging because the
criterion defining a good trajectory varies with users, tasks and interactions
in the environment. We represent trajectory preferences using a cost function
that the robot learns and uses it to generate good trajectories in new
environments. We design a crowdsourcing system - PlanIt, where non-expert users
label segments of the robot's trajectory. PlanIt allows us to collect a large
amount of user feedback, and using the weak and noisy labels from PlanIt we
learn the parameters of our model. We test our approach on 122 different
environments for robotic navigation and manipulation tasks. Our extensive
experiments show that the learned cost function generates preferred
trajectories in human environments. Our crowdsourcing system is publicly
available for the visualization of the learned costs and for providing
preference feedback: \url{http://planit.cs.cornell.edu}
