  This thesis considers sequential decision problems, where the loss/reward
incurred by selecting an action may not be inferred from observed feedback. A
major part of this thesis focuses on the unsupervised sequential selection
problem, where one can not infer the loss incurred for selecting an action from
observed feedback. We also introduce a new setup named Censored Semi Bandits,
where the loss incurred for selecting an action can be observed under certain
conditions. Finally, we study the channel selection problem in the
communication networks, where the reward for an action is only observed when no
other player selects that action to play in the round. These problems find
applications in many fields like healthcare, crowd-sourcing, security, adaptive
resource allocation, among many others. This thesis aims to address the
above-described sequential decision problems by exploiting specific structures
these problems exhibit. We develop provably optimal algorithms for each of
these setups with weak feedback and validate their empirical performance on
different problem instances derived from synthetic and real datasets.
