  Despite the significant interest and progress in reinforcement learning (RL)
problems with adversarial corruption, current works are either confined to the
linear setting or lead to an undesired $\tilde{O}(\sqrt{T}\zeta)$ regret bound,
where $T$ is the number of rounds and $\zeta$ is the total amount of
corruption. In this paper, we consider the contextual bandit with general
function approximation and propose a computationally efficient algorithm to
achieve a regret of $\tilde{O}(\sqrt{T}+\zeta)$. The proposed algorithm relies
on the recently developed uncertainty-weighted least-squares regression from
linear contextual bandit and a new weighted estimator of uncertainty for the
general function class. In contrast to the existing analysis that heavily
relies on the linear structure, we develop a novel technique to control the sum
of weighted uncertainty, thus establishing the final regret bounds. We then
generalize our algorithm to the episodic MDP setting and first achieve an
additive dependence on the corruption level $\zeta$ in the scenario of general
function approximation. Notably, our algorithms achieve regret bounds either
nearly match the performance lower bound or improve the existing methods for
all the corruption levels and in both known and unknown $\zeta$ cases.
