  The recently discovered cuprate superconductor Ba$_2$CuO$_{3+\delta}$
exhibits a high $T_c\simeq73$K at $\delta\simeq0.2$. The polycrystal grown
under high pressure has a structure similar to La$_2$CuO$_4$, but with
dramatically different lattice parameters due to the CuO$_6$ octahedron
compression. The crystal field in the compressed Ba$_2$CuO$_4$ leads to an
inverted Cu $3d$ $e_g$ complex with the $d_{x^2-y^2}$ orbital sitting below the
$d_{3z^2-r^2}$ and an electronic structure highly unusual compared to the
conventional cuprates. We construct a two-orbital Hubbard model for the Cu
$d^9$ state at hole doping $x=2\delta$ and study the orbital-dependent strong
correlation and superconductivity. For the undoped case at $x=0$, we found that
strong correlation drives an orbital-polarized Mott insulating state with the
spin-$1/2$ moment of the localized $d_{3z^2-r^2}$ orbital. In contrast to the
single-band cuprates where superconductivity is suppressed in the overdoped
regime, hole doping the two-orbital Mott insulator leads to orbital-dependent
correlations and the robust spin and orbital exchange interactions produce a
high-$T_c$ antiphase $d$-wave superconductor even in the heavily doped regime
at $x=0.4$. We conjecture that Ba$_2$CuO$_{3+\delta}$ realizes mixtures of such
heavily hole-doped superconducting Ba$_2$CuO$_4$ and disordered Ba$_2$CuO$_{3}$
chains in a single-layer or predominately separated bilayer structure. Our
findings suggest that unconventional cuprates with liberated orbitals as doped
two-band Mott insulators can be a direction for realizing high-T$_c$
superconductivity with enhanced transition temperature $T_c$.
