  We introduce a notion of generalized Serre duality on a Hom-finite
Krull-Schmidt triangulated category $\mathcal{T}$. This duality induces the
generalized Serre functor on $\mathcal{T}$, which is a linear triangle
equivalence between two thick triangulated subcategories of $\mathcal{T}$.
Moreover, the domain of the generalized Serre functor is the smallest additive
subcategory of $\mathcal{T}$ containing all the indecomposable objects which
appear as the third term of an Auslander-Reiten triangle in $\mathcal{T}$;
dually, the range of the generalized Serre functor is the smallest additive
subcategory of $\mathcal{T}$ containing all the indecomposable objects which
appear as the first term of an Auslander-Reiten triangle in $\mathcal{T}$.
  We compute explicitly the generalized Serre duality on the bounded derived
categories of artin algebras and of certain noncommutative projective schemes
in the sense of Artin and Zhang. We obtain a characterization of Gorenstein
algebras: an artin algebra $A$ is Gorenstein if and only if the bounded
homotopy category of finitely generated projective $A$-modules has Serre
duality in the sense of Bondal and Kapranov.
