  The goal of this paper is to develop an estimate for the entropy of random
long-range correlated symbolic sequences with elements belonging to a finite
alphabet. As a plausible model, we use the high-order additive stationary
ergodic Markov chain. Supposing that the correlations between random elements
of the chain are weak we express the differential entropy of the sequence by
means of the symbolic pair correlation function. We also examine an algorithm
for estimating the differential entropy of finite symbolic sequences. We show
that the entropy contains two contributions, the correlation and fluctuation
ones. The obtained analytical results are used for numerical evaluation of the
entropy of written English texts and DNA nucleotide sequences. The developed
theory opens the way for constructing a more consistent and sophisticated
approach to describe the systems with strong short- and weak long-range
correlations.
