  The diversity of intrinsic qualities of multimedia entities tends to impede
their effective retrieval. In a SelfLearning Search Engine architecture, the
subtle nuances of human perceptions and deep knowledge are taught and captured
through unsupervised reinforcement learning, where the degree of reinforcement
may be suitably calibrated. Such architectural paradigm enables indexes to
evolve naturally while accommodating the dynamic changes of user interests. It
operates by continuously constructing indexes over time, while injecting
progressive improvement in search performance. For search operations to be
effective, convergence of index learning is of crucial importance to ensure
efficiency and robustness. In this paper, we develop a Self-Learning Search
Engine architecture based on reinforcement learning using a Markov Decision
Process framework. The balance between exploration and exploitation is achieved
through evolutionary exploration Strategies. The evolutionary index learning
behavior is then studied and formulated using stochastic analysis. Experimental
results are presented which corroborate the steady convergence of the index
evolution mechanism. Index Term
