  We introduce the Pointer Q-Network (PQN), a hybrid neural architecture that
integrates model-free Q-value policy approximation with Pointer Networks
(Ptr-Nets) to enhance the optimality of attention-based sequence generation,
focusing on long-term outcomes. This integration proves particularly effective
in solving combinatorial optimization (CO) tasks, especially the Travelling
Salesman Problem (TSP), which is the focus of our study. We address this
challenge by defining a Markov Decision Process (MDP) compatible with PQN,
which involves iterative graph embedding, encoding and decoding by an
LSTM-based recurrent neural network. This process generates a context vector
and computes raw attention scores, which are dynamically adjusted by Q-values
calculated for all available state-action pairs before applying softmax. The
resulting attention vector is utilized as an action distribution, with actions
selected hinged to exploration-exploitation dynamic adaptibility of PQN. Our
empirical results demonstrate the efficacy of this approach, also testing the
model in unstable environments.
