  Cloud computing is a reliable solution to provide distributed computation
power. However, real-time response is still challenging regarding the enormous
amount of data generated by the IoT devices in 5G and 6G networks. Thus,
multi-access edge computing (MEC), which consists of distributing the edge
servers in the proximity of end-users to have low latency besides the higher
processing power, is increasingly becoming a vital factor for the success of
modern applications. This paper addresses the problem of minimizing both, the
network delay, which is the main objective of MEC, and the number of edge
servers to provide a MEC design with minimum cost. This MEC design consists of
edge servers placement and base stations allocation, which makes it a joint
combinatorial optimization problem (COP). Recently, reinforcement learning (RL)
has shown promising results for COPs. However, modeling real-world problems
using RL when the state and action spaces are large still needs investigation.
We propose a novel RL framework with an efficient representation and modeling
of the state space, action space and the penalty function in the design of the
underlying Markov Decision Process (MDP) for solving our problem.
