While Cyber-Manufacturing System security must involve three separate yet interrelated processes (prediction, detection, and prevention), the detection process is the focus of research presented in this paper. Current intrusion detection systems often result in high false positive and false negative rates. Also, the actual detection time may take long time—up to several months. The current intrusion detection systems rely heavily on the network data, but do not utilize the physical data such as side channel, sensor reading, image, keystrokes., which are generated during manufacturing processes. An adaptive intrusion detection system composed of two security layers is proposed to detect cyber-physical intrusions. Model-free deep reinforcement learning is used in the two security layers: the network layer and the physical layer. The capability of reinforcement learning through trial and error and a course of actions based on observations in an environment makes it more robust to the continuously changing attack vectors in current manufacturing industry. The proposed intrusion detection system demonstrates that it can reduce the false positive rate and generate alerts to a wide range of attack patterns.