Abstract: The Manufacturing Industry has been pressured to increase sustainability, flexibility, efficiency, and productivity. Industry 4.0 (I4.0) has enabled the development of smart manufacturing environments supported by evolving technologies. Scheduling is a critical decisionmaking process that enhances productivity, reduces costs, improves customer satisfaction, and meets production goals. This study sheds light on job shop scheduling, which requires defining which job to process on which machine. The focus in this research is to respond to the following question: Is deep reinforcement learning (Q-learning algorithm) and Optimization models an effective method for reducing total tardiness when machines break down and generate an optimized production schedule. This research aims to address Flexible Job Shop Scheduling Problem (FJSSP) under machine breakdown with the goal of minimizing the total tardiness