A Deep Reinforcement Learning-Based Caching Strategy for Internet of Things Networks with Transient Data

Internet of Things (IoT) has been on a continuous rise in the past few years, and its potentials are now more apparent. Transient data generation and limited energy resources are the major bottlenecks of these networks. Besides, conventional quality of service measurements are still valid requirements that should be met, such as throughput, jitter, error rate, and delay or latency. An efficient caching policy can help meet the standard quality of service requirements while bypassing the specific limitations of IoT networks. Adopting deep reinforcement learning (DRL) algorithms enables us to develop an effective caching scheme without the need for any prior knowledge or contextual information such as popularity distributions or the lifetime of files. In this thesis, we propose DRL-based caching schemes that improve the cache hit rate and energy consumption of IoT networks compared to some well-known conventional caching schemes. We also propose a hierarchical caching architecture that enables parent nodes to receive the requests from multiple edge nodes and make caching decisions independently. The results of comprehensive experiments show that our proposed method outperforms the well-known conventional caching policies and an existing DRL-based method in cache hit and energy consumption rates by considerable margins. In the fifth chapter, we seek machine learning-based caching solutions for the cases in which the file popularity distribution is not static but changing over time. Taking the same system model into consideration, we propose a concept drift detection method based on clustering and cluster similarity measurements. The drift detection can trigger a process that leads the DRL agent to adapt to the new popularity distribution; we have based this process on transfer learning techniques. Transfer learning can help us leverage the existing knowledge in trained models and significantly speed up the training phase of our machine learning models.