Towards a Fully Cooperative Multi-Agent Reinforcement Learning based Media Access Control Protocol for Underwater Acoustic Wireless Sensor Networks

Abstract

Underwater Acoustic Sensor Networks (UWASNs) has gain a widespread recognition recently due to some technological break- through, and thus, beginning a new era of research in the industry with potential for vast applications that are important to our livelihood. Despite all these potentials, deploying a reliable UWASNs based systems still remain very far from perfect and there are only limited experimental trials at the moment. This is due to challenges of reliabilty, QoS and energy efficiency, which is due to inherent characteristics of underwater acoustic channel. These pose signif- icant challenges for the design of network protocols, especially, the Media Access Control (MAC) protocol for UWASNs. Various MAC protocols have been developed for UWASNs and some few adopted from Wireless Sensor Networks (WSNs). However, most of these protocols do not provide acceptable QoS in terms of delay, throughput, fairness and energy efficiency. This paper presents a review of some of the prominent MAC protocols for UWASNs and adaptable WSNs based MAC protocols for UWASNs and propose a Fully Cooperative Multi-Agent Reinforcement Learning based MAC protocol for UWASNs. The proposed scheme will apply Multi-Agent based Reinforcement Learning (RL) to ALOHA MAC scheme to create a dynamic contention-free-like slotted MAC to aid nodes cooperation and interactions within themselves and the underwater environment to significantly achieve “self-organization” and “self-adaptability” to changes in the environment which would provide means for coping with long and variable propagation delay, low data rates and energy efficiency and in turn can significantly improve the QoS of UWASN systems by having better convergence time and Energy efficiency.