BZPEERReading, trust and collaboration for research
Menu

Discover

GraphFollow connections around a paper, topic or saved view.

Workspaces

HomePick up where your research flow left off.InboxHandle requests, replies and notifications from one place.CollectionsCurate reading lists, evidence sets and shareable dossiers.ResearchSee your private provenance graph, trail and imports.CollaborationMove from discovery into focused discussion rooms.PublishDraft, preview and publish full-text research articles.

Network

ExpertsFind specialists worth following or asking for help.CommunitiesJoin research circles organized around shared work.PartnersSee external organizations active around the same topics.

Opportunities

ListingsBrowse roles, calls and collaborations with clearer fit signals.

Account

Log inReturn to your reading and collaboration workspace.Create accountStart saving work, following people and joining teams.
Log inCreate account

Researcher profile

Melike Erol-Kantarci

Melike Erol-Kantarci contributes to research discovery and scholarly infrastructure.

ResearcherAffiliation not importedOpen to collaborate
Networking and Internet Architectureeess.SYSystems and Controleess.SPMachine LearningComputer Science and Game TheoryArtificial IntelligenceComputation and LanguageComputer VisionInformation Theorymath.ITMultiagent Systems

Trust snapshot

Quick read

Trust 21 - EmergingVerification L1Unclaimed author
23works
0followers
12topics
4close collaborators

Actions

Decide how to stay connected

Follow researcher0

Identity and collaboration

How to connect with this researcher

Claiming links this public author record to a researcher profile and unlocks direct collaboration workflows.

Log in to claim

Direct collaboration

Open a focused conversation when the fit is right

Claim this author entity first to unlock direct invitations.

Research graph

See the researcher in context

Open full explorer

Inspect adjacent work, topics, institutions and collaborators without jumping out to a separate graph page.

Building this graph slice

BZPEER is loading the nearby papers, people, topics and institutions for this page.

Published work

23 published item(s)

preprint2026arXiv

Foundation Model-Aided Hierarchical Control for Robust RIS-Assisted Near-Field Communications

The deployment of extremely large aperture arrays (ELAAs) in sixth-generation (6G) networks could shift communication into the near-field communication (NFC) regime. In this regime, signals exhibit spherical wave propagation, unlike the planar waves in conventional far-field systems. Reconfigurable intelligent surfaces (RISs) can dynamically adjust phase shifts to support NFC beamfocusing, concentrating signal energy at specific spatial coordinates. However, effective RIS utilization depends on both rapid channel state information (CSI) estimation and proactive blockage mitigation, which occur on inherently different timescales. CSI varies at millisecond intervals due to small-scale fading, while blockage events evolve over seconds, posing challenges for conventional single-level control algorithms. To address this issue, we propose a dual-transformer (DT) hierarchical framework that integrates two specialized transformer models within a hierarchical deep reinforcement learning (HDRL) architecture, referred to as the DT-HDRL framework. A fast-timescale transformer processes ray-tracing data for rapid CSI estimation, while a vision transformer (ViT) analyzes visual data to predict impending blockages. In HDRL, the high-level controller selects line-of-sight (LoS) or RIS-assisted non-line-of-sight (NLoS) transmission paths and sets goals, while the low-level controller optimizes base station (BS) beamfocusing and RIS phase shifts using instantaneous CSI. This dual-timescale coordination maximizes spectral efficiency (SE) while ensuring robust performance under dynamic conditions. Simulation results demonstrate that our approach improves SE by approximately 18% compared to single-timescale baselines, while the proposed blockage predictor achieves an F1-score of 0.92, providing a 769 ms advance warning window in dynamic scenarios.

0 citations0 reviewsNo code linkedOpen access
preprint2026arXiv

Multi-Modal Data-Enhanced Foundation Models for Prediction and Control in Wireless Networks: A Survey

Foundation models (FMs) are recognized as a transformative breakthrough that has started to reshape the future of artificial intelligence (AI) across both academia and industry. The integration of FMs into wireless networks is expected to enable the development of general-purpose AI agents capable of handling diverse network management requests and highly complex wireless-related tasks involving multi-modal data. Inspired by these ideas, this work discusses the utilization of FMs, especially multi-modal FMs in wireless networks. We focus on two important types of tasks in wireless network management: prediction tasks and control tasks. In particular, we first discuss FMs-enabled multi-modal contextual information understanding in wireless networks. Then, we explain how FMs can be applied to prediction and control tasks, respectively. Following this, we introduce the development of wireless-specific FMs from two perspectives: available datasets for development and the methodologies used. Finally, we conclude with a discussion of the challenges and future directions for FM-enhanced wireless networks.

0 citations0 reviewsNo code linkedOpen access
preprint2025arXiv

Wireless Multimodal Foundation Model (WMFM): Integrating Vision and Communication Modalities for 6G ISAC Systems

The emergence of multimodal foundation models has revolutionized learning paradigms by enabling joint understanding across diverse data types. In the context of next-generation wireless networks, integrating sensing and communication modalities presents a unique opportunity to develop generalizable and data-efficient models. In this work, we introduce the contrastive learning based Wireless Multimodal Foundation Model (WMFM), a large-scale framework that jointly learns from wireless channel coefficients and visual imagery. The WMFM is pretrained using contrastive learning, a self-supervised learning technique that aligns embeddings of camera and channel data without requiring explicit labels. The pretrained encoders are then frozen and employed as feature extractors, with lightweight task-specific heads, fine-tuned for downstream tasks, including user localization and LoS/nLoS classification. Extensive experiments on the DeepVerse6G dataset demonstrate that the proposed WMFM achieves a 17% improvement in balanced accuracy for LoS/nLoS classification and a 48.5% reduction in localization error compared to the end-to-end (E2E) benchmark, while reducing training time by up to 90-fold. Even when trained with as little as 20% of the data, the WMFM-based heads outperform the fully supervised E2E model, underscoring their robustness and data-efficient learning. The proposed approach establishes a foundation for scalable, multimodal learning in Integrated Sensing and Communication (ISAC) systems, paving the way for intelligent and adaptive 6G networks.

0 citations0 reviewsNo code linkedOpen access
preprint2023arXiv

Hierarchical Deep Q-Learning Based Handover in Wireless Networks with Dual Connectivity

5G New Radio proposes the usage of frequencies above 10 GHz to speed up LTE's existent maximum data rates. However, the effective size of 5G antennas and consequently its repercussions in the signal degradation in urban scenarios makes it a challenge to maintain stable coverage and connectivity. In order to obtain the best from both technologies, recent dual connectivity solutions have proved their capabilities to improve performance when compared with coexistent standalone 5G and 4G technologies. Reinforcement learning (RL) has shown its huge potential in wireless scenarios where parameter learning is required given the dynamic nature of such context. In this paper, we propose two reinforcement learning algorithms: a single agent RL algorithm named Clipped Double Q-Learning (CDQL) and a hierarchical Deep Q-Learning (HiDQL) to improve Multiple Radio Access Technology (multi-RAT) dual-connectivity handover. We compare our proposal with two baselines: a fixed parameter and a dynamic parameter solution. Simulation results reveal significant improvements in terms of latency with a gain of 47.6% and 26.1% for Digital-Analog beamforming (BF), 17.1% and 21.6% for Hybrid-Analog BF, and 24.7% and 39% for Analog-Analog BF when comparing the RL-schemes HiDQL and CDQL with the with the existent solutions, HiDQL presented a slower convergence time, however obtained a more optimal solution than CDQL. Additionally, we foresee the advantages of utilizing context-information as geo-location of the UEs to reduce the beam exploration sector, and thus improving further multi-RAT handover latency results.

0 citations0 reviewsNo code linkedOpen access
preprint2023arXiv

Hierarchical Reinforcement Learning Based Traffic Steering in Multi-RAT 5G Deployments

In 5G non-standalone mode, an intelligent traffic steering mechanism can vastly aid in ensuring smooth user experience by selecting the best radio access technology (RAT) from a multi-RAT environment for a specific traffic flow. In this paper, we propose a novel load-aware traffic steering algorithm based on hierarchical reinforcement learning (HRL) while satisfying diverse QoS requirements of different traffic types. HRL can significantly increase system performance using a bi-level architecture having a meta-controller and a controller. In our proposed method, the meta-controller provides an appropriate threshold for load balancing, while the controller performs traffic admission to an appropriate RAT in the lower level. Simulation results show that HRL outperforms a Deep Q-Learning (DQN) and a threshold-based heuristic baseline with 8.49%, 12.52% higher average system throughput and 27.74%, 39.13% lower network delay, respectively.

0 citations0 reviewsNo code linkedOpen access
preprint2023arXiv

Hierarchical Reinforcement Learning for RIS-Assisted Energy-Efficient RAN

Reconfigurable intelligent surface (RIS) is emerging as a promising technology to boost the energy efficiency (EE) of 5G beyond and 6G networks. Inspired by this potential, in this paper, we investigate the RIS-assisted energy-efficient radio access networks (RAN). In particular, we combine RIS with sleep control techniques, and develop a hierarchical reinforcement learning (HRL) algorithm for network management. In HRL, the meta-controller decides the on/off status of the small base stations (SBSs) in heterogeneous networks, while the sub-controller can change the transmission power levels of SBSs to save energy. The simulations show that the RIS-assisted sleep control can achieve significantly lower energy consumption, higher throughput, and more than doubled energy efficiency than no-RIS conditions.

0 citations0 reviewsNo code linkedOpen access
preprint2023arXiv

Traffic Steering for 5G Multi-RAT Deployments using Deep Reinforcement Learning

In 5G non-standalone mode, traffic steering is a critical technique to take full advantage of 5G new radio while optimizing dual connectivity of 5G and LTE networks in multiple radio access technology (RAT). An intelligent traffic steering mechanism can play an important role to maintain seamless user experience by choosing appropriate RAT (5G or LTE) dynamically for a specific user traffic flow with certain QoS requirements. In this paper, we propose a novel traffic steering mechanism based on Deep Q-learning that can automate traffic steering decisions in a dynamic environment having multiple RATs, and maintain diverse QoS requirements for different traffic classes. The proposed method is compared with two baseline algorithms: a heuristic-based algorithm and Q-learningbased traffic steering. Compared to the Q-learning and heuristic baselines, our results show that the proposed algorithm achieves better performance in terms of 6% and 10% higher average system throughput, and 23% and 33% lower network delay, respectively.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Deep Reinforcement Learning-based Radio Resource Allocation and Beam Management under Location Uncertainty in 5G mmWave Networks

Millimeter Wave (mmWave) is an important part of 5G new radio (NR), in which highly directional beams are adapted to compensate for the substantial propagation loss based on UE locations. However, the location information may have some errors such as GPS errors. In any case, some uncertainty, and localization error is unavoidable in most settings. Applying these distorted locations for clustering will increase the error of beam management. Meanwhile, the traffic demand may change dynamically in the wireless environment. Therefore, a scheme that can handle both the uncertainty of localization and dynamic radio resource allocation is needed. In this paper, we propose a UK-means-based clustering and deep reinforcement learning-based resource allocation algorithm (UK-DRL) for radio resource allocation and beam management in 5G mmWave networks. We first apply UK-means as the clustering algorithm to mitigate the localization uncertainty, then deep reinforcement learning (DRL) is adopted to dynamically allocate radio resources. Finally, we compare the UK-DRL with K-means-based clustering and DRL-based resource allocation algorithm (K-DRL), the simulations show that our proposed UK-DRL-based method achieves 150% higher throughput and 61.5% lower delay compared with K-DRL when traffic load is 4Mbps.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Energy-Efficient and Delay-Guaranteed Joint Resource Allocation and DU Selection in O-RAN

The radio access network (RAN) part of the next-generation wireless networks will require efficient solutions for satisfying low latency and high-throughput services. The open RAN (O-RAN) is one of the candidates to achieve this goal, in addition to increasing vendor diversity and promoting openness. In the O-RAN architecture, network functions are executed in central units (CU), distributed units (DU), and radio units (RU). These entities are virtualized on general-purpose CPUs and form a processing pool. These processing pools can be located in different geographical places and have limited capacity, affecting the energy consumption and the performance of networks. Additionally, since user demand is not deterministic, special attention should be paid to allocating resource blocks to users by ensuring their expected quality of service for latency-sensitive traffic flows. In this paper, we propose a joint optimization solution to enhance energy efficiency and provide delay guarantees to the users in the O-RAN architecture. We formulate this novel problem and linearize it to provide a solution with a mixed-integer linear problem (MILP) solver. We compare this with a baseline that addresses this optimization problem using a disjoint approach. The results show that our approach outperforms the baseline method in terms of energy efficiency.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Federated Deep Reinforcement Learning for Resource Allocation in O-RAN Slicing

Recently, open radio access network (O-RAN) has become a promising technology to provide an open environment for network vendors and operators. Coordinating the x-applications (xAPPs) is critical to increase flexibility and guarantee high overall network performance in O-RAN. Meanwhile, federated reinforcement learning has been proposed as a promising technique to enhance the collaboration among distributed reinforcement learning agents and improve learning efficiency. In this paper, we propose a federated deep reinforcement learning algorithm to coordinate multiple independent xAPPs in O-RAN for network slicing. We design two xAPPs, namely a power control xAPP and a slice-based resource allocation xAPP, and we use a federated learning model to coordinate two xAPP agents to enhance learning efficiency and improve network performance. Compared with conventional deep reinforcement learning, our proposed algorithm can achieve 11% higher throughput for enhanced mobile broadband (eMBB) slices and 33% lower delay for ultra-reliable low-latency communication (URLLC) slices.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Joint Sensing and Communications for Deep Reinforcement Learning-based Beam Management in 6G

User location is a piece of critical information for network management and control. However, location uncertainty is unavoidable in certain settings leading to localization errors. In this paper, we consider the user location uncertainty in the mmWave networks, and investigate joint vision-aided sensing and communications using deep reinforcement learning-based beam management for future 6G networks. In particular, we first extract pixel characteristic-based features from satellite images to improve localization accuracy. Then we propose a UK-medoids based method for user clustering with location uncertainty, and the clustering results are consequently used for the beam management. Finally, we apply the DRL algorithm for intra-beam radio resource allocation. The simulations first show that our proposed vision-aided method can substantially reduce the localization error. The proposed UK-medoids and DRL based scheme (UKM-DRL) is compared with two other schemes: K-means based clustering and DRL based resource allocation (K-DRL) and UK-means based clustering and DRL based resource allocation (UK-DRL). The proposed method has 17.2% higher throughput and 7.7% lower delay than UK-DRL, and more than doubled throughput and 55.8% lower delay than K-DRL.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing

Network slicing is a critical technique for 5G communications that covers radio access network (RAN), edge, transport and core slicing.The evolving network architecture requires the orchestration of multiple network resources such as radio and cache resources. In recent years, machine learning (ML) techniques have been widely applied for network management. However, most existing works do not take advantage of the knowledge transfer capability in ML. In this paper, we propose a deep transfer reinforcement learning (DTRL) scheme for joint radio and cache resource allocation to serve 5G RAN slicing. We first define a hierarchical architecture for joint resource allocation. Then we propose two DTRL algorithms: Q-value-based deep transfer reinforcement learning (QDTRL) and action selection-based deep transfer reinforcement learning (ADTRL). In the proposed schemes, learner agents utilize expert agents' knowledge to improve their performance on current tasks. The proposed algorithms are compared with both the model-free exploration bonus deep Q-learning (EB-DQN) and the model-based priority proportional fairness and time-to-live (PPF-TTL) algorithms. Compared with EB-DQN, our proposed DTRL-based method presents 21.4% lower delay for Ultra Reliable Low Latency Communications (URLLC) slice and 22.4% higher throughput for enhanced Mobile Broad Band (eMBB) slice, while achieving significantly faster convergence than EB-DQN. Moreover, 40.8% lower URLLC delay and 59.8% higher eMBB throughput are observed with respect to PPF-TTL.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

On the Impacts of Phase Shifting Design and Eavesdropping Uncertainty on Secrecy Metrics of RIS-aided Systems

This paper investigates the secrecy outage probability (SOP), the lower bound of SOP, and the probability of non-zero secrecy capacity (PNZ) of reconfigurable intelligent surface (RIS)-assisted systems from an information-theoretic perspective. In particular, we consider the impacts of eavesdroppers' location uncertainty and the phase adjustment uncertainty, namely imperfect coherent phase shifting and discrete phase shifting on RIS. More specifically, analytical and simulation results are presented to show that (i) the SOP gain due to the increase of the RIS reflecting elements number gradually decreases; and (ii) both phase shifting designs demonstrate the same PNZ secrecy performance, in other words, the random discrete phase shifting outperforms the imperfect coherent phase shifting design with reduced complexity.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Semantic Information Market For The Metaverse: An Auction Based Approach

In this paper, we address the networking and communications problems of creating a digital copy in the Metaverse digital twin. Specifically, a virtual service provider (VSP) which is responsible for creating and rendering the Metaverse, is required to use the data collected by IoT devices to create the virtual copy of the physical world. However, due to the huge volume of the collected data by IoT devices (e.g., images and videos) and the limited bandwidth, the VSP might become unable to retrieve all the required data from the physical world. Furthermore, the Metaverse needs fast replication (e.g., rendering) of the digital copy adding more restrictions on the data transmission delay. To solve the aforementioned challenges, we propose to equip the IoT devices with semantic information extraction algorithms to minimize the size of the transmitted data over the wireless channels. Since many IoT devices will be interested to sell their semantic information to the VSP, we propose a truthful reverse auction mechanism that helps the VSP select only IoT devices that can improve the quality of its virtual copy of objects through the semantic information. We conduct extensive simulations on a dataset that contains synchronized camera and radar images, and show that our novel design enables a fast replication of the digital copy with high accuracy.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Team Learning-Based Resource Allocation for Open Radio Access Network (O-RAN)

Recently, the concept of open radio access network (O-RAN) has been proposed, which aims to adopt intelligence and openness in the next generation radio access networks (RAN). It provides standardized interfaces and the ability to host network applications from third-party vendors by x-applications (xAPPs), which enables higher flexibility for network management. However, this may lead to conflicts in network function implementations, especially when these functions are implemented by different vendors. In this paper, we aim to mitigate the conflicts between xAPPs for near-real-time (near-RT) radio intelligent controller (RIC) of O-RAN. In particular, we propose a team learning algorithm to enhance the performance of the network by increasing cooperation between xAPPs. We compare the team learning approach with independent deep Q-learning where network functions individually optimize resources. Our simulations show that team learning has better network performance under various user mobility and traffic loads. With 6 Mbps traffic load and 20 m/s user movement speed, team learning achieves 8% higher throughput and 64.8% lower PDR.

0 citations0 reviewsNo code linkedOpen access
preprint2022arXiv

Variational Autoencoder Generative Adversarial Network for Synthetic Data Generation in Smart Home

Data is the fuel of data science and machine learning techniques for smart grid applications, similar to many other fields. However, the availability of data can be an issue due to privacy concerns, data size, data quality, and so on. To this end, in this paper, we propose a Variational AutoEncoder Generative Adversarial Network (VAE-GAN) as a smart grid data generative model which is capable of learning various types of data distributions and generating plausible samples from the same distribution without performing any prior analysis on the data before the training phase.We compared the Kullback-Leibler (KL) divergence, maximum mean discrepancy (MMD), and Wasserstein distance between the synthetic data (electrical load and PV production) distribution generated by the proposed model, vanilla GAN network, and the real data distribution, to evaluate the performance of our model. Furthermore, we used five key statistical parameters to describe the smart grid data distribution and compared them between synthetic data generated by both models and real data. Experiments indicate that the proposed synthetic data generative model outperforms the vanilla GAN network. The distribution of VAE-GAN synthetic data is the most comparable to that of real data.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Actor-Critic Learning Based QoS-Aware Scheduler for Reconfigurable Wireless Networks

The flexibility offered by reconfigurable wireless networks, provide new opportunities for various applications such as online AR/VR gaming, high-quality video streaming and autonomous vehicles, that desire high-bandwidth, reliable and low-latency communications. These applications come with very stringent Quality of Service (QoS) requirements and increase the burden over mobile networks. Currently, there is a huge spectrum scarcity due to the massive data explosion and this problem can be solved by helps of Reconfigurable Wireless Networks (RWNs) where nodes have reconfiguration and perception capabilities. Therefore, a necessity of AI-assisted algorithms for resource block allocation is observed. To tackle this challenge, in this paper, we propose an actor-critic learning-based scheduler for allocating resource blocks in a RWN. Various traffic types with different QoS levels are assigned to our agents to provide more realistic results. We also include mobility in our simulations to increase the dynamicity of networks. The proposed model is compared with another actor-critic model and with other traditional schedulers; proportional fair (PF) and Channel and QoS Aware (CQA) techniques. The proposed models are evaluated by considering the delay experienced by user equipment (UEs), successful transmissions and head-of-the-line delays. The results show that the proposed model noticeably outperforms other techniques in different aspects.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

AI-enabled Future Wireless Networks: Challenges, Opportunities and Open Issues

A plethora of demanding services and use cases mandate a revolutionary shift in the management of future wireless network resources. Indeed, when tight quality of service demands of applications are combined with increased complexity of the network, legacy network management routines will become unfeasible in 6G. Artificial Intelligence (AI) is emerging as a fundamental enabler to orchestrate the network resources from bottom to top. AI-enabled radio access and AI-enabled core will open up new opportunities for automated configuration of 6G. On the other hand, there are many challenges in AI-enabled networks that need to be addressed. Long convergence time, memory complexity, and complex behaviour of machine learning algorithms under uncertainty as well as highly dynamic channel, traffic and mobility conditions of the network contribute to the challenges. In this paper, we survey the state-of-art research in utilizing machine learning techniques in improving the performance of wireless networks. In addition, we identify challenges and open issues to provide a roadmap for the researchers.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Correlated Deep Q-learning based Microgrid Energy Management

Microgrid (MG) energy management is an important part of MG operation. Various entities are generally involved in the energy management of an MG, e.g., energy storage system (ESS), renewable energy resources (RER) and the load of users, and it is crucial to coordinate these entities. Considering the significant potential of machine learning techniques, this paper proposes a correlated deep Q-learning (CDQN) based technique for the MG energy management. Each electrical entity is modeled as an agent which has a neural network to predict its own Q-values, after which the correlated Q-equilibrium is used to coordinate the operation among agents. In this paper, the Long Short Term Memory networks (LSTM) based deep Q-learning algorithm is introduced and the correlated equilibrium is proposed to coordinate agents. The simulation result shows 40.9% and 9.62% higher profit for ESS agent and photovoltaic (PV) agent, respectively.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Decentralized Microgrid Energy Management: A Multi-agent Correlated Q-learning Approach

Microgrids (MG) are anticipated to be important players in the future smart grid. For proper operation of MGs an Energy Management System (EMS) is essential. The EMS of an MG could be rather complicated when renewable energy resources (RER), energy storage system (ESS) and demand side management (DSM) need to be orchestrated. Furthermore, these systems may belong to different entities and competition may exist between them. Nash equilibrium is most commonly used for coordination of such entities however the convergence and existence of Nash equilibrium can not always be guaranteed. To this end, we use the correlated equilibrium to coordinate agents, whose convergence can be guaranteed. In this paper, we build an energy trading model based on mid-market rate, and propose a correlated Q-learning (CEQ) algorithm to maximize the revenue of each agent. Our results show that CEQ is able to balance the revenue of agents without harming total benefit. In addition, compared with Q-learning without correlation, CEQ could save 19.3% cost for the DSM agent and 44.2% more benefits for the ESS agent.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Machine Learning-based Inter-Beam Inter-Cell Interference Mitigation in mmWave

In this paper, we address inter-beam inter-cell interference mitigation in 5G networks that employ millimeter-wave (mmWave), beamforming and non-orthogonal multiple access (NOMA) techniques. Those techniques play a key role in improving network capacity and spectral efficiency by multiplexing users on both spatial and power domains. In addition, the coverage area of multiple beams from different cells can intersect, allowing more flexibility in user-cell association. However, the intersection of coverage areas also implies increased inter-beam inter-cell interference, i.e. interference among beams formed by nearby cells. Therefore, joint user-cell association and inter-beam power allocation stand as a promising solution to mitigate inter-beam, inter-cell interference. In this paper, we consider a 5G mmWave network and propose a reinforcement learning algorithm to perform joint user-cell association and inter-beam power allocation to maximize the sum rate of the network. The proposed algorithm is compared to a uniform power allocation that equally divides power among beams per cell. Simulation results present a performance enhancement of 13-30% in network's sum-rate corresponding to the lowest and highest traffic loads, respectively.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Multi Agent Team Learning in Disaggregated Virtualized Open Radio Access Networks (O-RAN)

Starting from the Cloud Radio Access Network (C-RAN), continuing with the virtual Radio Access Network (vRAN) and most recently with Open RAN (O-RAN) initiative, Radio Access Network (RAN) architectures have significantly evolved in the past decade. In the last few years, the wireless industry has witnessed a strong trend towards disaggregated, virtualized and open RANs, with numerous tests and deployments world wide. One unique aspect that motivates this paper is the availability of new opportunities that arise from using machine learning to optimize the RAN in closed-loop, i.e. without human intervention, where the complexity of disaggregation and virtualization makes well-known Self-Organized Networking (SON) solutions inadequate. In our view, Multi-Agent Systems (MASs) with team learning, can play an essential role in the control and coordination of controllers of O-RAN, i.e. near-real-time and non-real-time RAN Intelligent Controller (RIC). In this article, we first present the state-of-the-art research in multi-agent systems and team learning, then we provide an overview of the landscape in RAN disaggregation and virtualization, as well as O-RAN which emphasizes the open interfaces introduced by the O-RAN Alliance. We present a case study for agent placement and the AI feedback required in O-RAN, and finally, we identify challenges and open issues to provide a roadmap for researchers.

0 citations0 reviewsNo code linkedOpen access
preprint2021arXiv

Radio Resource and Beam Management in 5G mmWave Using Clustering and Deep Reinforcement Learning

To optimally cover users in millimeter-Wave (mmWave) networks, clustering is needed to identify the number and direction of beams. The mobility of users motivates the need for an online clustering scheme to maintain up-to-date beams towards those clusters. Furthermore, mobility of users leads to varying patterns of clusters (i.e., users move from the coverage of one beam to another), causing dynamic traffic load per beam. As such, efficient radio resource allocation and beam management is needed to address the dynamicity that arises from mobility of users and their traffic. In this paper, we consider the coexistence of Ultra-Reliable Low-Latency Communication (URLLC) and enhanced Mobile BroadBand (eMBB) users in 5G mmWave networks and propose a Quality-of-Service (QoS) aware clustering and resource allocation scheme. Specifically, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) is used for online clustering of users and the selection of the number of beams. In addition, Long Short Term Memory (LSTM)-based Deep Reinforcement Learning (DRL) scheme is used for resource block allocation. The performance of the proposed scheme is compared to a baseline that uses K-means and priority-based proportional fairness for clustering and resource allocation, respectively. Our simulation results show that the proposed scheme outperforms the baseline algorithm in terms of latency, reliability, and rate of URLLC users as well as rate of eMBB users.

0 citations0 reviewsNo code linkedOpen access