Document Type
Thesis - Open Access
Award Date
2024
Degree Name
Master of Science (MS)
Department / School
Electrical Engineering and Computer Science
First Advisor
Kwanghee Won
Abstract
Unmanned aerial vehicles (UAVs) are increasingly employed as temporary base stations or access points to facilitate data transfer between ground terminals (GTs). However, in urban environments, UAV-GT communication links often face challenges due to obstructions from buildings and other obstacles, resulting in reduced data transfer efficiency. Reconfigurable intelligent surfaces (RIS) provide a promising solution by reflecting signals to enhance communication quality between UAVs and GTs. This thesis addresses the critical challenge of responsive UAV trajectory optimization in RIS-assisted communication networks. A novel approach is proposed, integrating federated learning with reinforcement learning techniques, specifically Double Deep Q-Network (DDQN) and Deep Deterministic Policy Gradient (DDPG) models, to optimize UAV trajectories while maintaining model quality. To tackle the complexities of training acceleration in distributed UAV networks, a RIS-assisted UAV trajectory simulation is developed, serving as the foundation for applying DDQN and DDPG models. The findings reveal that DDQN achieves rapid convergence but occasionally compromises trajectory precision, whereas DDPG generates more stable and logical trajectories at the cost of longer convergence times. To further enhance scalability and efficiency, federated learning frameworks, FedDDQN and FedDDPG, are designed. These frameworks enable multiple agents to collaboratively train models without requiring centralized data sharing, thereby preserving privacy. Comparative analysis demonstrates that federated learning with 10-agent parallel training accelerates the training process by a factor of 7.3 compared to single-agent training, though it incurs some loss in parallel efficiency. This approach not only improves training time but also ensures scalability and data privacy in distributed UAV networks. The proposed methodology represents a significant advancement in UAV trajectory optimization, combining the strengths of RIS, reinforcement learning, and federated learning to address the challenges of urban communication environments and distributed training systems.
Publisher
South Dakota State University
Recommended Citation
Li, Linsong, "Designing RIS-assisted UAV 3D Trajectory Using Deep Reinforcement Learning" (2024). Electronic Theses and Dissertations. 1344.
https://openprairie.sdstate.edu/etd2/1344
Rights Statement
