Self-Learning Optimal Control of Nonlinear Systems

1 Principle of Adaptive Dynamic Programming 1.1 Dynamic Programming 1.1.1 Discret¦Å-Time Systems 1.1.2 Continuous-Time Systems 1.2 Original Forms of Adaptive Dynamic Programming 1.2.1 Principle of Adaptive Dynamic Programming 1.3 Iterative Forms of Adaptive Dynamic Programming 1.3.1 Value Iteration 1.3.2 Policy Iteration 1.4 About This Book References2 An Iterative ¦Å-Optimal Control Scheme for a Class of Discret¦Å-Time Nonlinear Systems with Unfixed Initial State. 2.1 Introduction 2.2 Problem Statement 2.3 Properties of the Iterative Adaptive Dynamic Programming Algorithm 2.3.1 Derivation of the Iterative ADP Algorithm 2.3.2 Properties of the Iterative ADP Algorithm 2.4 The ¦Å-Optimal Control Algorithm 2.4.1 The Derivation of the ¦Å-Optimal Control Algorithm 2.4.2 Properties of the ¦Å-Optimal Control Algorithm 2.4.3 The ¦Å-Optimal Control Algorithm for Unfixed Initial State 2.4.4 The Expressions of the ¦Å-Optimal Control Algorithm 2.5 Neural Network Implementation for the ¦Å-Optimal Control Scheme 2.5.1 The Critic Network 2.5.2 The Action Network 2.6 Simulation Study 2.7 Conclusions References3 Discret¦Å-Time Optimal Control of Nonlinear Systems via Value Iteration-Based Q-Learning 3.1 Introduction 3.2 Preliminaries and Assumptions 3.2.1 Problem Formulations 3.2.2 Derivation of the Discret¦Å-Time Q-Learning Algorithm 3.3 Properties of the Discret¦Å-Time Q-Learning Algorithm 3.3.1 Non-Discount Case 3.3.2 Discount Case 3.4 Neural Network Implementation for the Discret¦Å-Time Q-Learning Algorithm 3.4.1 The Action Network 3.4.2 The Critic Network 3.4.3 Training Phase 3.5 Simulation Study 3.5.1 Example 1 3.5.2 Example 2 3.6 Conclusion References4 A Novel Policy Iteration-Based Deterministic Q-Learning for Discret¦Å-Time Nonlinear Systems 4.1 Introduction 4.2 Problem Formulation 4.3 Policy Iteration-Based Deterministic Q-Learning Algorithm for Discret¦Å-Time Nonlinear Systems 4.3.1 Derivation of the Policy Iteration-Based Deterministic Q-Learning Algorithm 4.3.2 Properties of the Policy Iteration-Based Deterministic Q-Learning Algorithm 4.4 Neural Network Implementation for the Policy Iteration-Based Deterministic Q-Learning Algorithm 4.4.1 The Critic Network 4.4.2 The Action Network 4.4.3 Summary of the Policy Iteration-Based Deterministic Q-Learning Algorithm 4.5 Simulation Study 4.5.1 Example 1 4.5.2 Example 2 4.6 Conclusion References5 Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm 5.1 Introduction 5.2 Problem Statement 5.3 Policy Iteration Q-Learning Algorithm for Optimal Tracking Control 5.4 Properties of the Policy Iteration Q-Learning Algorithm 5.5 Neural Network Implementation for the Policy Iteration Q-Learning Algorithm 5.5.1 The Critic Network 5.5.2 The Action Network 5.6 Simulation Study 5.6.1 Example 1 5.6.2 Example 2 5.7 Conclusions References6 Model-Free Multiobjective Adaptive Dynamic Programming for Discret¦Å-Time Nonlinear Systems with General Performance Index Functions 6.1 Introduction 6.2 Preliminaries 6.3 Multiobjective Adaptive Dynamic Programming Method 6.4 Model-Free Incremental Q-Learning Method 6.5 Neural Network Implementation for the Incremental Q-Learning Method 6.5.1 The Critic Network 6.5.2 The Action Network 6.5.3 The Procedure of the Model-Free Incremental Q-learning Method 6.6 Convergence Proof 6.7 Simulation Study 6.7.1 Example 1 6.7.2 Example 2 6.8 Conclusion References7 Multiobjective Optimal Control for a Class of Unknown Nonlinear Systems Based on Finit¦Å-Approximation-Error ADP Algorithm 7.1 Introduction 7.2 General Formulation 7.3 Optimal Solution Based on Finit¦Å-Approximation-Error ADP 7.3.1 Data-Based Identifier of Unknown System Dynamics 7.3.2 Derivation of the ADP Algorithm with Finite Approximation Errors 7.3.3 Convergence Analysis of the Iterative ADP Algorithm 7.4 Implementation of the Iterative ADP Algorithm 7.4.1 Critic Network 7.4.2 The Action Network 7.4.3 The Procedure of the ADP Algorithm 7.5 Simulation Study 7.5.1 Example 1 7.5.2 Example 2 7.6 Conclusions References8 A New Approach for a Class of Continuous-Time Chaotic Systems Optimal Control by Online ADP Algorithm 8.1 Introduction 8.2 Problem Statement 8.3 Optimal Control Based on Online ADP Algorithm 8.3.1 Design Method of the Critic Network and the Action Network 8.3.2 Stability Analysis 8.3.3 Online ADP Algorithm Implementation 8.4 Simulation Examples 8.4.1 Example 1 8.4.2 Example 2 8.5 Conclusions References9 Off-Pohcy IRL Optimal Tracking Control for Continuous-Time Chaotic Systems 9.1 Introduction 9.2 System Description and Problem Statement 9.3 Off-Policy IRL ADP Algorithm 9.3.1 Convergence Analysis of IRL ADP Algorithm 9.3.2 Off-Policy IRL Method 9.3.3 Methods for Updating Weights 9.4 Simulation Study 9.4.1 Example 1 9.4.2 Example 2 9.5 Conclusion References10 ADP-Based Optimal Sensor Scheduling for Target Tracking in Energy Harvesting Wireless Sensor Networks 10.1 Introduction 10.2 Problem Formulation 10.2.1 NN Model Description of Solar Energy Harvesting. 10.2.2 Sensor Energy Consumption 10.2.3 KF Technology 10.3 ADP-Based Sensor Scheduling for Maximum WSNs Residual Energy and Minimum Measuring Accuracy 10.3.1 Optimization Problem of the Sensor Scheduling 10.3.2 ADP-Based Sensor Scheduling with Convergence Analysis 10.3.3 Critic Network 10.3.4 Implementation Process 10.4 Simulation Study 10.5 Conclusion ReferencesIndex