INTELLIGENT ADAPTIVE OPTIMAL CONTROL OF DYNAMICAL SYSTEMS

Abstract

Most of the dynamical systems such as power systems, missile systems, robotic systems, inverted pendulum, industrial processes, chaotic circuits etc. are highly nonlinear in nature. The control of such systems is a challenging task. Intelligent adaptive optimal control is a viable recent approach. Intelligent adaptive optimal control has been emerged from the integration of adaptive control and optimal control methodologies with intelligent computational techniques. In this research work the performance investigation of intelligent adaptive optimal control of dynamical systems is presented. The applications of control schemes for dynamical systems control are implemented considering certain examples of linear and nonlinear dynamical systems to attempt this research investigation. The performance of controlled systems is desired to be optimal which should be valid also when applied in the real situation. Adaptive control which is able to deal with uncertainties is generally not optimal. Optimal control is off-line, and needs the knowledge of system dynamics for its design. Thus, to have both features of control design, it is desired to design online adaptive optimal control. Policy Iteration (PI) is a computational intelligence technique that belongs to a class of reinforcement learning (RL) algorithms; solves Hamilton-Jacobi-Bellman (HJB) equation by direct approach. Based on actor-critic structure, PI algorithm consists of two-step iteration: policy evaluation and policy improvement. These two steps of policy evaluation and policy improvement are repeated until the policy improvement step no longer changes the actual policy and thus converging to the optimal control. PI algorithm starts by evaluating the cost of a given initial admissible (stabilizing) control policy to converge towards state feedback optimal control. The infinite horizon optimal solution using HJB and algebraic Riccati equation (ARE) which gives linear quadratic regulator (LQR) require the complete knowledge of the system dynamics. Also these techniques give offline solution. The online PI algorithm solves online the continuous-time optimal control problem without using the knowledge of system internal dynamics, the information which is extracted from real-time dynamics by online measurement of sampled states along state trajectory. The knowledge of internal state dynamics is not needed for evaluation of cost or the update of control policy; and only the knowledge of input-to-state dynamics is required for updating the control policy. Thus, it is a partially model-free approach. The adaptive critic design (ACD) using online PI technique gives an online infinite horizon adaptive optimal control solution for continuous-time linear time invariant (LTI) systems and continuous-time affine nonlinear systems. By using neural networks to parameterize actor and critic for online implementation, this control scheme becomes a high-level intelligent control scheme. In the PI algorithm the critic is trained to approximate the solution of Lyapunov equation at the policy evaluation step, and the actor is trained to approximate the control policy at the ii policy improvement step, and the critic and actor are sequentially updated taking other one constant. In the generalized PI algorithm either one or both of the policy evaluation and policy improvement steps are not required to complete before the next step is started. The online synchronous policy iteration algorithm uses simultaneous continuous-time tuning for the actor and critic neural networks. The online synchronous PI algorithm which needs the knowledge of the system dynamics, solve the optimal control problem online using real-time measurements of closed-loop signals. Using neural networks approximations for critic and actor both it gives an online intelligent adaptive optimal control solution for the continuous-time dynamical systems. This research work contributes by presenting the comprehensive performance investigation of the different control schemes for continuous-time linear time-invariant (LTI) systems and affine nonlinear systems. The following objectives have been considered in this research work. 1. Optimal control of nonlinear inverted pendulum dynamical system using PID controller & LQR. 2. Intelligent control of nonlinear inverted pendulum dynamical system using Mamdani and TSK fuzzy inference systems. 3. Optimal control using LQR for automatic generation control of two-area interconnected power system. 4. Intelligent control using fuzzy-PI controller for automatic generation control of two-area interconnected nonlinear power system. 5. Intelligent control of process system using radial basis function. 6. Adaptive optimal control using policy iteration technique for LTI systems. 7. Adaptive optimal control using policy iteration technique for affine nonlinear systems. 8. Intelligent adaptive optimal control using synchronous policy iteration technique for LTI systems. 9. Intelligent adaptive optimal control using synchronous policy iteration technique for affine nonlinear systems. These research objectives are briefly described as below. Linear quadratic regulator (LQR), an optimal control technique and PID control method which are generally used for control of the linear dynamical systems have been used in this research work to control the nonlinear dynamical system. The modeling and control design of nonlinear inverted pendulum-cart dynamic system using PID controller & LQR have been presented for both cases of without and with disturbance input. The simulation results and performance analysis have been presented which justify the comparative advantages of optimal control using PID+LQR method. The pendulum stabilizes in upright position and cart iii reaches the desired position quickly & smoothly even under the continuous disturbance input justify that the control schemes are simple, effective & robust. Due to the capabilities of generalization, function approximation, learning and adaptation etc. the neural networks are applied for various control, identification, and estimation applications. In this research work the indirect adaptive control of a nonlinear process system using radial basis function neural networks (RBFNN) is presented. The liquid level control problem of a surge tank is considered as a process system. Two RBFNNs are used to model this affine nonlinear system to approximate the internal state dynamic function and input-to-state dynamic function respectively. The RBFNN controller provides a satisfactory response. Fuzzy control has an impact in the control community because of the simple approach it provides to use heuristic control knowledge for nonlinear control problems. Fuzzy control is an intelligent control technique which uses the human expert knowledge to make the control decisions. In this research work, the modeling, control design and performance analysis of fuzzy control for nonlinear inverted pendulum-cart dynamic system without & with disturbance input are presented. The fuzzy control methods using Mamdani and Takagi-Sugeno-Kang (TSK) fuzzy inference systems have been implemented to control the cart position and stabilize the inverted pendulum in vertically upright position. The comparative performance analysis of these fuzzy control methods have also been done with PID control method. The simulation results justify the comparative advantages of fuzzy control methods. The pendulum stabilizes in vertically upright position and cart approaches the desired position even under the continuous disturbance input justify that the control schemes are effective & robust. The analysis of the responses of the control schemes gives that the performance of PD-FLC using Mamdani type FIS is better than PID controller, and the performance of TSK FLC is better than both. The response of direct fuzzy control using TSK FIS is more smooth & fast than both PID control & Mamdani PD-fuzzy control. Electrical power systems are complex nonlinear dynamic systems. As the system parameters can’t be completely known under the presence of nonlinearities and uncertainties, the controller designed based on a fixed-parameter linearized model may not work properly for the actual plants. Thus, it is required to take into account the system nonlinearities and parametric uncertainties in the control design. In view of this aspect of investigation, this research work presents the modeling, simulation and performance analysis of automatic generation control (AGC) of two-area interconnected nonlinear power system using fuzzy-PI controller. The conventional integral control is also presented for comparing results. The simulation results and analysis justify the comparative advantages of fuzzy control method. iv In this research work the application of policy iteration technique based adaptive critic scheme for adaptive optimal control of continuous-time LTI dynamical systems is presented. The control scheme is implemented considering various practical examples of LTI systems- general SISO LTI system, higher order LTI system- a mechanical system, load frequency control of power system, automatic voltage regulator of power system, and DC motor speed control system. The systems modeling, analysis, and simulation results are presented for load frequency control of power system for both of system models without and with integral control, automatic voltage regulator of power system for both models of without and with sensor, DC motor speed control system for both of system models without and with integral compensator. Analyzing the simulation results obtained for these applications, it is observed that critic parameter matrix P and actor parameter K obtained from adaptive critic scheme using PI technique are converging adaptively to optimal values which are mostly same to that obtained from LQR approach. Also in case of change in system parameter in real situation the controller adapts it and converges to same optimal values. Thus the actor K and critic P parameters remain unchanged. The structural change introduced in system dynamics by including integral control/compensator is augmenting the system behavior such as of its credit that removing the steady state error in closed loop responses. The structural change in system will not be adapted by the proposed controller but it will adapt the change in system parameters in real situation at any moment of time which is demonstrated by simulating also with change in system parameters at certain instant of time. The comparative performance investigation of adaptive critic control scheme and linear quadratic regulator is also presented. Thus, adaptive optimal control scheme is partially model-free, effective & robust. In this research work the application of PI technique based adaptive critic scheme for adaptive optimal control of continuous-time affine nonlinear dynamical systems is presented. The cost function approximation using neural network is used for online implementation of PI algorithm. The application of control scheme is implemented considering the state regulation problem for certain general affine nonlinear systems and certain practical examples of affine nonlinear systems- single-link manipulator, inverted pendulum, Vander Pol oscillator. The simulation results and performance analysis are presented from which it is observed that the system states converge towards the equilibrium point at origin, and the control signal remains bounded converging towards zero. The cost function approximation neural networks weights are adjusted to the optimal values which give the critic parameters converging adaptively to optimal values and thus the control policy is adaptive optimal. The online PI algorithm requires an initial stabilizing controller for converging to the optimal solution. The simulation results and performance analysis demonstrate the effectiveness of online policy iteration technique based adaptive critic control scheme. v In this research work the applications of online synchronous PI technique using neural networks for adaptive optimal control of continuous-time LTI systems and affine nonlinear systems are presented. The application of online synchronous PI based control scheme is implemented for two practical examples of LTI systems- load frequency control of power system, and automatic voltage regulator of power system. The application of online synchronous PI based control scheme is also implemented for affine nonlinear systems considering the state regulation problem for certain general affine nonlinear systems and two practical examples of affine nonlinear systems- single-link manipulator, and Vander Pol oscillator. The simulation results and performance analysis are presented which demonstrate the effectiveness of online synchronous PI based adaptive critic control scheme. The online synchronous PI based adaptive critic design using neural networks provides an intelligent adaptive optimal control of continuous-time dynamical systems.