Abstract:

It is difficult for fuzzy Sarsa learning(FSL) to tune learning rate and balance exploration vs. exploitation, so an improved FSL(IFSL) method based on FSL is presented. In the method, an adaptive learning rate generator for tuning learning rate online and a fuzzy balancer for controlling the degree of exploration vs. exploitation are introduced. The diagram of IFSL is given, and the weight vector of IFSL with stationary action selection policy converges to a unique value is proved. Simulation results show that IFSL well manager balance, and outperforms FSL in terms of learning speed and action quality.

Key words: reinforcement learning, fuzzy control, fuzzy Sarsa learning, exploration, exploitation