Multi-use Energy Management Concept for PV Battery Storage Systems Based on Reinforcement Learning

DOI

10.2991/978-94-6463-156-2_15How to use a DOI?

Keywords

PV Battery Storage System (PVBSS); Energy Management; Multi-Use; Reinforcement Learning (RL); Artificial Neural Networks (ANN); Long Short-Term Memory (LSTM); Proximal Policy Optimization (PPO)

Abstract

This contribution introduces an energy management concept for multi-use applications of PV battery storage systems based on reinforcement learning (RL). The approach uses the state-of-the-art Proximal Policy Optimization algorithm in combination with recurrent Long Short-Term Memory networks to derive locally optimal energy management policies from a data-driven, simulation-based training procedure. For this purpose, an AC-coupled residential PV battery storage system is modelled and parametrized. Qualitative advantages of the RL-based approach compared to the commonly used model predictive control (MPC) approaches with regard to multi-use energy management applications, such as the ability to optimize a control policy over an infinite, discounted time horizon, are highlighted. From a large-scale training run of over 200 hyperparameter configurations, the five best energy management policies are selected and evaluated against state-of-the-art MPC and rule-based energy management concepts. In the evaluation over one year it is shown, that the energy management learned by the RL algorithm reduces curtailment losses from 5.70 % to 4.78 % , specific energy cost from 7.16 Cent kW h - 1 to 7.09 Cent kW h - 1 and increase the share of PV energy fed into the grid under a fixed feed-in limit from 49.95 % to 50.99 % compared to the MPC energy management, which is the second best one.

Copyright

© 2023 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Florus Härtel
AU  - Thilo Bocklisch
PY  - 2023
DA  - 2023/05/25
TI  - Multi-use Energy Management Concept for PV Battery Storage Systems Based on Reinforcement Learning
BT  - Proceedings of the International Renewable Energy Storage  Conference (IRES 2022)
PB  - Atlantis Press
SP  - 206
EP  - 214
SN  - 2589-4943
UR  - https://doi.org/10.2991/978-94-6463-156-2_15
DO  - 10.2991/978-94-6463-156-2_15
ID  - Härtel2023
ER  -