Machine learning predicting our future energy consumption ML

Machine learning predicting our future energy consumption ML saving environment and millions dollars Maher Selim, Ph. D. Machine learning Postdoctoral Fellow Professor Wenying Feng research group Trent University

Importance Energy Forecast ❖ Operational efficiency and cost control ❖ System stability and reliability ❖ Renewable energy management ❖ Energy efficiency and environmental issues ❖ Consumer engagement and service improvement Big data driven smart energy management: From big data to big insights Kaile Zhou, Chao Fu, Shanlin Yang 1

Forecast models based 1. Physical 1. Semi-physical 1. Data-driven methods (big data and machine learning) 2

Machine Learning Big data and machine learning Big data driven smart energy management: From big data to big insights Kaile Zhou, Chao Fu, Shanlin Yang 3

Smart Grid Architecture and main components of smart grid. A data mining based load forecasting strategy for smart electrical grids Ahmed I. Saleh, Asmaa H. Rabie, Khaled M. Abo-Al-Ez 4

A data-driven predictive model of cityscale energy use in buildings Constantine E. Kontokosta Christopher Tull 5

A review of data-driven building consumption prediction studies energy Kadir Amasyali, Nora M. El-Gohary 6

Conventional Deep learning Aarshay Jain https: //www. analyticsvidhya. com/blog/2016/03/introduction-deep-learning-fundamentals-neural-networks/ 7

Recurrent Neural Networks Time series forecasting with Recurrent Neural Networks (RNN) Taegyun Jeon 8

Bayesian deep learning New trend in Machine learning to improve the accuracy of deep learning 9

Bayesian statistics Bayesian neural networks https: //jmhl. org/research/ 10

Application of Bayesian deep learning algorithm for Energy Forecast RNN (LSTM) deep learning Then introducing Bayesian RNN (LSTM) deep learning Taegyun Jeon 11

Apply Bayesian deep learning A neural network (a deep learning tool) linearly transforms its input (bottom layer), applies some non-linearity on each dimension (middle layer), and linearly transforms it again (top layer). This model gives us point estimates with no uncertainty information. YARIN GAL http: //www. cs. ox. ac. uk/people/yarin. gal/website/blog_3 d 801 aa 532 c 1 ce. html#disqus_thread 12

A network with infinitely many weights with a distribution on each weight is a Gaussian process. The same network with finitely many weights is known as a Bayesian neural network. Using these is quite difficult though, and they haven't really caught-on. http: //www. cs. ox. ac. uk/people/yarin. gal/website/blog_3 d 801 aa 532 c 1 ce. html#disqus_thread 13

Gaussian process This is what a Gaussian process posterior looks like with 4 data points and a squared exponential covariance function. The bold blue line is the predictive mean, while the light blue shade is the predictive uncertainty (2 standard deviations). The model uncertainty is small near the data, and increases as we move away from the data points. http: //www. cs. ox. ac. uk/people/yarin. gal/website/blog_3 d 801 aa 532 c 1 ce. html#disqus_thread 14

Monte Carlo dropout as Bayesian deep learning 1. Easy way to Bayesian deep learning network and uncertainty estimation. 1. In the MC dropout technique a stochastic dropouts are applied after each hidden layer, and the model prediction output can be approximately as a random sample generated from the posterior predictive distribution. 15

Visual representation of Dropout, right from the paper. On the left there's the network before applying Dropout, on the right the same network with Dropout applied. The network on the left it's the same network used at test time, once the parameters have been learned. 16

Predictions Uncertainty The deep learning uncertainty can be estimated using Monte Carlo dropout by calculating variance of the model predictions in a few repetitions. Zhu, Lingxue, and Nikolay Laptev. "Deep and Confident Prediction for Time Series at Uber. " Data Mining Workshops (ICDMW), 2017 IEEE International Conference on. IEEE, 2017. 17

Zhu, Lingxue, and Nikolay Laptev. "Deep and Confident Prediction for Time Series at Uber. " Data Mining Workshops (ICDMW), 2017 IEEE International Conference on. IEEE, 2017. 18

EUNITE data set has been prepared and cleaned 19

Simple LSTM ANN val_loss Epoch Electric Load The real data in orange and the predicted data in blue 20

EUNITE+LSTM layers -Dropout 21

EUNITE data set Estimate the uncertainty of energy forecast using dropout as Bayesian deep learning Line Plot of Train and Test RMSE model The real data in black and the predicted data in blue and the Loss at end of each training epoch for 100 estimated prediction uncertainty using the above algorithm the realepochs 22 data within the uncertainty of the prediction

Low foot 10 clients 23

Customer ID 145593 Left side Line Plot of Train and Test RMSE model Loss at end of each training epoch for 100 epochs Right Side The real data in black and the predicted data in blue and the estimated prediction uncertainty using the above algorithm the real data within the uncertainty of the prediction 24