I attended my first international scientific conference as a listener. In February, I applied to this conference with the forecast topic “The Short-term Electricity Consumption Forecast Competition Under COVID-19 Lockdown Conditions.” My thesis was rejected, and I wanted to find out if this rejection was fair. I listened to four forecast topics in two sections: the “Forecasting” section on June 17, and the “Power Generations and Markets” on June 18. Below is my brief overview.

1. Isabel Figuerola-Ferretti: “Oil price analysts' forecasts”

The forecasting section opened with a presentation where no one had done any forecasts. Dr. Figuerola-Ferretti & Co picked up oil future prices and oil price prediction from various analysts (Bloomberg dataset) and compared them to find which was more accurate? The key finding was the following: the analyst predictions are hardly interpretable. Dr. Figuerola-Ferretti & Co invented some tricks and assumptions that allowed them to perform such comparisons. You know how it ended up? The future prices were more accurate.

1. Massimo Guidolin: “Distilling Large Information Sets to Forecast Commodity Returns: Automatic Variable Selection or Hidden Markov Models”

This was a very enthusiastic presentation about monthly return prediction for a number of commodities (coffee, gold, oil, etc.) The three-story regression with Markov’s model coefficient switching was developed by Dr. Guidolin & Co. Indeed, if you know how to apply a spade, you’ll hammer in nails with it! Unfortunately, the research had a poor outcome: in 50% of cases, AR(1) was more accurate than the developed “draconic” regression.

1. Bartosz Uniejewski: “Regularized Quantile Regression Averaging for probabilistic electricity price forecasting.”

I’ve spent years of my professional life solving that problem. Meanwhile, young Mr. (Dr.?) Uniejewski made up an extremely efficient model using what? You got it: regression, in particular, Quantile Regression Averaging. Moreover, he quickly proved that the model allows you to earn money if you have a 1 MW battery. Of course, I had two obvious questions.

• How does the accuracy of your model compete with production models from Energy Quantified, Bloomberg, Thomson Reuters, etc.? I’m a former senior analyst in the Thomson Reuters power department, I know that the company has spent millions of dollars developing their NordPool price forecast model during the last 15 years.
• If Mr. (Dr.?) Uniejewski knows how to earn money, why does he neither do it does nor propose it to investors? At least, he didn’t mention any such activities. Unfortunately, I wasn’t allowed to ask these questions because the discussant, Dr. Figuerola-Ferretti, was choking with delight for 15 minutes straight running out of time.

1. Grzegorz Marcjasz: “Artificial Neural Networks in EPF: Are deep structures beneficial?”

This presentation took place in the power generations and markets section. Who knows why? I don’t. Mr. (Dr.?) Marcjasz endeavored to prove that applying ANN is not beneficial because it takes so long to train the model. In particular, he offered a primitive three-layer feed-forward neural network with the number of neurons by layers 8-6-1 correspondingly. He claimed that to train such a model with a training dataset of 24,000 by 1,000 will take approximately 33 years. Years! I repeat, years! Hello Google! Do you hear it? You may throw away your TensorFlow right now! Another discovery of the study is that to compare different model performances for electricity prices, you need to use MAE instead of MAPE. Such a discovery took at least three plots and three tables to prove. I’m sure that says it all.

Neither of the studies above is reproducible nor practical, so the scientific level might be discussed. First, to be reproducible you need to publish both data and code. None of that has been done by researchers and there has been no explicit reference to any non-disclosure agreement. Second, to be practical, you need to prove high efficiency not against some benchmark model, developed by the same researcher, but against the existing production solution. None of the offered models are proved to be efficient in such a respect. Although I can’t judge the quality of the text because I don’t have access to it, the presentations shed some light on the matter, and I have highlighted the key outcomes above.

The outcome of my analysis was a complaint to CEMA organization about the unfair treatment of my thesis.