Electricity Market Prices Analysis of July 2020
ilaydacelenk_IE48A
8/12/2020
Introduction
The following analysis for July, 2020 is made acccording to MCP/SMP data from Energy Exchange Istanbul(EXIST) page. Data for the energy market is hourly.
Some Basic Definitions and Principles of the Energy System
- MCP: Market Clearing Price
- SMP: System Marginal Price
- Positive Imbalance Price: min(MCP,SMP)*0.97
- Negative Imbalance Price: max(MCP,SMP)*1.03
- If actual demand is higher than predicted demand, there is Energy Deficit and SMP is higher than MCP.
- If actual demand is lower than predicted demand, there is Energy Surplus and SMP is lower than MCP.
Every day, consumers declares next day’s hourly forecast on energy demand. Producers try to predict how much energy would be needed hourly as well. They do this in order to produce according to the demand. Just like all markets, when the demand is higher than supply, the price goes up and when the demand is lower than supply, the price goes down. Simple logic would suggest there is lower price at night, weekends and holidays. Therefore, it would be wise to consider it in this report. Since perfect forecast cannot be achieved numerically, imbalance prices are needed in order to solve the energy deficit and surplus problems. Forecasts can be improved using this report in machine learning processes.
Load Libraries and Read Dataset
library(readr)
library(tidyverse)
library(lubridate)
library(tinytex)
raw_df <- read.csv("https://raw.githubusercontent.com/pjournal/boun01-ilaydacelenk/gh-pages/energy_data_en.csv", header = TRUE, check.names=TRUE)Overview of Data
Columns of raw_df are renamed for simplicity, the date format is specified, SMP direction values are renamed and Dataset is created from raw_df according to these changes. Glimpse() function is called before and after in order to see the changes.
raw_df %>% glimpse()## Rows: 744
## Columns: 6
## $ Date <chr> "01.07.20 00:00", "01.07.20 01:00",…
## $ MCP. <dbl> 323.85, 326.95, 324.31, 322.11, 320…
## $ SMP. <dbl> 211.00, 201.00, 211.00, 211.00, 201…
## $ Positive.Imbalance.Price..TL.MWh. <dbl> 204.67, 194.97, 204.67, 204.67, 194…
## $ Negative.Imbalance.Price..TL.MWh. <dbl> 333.57, 336.76, 334.04, 331.77, 329…
## $ SMP.Direction <chr> "?Energy Surplus", "?Energy Surplus…Dataset <- raw_df %>%
rename(
dt = Date,
MCP = MCP.,
SMP = SMP.,
pos_imb_price = Positive.Imbalance.Price..TL.MWh.,
neg_imb_price = Negative.Imbalance.Price..TL.MWh.,
SMP_direction = SMP.Direction
)
Dataset$dt <- strptime(Dataset$dt,format="%d.%m.%y %H:%M")
for(i in 1:length(Dataset$SMP_direction)){
if(Dataset$SMP_direction[i] == "?Energy Surplus"){
Dataset$SMP_direction[i] = "Energy_Surplus"
} else if(Dataset$SMP_direction[i] == "? Energy Deficit") {
Dataset$SMP_direction[i] = "Energy_Deficit"
} else {
Dataset$SMP_direction[i] = "In_Balance"
}
}
Dataset %>% glimpse()## Rows: 744
## Columns: 6
## $ dt <dttm> 2020-07-01 00:00:00, 2020-07-01 01:00:00, 2020-07-01 0…
## $ MCP <dbl> 323.85, 326.95, 324.31, 322.11, 320.00, 286.21, 210.13,…
## $ SMP <dbl> 211.00, 201.00, 211.00, 211.00, 201.00, 181.00, 113.75,…
## $ pos_imb_price <dbl> 204.67, 194.97, 204.67, 204.67, 194.97, 175.57, 110.34,…
## $ neg_imb_price <dbl> 333.57, 336.76, 334.04, 331.77, 329.60, 294.80, 216.43,…
## $ SMP_direction <chr> "Energy_Surplus", "Energy_Surplus", "Energy_Surplus", "…Head and Tail of the Dataset
To understand the data, observe the entries.
head(Dataset)## dt MCP SMP pos_imb_price neg_imb_price SMP_direction
## 1 2020-07-01 00:00:00 323.85 211 204.67 333.57 Energy_Surplus
## 2 2020-07-01 01:00:00 326.95 201 194.97 336.76 Energy_Surplus
## 3 2020-07-01 02:00:00 324.31 211 204.67 334.04 Energy_Surplus
## 4 2020-07-01 03:00:00 322.11 211 204.67 331.77 Energy_Surplus
## 5 2020-07-01 04:00:00 320.00 201 194.97 329.60 Energy_Surplus
## 6 2020-07-01 05:00:00 286.21 181 175.57 294.80 Energy_Surplustail(Dataset)## dt MCP SMP pos_imb_price neg_imb_price
## 739 2020-07-31 18:00:00 310.78 206.18 199.99 320.10
## 740 2020-07-31 19:00:00 317.79 260.00 252.20 327.32
## 741 2020-07-31 20:00:00 324.94 205.00 198.85 334.69
## 742 2020-07-31 21:00:00 325.98 266.98 258.97 335.76
## 743 2020-07-31 22:00:00 325.07 340.07 315.32 350.27
## 744 2020-07-31 23:00:00 321.78 210.00 203.70 331.43
## SMP_direction
## 739 Energy_Surplus
## 740 Energy_Surplus
## 741 Energy_Surplus
## 742 Energy_Surplus
## 743 Energy_Deficit
## 744 Energy_SurplusSome Analysis
Summary of Dataset
Here the minimum, maximum and average values can be seen. According to the plot, above the blue line, SMP values are higher than MCP and this means for those data points there is energy deficit, ie. actual consumption is higher than the forecast.
There are 744 data points in total.
Dataset %>% summary## dt MCP SMP pos_imb_price
## Min. :2020-07-01 00:00:00 Min. :140.0 Min. : 10.0 Min. : 9.7
## 1st Qu.:2020-07-08 17:45:00 1st Qu.:300.0 1st Qu.:250.0 1st Qu.:228.2
## Median :2020-07-16 11:30:00 Median :306.6 Median :320.0 Median :293.4
## Mean :2020-07-16 11:30:00 Mean :296.4 Mean :299.3 Mean :268.6
## 3rd Qu.:2020-07-24 05:15:00 3rd Qu.:317.0 3rd Qu.:345.0 3rd Qu.:304.1
## Max. :2020-07-31 23:00:00 Max. :350.0 Max. :460.0 Max. :339.5
## neg_imb_price SMP_direction
## Min. :144.2 Length:744
## 1st Qu.:312.9 Class :character
## Median :331.9 Mode :character
## Mean :328.3
## 3rd Qu.:355.4
## Max. :473.8ggplot(Dataset, aes(x=MCP, y=SMP)) +
geom_point() + theme(aspect.ratio=1)+ xlim(0,400) + xlim(0,400) + ggtitle("MCP vs SMP Prices of July 2020") + labs(x = "MCP", y = "SMP") + geom_abline(slope=1, intercept=0, color="blue")
Number of points where there is energy surplus is 195. For these points SMP is lower than MCP, which indicates that the forecasts are higher than the actual required energy.
Number of points where there is energy deficit 528. For these points SMP is higher than MCP, which indicates that the usage is underestimated by the market.
Number of points where there is energy balance 21. These are the perfect forecasts.
Dataset %>% summary## dt MCP SMP pos_imb_price
## Min. :2020-07-01 00:00:00 Min. :140.0 Min. : 10.0 Min. : 9.7
## 1st Qu.:2020-07-08 17:45:00 1st Qu.:300.0 1st Qu.:250.0 1st Qu.:228.2
## Median :2020-07-16 11:30:00 Median :306.6 Median :320.0 Median :293.4
## Mean :2020-07-16 11:30:00 Mean :296.4 Mean :299.3 Mean :268.6
## 3rd Qu.:2020-07-24 05:15:00 3rd Qu.:317.0 3rd Qu.:345.0 3rd Qu.:304.1
## Max. :2020-07-31 23:00:00 Max. :350.0 Max. :460.0 Max. :339.5
## neg_imb_price SMP_direction
## Min. :144.2 Length:744
## 1st Qu.:312.9 Class :character
## Median :331.9 Mode :character
## Mean :328.3
## 3rd Qu.:355.4
## Max. :473.8plot_pie <- Dataset %>% group_by(SMP_direction) %>% summarise(count = n()) %>% mutate(proportion=count/sum(count)) %>% ungroup() %>% arrange(desc(count)) %>% mutate(SMP_direction=factor(SMP_direction, levels = as.character(SMP_direction)))
plot_pie## # A tibble: 3 x 3
## SMP_direction count proportion
## <fct> <int> <dbl>
## 1 Energy_Deficit 528 0.710
## 2 Energy_Surplus 195 0.262
## 3 In_Balance 21 0.0282ggplot(plot_pie, aes(x="", y= proportion, fill=SMP_direction)) + geom_col() + geom_text(aes(label = scales::percent(round(proportion,2))), position = position_stack(vjust = 0.5))+ coord_polar(theta = "y") + theme_void() + ggtitle("Energy Direction") + theme(plot.title = element_text(hjust = 0.5))
Hourly Average MCP and SMP
Since changes in the energy market happens hourly and in this report July 2020 data is analysed, there are 31 days with 24 data points for each day. The price changes regarding the hour of the day should be observed. In the graph, MCP vs. hour_of_day is shown to see the relationship between same hours of different days.
Due to lower demand, lower price is observed at night as expected.
Observe that for most of the hours, MCP over different days are very close to each other. This means there could be a relationship between same hours of different days.
plot1 <- Dataset %>% group_by(hour_of_day = lubridate::hour(dt), day=lubridate::date(dt))
ggplot(plot1, aes(x = hour_of_day, y = MCP, color=as.character(day))) + geom_point() + ggtitle("MCP vs Hour of Day for July 2020") + labs(x="Hour of Day", y="MCP") + theme(plot.title = element_text(hjust = 0.5))
After observing the relationship, it would be wise to calculate the hourly average values for each hour of the day. The plot below shows hourly average MCP and SMP values vs hour of day.
plot2 <- Dataset %>% group_by(hour_of_day = lubridate::hour(dt)) %>% summarise(avg_MCP = mean(MCP), avg_SMP = mean(SMP))
ggplot(plot2) + geom_line(aes(x = hour_of_day, y = avg_MCP, color="MCP")) + geom_line(aes(x = hour_of_day, y = avg_SMP, color="SMP")) + ggtitle("Hourly Average MCP and SMP vs Hour of Day for July 2020") + labs(x="Hour of Day", y="Average MCP and SMP") + theme(plot.title = element_text(hjust = 0.5))
Daily Average MCP and SMP
plot2 <- Dataset %>% group_by(day = lubridate::day(dt)) %>% summarise(daily_avg_MCP= mean(MCP), daily_avg_SMP= mean(SMP))
print(plot2)## # A tibble: 31 x 3
## day daily_avg_MCP daily_avg_SMP
## <int> <dbl> <dbl>
## 1 1 320. 268.
## 2 2 298. 357.
## 3 3 317. 373.
## 4 4 306. 321.
## 5 5 258. 198.
## 6 6 290. 307.
## 7 7 307. 298.
## 8 8 304. 191.
## 9 9 288. 189.
## 10 10 282. 292.
## # … with 21 more rowsggplot(plot2) + geom_point(aes(x = day, y = daily_avg_MCP, color="MCP")) + geom_point(aes(x = day, y = daily_avg_SMP, color="SMP")) + ggtitle("Daily Average MCP and SMP vs Hour of Day for July 2020") + theme(plot.title = element_text(hjust = 0.5)) + xlab("Day") + ylab("Daily Average Prices")
Average MCP and SMP over Weekdays
Lower price is expected at weekends. Looking at the plot, average MCPs are respectively lower at weekends. On the other hand, SMPs are not correlated with this.
plot3 <- Dataset %>% group_by(weekday = lubridate::wday(dt)) %>% summarise(wd_avg_MCP= mean(MCP), wd_std_MCP=sd(MCP), wd_avg_SMP= mean(SMP), wd_std_SMP=sd(SMP))
plot3## # A tibble: 7 x 5
## weekday wd_avg_MCP wd_std_MCP wd_avg_SMP wd_std_SMP
## <dbl> <dbl> <dbl> <dbl> <dbl>
## 1 1 275. 40.6 289. 70.2
## 2 2 302. 38.5 317. 62.6
## 3 3 299. 32.7 314. 49.2
## 4 4 307. 28.3 276. 69.8
## 5 5 297. 32.8 301. 69.4
## 6 6 299. 34.5 294. 82.1
## 7 7 292. 32.7 311. 38.6ggplot(plot3) + geom_line(aes(x=weekday, y=wd_avg_MCP, color="MCP")) + geom_line(aes(x=weekday, y=wd_avg_SMP, color="SMP")) + ggtitle("Average MCP and SMP vs Weekdays") + theme(plot.title = element_text(hjust = 0.5)) + xlab("Weekdays") + ylab("Average MCP and SMP")
Positive Imbalance Price and Negative Imbalance Price vs Energy Direction
When there is energy surplus, Positive Imbalance Price(PIP) means energy surplus, and Negative Imbalance Price(NIP) means energy deficit.
According to the graph below, Positive Imbalance Price is always higher than Negative Imbalance Price. Hourly Average PIP and NIP values behave very similar to Hourly Average MCP and SMP. This was also expected.
plot4 <- Dataset %>% group_by(hour_of_day = lubridate::hour(dt)) %>% summarise(avg_pos = mean(pos_imb_price), avg_neg = mean(neg_imb_price))
ggplot(plot4) + geom_line(aes(x = hour_of_day, y = avg_pos, color="MCP")) + geom_line(aes(x = hour_of_day, y = avg_neg, color="SMP")) + ggtitle("Hourly Average PIP and NIP vs Hour of Day for July 2020") + labs(x="Hour of Day", y="Average PIP and NIP") + theme(plot.title = element_text(hjust = 0.5))
References
Dataset: MCP/SMPdatafrom EXIST page
To see all my work related to IE 48A course, visit my Progress Journal website. You can also visit my personal Github Website.