Assignment1-Mef Big Data Msc
Import Data
library(knitr)
raw_df=read_csv("C:/Users/dell/Desktop/PTP-SMP.csv")##
## -- Column specification --------------------------------------------------------
## cols(
## Tarih = col_character(),
## PTF = col_double(),
## SMF = col_number(),
## `Pozitif Dengesizlik Fiyati (TL/MWh)` = col_double(),
## `Negatif Dengesizlik Fiyati (TL/MWh)` = col_number(),
## `SMF Yon` = col_character()
## )raw_df <- data.frame(raw_df,stringsAsFactors = FALSE)Format “Tarih” Column As Timestamp
In the Csv file, date(Tarih) column is character type. Converting Timestamp variable type() is needed for manipulating this column in the further steps.
raw_df$Tarih<-as.POSIXct(raw_df$Tarih, format ="%m/%d/%Y %H:%M")IN_USE_PRICING
SMP is always higher than MCP if system has Energy Deficit, and lower if there is Energy Surplus. Market operator also penalizes the operations in BPM by 3%. This is called Imbalance Price. Negative (Deficit) Imbalance Price is calculated as max(MCP,SMP)1.03 and Positive Imbalance Price is calculated as min(MCP,SMP)0.97.
IN_USAGE_PRICING column is created to get which price is preferred for any hour based on conditional rule above. For example, IF, SMF greater than PMF Negative Imbalance Market price value is choosen. Else if PMF higher than the SMF, Positive Imbalance Value is bringing. Otherwise, when the PMF equal to SMF, PMF is choosen in the code below.
raw_df <- raw_df %>% mutate(IN_USE_PRICING = ifelse(PTF==SMF, PTF,ifelse(SMF>PTF,Negatif.Dengesizlik.Fiyati..TL.MWh.,Pozitif.Dengesizlik.Fiyati..TL.MWh.) ))Select Only Necessary Columns
Now, I have created dataframe namely, “df” to show only essential columns which are “Tarih”,“SMF.Yon”,“IN_USE_PRICING” and get better readibility.
df<-raw_df %>% select(Tarih,SMF.Yon,IN_USE_PRICING)SMF.Yon Barchart
By making up histogram chart, it is easy to understand that the most occurred price type during trade-off is Energy Deficit price, after that Energy Suprlases is placed in second and then Balancing price is following them.
We can conclude from these during 30 days of September 2020, Electricty production companies’ price estimation is oftenly under the Market Price. Negative Imbalance Price(%3 is higher than SMF) is dominating the market
ggplot(data.frame(df), aes(x=SMF.Yon))+ geom_bar(width=0.7, fill="steelblue")+theme_minimal()+ theme(axis.title.x=element_text(size=15),axis.title.y=element_text(size=15),plot.title = element_text( size =15)) +labs(title="SMF.YON HISTOGRAM",x ="Type Of SMF.YON", y = "Count")
Monitoring Daily Prices on September
While 00:00- 05:00 hours prices are stabile direction and anomality datas is not observed for all 24 hours of 30 days.
The most outlier data points have been observed in 3rd, 7th and 17th of September.
28th,29th,30th days have the most fluctuated price graphs in a day.
’
ggplot(df,aes(x=hour(Tarih),y=IN_USE_PRICING))+geom_point(size=3)+
facet_wrap(~ day(Tarih))+ theme(axis.text.x = element_text(face="bold", color="#993333",size=25, angle=45),axis.title.x=element_text(size=35),
axis.text.y = element_text(face="bold", color="#993333",
size=25, angle=45),strip.text = element_text(size=25),axis.title.y=element_text(size=35),plot.title = element_text( size = 40)) +labs(title="Trade Price Change Over Hours",x ="Hours", y = "Price In Usage(Turkish Liras)")
Observing Daily Average Price with Max, Min and Std Metrics
avg_daily <- df %>% group_by(days=day(Tarih)) %>% summarise(avg_daily_price=mean(IN_USE_PRICING),.groups="drop")Max average daily price has come up in 3rd day which is 479 Turkish Liras.
avg_daily[which.max(avg_daily$avg_daily_price),]## # A tibble: 1 x 2
## days avg_daily_price
## <int> <dbl>
## 1 3 479.Max average daily price has come up in 19th day which is 216 Turkish Liras.
avg_daily[which.min(avg_daily$avg_daily_price),]## # A tibble: 1 x 2
## days avg_daily_price
## <int> <dbl>
## 1 19 216.Stdandard Deviation for each 30 average day values is 56 Turkish liras.
avg_daily %>% summarise(stdev=sd(avg_daily_price))## # A tibble: 1 x 1
## stdev
## <dbl>
## 1 56.0ggplot(avg_daily,aes(x=days,y=avg_daily_price))+ geom_line(color='steelblue', size=2, alpha=0.8)+ coord_cartesian(xlim=c(0, 30), ylim=c(200, 500)) +scale_x_continuous(breaks=seq(1,30))+theme(axis.title.x=element_text(size=15),axis.title.y=element_text(size=15),plot.title = element_text( size =15)) +labs(title="Average Daily Price Change Over Days",x ="Day", y = "Average Daily Price(Turkish Liras)")
Observing Hourly Average Price with Max, Min and Std Metrics
avg_hourly <- df %>% group_by(Hour=hour(Tarih)) %>% summarise(avg_hourly_price=mean(IN_USE_PRICING),.groups="drop")
avg_hourly## # A tibble: 24 x 2
## Hour avg_hourly_price
## <int> <dbl>
## 1 0 294.
## 2 1 292.
## 3 2 280.
## 4 3 273.
## 5 4 269.
## 6 5 258.
## 7 6 252.
## 8 7 249.
## 9 8 291.
## 10 9 341.
## # ... with 14 more rowsMaximum Hourly Average Price is occurred 16:00 hour with 458 Turkish Liras
avg_hourly[which.max(avg_hourly$avg_hourly_price),]## # A tibble: 1 x 2
## Hour avg_hourly_price
## <int> <dbl>
## 1 16 458.Minimum hourly Average Price is occurred 07:00 am with 249 Turkish Liras
avg_hourly[which.min(avg_hourly$avg_hourly_price),]## # A tibble: 1 x 2
## Hour avg_hourly_price
## <int> <dbl>
## 1 7 249.Average Hourly Standart Deviation between 24 hours value is 53.4 Turkish Liras
avg_hourly %>% summarise(stdev=sd(avg_hourly_price))## # A tibble: 1 x 1
## stdev
## <dbl>
## 1 53.4ggplot(avg_hourly,aes(x=Hour,y=avg_hourly_price))+ geom_line(color='steelblue', size=2, alpha=0.8) +scale_x_continuous(breaks=seq(1,24))+theme(axis.title.x=element_text(size=15),axis.title.y=element_text(size=15),plot.title = element_text( size =15)) +labs(title="Average Hourly Price Change Over Hours",x ="Hour", y = "Average Hourly Price(Turkish Liras)")
Summary
Negative Energy imbalanced Price situation is occurred more than 450 hours from total 720 hours.
In a normal day, Hourly Price values has been placed 350-400 TL price range.
The most Expensive average hour prices showed up at 16:00 hour.
12:00- 05:00 prices are represented in a straight line for 30 days. There is no anomaly price is detected.