Time Series

This document is an overview of time-series in the context of a flexibility value chain. It is by no means exhaustive or final, but an attempt to capture some important aspects and needs.

By time series we mean a sequence of data points collected, recorded or stored at successive points in time, often at regular intervals. The value of a data point is, in our context, electrical quantities.

Types

Time series can be of different types. The following is the definition of the types that we think are relevant in the context of a flexibility value chain.

Type	Norsk	Description	Origin
Measured values	Målte verdier, uavhenging av metode	A measure is a sampling/one-time value of a quantity or capacity. It is typically done by a sensor or dedicated metering device.	The service provider (technical aggregator) or end user has DMD data. The system operator also has so-called SCADA-data from sensors/local control system/IED.
Metering values	Målte verdier fra en måler	Metering is about keeping track of e.g. the flow of energy over a period using a meter. The recorded values are metered values and typically comes from smart meters (often referred to as AMS) or dedicated meters.	The system operator has data from conventional and smart meters. The metering values on accounting points are sent to Elhub and available there. The system operator also has meter data from other meters in the grid, e.g. on transformers.
Calculated values	Beregnede verdier	A general term for values that are based on some kind of formula or calculation. The consumption of a certain asset might be calculated by using a combination timeseries from different sources.
Baseline	Prognose / Prognosert volum over en tidsperiode	A counterfactual reference about the electrical quantities that would have been withdrawn or injected if there had been no activation of services.	The calculation of baseline can be done by the service provider (or its baseline provider), system operator(s) or by a trusted/neutral entity like FIS or third party/marketplace system.
Consumption plan	Forbruksplan	These are plans for consumption of energy that are set before any activation happens for system, balancing, local or voltage services.	Plans are provided by service providers.
Production plan	Produksjonsplan	These are plans for production of energy that are set before any activation happens for system, balancing, local or voltage services.	Plans are provided by balance responsible parties or service providers.
Timetable	Kjøreplan	This is the plan, after considering the delivery of services – the actual, continuously updated plan. Mostly useful for service providers but could also be relevant to communicate to system operators.
Activated volume	Aktivert volum	The delivered volume. Calculated using the respective baseline, when necessary, and taking into account compensation effects, if applicable
Delivered volume	Levert volum	Same as activated volume.
Quantified volume	Kvantifisert volum	Same as activated volume.
Verified volume	Verifisert volum	Same as activated volume but with additional verification done by system operator.
Traded volume	Handlet volum	This is the volume as agreed in the market/bids. Useful to compare against the actually activated volume.

There are other types of time series-like data that is highly relevant for baselining or optimization but is not included because we believe that it is not relevant for others in the value chain. Examples are weather data, moving holidays, spot-prices, grid-tariffs and so on.

Logical levels

Time series are usually linked to logical levels, typically the resources we use to model the flexibility domain. These logical levels are:

• Service Providing Group (SPG) / Portfolio
• Accounting Point (AP) / Metering point (MP)
• Controllable Unit (CU) / Asset
• Technical Resource (TR)

Origins

The origin of a time-series is the physical or concrete source of the data, i.e. where it comes from. We are using this nomenclature to distinguish between the different types of time-series origins.

Origin	Abbr	Type	Typical granularity	Responsible	Comment
Smart meter	SM	Meter	15M/60M	Connecting System Operator	Often talked about as AMS
Home Area Network-port	HAN	Measurement	Minimum every 10 seconds	Service Provider	HAN port of SM
Sub meter	SUB	Meter	15M/60M	Service Provider	Typically a MID compliant meter used to improve energy transparency and encourage energy-saving behaviors or a meter used in subgrid.
Dedicated Measurement Device	DMD	Measurement		Service Provider	See below
Dedicated Meter	DM	Meter	Realtime/1M/15M	Service Provider	See below
Calculated	CALC	Virtual	Realtime/1M/15M	Service Provider

DMD vs DM

Regulation (EU) 2024/1747 defines a Dedicated Measurement Device (DMD) as

a device linked to or embedded in an asset that provides demand response or flexibility services on the electricity market or to system operators

We have chose to "split" this term in two, depending on the nature of the measurement. If the measurement is a meter, i.e. a device that measures cumulative flow of energy, then we call it a Dedicated Meter (DM).

Measuring Instrument Directive (MID)

Directive 2014/32/EU is known as the Measuring Instrument Directive and outlines the requirements for a wide range of measuring instruments, including active electrical energy meters. MID compliance means that the meter gone through an independent conformity assessment.

Characteristics

The nature of a time-series can vary greatly, some characteristics to consider are

• Granularity - Time-resolution expressed as a duration like 1h, 15m and similar
• Quality – e.g. is it measured or metered and accuracy of measurements and baselines
• Reliability – in the data stream
• Frequency – delay in the data stream
• Timeline – e.g. future or historic data
• Audit – essentially bi-temporal data

Metering values

Metering values are the recorded data of energy production and consumption. These values are needed for creating baselines, calculating activated volume, verification and quantification. The needed time-resolution of the measured and metering values depends on their intended use.

Metering values measured by smart meters are collected by the connecting system operator. In some cases, a controllable unit (CU) represents only a small share of the total consumption behind a metering point. To accurately measure the flexibility provided by the CU, metering values must therefore be obtained directly from the CU itself using a sub meter or a dedicated meter. In such cases, the service provider collects the metering values.

Metering values collected by the connecting system operator from smart meters are stored in Elhub by 07:00 for the previous day. Current legislation requires that all residential custommers have metering values at 60-minute resolution, while commercial custommers with grid connection > 1000V have metering values at 15-minute resolution Metering values collected by sub meters or dedicated meters are not stored in Elhub and must be collected and managed by the service provider.

The procuring system operator (PSO) needs the metering values that make up a SPG in order to do both quantification and verification. These metering values must be made available through a time series service. The time series need to have IDs that can be linked to the CU and SPG IDs in the flexibility information system.

Baselines

As defined above, a baseline is a counterfactual reference about the electrical quantities that would have been withdrawn or injected if there had been no activation of services.

Baseline can be calculated on any logical level or origin, depending on the use. It is typically the service provider that has access to the most reliable (and possibly market sensitive) data that can be used to calculate an accurate baseline. They also often know the most about the assets general usage patterns. In situations where there is other loads on the same accounting point, it is unlikely that the service provider (or anyone else for that matter) can calculate a baseline that is accurate enough to be used for settlement.

Depending on method, baseline can also be calculated by another party, but that might require data from the service provider to be transferred in advance.

The fundamental problem with a baseline is that it is not possible to verify its correctness - it is counterfactural after all. We must rather determine a strategy that helps us to agree that the baseline is trustworthy or acceptable. There some main strategies to do this that can be used on their own or in combination:

• Standard method - calculated using a standard, deterministic, agreed-upon method where input data is trusted and the calculation can be repeted at any point in time.
• Auditable method - calculated using a method that can be audited, e.g. by providing the input data and method.
• Trusted party - calculated by a trusted (third-)party with no vested interest in the outcome
• Validate over time - calculated and communicated ahead-of-time and validated continuously/over time by comparing it to actual metered values when no trade is going on. This requires that the baseline is calculated continuously.

Each of these strategies allows different methods of calculating baselines as well as put different requirements on the systems and the service provider.

Baseline methods

The following sections outline some of the methods that can be used to calculate baselines. These are methods that has been surfaced in our work group discussions.

Black box

The baseline is calculated using an unspecified method that is considered a "black box" by the value chain. This is typically done by the service provider (SP), potentially based on confidential input, using a proprietary algorithm.

The black box method is the one that allows for the most innovation and allows the SP to use the best available data and methods, e.g. machine learning. However, it is also the most opaque and ensuring trust in this baseline can only be done by validating it over time.

Plan

This method uses a consumption or production plan as the baseline. Similar to the black box method, the plan can be validated over time to ensure trust in its accuracy. This method is only applicable to assets that are actively managed and have a clear operational profile, like industrial processes.

Meter before

The "Meter before" method uses the metered values before the activation as the baseline. This method should use the metered values for the same time period as the Imbalance Settlement Period (ISP) - 1h or 15m - as values typically will fluctuate between the ISPs.

Meter before and after

Similar to the "Meter before" method, but uses an average of the metered values before and after.

Sliding window

A sliding window baseline is a baseline that is calculated using a sliding window of historical data. The window is typically a fixed time period, e.g. the last 5 days, and the baseline is calculated using the data in that window.

This method can also use other inputs such as weather data, holidays, and other factors that can impact the consumption or production of energy.

Zero

The baseline is set to zero. Could be useful e.g. for fossil fuel generators or batteries. It can only be used if the asset is in contracted to not consume and/or produce energy for the periods it is part of a reservation contract.

Quantification and verification

After the delivery of a service, it must be established if the service was delivered according to the bid and the product requirements. We conceptually split this task in two, mainly based on why we do it and how often it is done. We call these activities quantification and verification.

The activities can be equal in practice, but different in what we are trying to achieve. The following table summarizes the key reasons why it is useful to think about it as two things. The following two chapters describe this in more detail.

Aspect	Quantification	Verification
Purpose	Settlement of service	Quality assurance of service delivery or instead of prequalification
Frequency	After each and every activation	Ad-hoc or at certain intervals
Level	SPG and AP/CU	SPG
Target	Volume	Volume and quality
Methods	Can use simplified methods	Usually more detailed and thorough methods

Note

How to do quantification and verification will vary between different products and service providing groups. Sometimes, e.g. when the group is "simple" and the quantification method is based on trusted data and a detailed method, then quantification and verification is the same activity.

Quantification

Quantification is the continuous activity of calculating the delivered volume after each and every activation. The purpose of quantification is to be able to do settlement. Quantification is about establishing the quantity of energy that was delivered, using a method that gives "good" and "fair" enough result for us to agree to using it.

There are three main methods for quantifying the activated volume:

• Assumed delivery - where the activated volume is assumed to be the same as what was traded in the market.
• Baseline comparison - where the activated volume is calculated by comparing the baseline to the metered values.
• Reported delivery - where the activated volume is reported by the service provider based on agreed upon terms.

Quantification is done to support three types of settlement:

• Quantification for service settlement - must end up on the SPG level since that is where the bids are.
• Quantification for imbalance adjustment and financial transfer - must be done at the level of accounting point (AP) or lower (CU/TR). NOTE - See the section Non- or over-consumed energy due to activation of flexibility for more details about imbalance settlement.
• Quantification for end user compensation - must be done at the level of the accounting point (AP) or lower (CU/TR), since that is where the end user is linked. While it is the service provider that is responsible for this and has the agreement with the end user, it is important that any potential flexibility volumes shown in a flexibility information system matches the quantities that SP is using to compensate the end user, to avoid confusion and distrust.

For some assets it is easy and practical to quantify the activated volume on the CU or AP level, e.g. a single battery facility or large boilers.

For other cases, such as a group of many small assets, it might not be practical to quantify the activated volume on the CU/AP level. By quantifying on the SPG level, we can use the "law of large numbers" to get a good prediction of the baseline (and thus the activated volume). Also, if quantification is done using the assumed delivery method, then the quantification happens on the SPG level per definition.

Having consistent quantification (e.i. the quantification "add up" between the levels) can follow one one of two main strategies:

• aggregate - sum - quantification is done on the lower level, e.g. CU or TR, and then aggregated to the SPG level.
• allocate - split . the quantification is done on the SPG level and then allocated to the lower levels, using some kind of distribution factor.

Allocation can be done with:

• fixed factors
  • even split
  • per SPG
• dynamic factors
  • assigned by SP per bid/activation - this required activation data
  • assigned by doing an approximate quantification on the lower level

Example of quantification

In order to calculate the delivered volume, a system operator can do a quantification analysis using metering values, baseline and bid volume.

First, the delivered volume can be calculated by subtracting the metering values from the baseline. Then the delivered volume is divided by the bid volume, giving a percentage of the delivery.

\[ \text{Delivered volume} = \text{baseline} - \text{metering values} \]

\[ \text{Delivery} = \frac{\text{delivered volume}}{\text{bid volume}} = x\% \]

The percentage required for settlement depends on the product-specific requirements.

This delivery calculation method can also be used for verification purposes, where checking the delivered volume serves as a quality assurance measure to confirm that the SPG delivered the agreed upon volume.

Verification

Verification is a quality assurance activity done by the system operator to ensure that the delivered service is of sufficient quality and that the agreed volume has been delivered.

Verification is targeted on the SPG level, where service is delivered.

It might be challenging to properly verify that a service was delivered. Some challenges are

• access to data of high enough quality and granularity
• a trusted source of data
• noise in data - for instance if the flexible asset represents only part of the energy consumption on an accounting point
• finding and applying the correct method to verify the service delivery

Verification can be done for every trade or activation but, given the challenges, it might be that it is feasible to only do it every now and then. This could for instance be once at application time (possibly to replace prequalification) and then ad-hoc or at certain intervals during the lifetime of the service providing group.

If, however, verification of volume is done for every trade, then we are basically doing quantification at the same time and the result should be used for settlement. It is however expected that we will see that quantification is done with a simplified method or with a higher degree of trust than what verification implies. Current TSO and DSO markets both quantify with simplified methods.

The verification might include

• Compare to trusted data - Comparing the quantified volume to what is observed in the accounting point.
• Product verification - Check if the quality of the delivered service is in accordance with the product requirements. E.g. by getting additional data from the service provider or others and doing analysis on that.
• Baseline quality - Check if the baseline is of sufficient quality based on the actual measurements.
• Consistency checks - Check delivery and bid volume over time to see if there are repeated discrepancies or systematic errors.

It is at the system operator's discretion to decide how and when to verify the service delivery, as long as it does not put an additional/undue burden on the service provider.

Activation data

Activation data is the data related to an activation of a flexible resource. For manual activation markets this is bid acceptance. For automatic reserves it is when thresholds for delivering service was met. It includes information about

• activated volume
• time period
• service providing group

To be able to quantify the activated volume for imbalance settlement, we might need to know, after the delivery of service, which CU that was actually activated. This can be:

• simple boolean - yes/no
• an actual contribution factor

Non- or over-consumed energy due to activation of flexibility

Activation of balancing and local services from fully independent service providers impacts the balance responsible party (BRP) and energy supplier (ES) on the accounting point(s) financially. Non-consumed or over-consumed energy can result in a financial loss or gain depending on prices, direction of activation and original balance of the BRP.

The BRP and ES will be impacted in two ways:

• Financial - due to less/more energy sold/bought
• Imbalance settlement - since the balance of the BRP will be affected by the activated volume

The imbalance settlement impact is of course also financial and sometimes work in “opposite direction” to the direct financial one. We should consider and handle the impacts independently since prices are different and impact is on ES vs BRP.

How to handle this in DSO markets is still pending regulation, and the current approach in the EuroFlex pilot is to do nothing. The current situation is also that a large majority of service providers are BRPs, and not independent.

In the TSO market, a split between roles for Balance Service Provider (BSP) and BRP is underway. This is not a fully independent service provider, but a small first step in that direction. Mitigation of the impact on imbalance settlement will be handled with imbalance adjustment, where the imbalance caused by activation is adjusted in the settlement. The financial impact will be handled by requiring bi-lateral agreements between BSP and the BRP. Also, all bids will be required to be with the same BRP.

Centrally done financial transfer, where money is transferred between the parties, based on a set price (spot), is another way to mitigate the financial impact. A more elaborate method, called financial compensation, could consider the actual cost of the BRP and compensate that.

One could also envision using the markets to mitigate the impacts. The BRP could be notified about activation and do intra-day trading to “cover the difference". The (D)SO could be required to make a counter-activation, i.e. an activation in the opposite direction to ensure system balance This could be used to mitigate the impacts if the counter-activation is for the correct BRP/ES. It will be close to impossible to ensure that as it will require additional methods for handling impact.

The methods described in this section are summarized in the table below. The columns imbalance settlement and financial show if the methods are applicable to mitigate that specific impact.

Method	Imbalance settlement	Financial	Comment
Do nothing	YES	YES	Simple, and can be sufficient if traded volume is not “noticeable”. This method is applicable, but not really a full-fledged "solution".
Bi-lateral agreement	NO	YES	Does not scale and is a barrier to entry for the service providers. This is a sort of bi-lateral financial transfer.
Counter-activation	YES	NO	Ensures system balance, but will be close to impossible to use for mitigating impact
Intra-day trading	YES	NO
Financial transfer	NO	YES	Centrally handled. Price will not be “correct”, but probably good enough?
Financial compensation	NO	YES	Complicated.
Imbalance adjustment	YES	NO

We believe that the most interesting methods to pilot for local flexibility markets and a common national flexibility register are do nothing, imbalance adjustment and financial transfer. It is ultimately regulation that will decide, but we can investigate if centrally handled imbalance adjustment and financial transfer is feasible at a reasonable complexity/cost.

Metered/Measured Data Administrators

We find the definition of Metered Data Administrator (MDA) e.g. in Implementing Regulation (EU) 2023/1162.

means a party responsible for storing validated historical metering and consumption data and distributing these data to final customers and/or eligible parties;

Elhub is the de-facto MDA in Norway for smart meters on the accounting point level. There is specific party assigned as administrators for sub-meter or dedicated meter/measurement device data.

Challenges

The following are some challenges that has been identified and must be addressed but are not directly in scope to “solve” in this overview.

Data privacy

The higher the granularity of data, the more sensitive it becomes. This is relevant only for personal data, i.e. values tied to a private address or a sole proprietorship company.

Data access and flow

Securing every entitled party access to time-series is one of the challenges we must solve. Does it flow via a Flexibility Information System (FIS), directly between the involved parties or via some other platform/mechanism.

Clock drift

If the time of different meter and measurement equipment is not in sync, then using those values together will produce incorrect results.

Granularity and frequency of AMS data

The AMS meters are considered of high integrity, but the lack of 15-minute data and delay in when the data is available via Elhub makes it less ideal to use for planning and baselining.

Needs

The rest of this document contains tables describing needs for different time-series in the flexibility value chain, based on the different aspects highlighted above.

In short:

• Who – needs and has data
• What – the needs are

We are making up some roles/responsibilities that is currently not defined elsewhere but since it is undefined who will do it, we single them out for clarity.

• Baseline provider – the party responsible for creating the baseline for a given (group of) asset(s)
• Activated volume calculator – the party responsible for calculating activated volume

This document does not intend to specify any technical requirements to data interchange formats, though we should not be scared of recording some details, e.g. about current implementation/requirements.

We generally want to express needs in a user-story like format. However, it also makes sense to structure the data in tabular format for easier filtering and mental processing.

As such, we have tried to specify tables that accommodate something that looks like a Five Ws user story template like phrase.

As <who>, I want <need/level/resolution/frequency> because <purpose>

Some abbreviations

• ISP – imbalance settlement period (1h / 15 m)

Who	Need	Level	Res.	Frequency	Purpose	Notes
Imbalance settlement responsible	Activated volume	AP	ISP	D+1	Do imbalance adjustment per balance responsible party.	Relevant for TSO markets if regulation allows fully independent service providers. Relevant for DSO markets if regulation states that imbalance adjustment must be done.
Balance Responsible Party and Service Providers	Activated volume	AP	ISP	D+1	Do financial transfers with energy suppliers.	When BRP/SP is responsible for the financial settlement with SP/ES due to relying on bi-lateral agreement between SP and BRP.
Energy Supplier	Activated volume	AP	ISP	D+1	To be informed about activations
Flexibility market	Activated volume	SPG	15m	Same day	Settle the activated flex.	Depending on the market/rules, this can be pay-as-bid, calculated with use of baseline.
Flexibility marked	Baseline	SPG	15m	Ahead of trading	Bid screening and ensuring that there is “available” flex	This is currently done in both NODES and Statnett markets.
Service provider	Activated volume	AP/CU	15m	Same day	Share rewards with end user.	It is not given that the SP will distribute rewards based on the actual activated volume per CU. Depends on the agreement between the SP and end user.
Service provider	Metered values	SM/AMS	15m	Real time	Planning and optimizations
Baseline provider	Measured values	TR	1m	Real time	Create good baselines
Baseline provider	Metered values	SM/AMS	15m	Real time	Create good baselines
Activated volume calculator	Metered values		15m	Same day	Calculate activated volume	It is not given that the Activated volume is derived from baseline/metered on the same level. An example can be for groups: Baseline and activated is calculated on the SPG level, then the activated is “distributed” on the CUs via some key/algorithm.
Activated volume calculator	Baseline		15m	Same day	Calculate activated volume
System operator	Baseline	CU/AP	At least ISP	Same day	Verifiy the provision of the service
System operator	Updated baseline	-	Ahead of trading	When baseline change	To have the best available data for verification
System operator	Corrected baseline	-	After trading	When baseline have errors	To be able to correct in case of errors
System operator	Parallell baselines	-	-	-	Assess different baseline methods over time
System operator	Baseline			Continuously ahead of trading	Validate baseline (if provided by service provider)	Validating baseline can be done by comparing the baseline to actual metered values over time.
System operator	Metered values	AP/AMS	At least ISP		Verify the provision of the service	Procuring system operator might not be the connecting system operator, so they might not have this data.