A stochastic dynamic programming model for hydropower scheduling with state-dependent maximum discharge constraints

We present a medium-term hydropower scheduling model that includes in ﬂ ow- and volume-dependent environmental constraints on maximum discharge. A stochastic dynamic programming algorithm (SDP) is formulated to enable an accurate representation of nonconvex relationships in the problem formu- lation of smaller hydropower systems. The model is used to assess the impact of including state-dependent constraints in the medium-term hydropower scheduling on the calculated water values. The model is applied in a case study of a Norwegian hydropower system with multiple reservoirs. We ﬁ nd that the maximum discharge constraint signi ﬁ cantly impacts the water values and simulated operation of the hydropower system. A main ﬁ nding is that the nonconvex characteristics of the environmental constraint are re ﬂ ected in the water values, implying a nonconvex objective function. Oper- ation according to the computed water values is simulated for cases with and without the environmental constraint. Even though operation of the system changes considerably when the environmental constraint is included, the total electricity generation over the year is kept constant, and the total loss in expected pro ﬁ t is limited to less than 0.8%. © 2022 Authors. Published Ltd. the


Introduction
Through the European Green Deal, the EU has set ambitious targets for both climate change mitigation and broader environmental sustainability [1]. To align capital flows with these policy goals, the EU is in the process of defining requirements for environmentally sustainable activities. Hence, power producers have strong incentives to operate in an environmentally sustainable way.
Like all power plants, hydropower operations may modify the surrounding ecosystems [2]. The environmental concerns of hydropower operations are often related to alteration of the flow regime downstream the plant [3], but can also be related to the changes in water temperatures or changed water volumes in the reservoirs. To protect ecological and recreational interests, regulation often imposes mitigation measures and limitations on operation, see e.g., Refs. [4,5].
Environmental regulation should be incorporated as constraints in optimisation-based hydropower scheduling models. Omitting such constraints can result in misestimation of hydropower electricity generation, revenues, and the amount of flexibility hydropower can provide to the electricity system [6]. The need for flexible resources is expected to increase as the transition of the European power system moves forwards. In the transition towards a low-carbon system, hydropower can play an important role as a flexibility provider by responding to rapid fluctuations in intermittent renewable generation and load [7]. Additional constraints on operation may reduce this flexibility potential. To correctly represent hydropower operation requires properly accounting for environmental constraints imposed on hydropower production.
In this research we are concerned with operational hydropower scheduling, i.e., the sequence of decisions that are made leading up to the actual operation of the system. Due to the computational complexity, the scheduling problem is normally solved for different planning horizons and technical details. Medium-term hydropower scheduling considers reservoir management under uncertainty over a planning horizon of several months up to a few years. In contrast, short-term scheduling usually concerns operational decisions over a period of days to weeks, and typically accounts for more technical details. In the decentralised Nordic system, medium-term scheduling models are used to compute water values, which are an essential input to the operational short-term models [8]. The purpose of water values is to reflect the longterm value of short-term operational decisions.
State-of-the-art methods to solve the medium-term scheduling of hydropower systems use stochastic dual dynamic programming (SDDP) [9] algorithms. These algorithms decompose the problem without discretising the state variables (such as reservoir volume and inflow), making it computationally tractable for systems with multiple reservoirs. For medium-term scheduling in the Nordic market, the combined SDP/SDDP method described in Ref. [10] is widely used. It combines SDDP with an outer layer based on stochastic dynamic programming (SDP) [11] to treat uncertainty in market prices. Still, a major drawback of the method is that nonconvexities cannot be easily treated in the modelling.
Nonconvex problem formulations are typically needed to represent the complex interaction between power output and water [12], and unit commitment of generators [13]. The challenge of representing nonconvex relationships in the SDDP algorithm has frequently been addressed in the literature by the use of approximations, e.g., Refs. [14e17]. A few studies also consider accurate modelling of non-convexities by the use of SDP [18] or stochastic dual dynamic integer programming (SDDiP), such as in Refs. [19,20].
SDP was used early on in hydropower planning, as it allows for explicit representation of uncertainty, e.g., in inflow and price [21e23]. The method has the advantage that it can represent nonconvex and nonlinear relationships. The main drawback of the method is that the state variables have to be discretised, causing the problem to grow exponentially in size with the number of state variables (e.g., reservoirs). The method is therefore best suited to solve systems with a small number of reservoirs, unless an aggregation technique is used like in Refs. [24e26]. Despite of this weakness, the SDP method is very well suited for accurate scheduling of hydropower systems with pronounced nonconvexities.
Regulators have imposed a wide range of environmental constraints on hydropower systems. Ecological flow requirements and maximum ramping rates are often applied, and may have significant impact on the flexibility of the hydro system. These constraints have been extensively studied in the technical literature, see e.g., Refs.
[27e29], and can be included in hydropower scheduling models without compromising the convexity requirement of the SDDP algorithm. However, fewer research studies consider environmental constraints that involve state-dependencies or logical conditions, which can not easily be treated in a convex model formulation [30]. Furthermore, only a very limited number of studies discuss the impact of environmental constraints on water values. To the best of our knowledge, the impact of environmental constraints on water values has only been discussed in Refs. [31,32]. In Ref. [32], an SDP-based model is used to evaluate the sensitivity of the water values to environmental flows and ramping restrictions. The authors of [31] find that incorporating the same environmental constraints into a linear programming based water value model has significant impacts on the profitability of hydropower plants with one or at most two turbines. This research considers the representation of a particular type of state-dependent environmental constraint in the medium-term scheduling of hydropower operation. Several European countries consider lake water alterations to be relevant mitigation measure to reduce impacts from water regulations [33], one example being statedependent maximum discharge regulations. In the Nordic region, such constraints are imposed on several reservoirs, and are likely to be implemented in other hydropower reservoirs in the future. The purpose of the regulation is to retain inflow during spring, to meet the ecological and recreational needs for high water levels in the reservoir through summer. In other regions, volume-dependent maximum discharge constraints are also used to allocate available water between irrigation purposes and electricity production [17].
Volume-dependent maximum discharge regulation in long-and medium-term hydropower scheduling has previously been studied by the use of SDDiP in Ref. [20] and approximated using SDDP in Ref. [17]. We accurately represent state-dependent maximum discharge constraints in a medium-term hydropower scheduling model based on SDP. Compared to Ref. [17], a different type of maximum-discharge regulation is considered. Furthermore, we use a different methodology (based on SDP) allowing an accurate formulation of state-dependent constraints. Compared to Ref. [20], we use a more mature methodology (SDP) and take a broader view of state-dependencies, studying a formulation of the maximum discharge constraint with a dynamically defined constraint period dependent on inflow. SDP-based models have previously been used to study environmental constraints in Refs. [31,32], but not for environmental constraints that include state-dependencies such as those discussed here.
The developed model is tested on a two-reservoir case study of a Norwegian hydro system. We discuss how the state-dependent environmental constraint modifies the resulting water value curves from the SDP algorithm, further distinguishing our research from Refs. [17,20], and use the water values to simulate operation of the system. The simulation results show the impact of the constraint on reservoir operation and economics. The novel contribution of this research is twofold in that we: Formulate a medium-term hydropower scheduling model based on SDP that accurately treats state-dependencies in the maximum allowed discharge from hydropower stations. Maximum discharge depends on the state variables reservoir volume, weekly inflow and a variable indicating if the lowinflow period has ended. Assess the impact of including such state-dependent maximum discharge constraints on the water value curves and shed light on the potential impacts that system operation guided by such curves may have. The assessment is carried out for a hydropower cascade in Norway.
The remainder of this paper is structured as follows: Section 2 describes the developed hydropower scheduling model; a case study is presented in Section 3; and concluding remarks are found in Section 4. Section 2 comprises subsections describing the weekly decision problem (Section 2.1), the state-dependent environmental constraint on maximum discharge (Section 2.2), the stochastic variables (Section 2.3) and the solution strategy (Section 2.4). The case study in Section 3 presents calculated water values (Section 3.2) and results from the simulations (Section 3.3).

Hydropower scheduling model
In the following we present a medium-term hydropower scheduling model that is formulated for a hydropower cascade operated by a single hydropower producer assumed to be a riskneutral price taker. The model maximises profit from operating the hydropower system for a presumed stationary future system state. The operation of the system is optimised for weekly decision stages over a horizon of one year.
The hydropower scheduling problem is a multi-stage stochastic optimisation problem. To solve the problem, we decompose the overall problem into several smaller subproblems, using the principles of SDP [11]. By decomposing the problem, we obtain one separate decision problem for each stage and state of the system. The SDP algorithm solves the decision problem, described in Section 2.1, for all stages and system states until convergence, as described in Section 2.4. The scope of potential system states is divided into a set of discrete states. The discrete states include all the information that is passed between the decision stages, from t À 1 to t. The set of states comprise subsets of endogenous states S p and exogenous stochastic state variables S u . The storage volume in the reservoirs is the endogenous state variable. The stochastic state variables are: the weekly average energy price l, the weekly total inflow into the systemẐ and the environmental state variable x. The stochastic variables are represented by a discrete Markov chain, as described in Section 2.3. The environmental state variable indicates if the socalled "low-inflow period" has ended. In practice, the variable indicates whether the inflow level has exceeded a certain threshold over a shorter period of time. The extension of the discrete Markov chain to include an environmental state variable is explained further in Section 2.3, while the environmental constraint is described thoroughly in Section 2.2. The implementation of the environmental constraint in the SDP algorithm is described in Section 2.4.

The weekly decision problem
The decision problem is solved for all system states, i.e., combinations of discrete reservoir volumes and stochastic nodes in the Markov chain. Since the stochastic variables are known at the beginning of the stage, each single decision problem is solved as a deterministic problem. The stochastic nature of the problem is managed in the SDP algorithm. Uncertainty is represented through the price and inflow states, and the uncertainty of future realisations of the stochastic variables are reflected in the expected water values. Each stage is divided into K number of time steps.
The objective function (1) maximises the immediate profit of the decisions and the impact on the expected future profit given by a tþ1 .
The expected future profit is a function of the stochastic state of the system and the resulting storage volume in the reservoirs at the end of the stage. Spillage of water is penalised according to a low cost C S . Furthermore, operation of the reservoirs below a filling degree of 10% is penalised to represent risk-aversion of the producer.
The energy production is a function of the discharge, q k,h,d , from each of the reservoirs, as given in (2). The maximum and minimum discharge is limited by (3) and (4). Furthermore, the discharge is bounded by the availability of water in the reservoirs. The reservoir balance (5), keeps track of the change in water volume in each reservoir, where the volumes, v k,h , are bounded by (6). To calibrate the reservoir management, a soft-constraint on minimum reservoir volume is used to reflect the risk-aversion of the producer.
While (1)e(6) describes the general decision problem, the following equations present the interpolation in the expected future profit function for a system with two reservoirs. The expected future profit a tþ1 is a function of the storage volume in the reservoirs at the end of the stage, and is bounded by (7)- (12). A two-dimensional, piecewise-linear approximation obtained by the triangle method is used to represent a tþ1 . The triangle method approximates multidimensional functions by the use of linear triangles, see e.g., Refs. [34,35]. The method was chosen for its ability to approximate nonconvex functions and its simplicity [36]. The formulation can be adapted to larger hydropower systems by expanding the dimensions of the expected future profit approximation.
The optimal expected future profit is obtained by convex combination of the expected future profit points FV n,m using the weighting variables g n,m , as given by (7)- (9). The points are calculated for each of the discrete reservoir states in the previous stage, and given as input to the optimisation problem. The sum of the weighting variables in each dimension are used to find the total weight of the discrete volume segments for each reservoir in (10)- (11), binding the expected future profit to the storage volumes in the reservoirs at the end of the stage. a tþ1 À X n2N X m2N g n;m FV n;m ¼ 0 Nonconvex characteristics in the expected future profit function are dealt with by restricting the weighing variables (g). In the optimal solution, a maximum of two adjacent weights (g) in each dimension can have non-zero values, thereby forming a square of adjacent weighting variables that can be active. Such behaviour can be enforced by special ordered sets of type 2 (SOS2) [37]. In SOS2, only two adjacent variables in the set can be non-zero. SOS2 are included in most commercial solvers, such as CPLEX, which is used in this research. In (14), b h is defined as one SOS2 for each dimension h (i.e., for each reservoir). By using four weights to describe a point in two dimensions, the model is given the freedom to decide which points to use. To ensure one unique solution, we force the model to use a predefined set of weights (3 out of 4) by defining the set c to be a SOS2 in (15) [37]. The set c comprises the sum of the weights g in the diagonal direction, hence forming a triangle of adjacent weighting variables. Piecewise-linear formulations of functions in two and three dimensions are thoroughly discussed in Ref. [35]. The SOS2 defined in (14) and (15) can be removed if the future profit function is concave, changing the formulation from an MILP to an LP.

Activation of the environmental regulation
The purpose of the considered state-dependent maximum discharge regulation is to meet the needs of ecological habitat and recreational use for high water levels in the hydropower reservoir in summer. Due to high seasonal and yearly variations in inflow, time-dependent minimum reservoir level constraints may lead to inefficient reservoir management during the low-inflow, winter period and increased system costs. To avoid this, the regulation is rather formulated as a state-dependent restriction on discharge from the reservoir. As discussed in section 1, formulations that include state-dependencies can present modelling challenges. We here present a general formulation of the regulation where a selection of different requirements can be included: Other than to honour minimum flow obligations, no discharge is allowed within a given period of time, unless the storage volume in the reservoir reaches a given threshold. If the required threshold is reached, discharge is permitted as long as the storage volume is kept above the threshold. The constraint can be activated by a given date or by the-end of the "low-inflow period", i.e., inflow levels above a given threshold over a short period. This is normally the beginning of the snow-melting in spring. From a given date and until the end of the restriction period, discharge from the reservoir is permitted as long as the storage volume in the reservoir does not decrease.
The above regulation can be expressed mathematically by a set of constraints. The activation and/or deactivation criteria defined in the regulation are often formulated as logical conditions dependent on the state variables, i.e., storage volume in the reservoirs and inflow. The state-dependent conditions are illustrated in Fig. 1. The dependencies on both inflow and storage volume in the reservoir are handled in the SDP algorithm, as described in section 2.4.
The different types of constraints imposed by the maximum discharge regulation are described by (16)- (19). Depending on the current state of the system, the constraints are added to the formulation before the decision problem is solved. The first part of the environmental regulation is a regulatory maximum discharge capacity constraint. When the constraint period is initiated, (16) replaces (3) to restrict release from the reservoir only to serve downstream requirements for minimum river flows. The constraint is activated by inflow above a certain threshold (in t A ) or by a given date, When (and if) the storage volume in the reservoir reaches the predefined threshold V lim h , constraint (16) is relaxed and replaced with a minimum reservoir level regulation, (17)e (18). (17) is active for the first week the storage volume in the reservoir reaches the wanted threshold (in t C ), while (18) becomes active from the following week. When (17) or (18) are active, discharge is permitted but the storage volume in the reservoir must be kept above the threshold.
Finally, for a given period, t D -t E , the storage volume in the reservoir is not permitted to decrease. This constraint is shown in (19). The formulation ensures that the storage level in the reservoir at the end of a stage is equal or higher than the storage level at the beginning of the stage. The storage level could also be bounded over each time step k.
The formulation assumes that the reservoir level can be maintained once above the threshold. This can be challenging in some hydropower systems due to minimum flow requirements or negative inflow (e.g., evaporation). While we did not encounter feasibility issues, these can be avoided by including a slack-variable that is penalised in the objective function in (18) and (19).

Stochastic variables
We consider three exogenous stochastic variables in this research; the total weekly inflow into the system, the weekly average energy price and the environmental state, i.e., if the inflow has been above a certain threshold. Inflow normally has a strong weekly correlation, while inflow and price tend to be negatively correlated in hydro-dominated systems. The environmental state variable is an extension of the inflow state. For computational simplicity, we assume that the stochastic variables can be described by discrete nodes using a Markov decision process. The following procedure is used to generate the Markov chain: Inflow and price data is given as input to the model, e.g., historical or forecasted data. An auto-regressive model is fitted to the input data. Serial correlation in inflow and cross-correlation between inflow and price is considered. 10 000 scenarios are sampled from the auto-regressive model. A given number of discrete nodes per week are generated from the scenarios, using K-means clustering. Each node represents one inflow value and one price value. The transition probabilities are determined by counting the share of scenarios transitioning between the different nodes from one week to the next.
In addition, information on the environmental state is required. The environmental state represents a binary variable, indicating whether the inflow has been above a certain threshold. For the weeks when this is applicable, the Markov chain is expanded with an additional environmental state (activated and not activated) in each of the nodes, as illustrated in Fig. 2. The transition probabilities are updated by multiplying with the probability of inflow above (or below) the threshold in each node, presuming that the inflow level has not previously been above the threshold. Once the environmental state is activated, it can only be deactivated by the reservoir storage level reaching above the given threshold or by a given date.

Solution strategy
The hydropower scheduling problem is solved using the SDP algorithm described in Algorithm 1. The algorithm is based on backwards recursion and solves the decision problem for each stage and state of the system for a planning horizon of one year. To account for end-of horizon effects, an iterative approach is used until the water values in the last and first stage converge. The algorithm Fig. 1. Illustration of the state-dependent conditions in the environmental regulation. Within the orange shaded period, the constraint can be activated by inflow above a defined threshold, illustrated by point t A . Similarly, the no discharge restriction can be deactivated within the green shaded area by storage volume above the reservoir threshold, as illustrated by point t C . procedure described in algorithm 2. The algorithm checks if any of the environmental conditions described in Section 2.2 are met, and solves the associated decision problem with the corresponding constraints included, i.e., (16), (17), (18) or (19). The inflow dependent early activation of the constraint is included as a stochastic state x in the discrete Markov chain. The environmental constraints are only included in a selection of the subproblems, but the non-concave characteristics of the expected future profit function may carry down to earlier stages before fading out. For efficiency, the expected future profit approximation is checked for concavity before each subproblem is solved. If the function is concave, (14) and (15) are relaxed and the problem is solved as an LP. If the problem is solved for a system without the environmental regulation, the decision problem is an LP described by (1)- (13). The solution of the optimisation problem for all stochastic states s u t is used to calculate the expected future profit, in line 16. The expected future profit is stored to matrix F. The water values are calculated and stored to the water value matrix J in line 18, following a similar approach as in Ref. [18]. When an iteration is completed, convergence is determined in line 23, by comparing the calculated water values in the last and first stage. If the algorithm has not converged in iteration j, the water value matrix and the expected future profit matrix for the last stage T is updated with the values from the first stage in iteration j, in lines 25 and 26, before the next iteration.

Case Description
This case study assesses the impact of the state-dependent maximum discharge regulation on water values, simulated reservoir operation and profit from the simulated operation of the system. First, water values are calculated in the SDP model, before optimal operation of the system is simulated for a selection of scenarios.
The described model is applied to the hydropower system shown in Fig. 3. The hydropower system is based on Bergsdalsvassdraget in Western Norway. The system comprise several hydropower reservoirs and power plants, of which the two upper reservoirs and power plants are modelled here. The modelled part of the system has a total generation capacity of approximately 55 MW and a reservoir storage capacity of up to 360 Mm 3 .
The lower of the two modelled reservoirs has a state-dependent environmental maximum discharge constraint. The constraint is of the type described in Section 2.2. The discharge limitation can be activated by inflow above the weekly average from week 15 and, at the latest, in week 19. The discharge limitation is active until week 32, or until the storage volume in the reservoir is above 146 Mm 3 . If the storage volume in the reservoir reaches 146 Mm 3 , discharge is permitted as long as the water level stays above the threshold. From week 33 to week 35, the reservoir storage is not allowed to be reduced. The constraint is deactivated in week 35. In addition, a minimum discharge of 3 m 3 /s is imposed on the lower reservoir.
Two cases are considered; without the environmental regulation, wo/Env, and with the environmental regulation, w/Env. In the w/Env case, the decision problem is solved with the environmental regulation, as given in Section 2.4. The SDP model is solved for one year of weekly decision stages, comprising three price periods of 56 h each. A high intra-week price variation is assumed. Each of the reservoirs are discretised into 20 equidistant points, giving 400 combinations of reservoir states. Since the focus of this research is on the modelling of the environmental constraint, a relatively coarse representation of uncertainty and time discretization is included. A discrete Markov chain with 10 nodes per stage is used, each comprising a unique price and inflow value. In the weeks when early activation of the environmental constraint could occur, the environmental state variable is added, leading to a total of 20 nodes per stage.
The discrete Markov chain is generated using inflow data from 58 historical years and power prices generated based on the same inflow data. Alternatively, forecasted weather data from climate models can be used, see e.g., Ref. [38]. The power prices were provided from the long-term hydropower scheduling model EMPS [25], based on a low emission dataset of the European power system for 2030 [39]. The penalty cost of spilling water is set low, C S ¼ 10 À3 V The simulation is conducted as weekly decisions in a sequence, solving the decision problem formulated in Section 2.1 as a shortterm operational problem for each week. The same technical details as in the SDP model are included. The water values calculated in the SDP model are used as input to the weekly decision problem to evaluate the value of the water in the reservoir at the end of each week. For the w/Env case, the logical conditions for activating the environmental constraint, as described in Section 2.2, are checked each time the weekly decision problem is solved. When (and if) the conditions are met, the associated constraints, i.e., (16), (17), (18) or (19), are added to the decision problem.
The model was implemented in Julia v1.5 using the Jump package [40] and the CPLEX 12.10 solver [41]. The relative MIP gap is set to zero and the absolute MIP gap to 10 À10 .The case study was carried out on an Intel Core i7-8650U processor with 16 GB RAM. One iteration of the wo/Env case solved 208k decision problems in approximately 440 s. One iteration of the w/Env case solved 224k decision problems in approximately 1860 s, whereof approximately 40% of the decision problems were solved as MILP. By reducing the number of problems solved as MILP, the solution time of the w/Env case was reduced with around 30%. The algorithm converged in 16e20 iterations with a convergence criterion of 10 À3 V Mm 3 . Advanced tuning of the applied MIP solver and use of parallel processing could serve to improve the computational efficiency [42].

Water values
The water values are the main result from the SDP model. We are especially interested in how the water values change when including the state-dependent maximum discharge constraint. This is of high importance in real-life hydropower scheduling, as the water values provide essential information for short-term decision making.
The upper reservoir is not directly impacted by the environmental regulation in the w/Env case but can be actively used to help manage the regulation. If it is optimal to adjust the reservoir management of the upper reservoir to reach the threshold in the lower reservoir more rapidly, this would be reflected in the water values of the upper reservoir. However, only minor or no changes were seen in the water values for the upper reservoir, implying that it is not economically profitable to release more water from the upper reservoir to reach the reservoir threshold in the lower reservoir earlier. This result is sensitive to several factors, such as the expected power prices in the weeks when the constraint is active, the strictness of the constraint and the efficiency of the lower power plant compared to the upper power plant. Since small or no changes were seen in the water values of the upper reservoir, the rest of this section only considers the water values of the lower reservoir. The water values presented for the lower reservoir are given for a medium storage level in the upper reservoir (i.e., $ 85Mm 3 ). Fig. 4 shows the calculated water values for the wo/Env case and the w/Env case. The water values change with the storage volume in the reservoir and the week of the year. In the wo/Env case, the water values are non-increasing with increasing storage volume in the reservoir. This will always be the result from linear hydropower scheduling models where the expected future profit is a concave function. For low storage volumes in the reservoir, the water value is high because of the risk of emptying the reservoir. For higher storage volumes in the reservoir, the water value decrease as the risk of spilling water increase. The water value is zero when water has to be spilled because of full reservoirs. In the w/Env case, the same behaviour of high and low water values can be observed for low and high storage volumes, respectively. However, the water values also sometimes increase with increasing storage volumes. For this case, the expected future profit is therefore a nonconcave function.
The increasing water values with increasing reservoir storage volumes in the w/Env case are a direct result of the state-dependent maximum discharge constraint. The largest differences in the water values between the two cases can be seen when the strictest part of the environmental regulation is active (week 18e32) and backwards in time (towards week 1). In the weeks when the constraint is active and the storage volume in the reservoir is lower than the threshold (146 Mm 3 ), the water values are higher in the w/Env case than in the wo/Env case. Since discharge from the reservoir is strictly limited over a longer period, the model cannot take advantage of the potential high prices within this period, unless the threshold is reached. As a result, the water values are higher in the w/Env case compared to the wo/Env case for storage volumes where the maximum discharge capacity restriction can be deactivated early.
The water values are calculated from the last to the first week of the year, hence the impact of the constraint on the water values from later weeks affects the earlier weeks. Week 32 is the last week when the maximum discharge capacity restriction is active. Fig. 5 compares the water values from the two cases for selected weeks. No large differences can be seen after the constraint is deactivated; this is, for example, shown in Fig. 5a. In week 32 (Fig. 5b), the difference between the water value curves for the two cases becomes more distinct. The local peak in the water value curve for the w/Env case in Fig. 5b is a result of discharge being permitted when the storage volume in the reservoir reaches the threshold. Moving forwards in time, this effect is being shifted towards lower storage volumes, as shown in Fig. 5c and d.

Simulation results
Simulations were run for 1000 scenarios, randomly selected from the originally sampled scenarios. The wo/Env case and the w/ Env case were simulated for the same scenarios, using water values from the SDP model for each of the cases accordingly. The final simulations are conducted to test the calculated water values in an operational decision-making setting, as well as to demonstrate the effect of the environmental constraint on operation of the system.
The activation and deactivation of the maximum discharge capacity restriction varies between the scenarios because of the statedependent nature of the conditions. Fig. 6 shows when the maximum discharge constraint is activated and deactivated in the w/Env case for the simulated scenarios. In approximately 85% of the scenarios, the storage volume in the reservoir reaches the threshold before week 33, deactivating the maximum discharge constraint. The discharge limitation is activated by high inflow before week 19 in 50% of the scenarios. Even so, inflow-dependent activation was found to only have muted impact on the strategy and simulated economic results for the case considered. Still, under other price assumptions inflow-dependent activation could be of higher importance.
The results from the simulations show a considerable change in optimal operation when the environmental constraint is included. Fig. 7 compares the change in storage volume in the lower reservoir over a year for the wo/Env and w/Env cases. In general, the storage volume in the reservoir is kept higher in the w/Env case. The largest difference can be seen in the spring and summer weeks, when the environmental regulation is active. In this period, no discharge is permitted if the storage level is below the reservoir threshold. For several weeks in this period, the median reservoir storage volume is raised from below 100 Mm 3 in the wo/Env case to over 150 Mm 3 in the w/Env case, demonstrating the effectiveness of the constraint to achieve the underlying purpose of reaching the threshold.
For most of the simulated scenarios, the storage volume in the lower reservoir reaches the reservoir threshold in week 25e30 in the w/Env case. This means that the optimal operation of the reservoirs lies within the reservoir volume segments where the maximum discharge capacity constraint has the largest impact on the water values, indicating the importance of including the constraint in the calculation of the water values. Furthermore, the reservoir storage volume is also kept higher in the autumn and winter weeks in the w/Env case compared to the wo/Env case. Higher storage volumes throughout the year, and not only when the constraint on discharge is active, can be explained by the    differences in water values. Higher water values for higher storage volumes in the w/Env case give the model an incentive to keep more water in the reservoir when coming into the constraint period.
A selection of average numeric results from the completed simulations are given in Table 1. The total yearly profit from the system is reduced by around 72 k(0.8%) in the w/Env case compared to the wo/Env case. In the period when the constraint is active, week 19e35, the total profit from electricity generation is reduced by approximately 17% or 406 k. By shifting the electricity generation, the model manages to keep the total generation in the w/Env case close to the total generation in the wo/Env case, significantly reducing the total loss in profit. This means that the loss in profit is mainly caused by a lower average realised price of electricity, and not reduced sales of electricity. Still, it should be mentioned that the economic results are sensitive to the power price assumptions used in the simulations. Higher power prices in the constrained period can increase the cost of restricting production in this period, and vice versa.

Conclusion
We present a medium-term hydropower scheduling model comprising an accurate representation of inflow-and volumedependent environmental constraints on maximum discharge. Such constraints cause a pronounced nonconvexity in the scheduling problem. The proposed model can solve nonconvex model formulations for smaller systems, by applying an SDP-algorithm, where binary variables are only required to represent the nonconvex characteristics of the expected future value function. By dynamically checking for nonconvexities in the value function, we find that the number of weekly decision problems solved as MILPs can be reduced significantly. Still, the required discretization of the state space leads the SDP-algorithm to scale poorly for larger systems with more reservoirs.
The model is applied to a case study of a Norwegian hydropower system with multiple reservoirs. Simulations of the system with and without the environmental constraint show a substantial difference in operation of the reservoir to which the constraint is imposed. In the case with the environmental constraint, the storage volume in the reservoir reaches the wanted threshold for most of the scenarios. Still, the total electricity generation over the year is maintained and total loss in profit is limited to approximately 0.8%.
The calculated water values were found to change considerably when the state-dependent maximum discharge constraint was included in the SDP model. For the reservoir to which the constraint was imposed, the water values were found to both increase and decrease with increasing storage volumes in the reservoir, reflecting that the expected future profit function is nonconcave. The distinct changes in the calculated water values when the environmental regulation was considered show the importance of accurate modelling of such regulations. This conclusion is further strengthened by the optimal reservoir operation being within the nonconcave area of the expected future profit function for most of the simulated scenarios, as demonstrated by the simulation results. If the future profit function is used as boundary condition in operational short-term scheduling models, the nonconcave shape may impact the results substantially.
The substantial impact on the calculated water values and reservoir operation observed in this study support further research on nonconvex environmental constraints. To strengthen the findings from this study, future research should investigate the impact of using accurate water values, compared to water values based on simplified problem formulations, in operation of hydropower systems with environmental regulation. Furthermore, a wider selection of cases and hydropower watercourses with different variations of the state-dependent maximum discharge constraint could be analysed. Finally, a broader analysis to evaluate how well the constraint meets the underlaying purpose of the environmental regulation, as well as other unintended consequences like flood risk, could be of interest.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.