Comparison of Two Convergence Criterion in the Optimization Process Using a Recursive Method in a Multi-Reservoir System
Abstract
:1. Introduction
2. Methodology
2.1. Background and Theoretical Framework
- (a)
- Value function.
- (b)
- Constraints.
- b.1
- Storage
- b.2
- Release
- (c)
- State transformation equations.
- (d)
2.2. Convergence Criterion
- Trial 1: The first stopping strategy involves adding the absolute value of the differences between two subsequent values, for all possible states, then taking the average and comparing it to the convergence criterion at the end of two cycles
- 2
- Trial 2: Instead of taking the absolute value, the second strategy was to square the differences
3. Case Study
3.1. Multi-Reservoir System
- (a)
- La Angostura was built between 1969 and 1975. Its main uses are flood control, hydroelectric power generation, agricultural irrigation and domestic water supply, fishing and tourism.
- (b)
- Chicoasén was built between 1977 and 1983. Its main uses are hydroelectric power generation, agricultural irrigation and domestic water supply, fishing, national and international tourism and water sports.
- (c)
- Malpaso was built between 1959 and 1964. Its main uses are electricity generation and flood control.
- (d)
- Peñitas is located in the municipality of Ostuacán, Chiapas. It was completed in 1987. The production of electricity is its primary function.
3.2. Optimization and Simulation Processes
- Six stages were defined to group the 12 months of the year (Table 1)
- A value of 600 hm3 (ΔV) was selected to discretize the state variables, thus, the number of states in which the normal storage capacity of La Angostura was discretized was 22, with 16 for Malpaso.
- The months of each stage and the maximum and minimum releases are shown in Table 1. These last values were determined as follows: the drinking water supply is a minimum release of 200 m3/s for La Angostura, and 300 m3/s for Malpaso. The maximum is determined by taking the maximum monthly turbine discharge, in this case, 2825 hm3 for La Angostura and 3732 hm3 for Malpaso. As example for stage 1 and La Angostura dam, the minimum and maximum releases are as follows: 200 m3/s must be expressed in hm3/month and then in volume discretization units to obtain the minimum release per month, i.e.,
- 4
- 5
- For both trials, the discount factor was fixed at 0.5.
- 6
- The maximum number of iterations is 100.
- 7
- To obtain the solution vectors, i.e., the optimum releases for each stage and each dam, the time was quantified.
- Year 1 (1959) has the dams full, with 10,000 hm3 at La Angostura and 7500 hm3 at Malpaso.
- The simulation process begins in January, ending in December.
- Using the initial storage, the corresponding time, and the file with the optimal release for each time and for each storage level of the reservoirs, the release is obtained.
- The operation of the system is based on the principle of continuity, i.e., the new filling level is calculated by adding the inflows and subtracting the releases and other outflows (mainly evaporation) from the initial storage for each time interval.
- The storage constraints are evaluated and it is determined whether the system is in a no-spill, spill, or deficit condition with the new storage value.
- The process continues until the number of years analysed is reached.
- The initial storage, spillage, deficit volumes and average energy generated were analysed. These variables were the evaluation parameters of the optimal policy.
4. Results
4.1. Evaluating the Convergence Criterion
- (a)
- The optimal policy was obtained when the number of iterations reached its limit (100 in this case); the computation time was 34 s without achieving the convergence criterion (fixed at 0.5 × 10−4).
- (b)
- The differences between each iteration decrease rapidly, but the values oscillate between 10−2 and 10−3 from the fifth iteration onwards, thereafter the convergence process is trapped in a loop that repeats values until the maximum number of iterations is reached (Figure 6).
- (a)
- The optimal policy was obtained in 5 s, meeting the convergence criterion in the sixth iteration. Figure 7 shows the details of the iteration cycle.
- (b)
- The difference between the first two iterations is significant at the beginning, but it decreases rapidly, and each iteration maintains a negative gradient that leads to finding the optimal policy and concluding the iterative process by meeting the convergence criterion.
- (a)
- In both trials the differences are gradually being reduced, towards a solution. However, in trial 1 this trend is seen only until iteration 9; from there, the method falls into a cycle that repeats values systematically, i.e., it reaches a point where it is not possible to improve the solution and the differences between two consecutive iterations do not satisfy the tolerance, without reaching the maximum number of iterations.
- (b)
- In trial 2, taking the squared differences amplifies the larger differences between the values of one iteration and the next. However, as these are always decreasing, they are better than those of trial 1, as they are increasingly smaller and thus reach one of the ways of stopping the algorithm, complying with the established tolerance value.
- (c)
- The minimum values that trial 1 manages remain in the range of thousandths, while the tolerance was set at a value in the range of ten thousandths. Therefore, here it is worth noting that if the tolerance is relaxed a little, trial 1 would manage to reach this, although trial 2 would be better, as the computational time is less.
4.2. Evaluating the System Operation with the Optimal Policy
- (a)
- The average release per fortnight was 390 hm3 in La Angostura, and 709 hm3 in Malpaso. In both dams, there were events when it was necessary to use the maximum capacity of the turbines, 11 times in La Angostura and only once in Malpaso.
- (b)
- In the 62 years simulated, the minimum release at La Angostura dam was 150 hm3, recorded eight times and always occurring in the second half of July. At Malpaso, the minimum release was 300 hm3, occurring 13 times in total: eight times in the first half of November, once in the second half of July, once in the first half of June, once in the second half of June and twice in the second half of May.
- (c)
- There are no periods in which the minimum required volume was not satisfied.
- (d)
- In only one year, 2010, there was a spill event, occurring at the Malpaso dam. The spill occurred twice: in the last half of September and the first half of October.
- (e)
- Both dams maintain average storage levels of a little over half their useful capacity.
- (a)
- For the 62 simulated years, the volumes released in La Angostura were 580,310 hm3 in trial 1, and 580,309 hm3 in trial 2, while in Malpaso these volumes were 1,043,924 hm3 and 1,043,917 hm3, respectively.
- (b)
- The only spill event occurred in September/October 2010 at the Malpaso dam, showing only a 2 hm3 difference, 155 hm3 in trial 1, 157 hm3 in trial 2.
- (c)
- The average storage volumes for Malpaso dam showed a slight difference: 4991 hm3 in trial 1, and 4997 hm3 in trial 2.
- (a)
- The average energy and deficit volume variables remain unchanged in the trials.
- (b)
- The minimum storage value reached in each dam is practically the same, with approximately 68 hm3 more in Malpaso between trials 2 and 1; however, this difference is minimal in showing a notable change in the performance of the other variables.
- (c)
- In general, the results of the simulation of the operation of the system with the two policies are similar.
4.3. Discussion of Results
5. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Bellman, R.; Kalaba, R. Dynamic programming and statistical communication theory. Proc. Natl. Acad. Sci. USA 1957, 43, 749–751. [Google Scholar] [CrossRef] [PubMed]
- Wu, X.; Cheng, C.; Lund, J.R.; Niu, W.; Miao, S. Stochastic dynamic programming for hydropower reservoir operations with multiple local optima. J. Hydrol. 2018, 564, 712–722. [Google Scholar] [CrossRef]
- Mousavi, S.J.; Ponnambalam, K.; Karray, F. Reservoir Operation Using a Dynamic Programming Fuzzy Rule–Based Approach. Water Resour. Manag. 2005, 19, 655–672. [Google Scholar] [CrossRef]
- Chang, F.-J.; Hui, S.-C.; Chen, Y.-C. Reservoir operation using grey fuzzy stochastic dynamic programming. Hydrol. Process 2002, 16, 2395–2408. [Google Scholar] [CrossRef]
- Raman, H.; Chandramouli, V. Optimal operation of multi-reservoir system using dynamic programming and neural network. WIT Trans. Inf. Commun. Technol. 1996, 16, 11. [Google Scholar]
- Kularathna, M.D.U.P. Application of Dynamic Programming for the Analysis of Complex Water Resources Systems: A Case Study on the Mahaweli River Basin Development in Sri Lanka; Wageningen University and Research: Wageningen, The Netherlands, 1992. [Google Scholar]
- Alaouze, C.M. The Optimality of Capacity Sharing in Stochastic Dynamic Programming Problemas of Shared Reservoir Operation 1. JAWRA J. Am. Water Resour. Assoc. 1991, 27, 381–386. [Google Scholar] [CrossRef]
- de los Reyes, A.G. Improved Stochastic Dynamic Programming for Optimal Reservoir Operation Based on the Asymptotic Convergence of Benefit Differences; University of Arizona: Tucson, AZ, USA, 1974. [Google Scholar]
- Brass, C. Optimising Operations of Reservoir Systems with Stochastic Dynamic Programming (SDP) under Consideration of Changing Objectives and Constraints; Ruhr Universitaet Bochum: Bochum, Germany, 2006. [Google Scholar]
- Lim, D.-G.; Kim, J.-H.; Kim, S.-K. A Stochastic Dynamic Programming Model to Derive Monthly Operating Policy of a Multi-Reservoir System. Korean Manag. Sci. Rev. 2012, 29, 1–14. [Google Scholar] [CrossRef]
- Macián Sorribes, H. Design of Optimal Reservoir Operating Rules in Large Water Resources Systems Combining Stochastic Programming, Fuzzy Logic and Expert Criteria; Universitat Politècnica de València: Valencia, Spain, 2017. [Google Scholar]
- Rougé, C.; Tilmant, A. Using stochastic dual dynamic programming in problems with multiple near-optimal solutions. Water Resour. Res. 2016, 52, 4151–4163. [Google Scholar] [CrossRef]
- Al-MohseenA, K.; Tawfiiq, A. Stochastic Dynamic Programming Model for Single Reservoir. AL-Rafidain Eng. J. (AREJ) 2014, 22, 1–14. [Google Scholar]
- Gjerden, K.S.; Helseth, A.; Mo, B.; Warland, G. Hydrothermal scheduling in Norway using stochastic dual dynamic programming; a large-scale case study. In Proceedings of the 2015 IEEE Eindhoven PowerTech, Eindhoven, The Netherlands, 29 June–2 July 2015; pp. 1–6. [Google Scholar]
- Leclère, V.; Carpentier, P.; Chancelier, J.-P.; Lenoir, A.; Pacaud, F. Exact Converging Bounds for Stochastic Dual Dynamic Programming via Fenchel Duality. SIAM J. Optim. 2020, 30, 1223–1250. [Google Scholar] [CrossRef]
- Khan, K.; Goodridge, W. Stochastic Dynamic Programming in DASH. Int. J. Adv. Netw. Appl. 2019, 11, 4263–4269. [Google Scholar] [CrossRef]
- Brandi, R.B.S.; Marcato, A.L.M.; Dias, B.H.; Ramos, T.P.; Junior, I.C.d.S. A Convergence Criterion for Stochastic Dual Dynamic Programming: Application to the Long-Term Operation Planning Problem. IEEE Trans. Power Syst. 2018, 33, 3678–3690. [Google Scholar] [CrossRef]
- Marescot, L.; Chapron, G.; Chadès, I.; Fackler, P.L.; Duchamp, C.; Marboutin, E.; Gimenez, O. Complex decisions made simple: A primer on stochastic dynamic programming. Methods Ecol. Evol. 2013, 4, 872–884. [Google Scholar] [CrossRef]
- Stedinger, J.R.; Sule, B.F.; Loucks, D.P. Stochastic dynamic programming models for reservoir operation optimization. Water Resour. Res. 1984, 20, 1499–1505. [Google Scholar] [CrossRef]
- Langen, H.-J. Convergence of Dynamic Programming Models. Math. Oper. Res. 1981, 6, 493–512. [Google Scholar] [CrossRef]
- Homem-de-Mello, T.; de Matos, V.L.; Finardi, E.C. Sampling strategies and stopping criteria for stochastic dual dynamic programming: A case study in long-term hydrothermal scheduling. Energy Syst. 2011, 2, 1–31. [Google Scholar] [CrossRef]
- Jaakkola, T.; Jordan, M.; Singh, S. Convergence of stochastic iterative dynamic programming algorithms. Adv. Neural Inf. Processing Syst. 1993, 6, 703–710. [Google Scholar]
- Hooshyar, M.; Mousavi, S.J.; Mahootchi, M.; Ponnambalam, K. Aggregation–Decomposition-Based Multi-Agent Reinforcement Learning for Multi-Reservoir Operations Optimization. Water 2020, 12, 2688. [Google Scholar] [CrossRef]
- Mendoza, R.; Arganis, M.; Domínguez, R. Políticas de operación con variación temporal en los coeficientes de castigo de las curvas guía en un sistema multiembalse. In Proceedings of the XXV Congreso Nacional de Hidráulica, Mexico City, México, 5–9 November 2018. [Google Scholar]
- Mendoza, R.; Domínguez, R.; Arganis, M. Influencia de curvas guía en las políticas de operación para el manejo de un sistema hidroeléctrico. In Proceedings of the XXV Congreso Latinoamericano de Hidráulica, San José, Costa Rica, 9–12 September 2012. [Google Scholar]
- Bellman, R. Dynamic Programming; Princeton University Press: Princeton, NJ, USA, 1957. [Google Scholar]
- Mamani, A.A.L. Un Método Recursivo Para el Problema de la Programación Lineal Dinámica; Universidad Nacional San Agustin de Arequipa: Arequipa, Perú, 2018. [Google Scholar]
- Mendoza, R.; Domínguez, R.; Arganis, M. Políticas de operación del sistema hidroeléctrico del río Grijalva considerando el efecto de la correlación en los volúmenes de ingreso. In Proceedings of the XXIII Congreso Nacional de Hidráulica, Puerto Vallarta, Jalisco, México, 15–17 October 2014. [Google Scholar]
- Alayo, H. Introducción a la Programación Dinámica Estocástica. Available online: https://hansroom17.files.wordpress.com/2016/12/dp.pdf (accessed on 1 June 2022).
- Domínguez-Mora, R. Metodología de Selección de Una Política de Operación Conjunta de Una Presa y Su Vertedor; Universidad Nacional Autónoma de México: Mexico City, Mexico, 1989. [Google Scholar]
- Larson, R.E.; Casti, J.L. Principles of Dynamic Programming: Advanced Theory and Applications; Marcel Dekker, Inc.: New York, NY, USA, 1982. [Google Scholar]
- Nandalal, K.; Bogardi, J.J. Dynamic Programming Based Operation of Reservoirs: Applicability and Limits; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
- Boutilier, C.; Dearden, R.; Goldszmidt, M. Stochastic dynamic programming with factored representations. Artif. Intell. 2000, 121, 49–107. [Google Scholar] [CrossRef] [Green Version]
- OMM/GWP. Gestión Integrada de Crecientes. Caso de Estudio México: Río Grijalva. Programa Asociado de Gestión de Crecientes. Unidad de Apoyo Técnico. Available online: http://www.floodmanagement.info/publications/casestudies/cs_mexico_full.pdf (accessed on 28 June 2022).
- Juan, E. Elaboración de Mapas Para la Cuenca del Rio Grijalva. Informe Interno. Internal Report. Restricted Circulation.; Instituto de Ingeniería, UNAM: Mexico City, Mexico, 2022. [Google Scholar]
- INECC. La Cuenca de Los Ríos Grijalva y Usumacinta. Available online: http://www2.inecc.gob.mx/publicaciones2/libros/402/cuencas.html (accessed on 28 June 2022).
Stage | Months | Releases R (ΔV) | |||
---|---|---|---|---|---|
Minimum | Maximum | ||||
La Angostura | Malpaso | La Angostura | Malpaso | ||
1 | Nov-Dec | 2 | 3 | 10 | 13 |
2 | Oct | 1 | 2 | 5 | 7 |
3 | Sept | 1 | 2 | 5 | 7 |
4 | Aug | 1 | 2 | 5 | 7 |
5 | June-July | 2 | 3 | 10 | 13 |
6 | Jan-Feb-Mar-Apr-May | 5 | 7 | 24 | 32 |
Variable | Reservoir | |||
---|---|---|---|---|
La Angostura | Malpaso | |||
Trial 1 | Trial 2 | Trial 1 | Trial 2 | |
Minimum initial storage (hm3) | 3717.4 | 3718.3 | 2394.4 | 2326.9 |
Spillage volume (hm3) | 0.0 | 0.0 | 154.7 | 156.6 |
Deficit volume (hm3) | 0.0 | 0.0 | 0.0 | 0.0 |
Average energy (GWh/fortnight) | 279.3 | 279.3 | 201.5 | 201.5 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Mendoza-Ramírez, R.; Silva, R.; Domínguez-Mora, R.; Juan-Diego, E.; Carrizosa-Elizondo, E. Comparison of Two Convergence Criterion in the Optimization Process Using a Recursive Method in a Multi-Reservoir System. Water 2022, 14, 2952. https://doi.org/10.3390/w14192952
Mendoza-Ramírez R, Silva R, Domínguez-Mora R, Juan-Diego E, Carrizosa-Elizondo E. Comparison of Two Convergence Criterion in the Optimization Process Using a Recursive Method in a Multi-Reservoir System. Water. 2022; 14(19):2952. https://doi.org/10.3390/w14192952
Chicago/Turabian StyleMendoza-Ramírez, Rosalva, Rodolfo Silva, Ramón Domínguez-Mora, Eduardo Juan-Diego, and Eliseo Carrizosa-Elizondo. 2022. "Comparison of Two Convergence Criterion in the Optimization Process Using a Recursive Method in a Multi-Reservoir System" Water 14, no. 19: 2952. https://doi.org/10.3390/w14192952
APA StyleMendoza-Ramírez, R., Silva, R., Domínguez-Mora, R., Juan-Diego, E., & Carrizosa-Elizondo, E. (2022). Comparison of Two Convergence Criterion in the Optimization Process Using a Recursive Method in a Multi-Reservoir System. Water, 14(19), 2952. https://doi.org/10.3390/w14192952