دوره 22، شماره 2 - ( مجله مهندسی برق و الکترونیک ایران - جلد 22 شماره 2 1404 )
جلد 22 شماره 2 صفحات 126-111 |
برگشت به فهرست نسخه ها
Download citation:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
BibTeX | RIS | EndNote | Medlars | ProCite | Reference Manager | RefWorks
Send citation to:
Ghavami Z, Mollaie Emamzadeh M, Maghfoori Farsangi M. An intelligent Gradient Method using Particle Swarm Optimization. Journal of Iranian Association of Electrical and Electronics Engineers 2025; 22 (2) :111-126
URL: http://jiaeee.com/article-1-1617-fa.html
URL: http://jiaeee.com/article-1-1617-fa.html
قوامی زهرا، ملایی امام زاده محمد، مغفوری فرسنگی ملیحه. هوشمندسازی الگوریتم گرادیان با الهام از روش بهینه سازی ازدحام ذرات. نشریه مهندسی برق و الکترونیک ایران. 1404; 22 (2) :111-126
دانشکده فنی و مهندسی- دانشگاه شهید باهنر کرمان
چکیده: (867 مشاهده)
الگوریتم گرادیان سادهترین و پرکاربردترین روش در بهینهسازی و یادگیری ماشین میباشد. سرعت همگرایی این روش بهشدت به انتخاب مقدار مناسب برای طولگام بستگی دارد. انتخاب طولگام بسیارکوچک میتواند باعث سرعت همگرایی کند شود. ازطرفی انتخاب طولگام بسیاربزرگ نیز ممکن است باعث واگرایی و نوسان حول نقطه بهینه گردد. معمولاً طولگام را در مراحل اولیه بهینهسازی بزرگتر انتخاب کرده و با پیشرفتن گام‌های اجرا و نزدیکی به جواب بهینه، مقدار آن کاهش مییابد که تنظیم بهینه مقدار این پارامتر بهصورت تجربی و با سعیوخطا برای هر مسئلهای باید انجام شود و زمان زیادی را میطلبد. ازطرفی در روشهای بهینهسازی مبتنیبر هوش جمعی، از جمله در الگوریتم بهینهسازی ازدحام ذرات (PSO) ، طولگامِ حرکت بهصورت خودکار و در‌‌حین اجرای روش تنظیم میشود. همچنین در این روشها، پارامترهای اندکی نیاز به تنظیم دارند و محدوده ازپیشتعیین شدهای برای این منظور موجود است. در این مقاله با ترکیب دو روش PSO و گرادیان، یک روش بهینهسازی هوشمند مبتنی بر گرادیان ارائه شده است که نیازی به تنظیم طولگام ندارد. عملکرد الگوریتم گرادیان پیشنهادی برروی ده تابع محک مورد بررسی قرارگرفته و مشاهده میشود که الگوریتم پیشنهادی سرعت همگرایی بهتری نسبت به الگوریتم گرادیان کلاسیک در رسیدن به جواب بهینه دارد.
واژه‌های کلیدی: الگوریتم گرادیان نزولی، الگوریتم بهینه سازی ازدحام ذرات (PSO)، طولگام، جهت حرکت بهینه، هوشمندسازی
فهرست منابع
1. [1] O. A. Al-Shahri et al., "Solar photovoltaic energy optimization methods, challenges and issues: A comprehensive review", Journal of Cleaner Production, vol. 284, p. 125465, 2021. [DOI:10.1016/j.jclepro.2020.125465]
2. [2] Nezamabadi-pour, Hossein. Genetic Algorithm: Basic and Advanced Concepts. Kerman: Shahid Bahonar University of Kerman, 2010.
3. [3] R. Bansal, "Optimization methods for electric power systems: An overview", International Journal of Emerging Electric Power Systems, vol. 2, no. 1, 2005. [DOI:10.2202/1553-779X.1021]
4. [4] A. Mustapha, L. Mohamed, and K. Ali, "An overview of gradient descent algorithm optimization in machine learning: Application in the ophthalmology field", in Smart Applications and Data Analysis: Third International Conference, Morocco, 2020, pp. 349-359. [DOI:10.1007/978-3-030-45183-7_27]
5. [5] T.T., Trung, H. T., Nguyen. "Backtracking gradient descent method and some applications in large scale optimisation. Part 2: Algorithms and experiments", Applied Mathematics & Optimization, Vol. 84, no. 3, pp. 2557-2586, 2021. [DOI:10.1007/s00245-020-09718-8]
6. [6] Y., Lei, K., Tang. "Learning rates for stochastic gradient descent with nonconvex objectives", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, no. 12, pp. 4505-4511, 2021. [DOI:10.1109/TPAMI.2021.3068154]
7. [7] Ghiassirad H, Aliyari Shoorehdeli M, Farivar F. "To Analysis and Compare 21 Weight Constraints in Stochastic Gradient Descent Algorithm Using Kernel Method", Journal of Iranian Association of Electrical and Electronics Engineers 2022; 19 (3) :145-152. URL: http://jiaeee.com/article-1-1276-fa.html [DOI:10.52547/jiaeee.19.3.145]
8. [8] S. Ruder, "An overview of gradient descent optimization algorithms", arXiv preprint arXiv:1609.04747, 2016.
9. [9] M. D. Zeiler, "Adadelta: an adaptive learning rate method", arXiv preprint arXiv:1212.5701, 2012.
10. [10] Z. Fu, S.-C. Chu, J. Watada, C.-C. Hu, and J.-S. Pan, "Software and hardware co-design and implementation of intelligent optimization algorithms", Applied Soft Computing, vol. 129, p. 109639, 2022. [DOI:10.1016/j.asoc.2022.109639]
11. [11] C. Blum, "Ant colony optimization: Introduction and recent trends", Physics of Life reviews, vol. 2, no. 4, pp. 353-373, 2005. [DOI:10.1016/j.plrev.2005.10.001]
12. [12] E. Kaya, B. Gorkemli, B. Akay, and D. Karaboga, "A review on the studies employing artificial bee colony algorithm to solve combinatorial optimization problems", Engineering Applications of Artificial Intelligence, vol. 115, p. 105311, 2022. [DOI:10.1016/j.engappai.2022.105311]
13. [13] M. G. Sahab, V. V. Toropov, and A. H. Gandomi, "A review on traditional and modern structural optimization: problems and techniques", Metaheuristic applications in structures and infrastructures, pp. 25-47, 2013. [DOI:10.1016/B978-0-12-398364-0.00002-4]
14. [14] A. G. Hussien et al., "Crow search algorithm: theory, recent advances, and applications", IEEE Access, vol. 8, pp. 173548-173565, 2020. [DOI:10.1109/ACCESS.2020.3024108]
15. [15] S. Mirjalili, S. M. Mirjalili, and A. Lewis, "Grey wolf optimizer", Advances in engineering software, vol. 69, pp. 46-61, 2014. [DOI:10.1016/j.advengsoft.2013.12.007]
16. [16] E. Rashedi, H. Nezamabadi-Pour, and S. Saryazdi, "GSA: a gravitational search algorithm", Information sciences, vol. 179, no. 13, pp. 2232-2248, 2009. [DOI:10.1016/j.ins.2009.03.004]
17. [17] H. Zhang, J. Li, M. Hong, and Y. Man, "Artificial intelligence algorithm-based multi-objective optimization model of flexible flow shop smart scheduling", Applications of Artificial Intelligence in Process Systems Engineering, pp. 447-472, 2021. [DOI:10.1016/B978-0-12-821092-5.00008-5]
18. [18] B. A. S. Emambocus, M. B. Jasser, and A. Amphawan, "A survey on the optimization of artificial neural networks using swarm intelligence algorithms", IEEE Access, vol. 11, pp. 1280-1294, 2023. [DOI:10.1109/ACCESS.2022.3233596]
19. [19] M. Jain, V. Saihjpal, N. Singh, and S. B. Singh, "An Overview of Variants and Advancements of PSO Algorithm", Applied Sciences, vol. 12, no. 17, p. 8392, 2022. [DOI:10.3390/app12178392]
20. [20] J. Kennedy and R. Eberhart, "Particle swarm optimization", in Proceedings of ICNN'95-international conference on neural networks, 1995, vol. 4: IEEE, pp. 1942-1948. [DOI:10.1109/ICNN.1995.488968]
21. [21] L. Zhen, Y. Liu, W. Dongsheng, and Z. Wei, "Parameter estimation of software reliability model and prediction based on hybrid wolf pack algorithm and particle swarm optimization", IEEE Access, vol. 8, pp. 29354-29369, 2020. [DOI:10.1109/ACCESS.2020.2972826]
22. [22] R. Padmanabhan and P. Seiler, "Analysis of Gradient Descent with Varying Step Sizes using Integral Quadratic Constraints", arXiv preprint arXiv:2210.00644, 2022.
23. [23] M. Bazaraa, H. Sherali, and C. Shetty, Nonlinear theory and algorithm, 3rd ed. USA: Wiley-Interscience, 2006.
24. [24] S. Biswas, S. Nath, S. Dey, and U. Majumdar, "Tangent-cut optimizer on gradient descent: an approach towards Hybrid Heuristics", Artificial Intelligence Review, pp. 1-27, 2022.
25. [25] E. Taş and M. MEMMEDLİ, "Near optimal step size and momentum in gradient descent for quadratic functions", Turkish Journal of Mathematics, vol. 41, no. 1, pp. 110-121, 2017. [DOI:10.3906/mat-1411-51]
26. [26] S. Guan and B. Biswal, "Spline adaptive filtering algorithm based on different iterative gradients: Performance analysis and comparison", Journal of Automation and Intelligence, vol. 2, no. 1, pp. 1-13, 2023. [DOI:10.1016/j.jai.2022.100008]
27. [27] Z. Gu, K. Jin, Y. Meng, L. Xue, and L.-H. Zhang, "On Kahan's automatic step-size control and an anadromic gradient descent iteration", Technical report, 2022, URL https://www. researchgate. net/publication/362126629_ On_Kahan's_automatic_ step-size_control_and_an_ anadromic_gradient_descent_iteration, 2022.
28. [28] A. Soodabeh and V. Manfred, "A learning rate method for full-batch gradient descent", Műszaki Tudományos Közlemények, vol. 13, no. 1, pp. 174-177, 2020. [DOI:10.33894/mtk-2020.13.33]
29. [29] Y. Qiao, B. van Lew, B. P. Lelieveldt, and M. Staring, "Fast automatic step size estimation for gradient descent optimization of image registration", IEEE transactions on medical imaging, vol. 35, no. 2, pp. 391-403, 2015. [DOI:10.1109/TMI.2015.2476354]
30. [30] Y. Xue, Y. Wang, and J. Liang, "A self-adaptive gradient descent search algorithm for fully-connected neural networks", Neurocomputing, vol. 478, pp. 70-80, 2022. [DOI:10.1016/j.neucom.2022.01.001]
31. [31] R. Padmanabhan and P. Seiler, "Analysis of Gradient Descent with Varying Step Sizes using Integral Quadratic Constraints", arXiv preprint arXiv : 2210.00644, 2022.
32. [32] M. Ravaut and S. Gorti, "Gradient descent revisited via an adaptive online learning rate", arXiv preprint arXiv:1801.09136, 2018.
33. [33] K. Zeng, J. Liu, Z. Jiang, and D. Xu, "A decreasing scaling transition scheme from Adam to SGD", Advanced Theory and Simulations, vol. 5, no. 7, p. 2100599, 2022. [DOI:10.1002/adts.202100599]
34. [34] X. Wei and H. Huang, "A survey on several new popular swarm intelligence optimization algorithms", Research Square, 2023, doi: 10.21203/rs.3.rs-2450545/v1. [DOI:10.21203/rs.3.rs-2450545/v1]
35. [35] F. Parandeh Motlagh, V. Khatibi Bardsiri, and A. Khatibi Bardsiri, "Human-Whale cooperation optimization (HWO) algorithm: A metaheuristic algorithm for solve optimization problems", International Journal of Nonlinear Analysis and Applications, vol. 14, no. 1, pp. 2279-2300, 2023.
36. [36] P. K. Gupta, B. Yadav, A. Kumar, and S. K. Himanshu, "Machine learning and artificial intelligence application in constructed wetlands for industrial effluent treatment: advances and challenges in assessment and bioremediation modeling", Bioremediation for Environmental Sustainability, pp. 403-414, 2021. [DOI:10.1016/B978-0-12-820524-2.00016-X]
37. [37] N. H. Phong, A. Santos, and B. Ribeiro, "PSO-convolutional neural networks with heterogeneous learning rate", IEEE Access, vol. 10, pp. 89970-89988, 2022. [DOI:10.1109/ACCESS.2022.3201142]
38. [38] D. A. Ejigu and X. Liu, "Gradient descent-particle swarm optimization based deep neural network predictive control of pressurized water reactor power", Progress in Nuclear Energy, vol. 145, p. 104108, 2022. [DOI:10.1016/j.pnucene.2021.104108]
39. [39] S. Liu and X. Huang, "Multi‐Objective Optimization for the Cross Brace of a Computer Numerical Control Gantry Machine Tool Based on Intelligent Algorithms", Advanced Intelligent Systems, vol. 5, no. 3, p. 2200368, 2023. [DOI:10.1002/aisy.202200368]
40. [40] A. Eleyan, M. S. Salman, and B. Al-Sheikh, "Application of optimization algorithms for classification problem", International Journal of Electrical and Computer Engineering, vol. 12, no. 4, p. 4373, 2022. [DOI:10.11591/ijece.v12i4.pp4373-4379]
41. [41] A. Mehboodi, H. Nezamabadi-pour, M. Soleimanpour-moghadam, "Multi-robot Path Planning in a 3D Environment by Modified Particle Swarm Optimization Algorithm", Journal of Iranian Association of Electrical and Electronics Engineers 2022; 19 (3) :163-174. URL: http://jiaeee.com/article-1-1073-en.html [DOI:10.52547/jiaeee.19.3.163]
42. [42] M. Vakilifard, A. Sahafi, A. M. Rahmani, P. Sheikholharam Mashhadi, "FRA-PSO: A two-stage Resource Allocation Algorithm in Cloud Computing", Journal of Iranian Association of Electrical and Electronics Engineers 2023; 20 (1) :43-49. URL: http://jiaeee.com/article-1-1237-fa.html [DOI:10.52547/jiaeee.20.1.43]
43. [43] M. P. Deisenroth, A. A. Faisal, and C. S. Ong, Mathematics for machine learning. Cambridge University Press, 2020. [DOI:10.1017/9781108679930]
44. [44] F. A. Hashim, E. H. Houssein, K. Hussain, M. S. Mabrouk, and W. Al-Atabany, "Honey Badger Algorithm: New metaheuristic algorithm for solving optimization problems", Mathematics and Computers in Simulation, vol. 192, pp. 84-110, 2022. [DOI:10.1016/j.matcom.2021.08.013]
45. [45] M. J. Goldanloo and F. S. Gharehchopogh, "A hybrid OBL-based firefly algorithm with symbiotic organisms search algorithm for solving continuous optimization problems", The Journal of Supercomputing, vol. 78, no. 3, pp. 3998-4031, 2022. [DOI:10.1007/s11227-021-04015-9]
46. [46] M. Dehghani, Š. Hubálovský, and P. Trojovský, "Tasmanian devil optimization: a new bio-inspired optimization algorithm for solving optimization algorithm", IEEE Access, vol. 10, pp. 19599-19620, 2022. [DOI:10.1109/ACCESS.2022.3151641]
ارسال پیام به نویسنده مسئول
| بازنشر اطلاعات | |
 |
این مقاله تحت شرایط Creative Commons Attribution-NonCommercial 4.0 International License (CC BY NC 4.0) قابل بازنشر است. |
