CC BY
0
MSC 68T07, 68W25
DOI: 10.14529/ mmp240402
KOLMOGOROV-ARNOLD NEURAL NETWORKS TECHNIQUE FOR THE STATE OF CHARGE ESTIMATION FOR LI-ION BATTERIES
M.H. Dao1, F. Liu2, D.N. Sidorov1'3'4
1Irkutsk National Research Technical University, Irkutsk, Russian Federation 2Central South University, Changsha, China
3Melentiev Energy Systems Institute of Siberian Branch of the Russian Academy of Sciences, Irkutsk, Russian Federation 4Harbin Institute of Technology, Harbin, China
E-mail: [email protected], [email protected], [email protected]
Kolmogorov-Arnold Network (KAN) is an advanced type of neural network developed based on the Kolmogorov-Arnold representation theorem, offering a new approach in the field of machine learning. Unlike traditional neural networks that use linear weights, KAN applies univariate functions parameterized by splines, allowing it to flexibly capture and learn complex activation patterns more effectively. This flexibility not only enhances the model's predictive capability but also helps it handle complex issues more effectively. In this study, we propose KAN as a potential method to accurately estimate the state of charge (SoC) in energy storage devices. Experimental results show that KAN has a lower maximum error compared to traditional neural networks such as LSTM and FNN, demonstrating that KAN can predict more accurately in complex situations. Maintaining a low maximum error not only reflects KAN's stability but also shows its potential in applying deep learning technology to estimate SoC more accurately, thereby providing a more robust approach for energy management in energy storage systems.
Keywords: state of charge (SoC); Kolmogorov-Arnold networks; energy storage; neural network.
Introduction
The Kolmogorov-Arnold Network (KAN) has emerged as an advanced method in the field of machine learning, particularly for addressing complex nonlinear problems. The development of KAN not only enhances the capabilities of predictive models but also improves the learning and representation abilities of neural networks, enabling them to efficiently handle nonlinear relationships in complex data. This has led to the application of KAN across various fields, including renewable energy, transportation, and electronics, particularly in predicting the State of Charge (SoC) of energy storage devices.
Accurate prediction of SoC plays a crucial role in optimizing energy usage, effectively managing remaining energy, extending battery life, and improving system reliability [1,2]. Furthermore, the ability to predict SoC helps prevent sudden power outages, ensuring continuous and safe operation.
In large systems such as smart grids, predicting SoC supports the efficient distribution of energy, aiding in cost reduction and minimizing energy waste [3]. In recent years, machine learning methods have become powerful tools for predicting and managing stored energy levels. Neural network models like Feedforward Neural Networks (FNN) and Long Short-Term Memory (LSTM) have been widely adopted due to their ability to learn nonlinear relationships and process sequential data [4,5].
However, these networks have certain limitations, such as reliance on linear weights and the requirement for large amounts of data to achieve high performance. Specifically, while
LSTM can retain long-term information, it still faces challenges regarding computational complexity and result interpretability. Consequently, the application of the Kolmogorov-Arnold Network (KAN) [6] for estimating the State of Charge (SoC) has shown great potential in enhancing predictive models in this domain.
The structure of this paper is organized as follows: the next section will explore the theoretical foundation of this study. This will be followed by a description of the network architecture and a detailed analysis of the Kolmogorov-Arnold Network (KAN) used for estimating the State of Charge (SoC). Section 4 will provide a comprehensive overview of the dataset utilized in this research, while Section 5 will present the results and key insights derived from the findings. Finally, we will conclude by summarizing the main points and suggesting potential directions for future research.
1. Theoretical Foundation
Kolmogorov-Arnold Network (KAN) represents a new neural Network architecture designed to replace traditional Multi-Layer Perceptrons (MLPs). It was inspired by the Kolmogorov-Arnold representation theorem [7, 8] whereas MLPs is inspired by the universal approximation theorem [9].
The Kolmogorov-Arnold representation theorem states that every multivariate continuous function f which depends on x = [x1 , X2,xn], in a bounded domain, can be represented as the finite composition of simpler continuous functions, involving only one variable. Formally, a real, smooth, and continuous multivariate function f (x) : [0,1]n ^ R can be represented by the finite superposition of univariate functions
2n+1
Here Фд : R ^ R amd ф, respectively.
f (x) = f (xl,-,xn) = $q ^q,p(xp ))• (1)
q=1 p=1
[0,1] ^ R are the so-called outer and inner functions,
q,p
2. Network Architecture
Regarding the architecture of KAN, a KAN layer can be defined as $ = I0q,p} , P = 1, 2,...,nin, q =1, 2,...,nout, where the functions 0q,p are parametrized functions with learnable parameters. This structure allows KAN to capture complex nonlinear relationships in the data more effectively than traditional Multi-Layer Perceptrons (MLPs).
To fully harness the power of KAN, deeper network architectures have been created. A KAN with depth L is constructed by stacking L KAN layer. The configuration of this deeper KAN is defined by an integer array [n0,n1,... ,nL] where ni indicates the number of neurons in the lth layers. Each lth KAN layer, takes an input of dimensions and produces an output of nl+1 dimensions, transforming the input vector accordingly xl G Rni to xl+1 G Rni+1 [6]
( 1,1 (■) 1,2 (■) ■ ■ ■ 0l,1,n; (■) \
0l,2,1 (-) 0l,2,2 (-) ■ ■ ■ 0l,2,n; (-)
1
\фг,пг+1,1 (•) Ф1,щ+1,2 (•)
(2)
■,щ+1,щ
n
where $ is the function matrix corresponding to the lth KAN layer and the KAN is essentially formed by combining multiple KAN layers
KAN (x) = ($L-i o $L-2 O ... O $0) X. (3)
The network depth, defined by the number of layers, allows it to capture more complex patterns and relationships within the data. Each KAN layer processes the inpu x through a series of learnable functions <frp,q enabling the network to be highly adaptable and resilient.
The Kolmogorov-Arnold (KAN) representation theorem allows for the creation of new neural network architectures by replacing conventional linear weights with univariate B-spline-based functions, which act as learnable activation functions. These B-splines are represented through the basis functions Nitj (t). The Oth-order basis function Ni,0(t) is defined as follows [10]:
N (t) = J1 if ti < t < ti+i and ti < ti+i, (4)
i,0( ) \ 0 otherwise. ( )
Higher-order basis functions Ni,j (t) are calculated using the recursive formula:
NiAt) = + Ni+itj-i(t), (5)
ti+j - ti ti+j+i - ti+i
where j = 1, 2,... ,p. The B-spline curve is defined by the following equation:
n
C (t) = £ Pi Ni,p(t) (6)
i=0
is call B-spline.
This method provides greater adaptability in creating the architecture of the neural network and improves the KAN model ability to learn and represent data, allowing it to better handle nonlinear relationships within intricate datasets.
3. Kolmogorov-Arnold Networks for SoC Estimation
Let us consider the application of the KAN network to the SoC estimation problem using the Panasonic NCR18650PF Data Normalized Li-ion Battery dataset. The dataset includes five inputs variables: Temperature, Voltage filtered at a frequency of 0,5 mHz, Current filtered at a frequency of 0,5 mHz, Voltage filtered at a frequency of 5 mHz, Current filtered at a frequency of 5 mHz, with and the State of Charge (SoC).
The connection between the inputs and the State of Charge (SoC) can be represented by a multivariable function as follows [11]:
SoC « F (T; Vo,5; Co,5; V5; C5), (7)
where T is temperature, V0,5 is voltage filtered at a frequency of 0,5 mHz, C0,5 is current filtered at a frequency of 0,5 mHz, V5 is voltage filtered at a frequency of 5 mHz, C5 is current filtered at a frequency of 5 mHz.
According to the Kolmogorov-Arnold theorem, this multivariable function can be expressed as a composition of univariate functions that depend independently on individual features:
X = f (0T(T) + (Vo,5) + 0Co,B (Co,5) + 0VB (V5) + 0CB (C5)), (8)
where X is the representation of univariate functions that reflect the influence of each input variable, is the function representing the influence of T, 0Vo,B is the function
representing the influence of V0,5, 0Co,5 is the function representing the influence of C0,5, 0yB is the function representing the influence of V5, 0C5 is the function representing the influence of C5.
More generally, the relationship can be extended across multiple functions:
m
Y = Y, (T) + 0VP,O,B(V0,5) + 0CP,0,5(Co,5) + 0vp,5 (V5) + 0cp,B (C5)), (9)
p=i
where Y is the sum of multiple nonlinear functions that describe the general relationship between the input variables and the State of Charge (SoC) value to be estimated, m is the number of functions fp that aggregate the input factors, fp is nonlinear functions. 0 is the univariate functions depend on each input variable, T, V0,5, C0,5, V5, C5 is the input variables.
Fig. 1. KAN network architecture for State of Charge (SoC)
The process of building a model to estimate SoC using KANs is carried out through several steps, as illustrated in Fig. 2. First, the data is meticulously prepared, including data validation, cleaning (removing NAN values), and normalization. Then, the dataset is divided into two parts: 80% for training and the remaining 20% for testing.
To improve signal quality and reduce noise, we applied additional filters to voltage and current at frequencies of 0,5 mHz and 5 mHz within the dataset. This not only smooths the data but also enhances prediction accuracy.
The model architecture is built upon the Kolmogorov-Arnold Networks (KAN) architecture, combined with carefully selected activation functions to enhance the accuracy in predicting the State of Charge (SoC). As illustrated in Fig. 1, the values of each variable are flattened and stacked into inputs, which are then processed through L layers of the KAN network to produce the estimated SoC result. The training and validation process focuses on optimizing the B-spline functions and network parameters with the goal of minimizing prediction errors and improving model performance. In particular, the B-spline functions are trained according to a detailed process as described in the diagram on the right.
The final stage involves evaluating and fine-tuning the model based on key metrics such as R-squared, RMSE, MAE, and MAX (Maximum Error). The final results are visualized by comparing the predicted values with the actual values, allowing for a comprehensive assessment of the model performance.
Prepare data
Data collection, preprocessing, and splitting the dataset into training and testing sets
I
Formation of feature vectors Identify the necessary features to capture the complex relationships in the data
1
Model Architecture
Design the KAN architecture and use activation functions to improve the accuracy of SoC predictions
1
Training and Validation
Optimize the B-spline functions and network parameters to minimize prediction error and evaluate the model's performance
i :
Evaluation and Fine Tuning
Evaluate the model's performance based on the following metrics: RMSE, MAE, MAXError, and fine-tune the model
1
Visualize the results
Compare the predicted values with the actual values
Fig. 2. Diagram of the training process using KAN networks 4. Dataset Description
This paper studies two types of Li-ion batteries from the Mendeley dataset: the cylindrical Panasonic NCR18650PF [12], commonly used in energy applications, and the pouch Turnigy Graphene 65C [13], known for its high-power output capability. Both types of batteries are used to train and test predictive models such as FNN, LSTM, and KAN, in order to evaluate the effectiveness of these methods in estimating the State of Charge (SoC) and performance-related parameters of the batteries.
The dataset is divided into two parts: training data and testing data, both within the temperature range of — 10°C to 25°C. This allows the model to be trained and tested under various real-world conditions. The training data uses mixed battery configurations,
including samples from Panasonic (Mix 1-4, US06) and Turnigy (Mix 1-8, US06), while the testing data uses other configurations such as Panasonic (LA92, NN) and Turnigy (LA92, HW) [14]. As illustrated in Fig. 3, the battery parameters are presented to assist the analysis and provide an objective dataset view.
0 1000 2000 3000 4000 5000 6000 7000
Fig. 3. Battery parameters chart 5. Results and Discussion
By using the KAN network model [15], we train and predict the State of Charge (SoC) for the Panasonic NCR18650PF Li-ion battery dataset. We then compare the predicted SoC results to those obtained from Feedforward Neural Networks (FNN) and Long Short-Term Memory (LSTM) methods, as shown in Table. For this comparison, we employ evaluation metrics including R-squared, RMSE, MAE, Maximum Error (MAX), Pearson Correlation Coefficient (R), and Spearman Correlation Coefficient (r). The findings indicate that while the KAN model excels in controlling maximum error, its accuracy, as measured by R-squared, RMSE, and MAE, is slightly lower than that of the LSTM model. Nevertheless, the KAN network exhibits strong predictive capability, supported by high Pearson and Spearman correlation coefficients.
Table
Error table of F NN, LSTM and KAN network models
Index FNN LSTM KAN network
R-squared 0,9382887175 0,9466769676 0,9350891130
RMSE 0,0593053601 0,0551275993 0,0608233635
MAE 0,0448270916 0,0400778923 0,0474486669
MAX Error 0,3506530296 0,2745016737 0,2647641933
Pearson Correlation Coefficient (R) 0,9691410083 0,9741125442 0,9676746864
Spearman Correlation Coefficient (r) 0,9739220118 0,9790702756 0,973298463
Conclusion
The experimental results show that the KAN Network outperforms traditional models like Feedforward Neural Networks (FNN) and Long Short-Term Memory (LSTM) in predictive accuracy, particularly in its ability to maintain a lower maximum error. However, to fully leverage the potential of KAN while minimizing the risk of overfitting, it is essential to carefully optimize the model's architecture and activation functions. This factor becomes even more critical when working with limited datasets or more complex modeling scenarios. Moving forward, we plan to further explore KAN in conjunction with other neural network architectures such as LSTM and RNN, as well as integrating KAN with various traditional neural networks. This approach aims to develop a hybrid model that achieves the highest performance in estimating the state of charge (SoC) in energy storage systems.
Acknowledgments. The work was funded by a grant from the Ministry of Science and Higher Education of the Russian Federation (Project No. 075-15-2022-1215) and in part by the National Foreign Experts Program of China under Grant DL2023161002L.
References
1. Chen Zhilong, He Ting, Mao Yingzhe, Zhu Wenlong, Xiong Yifeng et all. State of Charge Estimation Method of Energy Storage Battery Based on Multiple Incremental Features. Journal of The Electrochemical Society, 2024, vol. 171, no. 7, article ID: 070522, 12 p. DOI: 10.1149/1945-7111/ad5efa
2. Hu Xiaosong, Jiang Jiuchun, Cao Dongpu, Egardt B. Battery Health Prognosis for Electric Vehicles Using Sample Entropy and Sparse Bayesian Predictive Modeling. IEEE Transactions on Industrial Electronics, 2016, vol. 63, no. 4, pp. 2645-2656. DOI: 10.1109/TIE.2015.2461523
3. Dreglea A., Foley A., Hager U., Sidorov D., Tomin N. Hybrid Renewable Energy Systems, Load and Generation Forecasting, New Grids Structure, and Smart Technologies. Solving Urban Infrastructure Problems Using Smart City Technologies, 2021, pp. 475-484. DOI: 10.1016/B978-0-12-816816-5.00022-X
4. Bockrath S., Rosskopf A., Koffel S., Waldhor S., Srivastava K., Lorentz V.R.H. State of Charge Estimation Using Recurrent Neural Networks with Long Short-Term Memory for Lithium-Ion Batteries. IECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Society, Lisbon, 2019, pp. 2507-2511. DOI: 10.1109/IEC0N.2019.8926815
5. Tian Jinpeng, Chen Cheng, Shen Weixiang, Sun Fengchun, Xiong Rui. Deep Learning Framework for Lithium-Ion Battery State of Charge Estimation: Recent Advances and Future Perspectives. Energy Storage Materials, 2023, vol. 61, article ID: 102883. DOI: 10.1016/j.ensm.2023.102883
6. Liu Ziming, Wang Yixuan, Vaidya S., Ruehle F., Halverson J., Soljacic M., Hou T.Y., Tegmark M. KAN: Kolmogorov-Arnold Networks. arXiv: Computer Science, 2024, 48 p. Avialableat: https://arxiv.org/abs/2404.19756. DOI: 10.48550/arXiv.2404.19756
7. Kolmogorov A.N. On the Representation of Continuous Functions of Many Variables by Superposition of Continuous Functions of One Variable and Addition. Doklady Akademii Nauk SSSR, 1957, vol. 114, no. 5, pp. 953-956.
8. Arnold V.I. On the Representation of Continuous Functions of Three Variables as Superpositions of Continuous Functions of Two Variables. Doklady Akademii Nauk SSSR, 1957, vol. 114, no. 4, pp. 679-681.
9. Hornik K., Stinchcombe M., White H. Multilayer Feedforward Networks are Universal Approximators. Neural Networks, 1989, vol. 2, no. 5, pp. 359-366. DOI: 10.1016/0893-6080(89)90020-8
10. De Boor C. A Practical Guide to Splines. Mathematics of Computation, 1978, vol. 27. DOI: 10.2307/2006241
11. Dao Minh Hien, Sidorov D.N. Estimation of the State of Charge of Energy Storage Devices Using Kolmogorov-Arnold Networks. Dynamic Systems and Computer Sciences: Theory and Applications (DYSC 2024): Proceedings of the 6th International Conference, Irkutsk, 2024, pp. 206-209. DOI: 10.26516/978-5-9624-2309-8.2024.1-224
12. Kollmeyer P. Panasonic 18650PF Li-ion Battery Data. Mendeley Data, 2018. DOI: 10.17632/wykht8y7tg.1
13. Kollmeyer P., Skells M. Turnigy Graphene 5000mAh 65C Li-ion Battery Data. Mendeley Data, 2020. DOI: 10.17632/4fx8cjprxm.1
14. Vidal C., Kollmeyer P.J. Panasonic 18650PF Li-ion Battery Data and Example FNN and LSTM Neural Network SOC Estimator Training Script. Mendeley Data, 2021. DOI: 10.17632/xf68bwh54v.1
15. GitHub. Available at: https://github.com/KindXiaoming/pykan (accessed on 20.10.2024).
Received September 30, 2024
УДК 519.246.8+681.11.031.1 БЭТ: 10.14529/шшр240402
МЕТОД НЕЙРОННЫХ СЕТЕЙ КОЛМОГОРОВА - АРНОЛЬДА ДЛЯ ОЦЕНКИ СОСТОЯНИЯ ЗАРЯДА ЛИТИЙ-ИОННЫХ БАТАРЕЙ
М.Х. Дао1, Ф. Лю2, Д.Н. Сидоров134
1 Иркутский национальный исследовательский технический университет, г. Иркутск, Российская Федерация
2Центральный южный университет, г. Чанша, Китайская народная республика 3Институт систем энергетики имени Л.А. Мелентьева СО РАН, г. Иркутск, Российская Федерация
4Харбинский политехнический университет, г. Харбин, Китайская народная республика
Основная цель статьи - адаптация и приложение нейронных сетей типа Колмогорова - Арнольда в электроэнернетике. Нейросети Колмогорова - Арнольда являются новым подходом в области машинного обучения, основанном на классических результатах теории приближений. В отличии от традиционных нейронных сетей, сети Колмогорова - Арнольда задействуют унивариантные функции, параметризованные сплайнами, что позволяет гибко улавливать и изучать сложные активационные шаблоны более эффективно. Такая архитектура нейросетей позволяет существенно улучшать их прогнозирующую способность. В данном исследовании предлагается использовать сети Колмогорова - Арнольда в задаче оценивания уровня заряда в литий-ионных накопителях. Экспериментальные результаты на тестовых базах данных показывают, что нейросетевые модели Колмогорова - Арнольда демонстрируют меньшую
максимальную ошибку по сравнению с традиционными нейронными сетями, такими как LSTM и FNN, что показывает высокий потенциал использования нейросетевой модели в сложных ситуациях эксплуатации накопителей энергии. Поддержание низкой максимальной ошибки не только отражает устойчивость нейросетей Колмогорова - Арнольда, но демострирует потенциал в применении технологий глубокого обучения для более точной оценки уровня заряда, предоставляя более надежный подход к управлению системами хранения энергии.
Ключевые слова: состояние заряда (SoC); сети Колмогорова - Арнольда; накопители энергии; нейронная сеть.
Литература
1. Chen Zhilong. State of Charge Estimation Method of Energy Storage Battery Based on Multiple Incremental Features / Chen Zhilong, He Ting, Mao Yingzhe, Zhu Wenlong, Xiong Yifeng et all // Journal of The Electrochemical Society. - 2024. - V. 171, № 7. -Article ID: 070522. - 12 p.
2. Hu Xiaosong. Battery Health Prognosis for Electric Vehicles Using Sample Entropy and Sparse Bayesian Predictive Modeling / Hu Xiaosong, Jiang Jiuchun, Cao Dongpu, B. Egardt // IEEE Transactions on Industrial Electronics. - 2016. - V. 63, № 4. - P. 2645-2656.
3. Dreglea, A. Hybrid Renewable Energy Systems, Load and Generation Forecasting, New Grids Structure, and Smart Technologies / A. Dreglea, A. Foley, U. Hager, D. Sidorov, N. Tomin // Solving Urban Infrastructure Problems Using Smart City Technologies. - 2021. -P. 475-484.
4. Bockrath, S. State of Charge Estimation using Recurrent Neural Networks with Long Short-Term Memory for Lithium-Ion Batteries / S. Bockrath, A. Rosskopf, S. Koffel, S. Waldhor, K. Srivastava, V.R.H. Lorentz // IECON 2019 - 45th Annual Conference of the IEEE Industrial Electronics Society. - Lisbon, 2019. - P. 2507-2511.
5. Tian Jinpeng. Deep Learning Framework for Lithium-ion Battery State of Charge Estimation: Recent Advances and Future Perspectives / Tian Jinpeng, Chen Cheng, Shen Weixiang, Sun Fengchun, Xiong Rui // Energy Storage Materials. - 2023. - V. 61. -Article ID: 102883.
6. Ziming, Liu. KAN: Kolmogorov-Arnold Networks / Liu Ziming, Wang Yixuan, S. Vaidya, F. Ruehle, J. Halverson, M. Soljacic, T.Y. Hou, M. Tegmark // arXiv: Computer Science. -2024. - 48 p. - URL: https://arxiv.org/abs/2404.19756.
7. Колмогоров, А.Н. О представлении непрерывных функций нескольких переменных в виде суперпозиции непрерывных функций одного переменного и сложения / А.Н. Колмогоров // Доклады академии наук СССР. - 1957. - Т. 114, № 5. - С. 953-956.
8. Арнольд, В.И. О представлении непрерывных функций трех переменных суперпозициями непрерывных функций двух переменных / В.И. Арнольд // Доклады академии наук СССР. - 1957. - Т. 114, № 4. - С. 679-681.
9. Hornik, K. Multilayer Feedforward Networks are Universal Approximators / K. Hornik, M. Stinchcombe, H. White // Neural Networks. - 1989. - V. 2, № 5. - P. 359-366.
10. De Boor, C. A Practical Guide to Splines / C. De Boor // Mathematics of Computation. -1978. - V. 27, № 149.
11. Дао Минь Хиен. Оценка состояния заряда накопителей энергии с помощью сетей Колмогорова - Арнольда / Минь Хиен Дао, Д.Н. Сидоров // Динамические системы и компьютерные науки: теория и приложения (DYSC 2024): материалы 6-й Международной конференции. - 2024. - С. 206-209.
12. Kollmeyer, P. Panasonic 18650PF Li-ion Battery Data / P. Kollmeyer // Mendeley Data. -2018.
13. Kollmeyer, P. Turnigy Graphene 5000mAh 65C Li-ion Battery Data / P. Kollmeyer, M. Skells // Mendeley Data. - 2020.
14. Vidal, C. Panasonic 18650PF Li-ion Battery Data and Example FNN and LSTM Neural Network SOC Estimator Training Script / C. Vidal, P.J. Kollmeyer // Mendeley Data. -2021.
15. GitHub. - URL: https://github.com/KindXiaoming/pykan (дата обращения 20.10.2024).
Дао Минь Хиен, aспирант, Байкальский институт БРИКС, Иркутский национальный исследовательский технический университет (г. Иркутск, Российская Федерация), [email protected].
Фан Лю, профессор, школа автоматизации, Центральный южный университет (г. Чанша, Китайская народная республика), [email protected].
Денис Николаевич Сидоров, доктор физико-математических наук, профессор, Байкальский институт БРИКС, Иркутский национальный исследовательский технический университет (г. Иркутск, Российская Федерация); отдел прикладной математики №9G, Институт систем энергетики им. Л.А. Мелентьева СО РАН (г. Иркутск, Российская Федерация); кафедра электротехники и автоматизации, Харбинский политехнический университет (г. Харбин, Китайская народная республика), [email protected].
Поступила в редакцию 30 сентября 2024 г.
