CC BY
0УДК 004.415.53
Bobunov A.Yu.
Moscow University for Industry and Finance «Synergy» (Moscow, Russia)
ANALYSIS OF THE EFFECTIVENESS OF AUTOMATED TESTING FRAMEWORKS FOR BIG DATA IN FINANCIAL SYSTEMS
Аннотация: this study investigates the effectiveness of automated testing frameworks for Big Data in financial systems. The analysis covers various frameworks including Apache JMeter, Selenium, Apache Spark Testing Base, Robot Framework, TestNG, Gatling. The implications of these frameworks on financial operations, such as transaction processing and risk management, are explored. The study discusses future innovations in testing technologies, such as the integration of AI, real-time data processing and cloud-based testing.
Ключевые слова: automated testing, Big Data, financial systems, test frameworks, Artificial Intelligence, real-time processing.
Introduction.
In the modern financial industry, data plays a pivotal role, providing the foundation for decision-making, risk analysis, and asset management. The vast volumes of data, or Big Data, require sophisticated and high-performance technological solutions for their processing and analysis. The use of automated testing (AT) frameworks, which help ensure the reliability and accuracy of data within financial systems (FS), becomes particularly significant.
The aim of this study is to evaluate the effectiveness of various AT frameworks for working with Big Data in FS. The analysis includes both well-established and emerging technological solutions.
Main part.
The software testing market was valued at 45 billion dollars in 2022 and is projected to grow annually by 5% from 2023 to 2032 [1]. This growth is driven by the increasing use of mobile applications. AT frameworks are specialized tools designed to facilitate the process of software testing, enabling the execution of predefined test cases on software applications without manual intervention. Frameworks are crucial in handling the vast, complex datasets typical of Big Data environments, particularly within the financial sector where data integrity and accuracy are paramount. The automation provided by these frameworks allows for repetitive and extensive testing that would be impractical and time-consuming if done manually.
Overview of testing frameworks for Big Data in FS.
AT frameworks are essential tools in managing Big Data within FS. These frameworks facilitate the execution of comprehensive tests, which is important for maintaining data integrity and ensuring the security of financial transactions:
• Apache JMeter has evolved to support a variety of test functions, making it a versatile tool for load testing and measuring performance in Big Data applications. JMeter can simulate heavy loads on servers, networks, or objects to test their strength and analyze overall performance under different conditions. It is particularly useful in FS for stress testing transaction processing capabilities. As of 2023, Apache JMeter is used by over 30,500 companies, including SEO, Infosys Ltd, and Panasonic Corp. Financial services are among the leading industries where companies utilize this framework (figure 1).
Figure 1. Top industries that use Apache Jmeter [2].
In the software testing tools category, Apache JMeter holds approximately 11.8% of the market share.
• Selenium is predominantly used for automating web browsers, but its capabilities are useful in testing web-based financial applications (FA) that handle Big Data [3]. It supports multiple programming languages and integrates with other testing frameworks like TestNG to create a comprehensive testing environment. This makes Selenium ideal for functional and regression testing of complex transactional websites and applications in finance. Financial companies such as Capital One, American Express, JPMorgan Chase, and Barclays use Selenium for their web application testing.
• Apache Spark Testing Base is a library designed to simplify the process of writing scalable tests for Apache Spark applications, a common platform for Big Data processing in financial analytics. This framework helps verify the functionality of Spark programs, ensuring that all components operate correctly and efficiently with large data sets typical in financial contexts. In the Big Data category, Apache Spark holds approximately 2.4% of the market share. Apache Spark is used by companies such as JPMorgan Chase, Citigroup, and Morgan Stanley [4].
• Robot Framework is another versatile open-source automation tool that can be used for acceptance testing and acceptance test-driven development (ATDD). It is easy to use for all stakeholders involved in the development of financial software systems, including project managers, developers, and QA teams. Its keyword-driven testing approach is particularly effective for applications requiring continuous testing
and integration. Robot Framework is based on keywords for acceptance testing, test-driven development, and robotic process automation (RPA). It is written in Python but can also be used with Jython (Java) and IronPython (.NET). This framework is utilized by financial companies such as Wells Fargo, Bank of America, and HSBC.
• TestNG. Designed to cover a wider range of test categories: unit, functional, end-to-end, integration, etc., TestNG is a powerful testing framework that supports complex data-driven testing scenarios. It is highly configurable and allows for concurrent execution of tests, making it highly effective for FS that require rigorous testing across different environments and configurations. In 2022, in the software testing tools category, TestNG held about 0.9% of the market share. Among all its clients, the proportion of financial companies accounted for 7% [5]. TestNG is used for AT frameworks for Big Data in FS of companies such as Barclays, Deutsche Bank, and Morgan Stanley.
• Gatling is focused on performance testing, particularly useful for understanding how a system behaves under significant load, a typical scenario in the financial sector during peak trading hours or financial events. It uses a Scala-based scripting language and provides detailed performance metrics, which are crucial for optimizing Big Data operations in finance. Gatling is used by about 200,000 companies worldwide, including well-known financial companies such as Goldman Sachs, Citibank, and HSBC [6].
Each of these frameworks has unique features that can be strategically applied to meet specific requirements of Big Data processing in FS. The choice of a testing framework depends on the specific aspects of the application being tested, including the nature of the data, the complexity of the transactions, and the specific performance and integrity metrics that need to be validated. These tools collectively ensure that financial institutions can trust the accuracy and reliability of their Big Data systems, thereby safeguarding against errors that could lead to significant financial losses.
Comparison of functionality and applicability of AT frameworks for Big Data in FS.
Selecting the right AT framework for Big Data applications in FS is crucial due to the specific challenges posed by the volume, variety, and velocity of financial data. The functionality of these testing tools ranges from handling basic unit tests to managing complex integration and performance tests that simulate real-world financial transactions at scale. Understanding the strengths and limitations of each framework will assist financial institutions in deploying the most effective tools for their specific requirements (table 1).
Table 1. Comparative overview of AT frameworks for Big Data in FA [7].
Framework Testing capabilities Language support Integration ease Typical use in finance
Apache JMeter Load and performance testing Java Moderate Stress testing financial transaction processing systems
Selenium Functional and regression testing Multiple (Java, C#, Python, Ruby) High Testing web-based financial platforms and user interfaces
Apache Spark Testing Base Unit and integration testing Scala, Java High Data integrity and processing tests for financial analytics
Robot Framework Acceptance testing, RPA Python and others through libraries High Automating business processes and compliance testing
TestNG Unit, functional, integration, and end-to-end testing Java High Comprehensive multi-layered testing of FA
Gatling Performance and load testing Scala Moderate Simulating high-load environments for financial services
The integration of AT frameworks into FS handling Big Data comes with a range of advantages and challenges that must be carefully weighed by financial institutions.
Advantages:
• Enhanced accuracy and reliability: AT frameworks provide a systematic approach to testing that can repeat exact test procedures consistently, minimizing human error and increasing the reliability of results. This is essential in FS where even small errors can lead to significant financial losses.
• Efficiency and scalability: these frameworks can handle the vast amounts of data typical in FS efficiently.
• Cost-effectiveness: over time, AT reduces the cost of testing by decreasing the need for extensive manual testing resources. Although the initial setup cost might be high, the long-term savings in terms of reduced manpower and quicker testing cycles can be substantial.
• Improved regulatory compliance [8, 9]: financial institutions face stringent regulatory requirements for data accuracy, security, and privacy. Automated frameworks can be programmed to include regulatory compliance checks in their testing routines, ensuring that the software continuously complies with legal standards.
• Better risk management: by providing capabilities to perform stress tests and load tests, these frameworks help identify potential points of failure in FS.
While AT frameworks significantly enhance the capabilities of FS managing Big Data, they also introduce certain disadvantages that institutions must consider.
Disadvantages:
• High initial setup cost and complexity: implementing an AT framework can be costly and complex, especially in setting up and configuring the initial environment. There is also a need for skilled personnel capable of managing these tools effectively.
• Maintenance overhead: AT frameworks require regular updates and maintenance to cope with new testing requirements and updates in FA. This ongoing maintenance can add to the operational costs.
• Limitations in flexibility: while automated tests are excellent for repetitive scenarios, they might not be as effective in dealing with complex or unusual test cases.
• Compatibility issues: with legacy systems and the need for custom integration work can pose significant hurdles.
• Overreliance on automation: there is a risk of becoming too reliant on AT, which might lead to neglecting manual testing processes that are equally important for uncovering unexpected issues or understanding the user experience more deeply.
In the author's opinion, the decision to adopt a specific AT framework for Big Data in FS should be guided by a balanced evaluation of both the advantages and disadvantages presented. While these frameworks provide benefits such as improved efficiency, enhanced reliability, and strict adherence to compliance standards, they also pose challenges including high initial costs, maintenance complexities, and potential integration difficulties. Financial institutions need to weigh these factors carefully, considering both the short-term impacts and long-term operational needs. A strategic approach to selecting and implementing these frameworks can lead to substantial improvements in handling Big Data, optimizing testing processes, and maintaining system integrity in a regulated financial environment. This assertion is supported by the successful experiences of companies that have utilized these frameworks to manage large volumes of data, conduct scalable testing, and integrate complex testing scenarios essential for the financial industry.
Trends and innovations in AT for FS.
As the financial industry continues to evolve, so too does the landscape of AT, particularly for Big Data applications. One of the most significant advancements in this field is the integration of Artificial Intelligence (AI) and Machine Learning (ML) into testing frameworks. These technologies are being used to enhance the
capabilities of automated systems, such as predicting potential system failures before they occur, optimizing test cases based on historical data, and providing more in-depth analysis of test results. Companies like Google and IBM have been pioneers in integrating AI with their testing processes, demonstrating substantial improvements in detecting defects and reducing testing times.
Real-time testing frameworks are designed to process and analyze streaming data continuously, providing immediate feedback to developers [10]. Tools like Apache Kafka and Apache Storm have been instrumental in facilitating real-time data processing and testing, enabling financial firms to handle vast streams of transactional data effectively.
Security testing is also gaining prominence, driven by the increasing number of cyber threats targeting financial institutions. AT frameworks are now being developed with enhanced security testing capabilities to identify vulnerabilities that could lead to data breaches. Prominent security solutions providers like Symantec and McAfee are integrating these capabilities into their offerings.
The shift towards cloud-based testing solutions is another key trend. Cloud platforms offer scalability and flexibility, which are essential for testing Big Data applications. These platforms allow financial institutions to quickly set up and scale their testing environments as needed, without the upfront investment in physical infrastructure. Microsoft Azure and Amazon Web Services (AWS) have been at the forefront, providing robust cloud-based testing environments that support the dynamic needs of FA.
The area of Regulatory Technology (RegTech) is expanding within the AT domain. RegTech solutions are designed to ensure that FA are compliant with global financial regulations, which are constantly evolving. AT tools are being developed to include compliance checks within their standard testing routines. This is important for multinational banks like HSBC and Citigroup, which need to ensure compliance across different regulatory environments.
These innovations are shaping the future of AT in the financial sector, providing the tools necessary to manage the complexities of modern FS.
Conclusion.
The integration of AT frameworks designed specifically for Big Data into FS marks an evolution in financial technology. These frameworks not only streamline operations but also enhance the accuracy and efficiency of financial data processing, which is vital in a sector where precision is paramount. Financial institutions leveraging these frameworks benefit from improved data management capabilities, facilitating robust analysis and decision-making processes critical for maintaining competitive advantage and regulatory compliance.
As financial entities navigate the complexities of global markets, the importance of adopting advanced testing frameworks that can handle voluminous and complex data sets efficiently becomes even more pronounced. The practical application of these tools in major financial firms underscores their effectiveness in real-world scenarios, providing a blueprint for other institutions aiming to enhance their Big Data handling capabilities.
СПИСОК ЛИТЕРАТУРЫ:
1. Software Testing Market Size By Component (Application Testing, Services), By Application (IT, Telecom, BFSI, Manufacturing, Retail, Healthcare, Transportation & Logistics, Government & Public Sector, Consumer Electronics, Media), Type & Forecast, 2023 - 2032 / Global Market Insights URL: https://www.gminsights.com/industry-analysis/software-testing-market (date of application: 07.05.2024);
2. Top Industries that use Apache Jmeter / Enlyft URL: https://enlyft.com/tech/products/apache-jmeter (date of application: 07.05.2024);
3. Pursky, O., Babenko, V., Nazarenko, O., Mandych, O., Filimonova, T., Gamaliy, V. (2023). Framework Development for Testing Automation of Web Services Based on Python. In: Magdi, D., El-Fetouh, A.A., Mamdouh, M., Joshi, A. (eds) Green Sustainability: Towards Innovative Digital Transformation. ITAF 2023.
Lecture Notes in Networks and Systems, vol 753. Springer, Singapore. https://doi.org/10.1007/978-981-99-4764-5_24;
4. Apache Spark Market Share and Competitors in Big Data / Enlyft URL: https://enlyft.com/tech/products/apache-spark (date of application: 07.05.2024)
5. Top Industries that use TestNG Data / Enlyft URL: https://enlyft.com/tech/products/testng (date of application: 08.05.2024);
6. Worldwide community of 200,000 businesses & 20 million downloads / Gating URL: https://gatling.io/ (date of application: 08.05.2024);
7. Можаровский Е.А. ТЕХНОЛОГИИ МАШИННОГО ОБУЧЕНИЯ В МОБИЛЬНОЙ РАЗРАБОТКЕ // Научный сетевой журнал «Столыпинский вестник» №3/2024;
8. Kostoreva A.S. BAD FAITH ACTIONS IN INSURANCE LAW: ANALYSIS AND STRATEGIES FOR PROTECTING POLICYHOLDERS' INTERESTS// Proceedings of the XLI International Multidisciplinary Conference «Prospects and Key Tendencies of Science in Contemporary World». Bubok Publishing S.L., Madrid, Spain. 2024;
9. Kostoreva A. TORTS OFFENSES: HOW TO EFFECTIVELY DEFEND CLIENTS' INTERESTS IN COURT // Sciences of Europe #137, p. 48-51 (2024);
10. Tiumentsev, D. Application of cryptographic technologies for information protection in cloud services / D. Tiumentsev // Stolypin Annals. - 2024. - Vol. 6, No. 3. - EDN EDLWAT
