Improving the Performance of the ETL Jobs
  IJCTT-book-cover
 		          
 
© 2023 by IJCTT Journal
Volume-71 Issue-3
Year of Publication : 2023
Authors : Dhamotharan Seenivasan
DOI :  10.14445/22312803/IJCTT-V71I3P105

How to Cite?

Dhamotharan Seenivasan, "Improving the Performance of the ETL Jobs," International Journal of Computer Trends and Technology, vol. 71, no. 3, pp. 27-33, 2023. Crossref, https://doi.org/10.14445/22312803/IJCTT-V71I3P105

Abstract
ETL (extract, transform, load) jobs are responsible for extracting data from a variety of sources, transforming it into a consistent format, and loading it into a target data store. The performance of ETL jobs can significantly impact the overall performance of an organization's data management system. Several factors can affect the performance of ETL jobs, including the volume of data being processed, the complexity of the transformation logic, and the efficiency of the extraction and load processes. In this article, we will discuss some techniques for improving the performance of ETL jobs.

Keywords
Data warehouse, ETL testing, Extract Transform and Load (ETL), ETL performance, ETL optimization.

Reference

[1] Ralph Kimball, and Margy Ross, The Kimball Group Reader: Relentlessly Practical Tools for Data Warehousing and Business Intelligence, John Wiley & Sons, 2010.
[Publisher Link]
[2] Mitesh Athwani, “A Novel Approach to Version XML Data Warehouse,” SSRG International Journal of Computer Science and Engineering, vol. 8, no. 9, pp. 5-11, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Hameed Hussain et al., “A Survey on Resource Allocation in High Performance Distributed Computing Systems,” Parallel Computing, vol. 39, no. 11, pp. 709-736, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[4] Vishal Goar et al., “Improve Performance of Extract, Transform and Load (ETL) in Data Warehouse,” International Journal on Computer Science and Engineering, vol. 2, no. 3, pp. 786-789, 2010.
[Google Scholar] [Publisher Link]
[5] Vishal Goar et al., “Improve Performance of Extract, Transform and Load (ETL) in Data Warehouse,” International Journal on Computer Science and Engineering, vol. 2, no. 3, pp. 786-789, 2010.
[Google Scholar] [Publisher Link]
[6] Janne J. Korhonen et al., “Designing Data Governance Structure: An Organizational Perspective,” GSTF Journal on Computing (JoC), vol. 2, no. 4, 2014.
[Google Scholar] [Publisher Link]
[7] Baljit Singh, “Enterprise Reporting on SAP S/4HANA using Snowflake as Cloud Datawarehouse,” International Journal of Computer Trends and Technology, vol. 71, no. 1, pp. 28-39, 2023.
[CrossRef] [Publisher Link]
[8] Jayanthi Ranjan, “Business Intelligence: Concepts, Components, Techniques and Benefits,” Journal of Theoretical and Applied Information Technology, vol. 9, no. 1, pp. 60-70, 2009.
[Google Scholar] [Publisher Link]
[9] Vangipuram Radhakrishna, Vangipuram SravanKiran, and K. Ravikiran, “Automating ETL Process with Scripting Technology,” Nirma University International Conference on Engineering, pp. 1-4, 2012.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Dhamotharan Seenivasan, “ETL (Extract, Transform, Load) Best Practices,” International Journal of Computer Trends and Technology, vol. 71, no. 1, pp. 40-44, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Kamal Kakish, and Theresa A. Kraft, “ETL Evolution for Real-Time Data Warehousing,” In Proceedings of the Conference on Information Systems Applied Research, vol. 2167, pp. 1508, 2012.
[Google Scholar] [Publisher Link]
[12] Syed Muhammad Fawad Ali, and Robert Wrembel, “From Conceptual Design to Performance Optimization of ETL Workflows: Current State of Research and Open Problems,” The VLDB Journal, vol. 26, no. 6, pp. 777-780, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[13] [Online]. Available:https://www.integrate.io/blog/7-tips-improve-etl-performance/
[14] [Online]. Available: https://danischnider.wordpress.com/2017/07/23/10-tips-to-improve-etl-performance/
[15] [Online]. Available: https://medium.com/ziegert-group/etl-performance-improvement-c5a9bd65b6af
[16] [Online]. Available: https://blog.devart.com/how-to-optimize-sql-query.html
[17] [Online]. Available:http://www.ijmer.com/papers/(NCASG)%20-%202013/24.pdf
[18] [Online]. Available:https://dataintegrationinfo.com/improve-etl-performance/
[19] [Online]. Available: https://www.tridex.org/wp-content/uploads/Tridex-ETL.pdf
[20] [Online]. Available:https://www.researchgate.net/publication/341435560_Performance_Optimization_of_ETL_Process
[21] [Online]. Available:https://solutioncenter.apexsql.com/improve-the-performance-of-etl-process/
[22] [Online]. Available:https://elink.io/p/ways-to-optimize-the-performance-of-etl-process-9a0c9e9
[23] [Online]. Available:https://www.researchgate.net/publication/368300555_ETL_for_Data_Warehousing
[24] [Online]. Available:https://link.springer.com/article/10.1007/s00778-017-0477-2
[25] Dhamotharan Seenivasan, “Exploring Popular ETL Testing Techniques,” International Journal of Computer Trends and Technology, vol. 71, no. 2, pp. 32-39, 2023.
[CrossRef] [Google Scholar] [Publisher Link]
[26] [Online]. Available:https://www.disoln.org/search/label/Performance%20Tips
[27] [Online]. Available:https://www.timmitchell.net/etl-best-practices/
[28] [Online]. Available: https://medium.com/@data_analytics/etl-for-data-warehousing-1203dc346a4e