IJBTT

Addressing AI Drift in Fintech IoT Data Processing: Handling Leap Seconds with PySpark for Robust Predictive Analytics
	© 2025 by IJCTT Journal
	Volume-73 Issue-5
	Year of Publication : 2025
	Authors : Ram Ghadiyaram, Durga Krishnamoorthy, Vamshidhar Morusu, Jaya Eripilla
	DOI :  10.14445/22312803/IJCTT-V73I5P101

How to Cite?

Ram Ghadiyaram, Durga Krishnamoorthy, Vamshidhar Morusu, Jaya Eripilla, "Addressing AI Drift in Fintech IoT Data Processing: Handling Leap Seconds with PySpark for Robust Predictive Analytics," International Journal of Computer Trends and Technology, vol. 73, no. 5, pp. 1-6, 2025. Crossref, https://doi.org/10.14445/22312803/IJCTT-V73I5P101

Abstract
Fintech applications increasingly depend on real-time data from a growing network of IoT devices, where accurate timestamps are vital for reliable analytics and machine learning outcomes. However, timestamp irregularities, such as leap seconds, introduce subtle data inconsistencies that can lead to AI drift in machine learning (ML) models. This study introduces a PySpark-based streaming framework that automatically detects and corrects leap-second anomalies within high-frequency financial data streams, ensuring temporal accuracy and preventing AI drift. The cleaned data is then passed to AI/ML pipelines for reliable predictive analytics. The approach is demonstrated through a streaming data pipeline, with experimental results highlighting its effectiveness. As leap seconds are set to be phased out by 2035, this paper discusses a thought-forward approach by managing timestamp variations to provide a robust framework applicable in financial services and other time-sensitive systems across all domains.

Keywords
Leap seconds, Artificial Intelligence (AI), Machine Learning (ML), AI Drift, atomic time (TAI), astronomical time (UT1), Internet of Things (IoT), Apache Spark (PySpark, Streaming.

Reference

[1] Shengdong Zhang et al., “A Novel Ultrathin Elevated Channel Low-Temperature Poly-Si TFT,” IEEE Electron Device Letters, vol. 20, no. 11, pp. 569-571, 1999.
[CrossRef] [Google Scholar] [Publisher Link]
[2] Yi Wu, Jongwoo Lim, and Ming-Hsuan Yang, “Online Object Tracking: A Benchmark,” 2013 IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, pp. 2411-2418, 2013.
[CrossRef] [Google Scholar] [Publisher Link]
[3] Simeon M. Metev, and Vadim P. Veiko, Laser Assisted Microtechnology, 2nd ed., Germany: Springer-Verlag, pp. 1-270, 1998.
[Google Scholar] [Publisher Link]
[4] Jens Breckling, The Analysis of Directional Time Series: Applications to Wind Speed and Direction, Lecture Notes in Statistics, Berlin, Germany: Springer, vol. 61, no. 1, pp. 200-220, 1989.
[CrossRef] [Google Scholar] [Publisher Link]
[5] Ronald E. Sorace, Victor S. Reinhardt, and Steven A. Vaughn, “High-Speed Digital-to-RF Converter,” U.S. Patent US5668842A, pp. 1 8, 1997.
[Publisher Link]
[6] Hong-Ning Dai, Zibin Zheng, and Yan Zhang, “Blockchain for Internet of Things: A Survey,” IEEE Internet of Things Journal, vol. 6, no. 5, pp. 8076-8094, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[7] Emiliano Sisinni et al., “Industrial Internet of Things: Challenges, Opportunities, and Directions,” IEEE Transactions on Industrial Informatics, vol. 14, no. 11, pp. 4724-4734, 2018.
[CrossRef] [Google Scholar] [Publisher Link]
[8] Fan Liang et al., “Machine Learning for Security and the Internet of Things: The Good, the Bad, and the Ugly,” IEEE Access, vol. 7, pp. 158126-158147, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[9] Jie Lin et al., “A Survey on Internet of Things: Architecture, Enabling Technologies, Security and Privacy, and Applications,” IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1125-1142, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[10] Chaoyun Zhang, Paul Patras, and Hamed Haddadi, “Deep Learning in Mobile and Wireless Networking: A Survey,” IEEE Communications Surveys & Tutorials, vol. 21, no. 3, pp. 2224-2287, 2019.
[CrossRef] [Google Scholar] [Publisher Link]
[11] Paul Schulte, and Gavin Liu, “FinTech is Merging with IoT and AI to Challenge Banks: How Entrenched Interests Can Prepare,” The Journal of Alternative Investments, vol. 20, no. 3, pp. 41-57, 2017.
[CrossRef] [Google Scholar] [Publisher Link]
[12] Joseph Bamidele Awotunde et al., “Application of Big Data with Fintech in Financial Services,” Fintech with Artificial Intelligence, Big Data, and Blockchain, pp. 107-132, 2021.
[CrossRef] [Google Scholar] [Publisher Link]
[13] Adetoyese Omoseebi, Anderson Ella, and Jerry Henry, “Fintech Growth: Open Banking Fosters Innovation, Enabling Fintech Startups to Develop New Products and Services,” pp. 1-17, 2025.
[Google Scholar]