Online Transfer Learning for Real-Time Decision Making

Dr. Hua Zuo

Senior Lecturer, Australian Artificial Intelligence Institute (AAII)

Abstract

With the rapid development of deep learning and artificial intelligence, one major challenge in real-world applications is the continuous dynamic shift in data distributions. Traditional offline transfer learning methods assume a stable target domain post-training, often leading to rapid performance decline during testing or inference with evolving data. Online transfer learning addresses this by continuously updating the model at test time, maintaining adaptability to new data under resource constraints. However, it still faces issues like error accumulation, catastrophic forgetting, class imbalance, difficulties in detecting out-of-distribution (OOD) samples, and overfitting.

To tackle these challenges, we propose a multi-level framework. To mitigate error accumulation, we use multiple teacher models to generate pseudo-labels, whose outputs are weighted and averaged. To alleviate catastrophic forgetting, we employ a memory bank to store trainable prompts in large vision-language models (VLM), combined with a maximum gradient search strategy. This enables effective learning of new knowledge while maintaining overall performance in dynamic environments. For class imbalance, we adopt a fuzzy rule-based weighting mechanism that assigns flexible membership degrees to each sample, ensuring adequate updates for both minority and majority classes. To enhance OOD detection and robustness, we utilize large language models (LLM) to guide prompts for distinguishing in-distribution and out-of-distribution data, dynamically adjusting prompt parameters via a feedback mechanism. Finally, the Reply self-regularization strategy leverages the model's outputs as regularization constraints, comparing current predictions with historically stable states to mitigate overfitting. Experimental results demonstrate that our approach effectively addresses error accumulation, catastrophic forgetting, class imbalance, OOD detection, and overfitting, providing a robust solution for test-time adaptation in online transfer learning.

Artificial Intelligence has been widely used during the last two decades and has remained a highly-researched topic, especially for complex real-world problems. Evolutionary Intelligence (EI) techniques are a subset of artificial intelligence, but they are slightly different from the classical methods in the sense that the intelligence of EI comes from biological systems or nature in general. The efficiency of EI is due to their significant ability to imitate the best features of nature which have evolved by natural selection over millions of years. The central theme of this presentation is about EI techniques and their application to complex real-world problems. On this basis, first I will talk about an automated learning approach called genetic programming. Applied evolutionary learning will be presented, and then their new advances will be mentioned. Here, some of my studies on big data analytics and modelling using EI and genetic programming, in particular, will be presented. Second, EI will be presented including key applications in the optimization of complex and nonlinear systems. It will also be explained how such algorithms have been adopted to engineering problems and how their advantages over the classical optimization problems are used in action. Optimization results of large-scale towers and many-objective problems will be presented which show the applicability of EI. Finally, heuristics will be explained which are adaptable with EC and they can significantly improve the optimization results.

Biography

Dr. Hua Zuo is an ARC Discovery Early Career Researcher Award (DECRA) Fellow and a Senior Lecturer at the Australian Artificial Intelligence Institute within the Faculty of Engineering and IT at the University of Technology Sydney (UTS). She earned her PhD in Computer Science from UTS in August 2018 and subsequently joined UTS as a full-time Postdoctoral Research Associate in May 2018, supported by an ARC Discovery Project. She was promoted to Lecturer in May 2019 and, in 2022, secured an ARC DECRA, advancing to the position of Senior Lecturer. Dr. Zuo’s research interests include fuzzy machine learning, transfer learning, test-time domain adaptation, and transferable reinforcement learning. She has published nearly 50 papers in high-quality journals and conferences, including IEEE TFS, IEEE TNNLS, IEEE TKDE, IEEE TCYB, ACM TIST, ACM MM, IJCNN, FUZZ-IEEE, AJCAI, etc.