Online Transfer Learning for Real-Time Decision Making

Dr. Hua Zuo

Senior Lecturer, Australian Artificial Intelligence Institute (AAII)

Abstract

With the rapid development of deep learning and artificial intelligence, one major challenge in real-world applications is the continuous dynamic shift in data distributions. Traditional offline transfer learning methods assume a stable target domain post-training, often leading to rapid performance decline during testing or inference with evolving data. Online transfer learning addresses this by continuously updating the model at test time, maintaining adaptability to new data under resource constraints. However, it still faces issues like error accumulation, catastrophic forgetting, class imbalance, difficulties in detecting out-of-distribution (OOD) samples, and overfitting.

To tackle these challenges, we propose a multi-level framework. To mitigate error accumulation, we use multiple teacher models to generate pseudo-labels, whose outputs are weighted and averaged. To alleviate catastrophic forgetting, we employ a memory bank to store trainable prompts in large vision-language models (VLM), combined with a maximum gradient search strategy. This enables effective learning of new knowledge while maintaining overall performance in dynamic environments. For class imbalance, we adopt a fuzzy rule-based weighting mechanism that assigns flexible membership degrees to each sample, ensuring adequate updates for both minority and majority classes. To enhance OOD detection and robustness, we utilize large language models (LLM) to guide prompts for distinguishing in-distribution and out-of-distribution data, dynamically adjusting prompt parameters via a feedback mechanism. Finally, the Reply self-regularization strategy leverages the model's outputs as regularization constraints, comparing current predictions with historically stable states to mitigate overfitting. Experimental results demonstrate that our approach effectively addresses error accumulation, catastrophic forgetting, class imbalance, OOD detection, and overfitting, providing a robust solution for test-time adaptation in online transfer learning.

Biography

Dr. Hua Zuo is an ARC Discovery Early Career Researcher Award (DECRA) Fellow and a Senior Lecturer at the Australian Artificial Intelligence Institute within the Faculty of Engineering and IT at the University of Technology Sydney (UTS). She earned her PhD in Computer Science from UTS in August 2018 and subsequently joined UTS as a full-time Postdoctoral Research Associate in May 2018, supported by an ARC Discovery Project. She was promoted to Lecturer in May 2019 and, in 2022, secured an ARC DECRA, advancing to the position of Senior Lecturer. Dr. Zuo’s research interests include fuzzy machine learning, transfer learning, test-time domain adaptation, and transferable reinforcement learning. She has published nearly 50 papers in high-quality journals and conferences, including IEEE TFS, IEEE TNNLS, IEEE TKDE, IEEE TCYB, ACM TIST, ACM MM, IJCNN, FUZZ-IEEE, AJCAI, etc.

Accessibility

General information