Feature Selection and Transfer Learning in Network Data: Enhancing Anomaly Detection with Zero-Shot and Few-Shot Learning

The exponential growth of high-dimensional data across domains such as bioinformatics, healthcare, finance, and image processing has heightened the need for effective feature selection (FS) methods. These techniques improve model performance by identifying relevant features, reducing computational complexity, and mitigating overfitting.
This PhD thesis introduces Radian, a novel feature selection method that leverages the statistical properties of range and median to identify the most influential features. Radian effectively distinguishes between relevant and redundant attributes while also detecting anomalies, enhancing both model interpretability and data quality. Radian was rigorously evaluated on multiple benchmark datasets of varying size and complexity. The results show that it consistently outperforms conventional methods such as the Pearson correlation coefficient in three key areas: classification accuracy, feature reduction, and computational efficiency. Its ability to balance performance and simplicity enables the creation of compact, interpretable models that retain or improve predictive accuracy.

Beyond feature selection, this research advances transfer learning for tabular data, an area often underexplored in existing literature. Three innovative models TabLoRA, TabLoRA-ZS (zero-shot), and TabLoRA-FS (few-shot) are introduced by integrating TabNet, a deep learning architecture for tabular data, with Low-Rank Adaptation (LoRA) modules. The TabLoRA-ZS model enables generalisation to unseen tasks without prior data, while TabLoRA-FS fine-tunes efficiently with minimal data, addressing the challenges of data scarcity.

A major innovation lies in integrating Radian with TabNet and LoRA, allowing dynamic feature selection during transfer learning. This integration improves model adaptability, robustness, and scalability, particularly in environments with limited labelled data.

Comprehensive experiments demonstrate that these Radian-enhanced transfer learning models perform competitively with state-of-the-art approaches while maintaining interpretability and efficiency.

In conclusion, this thesis contributes to machine learning by (1) proposing Radian, a statistically driven, efficient feature selection method, and (2) developing Radian-integrated TabLoRA models for few-shot and zero-shot transfer learning. Together, they provide scalable, adaptable, and high-performing solutions for data-scarce domains, bridging the gap between feature selection and transfer learning in tabular data analysis.