A Benchmark of Activation Functions in Extreme Learning Machine for High-Dimensional Low-Sample-Size Microarray Classification

Abstrak

Microarray data classification presents significant challenges in machine learning due to its high-dimensionality, low-sample-size (HDLSS) characteristics and class imbalance. These conditions often lead to overfitting and result in models with low performance stability. Extreme Learning Machine Learning (ELM) is an efficient method, but its performance is highly dependent on the choice of activation function that determines the model's non-linearity. Therefore, this study aims to conduct a comparative benchmarking of eight activation functions (sigmoid, tanh, relu, hardlimit, identity, swish, cosine, and softsign) within the ELM framework on 11 cancer-related microarray datasets. Testing was conducted through 30 independent runs to ensure statistical robustness, and performance was evaluated using accuracy, F1-score, precision, and recall metrics. Experimental results show that the choice of activation function has a significant impact on microarray data classification performance. The sigmoid function consistently provides superior and most stable results across various datasets, achieving a global average accuracy above 70% for all evaluation metrics. The advantage of the sigmoid lies not only in its high average performance but also in its stability, as evidenced by its low standard deviation. These findings provide strong practical guidance, recommending the use of the sigmoid activation function for robust ELM implementations in microarray data classification.

Identifier

• OpenAlex: https://openalex.org/W7117246622
• DOI: https://doi.org/10.31572/inotera.vol10.iss2.2025.id593