Hybrid Ensemble of Large Language Models and Fractional Derivative Features for Domain-Specific Engineering Sentiment Analysis

Abstrak

This study addresses the need for applied sentiment analysis in engineering decision-support systems by presenting a hybrid framework for domain-specific engineering text. This study presents a hybrid sentiment classification framework by integrating transformer-based semantic embeddings with fractional-order feature modeling. The proposed BERTLR framework combines BERT and RoBERTa representations with Grünwald–Letnikov fractional derivative–enhanced TF-IDF features and logistic regression within a unified soft-voting architecture. Unlike conventional ensemble sentiment models that merely aggregate embeddings and handcrafted features, the proposed method introduces fractional-order feature transformation to capture non-local dependency patterns and memory-aware lexical variations that are often overlooked in technical review text. This design provides a structured fusion of contextual semantic information and fractional statistical representations, supported by SHAP-based explainability and ablation analysis. Experiments conducted on six real-world engineering application domains show consistent improvements over conventional TF-IDF models, LSTM baselines, and non-fractional transformer variants. The framework achieves up to 91% accuracy, together with strong precision, recall, and F1-score performance. These results demonstrate that fractional-order feature augmentation can provide a meaningful complementary signal to transformer embeddings, offering an interpretable and effective sentiment analysis solution for engineering and industrial decision-support applications.

Identifier

• DOI: https://doi.org/10.3390/app16094266
• OpenAlex: https://openalex.org/W7157616722