Leveraging Transformer-based Models for Classification and Large Language Models for Information Extraction from Medical Claims Documents

Abstrak

The study explores the use of advanced AI models for document classification and information extraction in Indonesian medical claims documents. Transformer-based models like Layout LMv3 show good performance with an F1 score above 93% for classification. LLMs like Qwen2.5-VL 7B yield a high F1 score (92-97%) for printed documents but face challenges in processing handwritten medical resumes. The methodology involves extensive data preprocessing and hyperparameter optimization to improve model performance. The research emphasizes the importance of balanced data sampling, valid ground truth data, and careful model selection for accuracy and efficiency. Future studies should integrate deep learning techniques and use pre-trained or fine-tuned BERT models for biomedical and clinical domains. These findings support the potential of transformer-based models and LLMs to automate and simplify document processing workflows in health administration.

Identifier

• DOI: https://doi.org/10.24167/sisforma.v12i2.14285
• OpenAlex: https://openalex.org/W4416455467