ModBERTBr: A ModernBERT-based Model for Brazilian Portuguese

Abstrak

A key model in the Large Language Model (LLM) field is the Bidirectional Encoder Representations from Transformers (BERT), known for its effectiveness and versatility. The current state-of-the-art variant of BERT is ModernBERT, and despite excelling in efficiency and performance, it is limited to English. This paper addresses this notable gap by introducing ModBERTBr, a novel pre-trained model based on the ModernBERT architecture, which is explicitly tailored for Brazilian Portuguese and incorporates cutting-edge research and technologies. Through both intrinsic and extrinsic evaluations, ModBERTBr was assessed against multiple baseline models, showing consistent improvements and competitive performance compared to its predecessors.

Identifier

• DOI: https://doi.org/10.5753/eniac.2025.14516
• OpenAlex: https://openalex.org/W7108344090