Comparative Analysis of TensorRT Inference Speeds Across Various Neural Network Block Architectures

Abstrak

This work presents a systematic investigation into the inference speed behavior of common neural network block architectures—VGG, ResNet, MobileNet, and ConvNeXt—when deployed using NVIDIA TensorRT in a C++ environment. The study evaluates the performance impact of different normalization layers (Batch Normalization vs. Layer Normalization), activation functions (ReLU vs. GELU), numerical precision (FP32 and FP16), and input resolutions. To focus on deployment-side characteristics, experiments are conducted using dummy inputs rather than real-world datasets. Key findings include:- MobileNet consistently achieves the fastest inference speeds across all tested configurations.- Batch Normalization (BN) significantly outperforms Layer Normalization (LN), largely due to TensorRT's layer fusion optimizations.- Anomalous behavior is observed at low input resolutions (e.g., 64×64), where FP16 precision can be slower than FP32 in specific configurations such as MobileNet with LN, likely due to memory access overhead and numerical stability issues. This record is intended as a research note and long-term engineering reference, rather than a benchmark for model accuracy or a finalized academic publication. The source code for reproducing these benchmarks is available at:https://github.com/Hwa-Jong/PyTorch-to-TensorRT-Block-Comparison

Identifier

• OpenAlex: https://openalex.org/W7117568985
• DOI: https://doi.org/10.5281/zenodo.18082201