Design of Dimensionality Reduction Algorithm for High-Dimensional Large-Scale Translation Corpora and Lightweight Translation Model Training

Abstract

Addressing the "curse of dimensionality" problem caused by the exponential growth of translation corpora in current machine translation research, and the practical bottlenecks of large-scale model deployment difficulties and high inference latency in resource-constrained scenarios, this paper designs a dimensionality reduction algorithm that integrates feature selection and deep reconstruction, and constructs a lightweight translation model training framework by combining knowledge distillation and structured pruning. First, feature importance is evaluated based on a self-attention mechanism to remove noisy and redundant features. Next, a variational autoencoder is used to perform deep reconstruction on the selected features to extract low-dimensional dense semantic representations. Then, the dimensionality-reduced features are input into a lightweight Transformer student network, which learns knowledge from the teacher model through knowledge distillation. Finally, structured pruning is used to further eliminate attention heads and neuron redundancy. Experimental results show that, while maintaining a BLEU score of 28.9, the feature dimension is compressed by 92.5%, the model parameter size is reduced to 26M, and the inference speed is improved by 2.3 times. Furthermore, it outperforms comparative methods at different compression rates, providing an efficient and feasible technical path for translation deployment in resource-constrained environments.