Analisis Perbandingan Nilai Akurasi Mekanisme Attention Bahdanau dan Luong pada Neural Machine Translation Bahasa Indonesia ke Bahasa Melayu Ketapang dengan Arsitektur Recurrent Neural Network

Wahyu Gunawan, Herry Sujaini, Tursina Tursina

Abstract


Di Indonesia, penerapan mesin penerjemah masih banyak dilakukan dengan berbasis statistik khususnya dalam eksperimen penerjemahan bahasa daerah. Dalam beberapa tahun terakhir, mesin penerjemah jaringan saraf tiruan telah mencapai kesuksesan yang luar biasa dan menjadi metode pilihan baru dalam praktik mesin penerjemah. pada penelitian ini menggunakan mekanisme attention dari Bahdanau dan Luong dalam bahasa Indonesia ke bahasa Melayu Ketapang dengan data korpus paralel sejumlah 5000 baris kalimat. Hasil pengujian berdasarkan metode penambahan secara konsisten dengan jumlah epoch didapatkan nilai skor BLEU yaitu pada attention Bahdanau menghasilkan akurasi 35,96% tanpa out-of-vocabulary (OOV) dengan menggunakan jumlah epoch 40, sedangkan pada attention Luong menghasilkan akurasi 26,19% tanpa OOV menggunakan jumlah 30 epoch. Hasil pengujian berdasarkan k-fold cross validation didapatkan nilai rata-rata akurasi tertinggi sebesar 40,25% tanpa OOV untuk attention Bahdanau dan 30,38% tanpa OOV untuk attention Luong, sedangkan pengujian manual oleh dua orang ahli bahasa memperoleh nilai akurasi sebesar 78,17% dan 72,53%. 


Keywords


Neural Machine Translation; Attention Mechanism; Deep Learning; Natural Language Processing; BLEU; K-Fold Cross Validation

Full Text:

PDF

References


M. Junczys-Dowmunt, T. Dwojak, and H. Hoang, “Is Neural Machine Translation Ready for Deployment? A Case Study on 30 Translation Directions,” 2016, [Online]. Available: http://arxiv.org/abs/1610.01108.

A. Sulissusiawan, Struktur Bahasa Melayu Dialek Ketapang. Pusat Pembinaan Dan Pengembangan N Nasional, 1998.

Z. Abidin, A. Sucipto, and A. Budiman, “Penerjemahan Kalimat Bahasa Lampung-Indonesia Dengan Pendekatan Neural Machine Translation Berbasis Attention Translation of Sentence Lampung-Indonesian Languages With Neural Machine Translation Attention Based,” J. Kelitbangan, vol. 06, no. 02, pp. 191–206, 2018.

N. Kalchbrenner and P. Blunsom, “Recurrent continuous translation models,” EMNLP 2013 - 2013 Conf. Empir. Methods Nat. Lang. Process. Proc. Conf., no. October, pp. 1700–1709, 2013.

K. Cho et al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” EMNLP 2014 - 2014 Conf. Empir. Methods Nat. Lang. Process. Proc. Conf., pp. 1724–1734, 2014, doi: 10.3115/v1/d14-1179.

I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” Adv. Neural Inf. Process. Syst., vol. 4, no. January, pp. 3104–3112, 2014.

D. Bahdanau, K. H. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1–15, 2015.

M.-T. Luong, H. Pham, and C. D. Manning, “Effective Approaches to Attention-based Neural Machine Translation,” arXiv, 2015, [Online]. Available: https://re-work.co/blog/deep-learning-ilya-sutskever-google-openai.

N. Kalchbrenner, L. Espeholt, K. Simonyan, A. van den Oord, A. Graves, and K. Kavukcuoglu, “Neural Machine Translation in Linear Time,” 2016, [Online]. Available: http://arxiv.org/abs/1610.10099.

J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin, “Facebook使用CNN替代RNN Convolutional Sequence to Sequence Learning,” 2017, [Online]. Available: http://arxiv.org/abs/1705.03122.

A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 2017-Decem, no. Nips, pp. 5999–6009, 2017.

A. Karpathy and L. Fei-Fei, “Deep Visual-Semantic Alignments for Generating Image Descriptions,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 4, pp. 664–676, 2017, doi: 10.1109/TPAMI.2016.2598339.

Y. Marathe, “Neural Machine Translation using Bahdanau Attention Mechanism,” medium.com, 2020. https://medium.com/analytics-vidhya/neural-machine-translation-using-bahdanau-attention-mechanism-d496c9be30c3 (accessed Jan. 20, 2021).

G. Loye, “Attention Mechanism,” blog.floydhub.com, 2019. https://blog.floydhub.com/attention-mechanism/ (accessed Jan. 20, 2021).

S. T. Dinar, H. Sujaini, and N. Safriadi, “Optimasi Korpus untuk Meningkatkan Nilai Akurasi Mesin Penerjemah Statistik ( Studi Kasus Bahasa Indonesia ke Bahasa Melayu Ketapang ) Corpus Optimizing To Increase Value Of The Accuracy For Statistical Machine Translation ( Study Case Indonesia Language,” 2020, doi: 10.26418/justin.v.

D. T. Rahayu, H. Sujaini, and A. B. P. Negara, “Perbandingan Metode Evaluasi Otomatis Terhadap Mesin Penerjemah Statistik Menggunakan Part of Speech,” pp. 1–8, 2018.

J. Brownlee, “What is Teacher Forcing for Recurrent Neural Networks?,” machinelearningmastery.com, 2021. https://machinelearningmastery.com/teacher-forcing-for-recurrent-neural-networks/ (accessed Jan. 20, 2021).

M.-T. Luong, “Neural Machine Translation,” Stanford University, 2016.

M. S. Islam and B. S. Purkayastha, “English to Bodo Phrase-Based Statistical Machine Translation,” in Advanced Computing and Communication Technologies, Springer, 2018, pp. 207–217.




DOI: https://doi.org/10.26418/jp.v7i3.50287

Refbacks

  • There are currently no refbacks.