Investigation of Two Types of Machines Translations Google and Targman in Five Scientific Disciplines based on BLEU Model

Ali Ashrafi

MT; NLP; BLEU; Google Translate; Targoman; IBM

In recent years, automatic translation as one of the sub-branches of natural language processing science in our country has been considered by many researchers, including the automatic translators of Targman, Faraazin, etc. In order to localize this technology, these automatic translators need to be evaluated and studied accurately and dynamically. However, large companies such as Google have also worked in this field in order to translate other languages into Persian and vice versa, but due to reasons such as inappropriate figures, calligraphy problems and other problems of Persian language in providing a good and even average translation in Persian language, Google cannot be a good machine translation for Persian language. The purpose of this study is to evaluate different translation machines including Google Translate and Targoman. For this purpose, two sentences in English and Persian in five scientific branches of linguistics, computer, psychology, genetic engineering and chemistry have been randomly selected from the scientific books of these branches. The evaluation criterion in this paper is the BLEU test, which was introduced as a standard method by IBM in 2001. After performing BLEU test on the scores obtained by each translation machine, Google Translate and Targman were ranked first to second .As the results show in a completely statistical and general way, the scores obtained by these machine translators are not satisfactory and the development of these translation machines to reach the desired level requires the efforts of researchers in this field. In addition, the goal of the current research is to examine the methods of improving machine translation using two-level sorting, linguistic features, machine translation evaluation system, semantic ambiguity, semantic similarity, structural reconstruction, as well as computerized linguistics and machine translation software. Due to the widespread increase in regional and international communications and the need for information exchange, the demand for translation has increased in recent years. They also have common and repetitive words, in which case machine translation can be used as an alternative to human translation. There are several ways to improve machine translation which this proposal deals with it.

[1] Hutchins, W.J. (2000).Early years in machine translation: memoirs and biographies of pioneers. John Benjamins Publishing Company,87-386. https://doi.org/10.1075/sihols.97.
[2] White, J, O’Connell, T. and O’Mara, F. (1994).The ARPA MT Evaluation Methodologies: Evolution, Lessons, and Future Approaches. Proceedings of the 1st Conference of the Association, 193-205. https://citeseerx.ist.psu.edu/viewdoc/download?doi= 10.1.1.137.1288&rep=rep1&type=pdf.
[3] King, M. (1996). Terminology, LSP and Translation. Studies in language engineering in honour of Juan C. Sager, 85-243. https://doi.org/10.1075/btl.18.
[4] Liu, CH., Karakanta, A., Tong, A.N. et al(2021). Introduction to the second issue on machine translation for low-resource languages. Machine Translation 35, 1–2. https://doi.org/10.1007/s10590-021-09265-1.
[5] Armengol-Estapé, J., & Costa-jussà, M. R. (2021). Semantic and syntactic information for neural machine translation. Machine Translation, 35(1), 3–17. https://doi.org/10.1007/s10590-021-09264-2.
[6] Abazyan, S., Mamikonyan, N., & Janpoladov, V. (2020). Interlanguage Translation Utility with Integrated Machine Learning Algorithms. OALib, 07(05), 1–5. https://doi.org/ 10.4236/ oalib.1106318.
[7] Yu, J. (2019). A Study on the translation and introduction of J.M.G. Le in China. Open Journal of Social Sciences, 07(12), 290–299. https://doi.org/10.4236/jss.2019.712021.
[8] White, J. S. (2003). How to evaluate machine translation. John Benjamins Publishing Company, 211–244. https://doi.org/10.1075/btl.35.16whi.
[9] Ren, H., Wang, J., Pang, J., Wu, L., & Shi, J. (2020). Review on Machine Translation Post-Editing of Science and Technology Texts in China. Open Journal of Modern Linguistics, 10(01). https://doi.org/10.4236/ojml.2020.101001.
[10] Golino, H. F., Gomes, C. M. A., & Andrade, D. (2014). Predicting Academic Achievement of High-School Students Using Machine Learning. Echo Psychology, 05(18), 2046–2057. https://doi.org/10.4236/psych.2014.518207.
[11] Ülker, M., Güngör, H., & Çakıroğlu, Y. (2021). The Effect of Conducting Introduction Activities with Native Language and Video Learning on Academic Success in Teaching. Creative Education, 12(05), 1169–1185. https://doi.org/10.4236/ce.2021.125087.
[12] Domingo, M., Peris, Á, & Casacuberta, F. (2017). Segment-based interactive-predictive machine translation. Machine Translation, 31(4), 163-185. Retrieved July 2, 2021, from http://www.jstor.org/stable/44987848.
[13] Banerjee, S. and A. Lavie.(2005).METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments.Proceedings of Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and Summarization at the 43th Annual Meeting of the Association of Computational Linguistics,65-72. https://aclanthology.org/W05-0909.pdf.
[14] Turian, J. P., Shen, L,Melamed, I.D.(2003).Evaluation of Machine Translation and its Evaluation. Proceedings of MT Summit IX.1-8.
[15] https://nlp.cs.nyu.edu/publication/papers/turian-summit03eval.pdf, visited on 16.07.2022.
[16] Oliver, A. (2017). A system for terminology extraction and translation equivalent detection in real time: Efficient use of statistical machine translation phrase tables. Machine Translation, 31(3), 147-161.http://www.jstor.org/stable/44987845.
[17] Qian, D. (2017). Machine Translation, 31(4), 257-260.http://www.jstor.org/stable/44987852.
[18] Sanchis-Trilles, G. (2017). Machine Translation, 31(4), 251-255. http://www.jstor.org/stable/44987851.
[19] Schubert, Lenhart.(2020).Computational Linguistics. The Stanford Encyclopedia of Philosophy.Edward N. Zalta.https://plato. stanford.edu/archives/spr2020/entries/computational-linguistics
[20] Mathias, M.(2009).The Limits of Machine Translation (Thesis). University of Copenhagen.11. https://www.semanticscholar.org/
paper/The-Limits-of-Machine-Translation-Madsen/c3ec15cd59199 8821af5e731739083a5070ef063.
[21] Kishore A, Papineni,K, Roukos,S, Ward,T, Zhu,W, Zhu,W.J.(2002).BLEU: a method for automatic evaluation of machine translation.ACL ’02: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics,Pages 311–318. https://doi.org/10.3115/1073083.1073135.
[22] Farzi, S., Faili, H., Khadivi, S., & Maleki, J. (2013). A Novel Reordering Model for Statistical Machine Translation. Res. Comput. Sci., 65, 51-64.https://pdfs.semanticscholar.org/89c3/8d10290876 7214ea7db1ab8195f272c15b96.pdf.
[23] Koehn, p, Och, F.J,Marcu,D.(2003).Statistical phrase based translation. Proceedings of the Joint Conference on Human Language Technologies and the Annual Meeting of the North American Chapter of the Association of Computational Linguistics. 1-7. https://aclanthology.org/N03-1017.pdf.
[24] Doherty, S., O’Brien, S., & Carl, M. (2010). Eye tracking as an MT evaluation technique. Machine Translation, 24(1). 1-13. http://www.jstor.org/stable/40926408.
[25] Specia, L., Raj, D., & Turchi, M. (2010). Machine translation evaluation versus quality estimation. Machine Translation, 24(1), 39-50. http://www.jstor.org/stable/40926411.
[26] Back Matter. (2010). Machine Translation. 24(1).http://www.jstor.org/stable/40926413.
[27] Tsai, C., Mayhew, S., Song, Y., Sammons, M., & Roth, D. (2018). Illinois CCG LoReHLT named entity recognition and situation frame systems. Machine Translation, 32(1–2), 91–103. https://doi.org/10.1007/s10590-017-9211-5.
[28] Jurafsky, D, Martin,J.H.(2009).SPEECH and LANGUAGE PROCESSING.An Introduction to Natural Language Processing,Computational Linguistics and Speech Recognition in University of Colorado at Boulder, 500-600. https://web.stanford.edu/~jurafsky/slp3/ed3book.pdf.
[29] Stahlberg, F. (2020). Neural Machine Translation: A Review. Journal of Artificial Intelligence Research, 69, 343–418. https://doi.org/10.1613/jair.1.12007.
[30] Ni, Y., Saunders, C., Szedmak, S., & Niranjan, M. (2010). The application of structured learning in natural language processing. Machine Translation, 24(2), 71-85. http://www.jstor. org/stable/40926416.
[31] Wang, L., Tu, Z., Zhang, X., Liu, S., Li, H., Way, A., & Liu, Q. (2017). A novel and robust approach for pro-drop language translation. Machine Translation, 31(1–2), 65–87. https://doi.org/10.1007/s10590-016-9184-9.
[32] Crego, J., & Yvon, F. (2010). Factored bilingual n-gram language models for statistical machine translation. Machine Translation, 24(2), 159-175. http://www.jstor.org/stable/40926421.
[33] Back Matter. (2010). Machine Translation, 24(2). http://www.jstor.org/stable/40926422.
[34] Ronald Wardhaugh.(2009).An Introduction to the Sociology of Language.Wiley-Blackwell.Oxford. Translated by Reza Amini. Bouye Kaghaz Publication.
[35] MaiRady, Tamer, Abdelkader Rasha, Ismail. (2019).Integrity and Confidentiality in Cloud Outsourced Data. Ain Shams Engineering Journal.P 275-285.V 10.
[36] Stephen Guise. (2015).How not to be perfectionist.Kindle Edition. Translated by Narges Mohammadi. Shemshad publication.
[37] Helen M. Kingston. (1994).ABC of Clinical Genetics. Login Brothers Book Co.Translated by Jafar Vatandoost. Khamseh Publication.
[38] John McMurry.(2011).Organic Chemistry.Cengage Learning press.Translated by Eisa Yavari. Norpardazan Publication.