Preprocessing of Subword Encoding for NMT
Předzpracování podslovních jednotek pro neuronový strojový překlad
diploma thesis (DEFENDED)
View/ Open
Permanent link
http://hdl.handle.net/20.500.11956/192387Identifiers
Study Information System: 257121
Collections
- Kvalifikační práce [11199]
Author
Advisor
Referee
Stephen J., Abishek
Faculty / Institute
Faculty of Mathematics and Physics
Discipline
Computer Science - Language Technologies and Computational Linguistics
Department
Institute of Formal and Applied Linguistics
Date of defense
12. 6. 2024
Publisher
Univerzita Karlova, Matematicko-fyzikální fakultaLanguage
English
Grade
Excellent
Keywords (Czech)
neuronový strojový překlad|segmentace na podslovní jednotky|Byte Pair Encoding|tokenizaceKeywords (English)
neural machine translation|subword segmentation|Byte Pair Encoding|tokenizationMASTER THESIS Kirill Semenov Pre-processing of the Subword Encoding for the Neural Machine Translation Institute of Formal and Applied Linguistics Supervisor of the master thesis: Mgr. Martin Popel, Ph.D. Study programme: Computer Science Study branch: Language Technologies and Computational Linguistics Prague 2024 I declare that I carried out this master thesis independently, and only with the cited sources, literature and other professional sources. It has not been used to obtain another or the same degree. I understand that my work relates to the rights and obligations under the Act No. 121/2000 Sb., the Copyright Act, as amended, in particular the fact that the Charles University has the right to conclude a license agreement on the use of this work as a school work pursuant to Section 60 subsection 1 of the Copyright Act. In . . . . . . . . . . . . . date . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Author's signature i Firstly, I would like to thank Martin Popel for encouraging me to take on the topic that combines the technical aspects of NLP and the linguistic analysis of tokenization. Since the field of NLP was relatively new to me, I am grateful for the attentive guidance in my work. This included both suggesting the directions of the research...