Textual Ciphers as a Tool for Better Understanding the Transformers

Abstract

Architektura Transformer je velmi popula ́rnı ́, takz ̌e mu ̊z ̌e by ́t potencia ́lne ̌ vy ́znamne ́ interpretovat, co ovlivn ̌uje jejı ́ vy ́kon. Testujeme hypote ́zu, z ̌e mo- del se pr ̌i pra ́ci s textem spole ́ha ́ na jeho lingvisticke ́ vlastnosti. Abychom eli- minovali vliv kultury na vy ́znam, pouz ̌ı ́va ́me u ́lohu pracujı ́cı ́ na u ́rovni znaku ̊ s Transformer modelem ByT5. Dotre ́nujeme ByT5-small na des ̌ifrova ́nı ́ ve ̌t zas ̌ifrovany ́ch pomocı ́ textovy ́ch s ̌ifer (Vigene ̀re, Enigma). Anotujeme eva- luac ̌nı ́ dataset ve ̌t pomocı ́ publikovany ́ch na 'stroju ̊ pro NLP. Na evaluac ̌nı ́m datasetu zkouma ́me vztahy mezi lingvisticky ́mi vlastnostmi a c ̌etnostı ́ chyb dotre ́novane ́ho ByT5 pr ̌i des ̌ifrova ́nı ́ ve ̌t. Analyzujeme korelace, tre ́nujeme ML modely na predikci c ̌etnosti chyb ve ̌ty z jijı ́ch lingvisticky ́ch vlastnostı ́ a interpretujeme du ̊lez ̌itost vlastnostı ́ pomocı ́ SHAP. Nacha ́zı ́me male ́ signifi- kantnı ́ korelace, ale predikce c ̌etnosti chyb z vlastnostı ́ selha ́va ́. Dospı ́va ́me k za ́ve ̌ru, z ̌e identifikovane ́ vlastnosti neposkytujı ́ vhled do vy ́konu Transfor- meru ̊.

The Transformer architecture is very popular, so it is potentially im- pactful to interpret what influences its performance. We test the hypothesis that the model relies on the linguistic properties of a text when working with it. We remove interference with cultural aspects of meaning by using a character-level task with the ByT5 Transformer model. We train ByT5 to decipher sentences encrypted with text ciphers (Vigenère, Enigma). We annotate a sentence dataset with linguistic properties with published NLP tools. On this dataset, we study the relationships between the linguistic properties and the fine-tuned ByT5 decipherment error rate. We analyze correlations, train ML models to predict error rates from the properties and interpret them with SHAP. We find small significant correlations but can- not predict error rates from the properties. We conclude the properties we identified do not give much insight into the performance of the Transformer.