Title:	Transfer Learning of Transformers for Spoken Language Understanding
Authors:	Švec, Jan Frémund, Adam Bulín, Martin Lehečka, Jan
Citation:	ŠVEC, J. FRÉMUND, A. BULÍN, M. LEHEČKA, J. Transfer Learning of Transformers for Spoken Language Understanding. In Text, Speech, and Dialogue 25th International Conference, TSD 2022, Brno, Czech Republic, September 6–9, 2022, Proceedings. Cham: Springer International Publishing, 2022. s. 489-500. ISBN: 978-3-031-16269-5 , ISSN: 0302-9743
Issue Date:	2022
Publisher:	Springer International Publishing
Document type:	konferenční příspěvek ConferenceObject
URI:	2-s2.0-85139041956 http://hdl.handle.net/11025/50928
ISBN:	978-3-031-16269-5
ISSN:	0302-9743
Keywords in different language:	Wav2Vec model;Speech recognition;T5 model;Spoken language understanding
Abstract in different language:	Pre-trained models used in the transfer-learning scenario are recently becoming very popular. Such models benefit from the availability of large sets of unlabeled data. Two kinds of such models include the Wav2Vec 2.0 speech recognizer and T5 text-to-text transformer. In this paper, we describe a novel application of such models for dialog systems, where both the speech recognizer and the spoken language understanding modules are represented as Transformer models. Such composition outperforms the baseline based on the DNN-HMM speech recognizer and CNN understanding.
Rights:	Plný text je přístupný v rámci univerzity přihlášeným uživatelům. © Springer Nature Switzerland AG
Appears in Collections:	Konferenční příspěvky / Conference papers (NTIS) Konferenční příspěvky / Conference Papers (KKY) OBD

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.