A Method for Improving Unsupervised Intent Detection using Bi-LSTM CNN Cross Attention Mechanism

Sadeghi, Hamid reza, SHIRY GHIDARY, Saeed and BAKHTIARI BASTAKI, Benhur (2021) A Method for Improving Unsupervised Intent Detection using Bi-LSTM CNN Cross Attention Mechanism. In: ICAAI 2020 Conference Proceedings. Association for Computing Machinery New York,NY, United States, USA, pp. 41-46. ISBN 978-1-4503-8784-2

Preview

Text BCCAM.pdf - AUTHOR'S ACCEPTED Version (default) Available under License All Rights Reserved. Download (2MB) | Preview

Abstract or description

Spoken Language Understanding (SLU) can be considered the most important sub-system in a goal-oriented dialogue system. SLU consists of User Intent Detection (UID) and Slot Filling (SF) modules. The accuracy of these modules is highly dependent on the collected data. On the other hand, labeling operation is a tedious task due to the large number of labels required. In this paper, intent labeling for two datasets is performed using an unsupervised learning method. In traditional methods of extracting features from text, the feature space that is obtained is very large, therefore we implemented a novel architecture of auto-encoder neural networks that is based on the attention mechanism to extract small and efficient feature space. This architecture which is called Bi-LSTM CNN Cross Attention Mechanism (BCCAM), crosswise applies the attention mechanism from Convolutional Neural Network (CNN) layer to Bi-LSTM layer and vice versa. Then, after finding a bottleneck on this auto-encoder network, the desired features are extracted from it. Once the features are extracted, then we cluster each sentence corresponding to its feature space using different clustering algorithms, including K-means, DEC, Agglomerative, OPTICS and Gaussian mixture model. In order to evaluate the performance of the model, two datasets are used, including ATIS and SNIPS. After executing various algorithms over the extracted feature space, the best obtained accuracy and NMI for ATIS dataset are 86.5 and 91.6, respectively, and for SNIPS dataset are 49.9 and 43.0, respectively.

Item Type:	Book Chapter, Section or Conference Proceeding
Faculty:	School of Computing and Digital Technologies > Computing
Event Title:	The 4th International Conference on Advances in Artificial Intelligence (ICAAI 2020)
Event Location:	Online
Event Dates:	13-15 Nov 2020
Depositing User:	Saeed SHIRY GHIDARY
Date Deposited:	03 Mar 2021 09:46
Last Modified:	24 Feb 2023 14:00
Related URLs:	Organisation
URI:	https://eprints.staffs.ac.uk/id/eprint/6635

Actions (login required)

View Item