Luganda speech intent classification for IoT applications

Abstract

The ability to control Internet of Things (IoT) devices using voice commands has gained popularity in recent years. However, most existing systems are designed for high-resource languages, and less work exists in low-resource languages like Luganda. This research project focuses on developing a Luganda speech intent classification system for IoT applications, aiming to integrate local languages like Luganda into smart home environments. This project involved collecting a dataset of Luganda voice commands, building and training natural language processing (NLP) models based on both speech-based and automatic speech recognition (ASR) approaches, and implementing the best-performing model on a Raspberry Pi which is used to control devices. The project used hardware components such as Raspberry Pi, Wio Terminal, and ESP32 nodes as microcontrollers. The Raspberry Pi acts as the central hub for processing and interpreting Luganda voice commands, while the Wio Terminal serves as a display device for the output providing a user-friendly interface for interacting with the smart home system. The ESP32 nodes are used for controlling and coordinating the IoT devices for the smart home setup. The IoT devices use the MQTT (Message Queuing Telemetry Transport) protocol as a communication protocol for wireless communication. The model incorporates Mel Frequency Cepstral Coefficients (MFCCs) as acoustic features and utilizes a Convolutional Neural Network (Conv2D) architecture for speech intent classification. Our proposed system aims to allow Luganda-speaking individuals to interact with smart home devices using voice commands in their native language, improving their experience and enabling more efficient control of their devices. This project will contribute to the fields of natural language processing and IoT and promote the inclusion of low -resource language speakers in the design of smart home systems. The developed system can be extended to other languages with limited resources and can serve as a starting point for future research. This project addresses the localization challenges and linguistic diversity in IoT applications by using Luganda voice commands for device control which allows users to interact seamlessly and naturally with their smart home devices. The project findings contribute to the application of low-resource languages to control Internet of Things (IoT) devices using voice commands.