The use of artificial intelligence for sign language recognition in education: from a literature overview to the ISENSE project

Abstract

Common strategies to guarantee the inclusion of d/Deaf students in university are still absent. Recent studies have demonstrated how innovative technologies, such as artificial intelligence and virtual/augmented reality, could be applied to solve this issue. This paper aims at providing an overview of the AI-based algorithms for sign language recognition, as well as the description of the project ISENSE. As concerning the AI application, the already implemented procedures are generally based on three steps, starting from sign language capture to sign language translation, passing through sign language recognition. The state-of-the art for each step has been reported, highlighting the most widespread solutions and the related pros and cons. Focusing on ISENSE, the project aims at implementing services and tools based on artificial intelligence and immersive technologies for deaf students at university. The project framework has been introduced, reporting both the main outcomes and the operating steps.