The next speaker in this AoIR 2019 session is Indra Mckie, who shifts our focus to chatbots – which to date have often been found to be somewhat disappointing in their performance. One type of such chatbot are the dialogue systems that are used to complete bookings or make purchases, and speed up customer interaction; another are chat(ter)bots like the famous Eliza that are set to mimic unstructured human conversations.
Indra has been working with student-facing, library, and human resources chatbot projects at the University of Technology Sydney, and one challenge in progressing such projects has also been to assess their actual performance – what are the indicators of success here? One of the projects targetted the needs of first-year undergraduate students, in particular, who often experience a kind of ‘library anxiety’ and sometimes have poor time planning skills. The hope is that a chatbot might be able to help alleviate some of these issues.
Indra worked with data from UTS’s library Q&A site and (human) live chat tool to identify the typical interactions between students and librarians, and translated some of the most common query scenarios to the format required by the university’s chatbot system. But this can be complicated, due to the polysemic nature of everyday language (in the library context, for example, ‘book a room’ uses the word ‘book’ in a completely difference sense from ‘request a book’).
Such chatbots might also be used to anticipate certain information needs, and proactively contact students to alert them to upcoming deadlines or other needs. It is often useful to frame such content in a narrative that reflects the student experience, in order to create a better sense of belonging for undergraduate students – but to do this well also requires appropriate resourcing and may require lengthy iterative processes. There is also limited incentive for human staff to work on such projects as they might feel that such chatbots will become an automated replacement for their own positions.
