CAICCAIC: Center for Artificial Intelligence Challenge on Conversational AI Correctness

Challenges for Natural Language Processing (CNLPS)

The goal of this competition is to develop Natural Language Understanding models that are robust against speech recognition errors. Regardless of near-human accuracy of Automatic Speech Recognition in general-purpose transcription tasks, speech recognition errors can significantly deteriorate the performance of a Natural Language Understanding model that follows the speech-to-text module in a virtual assistant. The problem is even more apparent when an ASR system from an external vendor is used as an integral part of a conversational system without any further adaptation. The contestants are expected to develop NLU models that maintain satisfactory performance regardless of the presence of ASR errors in the input.

The data for the competition consist of natural language utterances along with semantic frames that represent the commands targeted at a virtual assistant. The detailed specification of the task and the description of the evaluation procedure are available here.


The solutions for the task are to be submitted via the Gonito platform challenge. In order to be included in the final ranking the participants are expected to provide the report that describes their solution. The reports should conform to the requirements for papers submitted to FedCSIS conference and should not exceed 4 pages.

Presentations and publication

The authors of selected submissions will be invited to prepare the extended versions of their reports for publication in the conference proceedings and presentation at FedCSIS 2023. The selection will be made by a Jury on the basis of final evaluation results, quality of the submitted reports and originality of the presented methods.

Important Dates

  • Feb 13, 2023: Training data available (English)
  • Feb 20, 2023: Training data available (other languages)
  • May 17, 2023: Test data available
  • June 21, 2023: Deadline for submitting the results
  • June 23, 2023: Announcement of the final results, sending invitations for submitting papers
  • July 9, 2023: Deadline for submitting invited papers
  • July 16, 2023: Author notification
  • July 31, 2023: Final paper submission, registration
  • Sept 20, 2023: FedCSIS conference


  • Marek Kubis, Adam Mickiewicz University, Poland
  • Paweł Skórzewski, Adam Mickiewicz University, Poland
  • Marcin Sowański, Samsung Research Poland
  • Tomasz Ziętkiewicz, Samsung Research Poland

Important dates

  • Track proposal submission: November 14, 2022
  • Paper submission (no extensions): May 23, 2023
  • Position paper submission: June 7, 2023
  • Author notification: July 11, 2023
  • Final paper submission, registration: July 31, 2023
  • Discounted payment: August 18, 2023
  • Conference date: September 17⁠–⁠20, 2023

FedCSIS 2023 is organized by

Under patronage of

Warsaw University of Technology

Prof. Krzysztof Zaremba
Rector of Warsaw University of Technology