Assessing students' knowledge has always been a significant challenge for both students and teachers. Beyond the traditional method, which relied solely on pen-and-paper tests and students' memories, there are other approaches to testing. An "open-book exam" is a test where students are allowed to use their textbooks or notes to find the answers, which can seem more appealing. It may create the impression that they need to know less or even nothing to pass. However, many students find that this is not the case during the actual test. I see some analogy when students are allowed to use AI tools during the tests.

I recently experienced a similar situation when teaching a course on Data Visualization to 91 Chinese students. After conducting 16 online classes, I traveled to China to deliver the remaining classes in person, with tests also administered in the classroom. The course focuses on creating powerful visualizations and dashboards using specific software (Tableau from Salesforce); it is more hands-on than theoretical. To gain new insights or reveal hidden patterns in the different datasets, students must be familiar with the software's features and have some proficiency as well. Acquiring new knowledge in a foreign language is always a significant challenge; thus, to address language barriers and the challenges of intensive learning (four classes per day), I was forced to allow the students to use AI tools during the tests. Given the difficulty of preventing students from accessing AI, they were permitted to use it during the second test. However, I emphasized that without a solid understanding of the software, AI tools would not be effective in creating graphs, charts, and dashboards.

It was clear that students feel themselves much more self-confident with the possibility of using AI. During the assessment of their work, I collected and classified the typical traces of AI use. I identified three levels of unsuccessful AI assistance in the students' work.

The lowest level is when students received detailed instructions about how to create a visualization. This means that AI was familiar with the user interface and could provide a step-by-step description for the students. Unfortunately, some of them copied this information as an answer. These students were unable to follow the instructions, which demonstrated a lack of basic software skills.

The second level is when students received detailed instructions and tried to follow them. The time limit was a restriction for them, as it took them much longer to understand the steps and locate the menu items. The quality of their solution depended on their level of knowledge.

Students from the top level asked for help from AI to solve smaller problems while building the visualizations. Appropriate prompting played a crucial role in getting the right help. Prompting skills are related to students’ knowledge. Some results of bad prompting: AI suggested creating calculations, although the aggregation function was available, or data filtering was achieved with embedded calculations.

In my experience, artificial intelligence cannot fully replace students' knowledge, although it may take time for students to realize this. Educators must adapt to this new situation by revising their testing methods to accurately measure students' true skills.