Knowledge Construction through annotation: Categorizing annotation output

After your students have annotated a text in your course, how could you go about classifying those annotations?  Do you have a classification scheme in mind, or should you adapt from others? There is an annotation Knowledge Construction categorization scheme that has appeared in multiple papers thus far…….

Most recently in Morales, E., Kalir, J. H., Fleerackers, A., & Alperin, J. P. (2022). Using social annotation to construct knowledge with others: A case study across undergraduate courses. F1000Research, 11.

KC activity analysis in annotations

These same seven Knowledge Construction categories appear previously in: Plevinski, J., Weible, J., & DeSchryver, M. (2017). Anchored annotation to support collaborative knowledge construction. Philadelphia, PA: International Society of the Learning Sciences.

Plevinski et al table 1 segment

They also appear in Eryilmaz, E., van der Pol, J., Ryan, T., Clark, P. M., & Mary, J. (2013). Enhancing student knowledge acquisition from online learning conversations. International Journal of Computer-Supported Collaborative Learning, 8, 113-144.

Eryilmaz Framework

And they first appear in a slightly expanded form (10 categories) in Gay, G., Pena-Shaff, J., & Martin, W. (2001). An epistemological framework for analyzing student interactions in computer-mediated communication environments. Journal of interactive learning research, 12(1), 41-68.

10 categories of KC classification

Of the four works, the Morales paper featured the largest annotation corpus (2121 annotations), and uses a platform that is widely accessible now (hypothes.is). Another recent paper features a similarly sized corpus of annotations (~2,200 annotations), with longitudinal analysis for output, % isolated and threaded annotations, subjective annotation quality, and additional qualitative feedback from students regarding collaborative annotation.

In the courses analyzed by Morales et al., Interpretation was the most common type of Knowledge Construction activity, followed by Elaboration, and Clarification. Conflict was rare.

Morales et al KC activities all courses

If you are analyzing annotation output of different courses, or isolated versus threaded annotations, this furnishes the opportunity for comparison of the Knowledge Construction category percentages within those sub-categories. The authors did both of these analyses.

Future analysis could be done:

  • over time (……how do annotation Knowledge Construction activities change in a course from assignment 1 to assignment 5?)
  • over different group compositions
  • over different group sizes
  • over different institutions

76% of student social annotations in the Morales et al. study received no peer responses and were thus not part of any threaded discussion. In one of the courses analyzed, only 6% of student social annotations were in threads. If you are using an annotation platform, how does your course compare?  Many are familiar with the quotation: If a tree falls in the forest, and no one is around to hear it, does it make a sound? Those in the annotation research community can consider: If an annotation remains solitary, has it been read? Has it had an impact?.......these are interesting questions to consider……..

Morales et al. provide pedagogical tips at the end of their manuscript……..

“Aiding Instructor knowledge of student social annotation through efficient feedback processes would also help inform the ways in which instructors participate in online discussion to clarify student misunderstandings and build connections to relevant disciplinary literature and methods.”

  • Instructors might need help monitoring annotation output, so they can shape it constructively
  • Instructors in this study contributed only 1% of the corpus, so perhaps future instructors could use a nudge……

“Instructors model and transparently assess how SA enables productive, discipline-specific online discussion in accordance with course learning objectives”

  • This is a good reminder for instructors to ask themselves whether they have constructive alignment between collaborative annotation and course learning objectives……

The categories originally articulated by Gay et al. (2001) manuscript, and re-appearing in the Erilymaz (2013), Plevinski (2017), and Morales (2022) manuscripts are logical and seem reasonable to implement. The annotations have been coded by humans in these works, but perhaps some technological help is on the way for future annotation classification. Shared classification approaches among researchers will support replication and comparison across courses, disciplines, and institutions – a good thing!