Annotation for professional development, collaboration, and critique

Centering scholarly communication in the discussed text.....

The first project to mention is: The Marginal Syllabus: Educator Learning and Web Annotation across sociopolitical texts and contexts; see also the The Marginal Syllabus webpage

Why is it important?

It demonstrates that annotation can function as an outlet for professional development, in being:

  • contemporary and relevant (pick a recent paper, get a group together, annotate!!!)
  • collaborative (varied voices of critique, potentially internet-wide) as opposed to top-down/authoritative

The marginal syllabus has a double entendre in that the paper selection has focused on issues pertaining to marginalization in education, and the commentary on those works occurs in the margins of the papers/blogs/websites, using hypothes.is web annotation. As with any initiative, the impact varies on the level of participation and engagement of the annotators.

Annotation in peer review/post-publication peer review

Most academics are well aware of the peer review process, and have been frequent referees or recipients of critique. Is a once-and-for-all review occuring prior to publication sufficient? As time passes, the work might be seen in a different light (better or worse), and education research can always benefit from context (how does research finding X work in Beijing vs. Berlin vs. Boston?). This is where additional post-publication peer review (or PPPR) can come into play.

Great work on PPPR comes from:

Jane Hunter's Post-publication peer review: opening up scientific conversation. This provides the viewpoint of the Facutly of 1000 (F1000), which engages in open, post-publication review with named reviewers, and "version control" of what is published. If you are new to this journal or approach, go see how much insight you derive from reading the reviews that are included alongside a paper. These public reviews also furnish an opportunity for the reviewers to showcase a fair, yet critical, approach, and be viewed as helpful collaborators in bringing a better version of a paper into existence. To see how PPPR occurred with an F1000 research paper on annotation, see: Using social annotation to construct knowledge with others: A case study across undergraduate courses. There are multiple versions there, and reviewer comments are there for all to see.

For an lengthy, in-depth discussion on PPPR, see Nikolaus Kriegeskorte's: Open evaluation: a vision for entirely transparent post-publication peer review and rating for science. I'm a huge fan of the quotation in the introduction:

"A scientific publication system needs to provide two basic functions: access and evaluation. Access means we can read anything, evaluation means we do not have to read everything".

In regards to access, the idea that government (tax payer)-funded research work resulting in publications, should not then be placed behind a paywall, has existed for some time. University pre-print depositories have freed up some work that might be paywalled at a journal, and those in the scientific field might have heard of the SciHub depository.

In regards to evaluation, the idea is a universal one of separating the "wheat" from the "chaff" and highlighting high-quality publications for consumption in our limited time. PPPR work could itself become a citable document, and something that an academic takes pride in (as opposed to ending up in a trash can, as has been the case for pre-publication peer review)

closed reviews into trash can

CC by 4.0 Figure 1 The Current system by Nikolaus Kriegeskorte

Many are dissatisfied with the status quo surrounding publication (hidden reviews that end up in the trash). Post publication reviews like PubPeer, have found an audience of watch dogs, who identify "chaff", leading to some high profile paper retractions. One wonders whether a directly text-linked/marginal interface for PPPR (as could be provided via hypothes.is), would be favorable.

For work on PPPR that takes hypothes.is into account, see Heather Staines': Digital Open Annotation with Hypothesis: Supplying the Missing Capability of the Web

Why is it important?

  • Provides a history of hypothes.is and states its driving raison d'etre: conversations should take place where the content lives online, and not relegated to other places (blogs, Twitter, Reddit, PubPeer).
  • discusses the integration of hypothes.is with multiple publishers (i.e. eLife)
  • gives the author's view of how annotation can facilitate the process of scholarly reading (for later writing.....)