Two weeks ago I attended the 5th ACM Learning Analytics and Knowledge Conference in Poughkeepsie, NY, USA to present my short paper titled: “Correlations between automated rhetorical analysis and tutors’ grades on student essays”. This paper was written jointly with our research collaborator Dr. Agnes Sandor from Xerox Research Centre Europe, and my supervisors Drs. Rebecca Ferguson, and Anna De Liddo, Prof. Denise Whitelock and my ex-main supervisor Prof. Simon Buckingham Shum.
When assessing student essays, educators look for the students’ ability to present and pursue well-reasoned, strong, valid and logical arguments, and for their ability to use examples, evidence and personal interpretations for and against a particular position throughout a written piece. Such scholarly argumentation is often articulated by rhetorical metadiscourse. Educators will be necessarily examining metadiscourse in students’ writing as signals of the intellectual moves that make their reasoning visible. Therefore students and educators could benefit from available powerful automated textual analysis that is able to detect rhetorical metadiscourse. However, there is a need to validate such language technologies in higher education contexts, since they were originally developed in non-educational applications. This paper describes an evaluation study of a particular language analysis tool, the Xerox Incremental Parser (XIP), on undergraduate social science student essays, using the mark awarded as a measure of the quality of the writing. As part of this exploration, the study presented in this paper seeks to assess the quality of the XIP through correlational studies and multiple regression analysis. As a result, we found statistically significant correlations between the output of the XIP and grades of student essays.
Although I was the last presenter in the last session of the last day of the conference (I know!), luckily there was a reasonable number of audience who vividly participated with their questions followed by after-talk. I established some good connections with researchers around the world who share similar interests with me; hoping to update you regarding these collaborations in near future. All in all, I personally found LAK’15 totally interesting especially with the high volume of talks on discourse and writing analytics. From one day pre-conference workshop in LAK’13 to full 1,5 day of main conference talks on this area really demonstrates the trend towards discourse. Conference organisers also mentioned the trend towards discourse in LAK submissions over the past few years which is exciting! Even one of the keynotes was directly related with automated analysis of written text, discourse analysis and writing analytics. On the last day, Danielle McNamara gave her keynote “Getting a Big Picture from Big Data: Use Your Words!” and talked about her interests in discovering new methods to improve students’ ability to understand challenging text, learn new information, and convey their thoughts and ideas in writing. She mentioned various approaches and methodologies have been used in her lab including the development of game-based, intelligent tutoring systems (e.g., iSTART, Writing Pal), and the development of natural language processing tools (e.g., iSTART, Writing Pal, Coh-Metrix, the Writing Assessment Tool). Basically, day 2 & 3 were filled up with various sessions on discourse, discourse and discourse. Below is the list of papers presented in this area that I also attended:
- 4B: Writing and Discourse Analysis
- BECOMING STRATEGIC WRITERS: CAPTURING AND DISPLAYING STUDENTS’ STRATEGIES FOR WRITING TO ENHANCE FEEDBACK (PRACTITIONER) Ge Vue and Tracey Hall
- YOU’VE GOT STYLE: DETECTING WRITING FLEXIBILITY ACROSS TIME (FULL) Erica L. Snow, Laura K. Allen, Matthew E. Jacovina, Cecile A. Perret and Danielle S. McNamara
- PSSST… TEXTUAL FEATURES… THERE IS MORE TO AUTOMATIC ESSAY SCORING THAN JUST YOU! (SHORT) Scott Crossley, Laura K. Allen, Erica Snow and Danielle S. McNamara
- OPENESSAYIST: A SUPPLY AND DEMAND LEARNING ANALYTICS TOOL FOR DRAFTING ACADEMIC ESSAYS (SHORT) Denise Whitelock, Alison Twiner, John T.E. Richardson, Debora Field and Stephen Pulman
- 5B: Text & Discourse Analysis
- TOPIC FACET MODELING: SEMANTIC VISUAL ANALYTICS FOR ONLINE DISCUSSION FORUMS (SHORT) I-Han Hsiao and Piyush Awasthi
- EFFECTS OF SEQUENCES OF SOCIALLY REGULATED LEARNING ON GROUP PERFORMANCE (SHORT) Inge Molenaar and Ming Ming Chiu
- DEVELOPING A MULTIPLE-DOCUMENT-PROCESSING PERFORMANCE ASSESSMENT FOR EPISTEMIC LITERACY (SHORT) Simon Knight and Karen Littleton
- ARE YOU READING MY MIND? MODELING STUDENTS’ READING COMPREHENSION SKILLS WITH NATURAL LANGUAGE PROCESSING TECHNIQUES (FULL) Laura K. Allen, Erica L. Snow and Danielle S. McNamara
- 7B: Curricula, Network and Discourse Analysis
- DISCOURSE COHESION: A SIGNATURE OF COLLABORATION (SHORT) Mihai Dascalu, Stefan Trausan-Matu, Philippe Dessus and Danielle McNamara
- And me 🙂
CORRELATIONS BETWEEN AUTOMATED RHETORICAL ANALYSIS AND TUTORS’ GRADES ON STUDENT ESSAYS (SHORT) Duygu Simsek, Agnes Sandor, Simon Buckingham Shum, Rebecca Ferguson, Anna De Liddo and Denise Whitelock
I’ll be posting soon regarding the review of some of these fantastic talks!
It would be unfair to finish this post without saying how wonderful the host was. Conference was professionally held in Marist College, Poughkeepie which had lovely oak, wood decor everywhere and some of the rooms welcomed the nice view of misty air above the frozen Hudson River! I should also thank to all student-volunteers of Marist College as they were real assets in spotless organisation with no doubt! Anyway, overall it was lovely meet with new people and catch up with some old friends from LAK’14 as well as my previous main supervisor Simon Buckingham Shum, who left the OU after 20 years to establish a new research centre. I found a chance to update him regarding the last 7 months of my PhD that he missed as we never had a real chance to talk since he left for Oz at all. I learned that he also keeps continue to work on my research area over Aussie with new researchers (don’t know whether this is a good thing or not but it urges me to finish ASAP for keeping the originality of this work!). If you are interested, which I assume as you’re reading my page 🙂 you can read about UTS’ Evaluating Writing Analytics post here.
At last but not least, LAK’16 will be held in one of my favourite cities, Edinburgh, UK 🙂 So I’m glad it doesn’t require much budget to fly overseas and hopefully there will be more local attendees who couldn’t join us in the previous years. My greates wish of course is to join LAK’16 as Dr. Duygu to present the outcomes of my PhD work.
Here are some useful links:
- My short paper: http://oro.open.ac.uk/42042/
- Slides of my talk: http://www.slideshare.net/dsimsek/lak15-short-paper-talk
- YouTube video of my talk: Will be available soon in SoLAR’s youtube channel
- Read about OU work at LAK’15: http://kmi.open.ac.uk/news/article/18702
- LAK’15 Twitter Stream: Check out #lak15 for getting really good insights about the sessions
- Conference WebPage: http://lak15.solaresearch.org/