Member spotlight: From psychometrics to advocating for ethical assessments – an interview with Tonia Parker

Could you tell us about your current role and the nature of your work on the student assessment and program assessment there?

"I am Department Chair for Psychology for American Public University System. We currently have a bachelor’s program and a master’s program in general psychology. As an online university, our courses are what we refer to as 'turnkey.' A faculty member who is a subject-matter expert develops the content for the class, including assessments. Once the class is developed, all faculty teaching the course teach it from that template. They can add to what is in there, but they cannot take away from it. I oversee the process of the courses being developed as well as how they are taught.

In terms of assessment, I obviously have oversight over how students are assessed in their psychology classes. I am also responsible for evaluation of my program to ensure we are meeting program and institutional learning objectives. We do that in part through mapping of course assignments to those objectives and using student performance on those assessments as an indication of how they are doing on the program and institutional learning objectives."

You have a background as a counsellor, is there anything from that side of your practice that you reflect on in your assessment practice?  

"As a counselor, one of the required courses in my training was a course in tests and measurements. While this was focused on administration of standardized psychological and career assessments, it did introduce me to the concepts of test validity and reliability. I have administered some measures in clinical practice, but that is different from developing assessments or the use of assessments in education. 

That is a big part of the reason why I started taking the trainings offered by Cambridge’s Assessment Network. It was the only place outside of a graduate program in something like quantitative psychology where I could find training in psychometrics (A104) that went beyond just an overview of the concepts."

You’ve spoken on the member forum about your passion for the ethical use of data in assessing students. Could you tell us a bit more about this? 

"It should be no secret to anyone at this point that there is very little information about us that isn’t online in some fashion, but some people may be surprised at how it might be used and how intrusive that can be. 

I came across a situation a few years ago where a program was going to be implemented as a way to evaluate student and faculty engagement. To shorten a very long story, the program was going to generate a five-factor personality assessment based on a text analysis of things students and faculty posted in the classroom, and it was going to be used to determine which students were at risk and likely to respond to intervention, and how friendly faculty appeared to be in their interactions with students. It was also going to be implemented without informing students or faculty. 

I had questions about the science behind it. I also had serious concerns about the idea of doing a personality assessment on people without their knowledge or explicit consent. To be clear, it was the institution considering implementing it that did not plan to inform students or faculty and not a condition of the program itself. 

Fortunately, the program was not implemented at that institution, but it really opened my eyes to things like function creep and issues around consent. Here in the U.S., we have a lot fewer protections than what you see in Europe with GDPR. I am currently advocating for my organization to develop a data code of ethics and encourage others to do the same."

Did you have any ‘lightbulb moments’ as a result of studying on A104? What were your key takeaways from the course?

"My biggest lightbulb moment from that class, and some of the other trainings I have taken through Cambridge Assessment Network, is realizing how little I knew about developing assessments. 

I don’t know how things are in other countries, but here in the U.S. faculties in higher education get little to no training on how to teach, much less how to develop an assessment. We tend to assess students in the same way that we were assessed without looking at whether that is the best or most valid way to do it. 

I really appreciated learning about classical test theory and item response theory as well as parallel tests. Between A104 and the Effective Assessment Design & Delivery in Higher Education course that I took, I am taking a different look at how my department does assessment and planning to work with my faculty on improving that process."

Do you think of yourself as having an ‘assessment identity’ and if so, what has informed that?

"I think that I am working on expanding my assessment identity as I expand my assessment literacy. From a Bloom’s taxonomy perspective, I think I am moving from an application level with some analysis to being better able to evaluate and create new and better assessments. 

I’m also getting to be in a better position to advocate for better and more ethical assessments because I am in a better position to demonstrate why an assessment may be deficient or why another assessment might be a better choice. 

That is a big part of what started me down this road. I intuitively knew that some of the assessments we were using lacked validity or the ability to discriminate, for example, but now I am in a position to better support my argument with evidence."