I’m doing the tab for GW this weekend, and I regret it already. Tabbing makes me feel terrible on every level: I feel like I’m wasting my time, I feel like I am ignoring my students, but worst of all, when I look to the way judges are ranked and assigned in competitions, I feel that BP debate tournaments really do not belong in the realm of university or collegiate activity. There is no other metric than familiarity with which judges are ranked and assigned.
If there is a metric, judges are assigned by the metric of tournament success, which is defined by the CA Class as wins and awards. There is no other way to understand success at these competitions. If someone is unfamiliar but has won a number of competitions, that person is placed in a position of power in the competition. If they have not won, but are familiar, the same. But what about those who do not win and are unfamiliar to the CAs?
The CA Class has no method for dealing with this. I wonder how many amazing people turn up to judge competitions for different reasons, but serve as wings and are never taken seriously by their chairs. I hope it’s a small number. But it really shouldn’t be any number. There are ways that this bias could be dealt with.
Feedback forms are not the answer as they are always read through confirmation bias. The forms are built in with numerous levers to pull to jettison their validity. They are not treated that seriously anyway by anyone, except really angry teams that were certain they were right and their judge wasn’t.
There’s no incentive for judges to try out new approaches or ways of judging debate, no motive for including new information or approaches to debate at all in the present system. The only motive for those who want to judge, chair, and sit on elimination panels is to listen and learn and conform to the ways that the top judges behave. There are no opportunities for exception. Conform or be relegated to the borders of the tournament. What is most vital here is that learning how to judge is conflated with conforming to the norms of judging demonstrated by those most familiar to the group of CAs at the competition, or to those who demonstrate they can meet those norms via a trophy or three.
Most of you probably have no reason or incentive to reverse this system, but here are some ideas for reversal that could make things much more interesting
International/National Database of Rankings
This would be a source of information where judge rankings are stored from all tournaments and then averaged across the year, the season, or geography. CAs could examine these rankings and assign them accordingly to unproven and unfamiliar judges at competitions they control.
Debate Leaders/Professors/Coaches Poll
Used as a metric, this monthly poll by those who are the leaders, CAs, coaches, professors, and teachers of debate recognized as a leader in their debate society offers the rankings per month of judges. Judges submit their names for consideration for the poll at the start of the season. This is a non-binding measure of what people think and feel who are not always in the driver’s seat on assigning rankings.
The People’s Champion
Debaters should start something similar to rate my professor for judges. This could be another great guide to get the other side of the judge’s ranking and what the teams who had this person, or the chairs who worked with this person, thought about the judge’s ability. Another model would be the now outmoded PIck A Prof (I wonder if it is still around in some form) where data which is aggregated is offered and the user can look at it in any arrangement they wish to help their decision (grade distributions, number of assignments, pages per week, etc). All this data was scraped from the university wesbite where faculty were required to post syllabi, etc. Something similar could be done for BP tournaments.
When I think about judging, there is no other thing it can be in my mind than an opportunity to teach. We often forget this when assigning judges, so we must keep that notion in our heads when assembling panels. You might not agree with me, but you have to concede that some part of judging involves teaching. It’s happening whether you want it to or not, and it’s happening at some level, even if you are just rehearsing the arguments of the debate.
As an opportunity to teach, we are right to examine teacher qualifications as another area to determine who should judge. I can’t imagine my university staffing positions based on awards, grade distribution, or the fact that the Dean knows them really, really well and they are really a great teacher. That’s not good enough. We should work to develop deeper metrics that are more worthy of the power and potential of BP debate, and stop relying on methods that to anyone outside of the organization look like collusion and favoritism.