© 2006, Colm Flynn. All rights reserved.
Page design by GetTemplate
Guide to chairing and adjudication
This booklet is intended as a guide, to assist you in performing effectively in your principle role as an adjudicator in this competition, and to help you fulfil the other important responsibilities that are likely to be asked of you. These include things like: convening and chairing a debate, keeping time, conducting a post-debate adjudicators' discussion, and finally giving feedback and results to debaters. We understand that every experienced adjudicator will have developed an individual method for the way in which he or she runs a debate, records that debate, and gives feedback to teams and individual speakers. We are also aware that the type, quality and duration of experience will vary considerably from one individual adjudicator to another in a tournament of this type.
For the first time, in 1998, adjudicators were asked to give an oral
adjudication, or feedback, at the conclusion of each of the first six rounds of
debating (preliminaries). It is designed to accomplish a number of objectives;
all of these being established by Council in response to the needs of debaters
as they have been expressed over the years. The first of these relates, of
course, to the development of better debating. It has been a criticism of the
World's format in the past that debaters, teams and coaches have almost no
access to the kind of constructive criticism that would allow them to hone their
skills during the preliminary rounds of the competition. Moreover, teams could
only guess at how well they were doing during this stage, based primarily on the
kind of company they were debating in as the early rounds progressed.
With the introduction of an oral adjudication, delivered by the Chairperson, at
the end of a debate, the debaters will know their finishing position (first to
last) and the points (3 to 0) that they will have been awarded for that
particular debate. Similarly, the adjudication will indicate how and why the
adjudicators have arrived at their decision and precisely what teams and
individual speakers did well, and what they did not do so well (constructive
criticism). The oral adjudication then provides debaters with exactly the kind
of constructive criticism that they need.
The second group of objectives relates to the development and refinement of
adjudication at Worlds. Oral adjudication provides an insight into the way that
adjudicators observe and adjudicate debates; one that will not only benefit
debaters, but also adjudicators. The discussion leading to the decision-making
stage gains a new importance as the Chair now has to advance the collective
opinions of the panel in order to justify the unanimous or consensus decisions
that are made when the feedback is given. This encourages all of the
adjudicators on a panel to be particularly considerate and careful in the
processes of observation, recording, decision-making, justification and
tabulation.
In every major international competition these days, all those registered as
adjudicators for the duration of the competition will have to attend a
seminar/workshop. It is important that you attend this seminar, even though you
have a wealth of experience in World's adjudication. This is because the Chief
Adjudicator for the competition will have certain specific things that he/she
will want you to focus upon in your adjudication and, as these will differ in
perspective from previous competitions that you have attended, you will need to
know them too.
Similarly, once you register as an adjudicator, you can expect to adjudicate in
all of the preliminary rounds of that competition. If you are adjudicating well,
and the feedback that the Chief Adjudicator's Panel is getting on your
post-debate discussions is good, then you might be honoured with selection to
adjudicate after the break. In this light, once you register as an adjudicator,
you should commit yourself to acquitting that responsibility until the Chief
Adjudicator indicates that your services are no longer required. This means
turning up to every briefing on time and in an appropriate physical and mental
state.
As an adjudicator, you should take some to familiarise yourself with the
rules of the competition. Any questions that you think you might want to ask
during the seminar should be noted down, no matter how silly you might think
them to be. Even if you don't ask them during the seminar proper, you can always
approach one of the adjudication panel immediately after the seminar is over.
Each of these pre-competition seminars will end in an examination or test. This commonly takes the form of an adjudication of a live exhibition debate, staged there and then, or the observation and assessment of prerecorded videotape of a selected World's style match. At the end of the test-debate, you will be given some time to go through your notes, arrive at a decision (finishing positions) and then give your justification for this in written form. Your familiarity with Worlds’ Rules will also be tested. This will result in your name joining a pool of adjudicators with similar levels of skill, something which will in turn permit the Panel adjudicator in charge of the adjudicators tab to balance the panels (members) in terms of experience and skills.
Adjudicators should get into the habit of carrying around what might be
recognised as 'the tools of the trade', or an adjudicator's kit, if you prefer.
At the very least, this must consist of a pad of paper and a writing implement.
A watch is fairly essential. You should have a digital watch if no stopwatch is
available to you, just so that you can time speeches for yourself.
You will be part of a briefing that precedes each and every round. This is your
opportunity to ask the Chief Adjudicator and members of his/her panel for any
further clarification of the rules, their application and for help in solving
any problems that you are having in your adjudication of your rounds. This also
an opportunity to address your particular concerns to that same panel.
Similarly, listen to any announcements regarding adjudication processes that are
made during these briefings.
At some time during the briefing, the match-ups will be either displayed on a
screen (via OHP transparency or 'Power Point' slides), or photocopies of the
draw will be handed out. These lists will tell you which room you will be
adjudicating in, who will be on the panel with you and which one of you will be
chairing the panel. You will also know which teams you will be adjudicating and
the respective positions that they will be debating in.
The other things that you should consider, as the list of matches is revealed;
At the appointed time, the motion will be released to both debaters and
adjudicators. You should write this down as well, checking tomake sure that you
have the exact wording, as it is given. Debates should commence fifteen minutes
after the motion has been announced (Worlds’ Rulebook 3:1.3), so you should
arrive at the venue of your match at least two or three minutes before that.
When the time has come for the debate to start, the Chair of the panel of
adjudicators should start things off by calling teams into the room and saying
something like, "I call this house to order". The Chair may then make
some opening remarks.
The panel member responsible for timing speeches starts his or her watch as soon
as the speaker starts speaking (not as soon as he or she stands up, clears the
throat or shuffles some papers).
From that point onwards, the debate progresses with speakers being thanked
for speaking by the Chair (functioning as nominal Speaker of the House) as they
conclude their speeches, and subsequent speakers being introduced by title,
position or name, or combinations of these, as their turn comes to speak.
The panel member responsible for keeping time should try to give clearly audible
signals (Worlds’ Rulebook 4:1.6). A sharp slap or knock on a flat surface
(such as a table or a book-rest) with the flat of the hand will normally
suffice. If a speaker begins to run overtime, it is not necessary to knock
continuously, or otherwise signal that the prescribed optimum time is being
exceeded. Good time management should be the responsibility of individual
speakers and their teams, not the timekeeper. In this regard, it might be a good
idea for the Chair of the panel to remind speakers during the opening commentary
that it is acceptable for speakers to receive time signals from their teammates.
Other than these invitations, thanks and time-signals, the adjudicators do not
interfere in the debate, being involved in taking notes which detail the process
and progress of the debate and observing those aspects detailed in the Worlds’
Rulebook 8-12: 3.1-4.4. The only time at which an interjection may become
necessary from the Chair of the panel is in the event that teams or individuals
are becoming unacceptably and inappropriately obtrusive during the speeches of
other members. This will be times at which the members not holding the floor
have begun to indulge in behaviour that amounts to things like heckling,
barracking and the advancement of otherwise malicious interruptions in the
speech of the member holding the floor.
It should be noted that these terms are subjective, and that the competition
attracts many different styles of debating which are acceptable and appropriate
in such a forum (Worlds’ Rulebook 11: 4.4.2). However, when adjudicators on a
panel begin to feel that the manner of members is becoming inappropriate in such
cases, then the issuance of a verbal warning to that effect, directed towards
the individual, team or bench that is behaving in such a way, allows those thus
warned to amend such behaviour before adjudicators begin to penalise them for
the perceived breach of debating decorum. At this point, the Chair may call for
“order” to be restored to the round. In this way, a clear signal is sent to
those verging on the offensive and they have the option to curtail that
behaviour before it begins to affect their own team's manner marks.
Remember that what is, or is not, acceptable to you in this context is largely a
matter of common sense, but it is better to send a clear signal to debaters in
danger of overstepping these bounds before it starts affecting their
marks/grades for the debate and allow them the benefit of the doubt up to that
point.
If you are concerned that someone has overstepped these bounds, whether
subjective or not, discuss this matter with the others on your panel at the
conclusion of the debate before you reach a hard and fast conclusion.
The note-taking/making process is an important one. Not only should such
notes provide you with a fairly complete description of the debate after it has
been concluded, it should also present you with concrete reasons why you have
reached your own particular conclusions as to how individual speakers and each
of the four teams has performed. You should try to record, for example, the
degree to which individuals we keeping in touch with the dynamics of the debate
through things like POIs and intersections. You should also be able to indicate,
within a particular speech, whether POls have been accepted, when, what they
consisted of and how the speaker holding the floor at the time responded to
them.
You should also be able to track the logic and flow of an argument or idea
through your own notation and determine whether statements have been left
largely unsupported (asserted) (Worlds’ Rulebook 8: 3.3), whether speeches
have a reasonable balance and are consistent (Worlds’ Rulebook 8: 3.3.3-3.3.4)
and whether speakers have misrepresented things said earlier in the debate,
among other things.
An individual adjudicator’s approach to note taking is likely to be markedly
different from person to person. The main thing is that you develop a means of
accurately charting what has happened in the debate.
The observation process is also important. You should be watching how readily
a speaker's manner develops a rapport with the audience (if any - or your panel,
if not), how she or he stands, gesticulates and is expressive during the
delivery of their speech. Similarly, you should watch for things such as how
members not holding the floor continue to communicate with each other during the
course of the debate and maintain contact with it through the POI and more
general interaction (appropriate reactions to statements being made; laughter,
etc.).
When the last speaker has concluded his/her remarks and retaken his/her seat,
it is customary for the Speaker, or Chair of Adjudicators (in the event that
he/she is taking the role of the Speaker of the House) to give the debaters
"permission to cross the floor". This is so that teams can shake hands
and congratulate each other on a successful debate.
It is pertinent at this point to tell members that they can withdraw while a
decision is made by adjudicators, in which case they must all withdraw until
asked to return to the room.
Either way, the discussion that is then held between adjudication panel
members is confidential, and its course and specifics should not be made known
to individual debaters. This confidentiality is essential if adjudicators are to
maintain a degree of professionalism, and neither to undermine, nor be
undermined by, their fellow adjudicators.
Consensus decisions are exactly that. Different adjudicators see debates in
different ways. That's exactly why we have panels of adjudicators. However, we
should avoid making individual perceptions about a particular debate, or a
particular adjudicator, common knowledge. This in no way restricts the kind of
advice that you may be asked for by a particular speaker or team: it merely asks
of you that you are considerate of your colleagues in advancing your own
comments and suggestions.
At the end of the debate, your panel begins the process of discussion and
decision-making. While the following is not presented as either a schedule or a
checklist for this process, it is clear that these major components will each
have to feature somewhere in the process of your deliberations.
The first thing that should happen, after the debaters, audience and television crew (it happens!) have left the room, is that the panelists should take a few minutes to review their notes before any form of discussion begins. During this 'quiet time' individual panelists should highlight items, arguments, comments and so on, that they consider to be critical in terms of the debate, its outcomes and their respective decisions.
Panelists should then move on to confer on grades for teams and speakers. You
should reach agreement on these things if you can, because it makes the work of
the tabulation crew that much less complicated, and they can look forward to
living longer and more productive lives. While the rules allow for a degree of
flexibility within the grade bandwidths that you have already decided upon,
you've managed to achieve consensus thus far, so why not push your luck a little
further!
One way to approach this is to try and agree on the standard of the debate as a
whole. As the power-matching software starts to spread things out nice and
evenly after about round three, you should find this progressively easier to do
as the competition goes on, because there should be an increasing level of
similarity in the strengths and skills of teams debating in each match. Remember
that you still have a little flexibility within a particular grade (or band) in
terms of the marks that can be awarded to an individual speaker, so you can
still use this range to reflect your own opinions. However, remember also that
the marks of the two speakers, when added together, must still equate with the
overall grade that has been agree for the team.
At this point, the panelists can begin to fill in their adjudication sheets, with perhaps one last communal cheek through what has been agreed and what the final decision is, just to make absolutely sure. It may also be a good idea at this stage for the Chair to ask for any points that the panelists would like incorporated into the oral adjudication of the debate.
While the broad categories of “matter” and “manner” serve as
touchstones for evaluation, they focus mainly on assessing the qualities of an
individual’s performance in the round. Manner is concerned with the style and
structure of a speaker’s presentation—delivery, organization, and language
use—while matter is concerned with the content and analysis of an
individual’s presentation—that a speaker’s arguments are relevant, logical
and consistent with his or her team’s or side’s positions.
While useful, these categories do not provide the adjudicator guidance on how to
weigh competing lines of argument. Though the adjudication of a Worlds round
does not require that the adjudicators declare a winning “side” in the
debate, the debaters involved will almost unfailingly be concerned with whether
the Government or Opposition prevailed on the question. Moreover, successful
adjudication of a round in the Worlds style—that is, the ranking of teams from
best to worst—must be concerned with the comparison of each team’s positions
relative to the other teams’ positions.
Adjudicators, both when adjudicating the debate and when articulating the
panel’s decision to the debaters in the oral adjudication, should pay
particular attention to three elements of the argument: the issue over which the
debate is contested; the standard by which the arguments on either side of the
issue may be assessed; and the appraisal of each team’s arguments relative to
that standard.
All debates may be characterized as a clash of arguments over an issue—some
statement that serves as the ideological dividing line between Government and
Opposition argumentative ground. Identifying the issue in a debate is the first
step toward successfully adjudicating the competing arguments in the round.
Ideally, the issue is made clear early in the debate, either by the motion or by
the Government. The majority of Worlds-style debates will have as their central
issue the motion as announced. This is particularly true when the motion is
exceptionally clear: “This house believes that making Yassar Arafat a partner
in peace was a mistake” or “This house would make company directors
criminally liable for the wrongs of their companies” are examples of motions
that define clear ground for the Government and Opposition and, therefore, serve
as the primary issue in the debate.
Other motions are less useful as issue statements. Motions that allow the
Government room to interpret the topic and define the focus of the debate are
less likely to function as issue statements. A motion such as “This house
believes that religious leaders should listen to public opinion” may be
supported by a general case in which a Government offers arguments that clergy
should be responsive to their followers or it may motivate a Government to run a
specific case that is derived from the motion. When presented with the motion
above, for example, the Government could choose to run a case that argues the
Catholic church should be more proactive in acknowledging and addressing issues
of sexual abuse of minors by Catholic priests. When the Government chooses to
define a case that is more specific than the motion offered, the central issue
in the round typically is the thesis of the case offered by the Government, not
the motion itself.
While the issue statement will usually be explicit in the round, there will be
cases in which neither side makes apparent the central issue in the round. In
this case, the adjudicator must articulate an issue as a starting point for his
or her adjudication. When doing so, an adjudicator should phrase an issue
statement that is clear and balanced. To be clear, an issue statement should
define ground for both the Government and Opposition team in a way that makes
obvious their responsibilities. A balanced issue statement will avoid expressing
the controversy in a way that might be weighted toward one side or the other.
In all decisions, the adjudicator will utilize some criterion or criteria to
make his or her assessment of the arguments advanced by the debaters. For
example, when adjudicating a debate on the motion “This House believes the
International Monetary Fund has done more to harm than help the global
condition,” an adjudicator must be able to determine how to evaluate
relatively the instances of the IMF “harming” and “helping” the global
condition as argued by the debaters. Should impact to local economies be
prioritized over facilitating the transition to a global economy? Should concern
for effects on the environment and workers’ rights be subjugated to the
long-term benefits of capitalism? How should the deterioration of state
sovereignty be weighed against the benefits of global trading opportunities?
The issue statement for the debate will usually contain some term or phrase that
will serve as the standard for evaluating the competing arguments of each side.
This term or phrase typically expresses some evaluation of the subject under
consideration. Consider the previously mentioned motion: “This house believes
that making Yassar Arafat a partner in peace was a mistake.” In this example,
the term that proposes an evaluation of Arafat’s performance is “mistake.”
In other words, to evaluate the competing arguments in the round, the
adjudicator will employ as his or her standard whether the examples of
Arafat’s performance offered by each side constitute a “mistake.” For this
term to function as a standard, the adjudicator must know what constitutes a
“mistake.” For the IMF motion, the adjudicator must understand what is meant
by “the global condition” before he or she may determine which side has best
substantiated the impact of the IMF on the global condition. Thus, the nature
and definition of the “global condition” becomes the standard for evaluating
the competing arguments.
In an ideal situation, the debaters would make clear the standard to be used to
weigh competing arguments in the round. The definition of the pertinent term or
phrase would be made clear by the Government side and their all arguments would
be made relevant to that definition. Similarly, the Opposition would recognize
the Government’s definition and orient their arguments toward that standard as
well. For example, if “mistake” in the Arafat motion was defined as
“anything that has served to impede the progress toward peace,” both the
Government and Opposition would align their arguments for Arafat’s influence
with an eye toward proving that his presence has affected—either positively or
negatively—the progress toward peace.
More typically, however, both sides in a round will have competing standards for
evaluating their arguments. With the IMF motion, the Government may defend the
escalating Gross Domestic Product of IMF beneficiaries while the Opposition may
argue that the austerity measures imposed by the IMF cause significant damage to
social programs. Without a clear standard advanced by either side, the
adjudicator is left to decide how to evaluate these competing positions. In
cases where the respective sides in the debate have failed to “agree” on a
particular standard, the adjudicator must determine the standard for evaluating
the competing arguments.
When determining a standard, the adjudicator should acknowledge the
Government’s responsibility to define the terms of the motion. Assuming that
the Government has defined the terms, particularly the term or phrase that will
serve as the standard for the competing arguments, some presumption should be
given their definition regardless of whether the Opposition chooses to orient
their arguments toward that standard. The criteria of clarity and balance
applied to the issue statement are also relevant to the definitions offered by
the Government: any definition of a standard should increase the clarity of the
debate and should not exclude the potential for Opposition argument.
Frequently, however, the Government will fail to define the pertinent term or
phrase and the adjudicator will be required to extract a standard for evaluation
from the arguments made by both sides in the debate. In the IMF example, the
arguments that a beneficiary country’s GDP has improved following IMF
intervention and that austerity measures have had a detrimental impact on the
social welfare of a country may both be true. The adjudicator must then decide
how to compare the competing arguments. Ideally, the debaters will give cues on
how to do so in their own argumentation. In this instance, the arguments
relevant in the round are focused on the effect of IMF involvement on the
beneficiary country. While this may not evaluate the consequences of IMF actions
on “the global condition” as most would understand the “global
condition” (i.e.: as more broad reaching than the effect of the IMF on a
single country) it is what the debaters have opted to focus on. To penalize
either side for failing to make the arguments the adjudicator believes would be
most appropriate is not sound adjudication.
In such a case, directed by the arguments the debaters have made, the
adjudicator may extract a standard of “impact to the beneficiary country.”
He or she would then evaluate competing arguments about the benefits or harms of
IMF involvement from the perspective of how those outcomes may affect a
beneficiary country.
Once a standard has been determined, the adjudicator must compare the
arguments made in the round to that standard. At this point, the adjudicator
should appraise each argument for its relevance to the standard. That relevance
may be measured in two ways.
Initially, relevance may be measured from a quantitative perspective. The
adjudicator may appraise a side’s arguments for the impact the totality of
those arguments has on the standard. More positions relevant to the standard,
using a strict quantitative perspective, mean that a particular side should
prevailed. If the Government offers five examples of how Arafat’s presence has
diminished the prospects for peace to the Opposition’s two examples of how
Arafat has improved the prospects for peace, the Government would likely
prevail. This perspective, however, has limited utility on its own.
Arguments must also be appraised from a qualitative perspective in which the
adjudicator assesses the significance of each argument’s impact to the
standard. Some examples or arguments will be more relevant the standard than
others. Building off the previous example, the adjudicator may believe that the
two arguments offered by the Opposition are more relevant to the
standard—perhaps those two examples of how Arafat benefited the peace process
were very detailed and specific whereas the Government’s five examples of
Arafat’s detractions from the peace process were vague and ill developed. In
this case, the Opposition would likely prevail.
Thus, a Worlds round might have a team present seven arguments, but have the
opposing team address all of them sufficiently with just one. The most important
point here is that the adjudicator account for each major line of argument
advanced by the Government and Opposition and assess the merit of each of those
arguments relative to the standard.
This approach to adjudication of the round is most useful for clearly
articulating a basis for decision in a verbal adjudication. Verbal adjudications
may be structured around these three concepts quite simply.
The Chair of a panel may open an oral adjudication by identifying the issue that
divided the Government and Opposition ground in the debate; this may be as
simple as saying “The central issue in the round was whether involving Arafat
in the peace process was a mistake.” From there, the Chair would articulate
the panel’s consensus as to the standard employed for evaluating the competing
lines of argument: “The Panel understood that whether Arafat’s involvement
would be considered a mistake depended upon whether he had contributed to or
detracted from the peace process.” Finally, the Chair would sort through the
major lines of argument advanced by each side to offer an appraisal of those
each of those arguments relative to this standard.
An oral adjudication structured around these concepts will provide the debaters
with the certainty that each of their respective arguments was weighed in the
adjudicators’ consideration. That certainty will, in turn, demonstrate that
the adjudicators were discharging their duties responsibly.
As with things like note taking, individual adjudicators will each have their
own way of giving an oral adjudication.
There is a division of opinion over whether it is best to announce results
first and then give the feedback, or whether to give the feedback first and then
announce the result. Our advice would be to adopt the former method, because it
is questionable how much benefit teams and speakers can get if they are
anxiously waiting for the result and you are, unconsciously perhaps, trying to
give nothing away.
You may like to preface your remarks with a few comments on the quality and
standard of the debate (coming from your discussions on an overall debate
grade?). You may also indicate whether there was a unanimous agreement, or
whether the panel encountered some resolvable disagreements in the course of its
discussion (thereby indicating that the match might well have been very close in
some respects).
As with the set-up for a debater's speech, an adjudicator's feedback should
have 'matter' and 'manner'. You should also 'structure' your own intended
feedback.
Give the finishing order, from team placing first in the debate (and therefore
&winning' it), to that placing last.
Then, proceed with the overview of the debate that your panel has assembled
during your discussions, but keep it brief. Focus on the definition, the
parameters and demands that this set up, the cases and major arguments that
followed this, the challenges that these represented and the way that these
challenges were met.
You should be able to trace the major issue(s) or themes that ran through the
debate through this overview, as well as focusing on the ways in which various
teams dealt with these.
It would then be a good idea to explain exactly why the debate has been
awarded to a particular team, and consider the positions of the other teams
relative to this. The reasons why teams have finished in the particular order
that you have determined should then follow, with the relevant explanations
offered as you go. You should conclude this phase by summarising what you have
said, but by means of reference to the key arguments and issues that you
outlined in your opening commentary. Comments about eye contact, off-key humming
and torn jeans are probably not appropriate at this point.
Your adjudication feedback might then move towards a conclusion with any specific comments on the roles, performance and style of individual speakers being offered. However, this should only be necessary in the event that an individual's speech has affected the debate, or a team's role, in a particularly critical way. Please try to keep your remarks in these cases constructively critical, perhaps softening what might be construed as negative criticism by picking out some positive aspects as well and mentioning them.
The main thing is that you enjoy the experience of adjudicating at Worlds and profit from this in the context of your own development as an adjudicator, and perhaps even as a debater. It comes down to one thing: common sense. If you continually apply that particular quality to the process of running, observing, discussing and assessing the debates that you will see, it will not only be you that gains. The debaters, the organisers and the competition that is Worlds Universities Debating will profit too.