语言测试学资料3

1 Chapter 3

(第三章)

The Reliability of Testing

(测试的信度)

• The definition of reliability

• The reliability coefficient

• How to make tests more reliable

What is reliability?

Reliability refers to the trustworthiness and stability of candidates‟

test results.

In other words, if a group of students were given the same test twice

at different time, the more similar the scores would have been, the more

reliable the test is said to be.

How to establish the reliability of a test?

It is possible to quantify the reliability of a test in the form of a

reliability coefficient.

They allow us to compare the reliability of different tests.

The ideal reliability coefficient is 1.

---A test with a reliability coefficient of 1 is one which would give

precisely the same results for a particular set of candidates regardless of 2 when it happened to be administered.

---A test which had a reliability coefficient of zero would give sets of

result quite unconnected with each other.

It is between the two extremes of 1 and zero that genuine test

reliability coefficients are to be found.

How high should we expect for different types of language tests?

Lado says

Good vocabulary, structure and reading tests are usually in the 0.9 to

0.99 range, while auditory comprehension tests are more often in the 0.8

to 0.89 range.

A reliability coefficient of 0.85 might be considered high for an oral

production test but low for a reading test.

The way to establish the reliability of a test:

1. Test-retest method

It means to have two sets of scores for comparison. The most

obvious way of obtaining these is to get a group of subjects to take the

same test twice.

2. Split-half method

In this method, the subjects take the test in the usual way, but each

subject is given two scores. One score is for one half of the test, the

second score is for the other half. The two sets of scores are then used to

obtain the reliability coefficient as if the whole test had been taken twice. 3 In order for this method to work, it is necessary for the test to be spilt into

two halves which are really equivalent, through the careful matching of

items (in fact where items in the test have been ordered in terms of

difficulty, a split into odd-numbered items and even-numbered items may

be adequate).

3. Parallel forms method

(the alternate forms method)

It means to use two different forms of the same test to measure a

group of students continuously or in a very short time. However, alternate

forms are often simply not available.

How to make tests more reliable

As we have seen, there are two components of test reliability: the

performance of candidates from occasion to occasion, and the reliability

of the scoring.

Here we will begin by suggesting ways of achieving consistent

performances from candidates and then turn our attention to scorer

reliability.

1.Take enough samples of behavior

Other things being equal, the more items that you have on a test, the

more reliable that test will be.

e.g.

If we wanted to know how good an archer someone was, we 4 wouldn‟t rely on the evidence of a single shot at the target. That one shot

could be quite unrepresentative of their ability. To be satisfied that we had

a really reliable measure of the ability we should want to see a large

number of shots at the target.

The same is true for language testing.

It has been demonstrated empirically that the addition of further

items will make a test more reliable.

The additional items should be independent of each other and of

existing items.

e.g.

A reading test asks the question:

“Where did the thief hide the jewels?”

If an additional item following that took the form: “What was

unusual about the hiding place?”

Would it make a full contribution to an increase in the reliability of

the test?

No.

Why not?

Because it is hardly possible for someone who got the original

questions wrong to get the supplementary question right.

We do not get an additional sample of their behavior, so the

reliability of our estimate of their ability is not increased. 5 Each additional item should as far as possible represent a fresh start

for the candidate.

Do you think the longer a test is, the more reliability we will get?

It is important to make a test long enough to achieve satisfactory

reliability, but it should not be made so long that the candidates become

so bored or tired that the behavior that they exhibit becomes

unrepresentative of their ability.

2. Do not allow candidates too much freedom

In general, candidates should not be given a choice, and the range

over which possible answers might vary should be restricted.