ASSESSING SPEAKING

Speaking is a valuable skill to develop in language learners for communicative purposes. Thus, assessing speaking becomes relevant; otherwise, if we neglect assessing it, we would be sending a double message. Nevertheless, objective testing may be challenging due to reliability issues concerning possible subjectivity in grading it by trained (or not) raters, plus other issues like lack of time, number of students, administrative difficulties. But with the importance of Rnglish as a universal language, language teachers must test speaking progress in our students.

THEORY OF SPEAKING ASSESSMENT

Different abilities are simultaneously used when speaking. These are:

-Grammatical competence

-Discourse competence

-Sociolinguistic competence

-Strategic competence

CATEGORIES OF ORAL SKILLS

Oral skills can be categorized as routine skills, which are associated to spoken language that is used for daily routines, such as asking for directions, and improvisational skills, which are used to keep a conversation or for negociation.

DESIGNING SPEAKING ASSESSMENTS

Prior considerations:

-Place and equal focus on fluency and accuracy

-To ensure inter-rater reliability, give relative weight to accuracy (grammar), vocabulary, linguistic ability (pronunciation, intonation, and stress), fluency (ability to express ideas), and content or ideas.

-Use multiple raters for reliability. For the exam, there could be two raters, the interlocutor, who interacts with the student being tested, and the assessor, who writes scores and makes notes. Then, both negociate the final score or take an average of the two marks.

-Simplify by using a scoring sheet with criteria for assessment.

-Leave a space for comments that wil serve as feedback to the student.

DESIGNING SPEAKING ASSESSMENT

Start with a simple task to calm students during the test, and use a variety of methods and techniques.

Formal Speaking Assessment Techniques

All students should be tested under reliable and standard conditions at least once during the course. According to Canale (1984), students perform better if they follow these steps:

-Warm up to relax students and to obtain basic information from the student.

-Level check the assessor gives questions or situational activities to determine the student´s level of proficiency.

-Probe to check the student´s level or to make him/her go beyond his/her abilities.

-Wind down by talking a little to the student. This part is not scored.

VARIATIONS ON THE FRAMEWORK

The following is a list of common tasks that may be used fot the level-check stage.

-Picture cue

-Prepared monologue

-Role play

Information-gap activities

CLASSROOM SPEAKING ASSESSMENT TECHNIQUES

-Oral presentations

-Debate on controversial topic

-Reading aloud

-Retelling stories

-Verbal essays

-Extemporaneous speakinG

A LAST TIP REGARDING ADMINISTRATIVE ISSUES

In regular classrooms, a tip is to assess speaking skills by selecting two or three students in a regular class period and focus on their performance. do this regularly to track students´progress.

ASSESSING LISTENING

Although listening is a very important skill to develop, this area is one of the least developed ones. This is a process that is given internally, which makes it difficult to observe and study. In order to teach and assess listening skills, we must be aware of the latest theoretical models of listening. We are following Davis Nunan´s theories into consideration.

According to Nunan, the processing of this skill may be bottom-up or top-down. In bottom-up processing, it is data-driven, linear. Comprehension takes place when learners take and decode phonemes, words, chunks of words, and later sentences, decode them, and link them with other ones.

In top-down listening, meaning is built upon input using background knowledge of situation and context.

APPROACHES TO LISTENING ASSESSMENT

Three major approaches to assess the skill of listening are described.

a. Discrete-point approach, along with the audiolingual method, where two beliefs were the rationale: being able to isolate one element from the stream of speech, and belief that spoken and written language were the same, except for one was presented orally. Tests assessed language knowledge by separating elements, using common question types such as responsive evaluation, phonemic discrimination, or paraphrase recognition.

b. Integrative approach assesses capability to use many little parts at the same time. Tests used dictation and cloze items.

c. Communicative approach assesses capability to understand the messageand then use it in context. Tests include authentic communicative question formats.

General and academic listening are separates with a taxonomy of micro-skills.

DESIGNING LISTENING TASKS

Before we design listening tasks, we must consider the course objectives. Also, we should focus on meaning rather than on form. Other aspects we must consider are background knowledge, test content (take texts you like and infuse oral characteristics into them,), vocabulary (lexical overlap can affect difficulty), test structure (questions in the order information is heard), formats (do not use new formats in tests), item writing (place items separately to give time to respond, frame new sections, record specific instructions.), timing (give time for pre reading the question.), and skill contamination, or skill integration (first read the question,then write the answer.)

TECHNIQUES FOR ASSESSING LISTENING COMPREHENSION

-Phonemic discrimination

-Paraphrase recognition

-Objective formats: MCQs and T/F, short answer questions, cloze, dictation.

-Information Transfer tasks (transfer information to a chart or visual)

-Note-taking

LISTENING TEST DELIVERY

The equipment used, as well as the conditions of the room where the test is taken influence in the validity and reliability of the test. The modes of delivery used may also affect these. Also, non standard procedures.

SCORING

The most reliable tests are are dichotomous scoring.

ASSESSING WRITING

The following is a summary of chapter 4 in the book, "Assessing English Language Learners," by Christine Coombe et al., Michigan Teacher Training.

PRACTICAL ISSUES TEACHERS FACE ASSESSING WRITING

Good writing abilities are highly appreciated in students who aspire for higher education and/ or better jobs. That is why language teachers must assure that our writing assessment practices be valid and reliable.

There are two major approaches in assessing writing, which are:
a. indirect measures of writing assessment, and
b. direct measures of writing assessment.

Indirect measures of writing assessment is the approach that refers to the correct use of sentence construction, spelling, punctuation, through multiple choice and cloze test formats, to measure writing sub-skills in accuracy of sentence construction and grammar.

Direct measures of writing assessment is the approach is the approach which assesses the ability to communicate through production of written texts, integrating all elements of writing, such as organization of ideas, use of appropriate vocabulary, and syntax.

There are four basic elements in designing good writing assessment tests, according to Hyland (2003), which follow.

1. Rubric: the instructions to carry out the writing task and/ or the set of criteria by which a paper or a project is evaluated. Most of the information comes from the test specifications like topic, text type, length, areas to be assessed, timing, weighting (percentage), and pass level.

2. Prompt: the writing prompt, which can be base prompt - states the entire task in direct and simple terms, framed prompt -presents the frame to interpret the task, and text-based - presents a text to respond or use in their writing.

3. Expected response: description of what the teacher intends students to do with the task.

4. Post-task evaluation: assessing the effectiveness of the writing task.

Some relevant issues in writing assessment are the following.

Time allocation. This is how much time is allowed for the sttudent to complete the task.

Process versus Product. More value has been given to the process of writing, so the process approach, such as working with a portfolio that includes all drafts, is suggested.

Use of Technology. The use of technology allows grammar and spelling checkers, which could plasce students with no access to technology in a disadvantage, so be consistent with all students.

Topic restriction. Some teachers provide a list of topics for students to select one, which allows them to write about what they like and know, while limiting to one topic does not produce variance in scores. It is recommended to select topics of same genre and rhetorical pattern.

READING: FACTORS THAT AFFECT IT

Towards the insights that any language teacher may reflect upon reading, I have chosen to briefly comment on three factors that may affect this interactive skill. These factors are the following.

1. The metacognitive knowledge.
Any person may take a reading and analyze it from top to bottom - the process whole texts, from the largest parts to the smallest ones- or bottom-up - the recognition of letters, words, and sentences - or both at the same time.

2. The schemata.
Readers bring background knowledge - schemata - which is constantly incorporating with knew information. The degree of differences between the native language of the learner and the target language, includes all cultural differences involved.

3. The proficiency.
Reading skills that have already been acquired by the individuals in their native language are going to be transfered to process the texts.

ASSESSING READING

Some difficulties in assessing language learners imply the issues regarding invisible skills such as reading and luistening, since we cannot see the process in our students´ minds. That is why we must consider construct models or models of receptive skills to design assessment. A relevant aspect to take into account is the importance of reading in the overall institutional assessment of our learners.

Reading is a complex skill that is challenging to those who assess. It is fundamental for most overall assessments at the K-12 levels in USA, and a mayor component in an intensive English program for students´success in tertiary level of education, while it is not important in conversational classes. In all cases, teachers asure that students are familiarized with the format and directions of the examinations.
The processes that occur during reading may vary according to the texts. Bottom-up skills refers to the recognition and decoding letters, words, and sentences, while top-down refers to the process of entire texts. "Text applies to both linear passages of prose as well as a wide variety of non-linear sources of information such as maps and pie charts." (p.45) A starting point for reading assessment is the target skills that our students develop, as well as the subskills and strategies that are relevant within that teaching/learning context. Classroom assessment requires focus on mayor and minor (contibuting) skills in reading. These are described as follows.

MAYOR READING SKILLS
-skimming and scanning, establishing overall organization of the text
-careful reading for main ideas, supporting details, author´s arguments and purpose, relationship of paragraphs, and facts versus opinion
-informatiuon transfer from nonlinear texts
-drawing inferences from stated / implied content

MINOR OR ENABLING READING SKILLS
-understanding at the sentence level - vocabulary, syntax, cohesive markers
-understanding at inter-sentence level - identifying what pronouns refer to, recognizing discourse markers
- understanding components of nonlinear texts - the meaning of graph or chart labels, keys, and the ability to find and interpret intersection points

ASSESSMENT AND EDUCATION IN OUR MEDIA

Teaching practices involve many aspects that encompass teaching/learning processes. Data concerning (Informe del Estado de la Educación, Costa Rica, 2010) our students show an index of educational opportunities where 40% of the 17 to 21 year olders complete high school due to factors such as resources and characteristics of educational climate from their homes in a country where youngsters need at least complete high school and a year of collegeor tertiary education. Thus, evaluating this entire process takes mayor relevance, focussing on assessment for it "refers to a v ariety of ways of collecting information on a learner´s language ability or achievement."(p.xv) being testing the part of measuring the learner´s achievement, and an essential issue in our quotidian endeavor.

The etire curriculum cycle in education enfolds the process of developing assessment, embracing planning, development, administration, analysis, feedback, and reflection. Planning involves the consideration of the goals and objectives of the linguistic abilities we expect in our students, deciding "the best means of assessment for those objectives."(p.5)

OTHER TEST ITEMS

In language tests, there are other options of common subjective items we can use. Among these, we may use cloze and gap fill, short answer and completion items, and essay questions. The following is a brief description of these items.

CLOZE AND GAP-FILL ITEMS

Cloze testing

The fist time that Cloze testing was used was around 1950´s to evaluate reading comprehension. In this test, a word is systematically deleated from any written text. Now, in a conventional cloze test, we remove a word every five to eight words, and the student has to read around the word to complete the text with an appropriate term, in meaning and structure.

Gap-fill

Gap-fill testing has a blank in a sentence. The student has to fill in the missing word or phrase. The gap may be a function word (articles, prepositions, conjuntions) with only one correct answer, or semantic (noun, adjectives, verbs, adverbs) with a variety of possible correct answers.

Rational deletion cloze

This is an item similar to gap fill. It has words deleted for assessment purposes, for example, focused on certain grammar points like articles, prepositions, pronouns, verb tense, and agreement, within one same paragraph. It can be formatted into a multiple choice cloze, too, so that the student has a limitted amount of options to select the word that is best for each space.

Some general recommendations to write cloze/gap-fill items are the following:

- Make sure there is enough space to write the answers (which should not be very long).

-Offer enough context for students to presume what goes in the blank.

-Provide the same length of blanks. The main body of thw question should predeed the blanks.

-Allow more than one possible answer when grading.

-Let the initial sentence in the paragraph complete to set context, so put the gap after it.

SHORT ANSWER/ COMPLETION ITEMS

There are some advantages and disadvantages in short answer/completion items. Among the advantages, these items help students not only to recognize but to learn and know the answer. Another advantage is less guessing, for they must produce the answer. Also, they are easy to construct. Besides, they are valuable to check what, where, when, who content, intense text comprehension, gist, and higher order thinking skills. Last, they are easy to construct. On the other hand, some disadvantages include discouraging memorization of fact since students produce language to answer, the time it takes the student to answer, and the time it takes the teacher to score them, and the potential unreliability through inter-rater and intra-rater issues. Inter-rater reliability refers to consistency between two or more graders, and intra-rater reliability alludes inner consistency between one marking session and another.

Some recommendations to write these types of items are:

-Only one short, concise response should be allowed.

-Partial credit should be given to the various degrees of correctness.

ESSAY QUESTIONS

There are some advantages and disadvantages in essay questions. Some advantages include their usefulness to assess higher-order cognitive processes such as analysis, evaluation, summary and synthesis, and the autonomy the student has to decide the organization of the answer regarding the approach, ideas, contents and conclusions. Contrary to these, the disadvantages may be the required writing ability, they are time consuming, their potential reliability problems emerging from subjective scoring, and the scoring workload for the teacher.

The following are recommendations to write essay questions.

-The level of difficulty in essay questions should be similar.

-Essay questions should force students to use higher-order cognitive processes such as, evaluating, summarizing, alayzing, and synthesizing.

-Allow plenty linear space for students to write their essays.

-Specify the type and amount of information required in the essay.

-Asses content that is appropriate for the essay format.

-The specific scoring rubric should be shared wih the students before or during the exam to decreasse student´s anxiety.

According to Coombe et al, these are the ten things to remember about testing techniques.

"1. Design tests and assessmentt tasks based on blueprints or test specifications."

"2. Ensure the format remains the same within one section of the exam."

"3. Make sure the item format is correctly matched o the test purpose and course content."

"4. Include items of varying levels of difficulty." (Follow the 30/40/30 principle of students´ range abilities.)

"5. Start with an essay question first."

"6. Avoid ambiguous items, negatives, and especially double negatives."

"7. Avoid race, gender, and ethnic background bias."

"8. Prepare answer keys in advance of test administration."

"9. Employ items that test all levels of thinking."

"10. Give clear instructions."

CORRECTING TEST ITEMS

Correcting tests elaborated by other colleagues or by ourselves in previous years seems rather unusual in our daily work. In my years of teaching experience, the regular procedure requires handing in a copy of the test to our institution´s evaluation committee and file a copy in our personal test bank. Now we have a practical exercise of our work´s insight - or that from our colleagues- by revising, correcting or redesigning any old - or not so old- test. The procedure is very simple, since all we do is write observations on the margins of the pages. This gives us the opportunity to go back to that test, read the observations, and correct or redesign them so we can use them with a new group of students, if it is suitable.

The following is an example of a test with some observations.

XXXXXXXXXXXXXX High School Total Points: 50
7th Grade I Mid-Term Exam I Trimester Points Obtained:

Teacher:XXXXXXXXXXXX - 2009 Score: ______
Time allowed: 80 minutes. 35%Obtained:
Parent´s Signature: _____________________________
Name: _________________________Date: ______________ Group: ______
General Instructions: Read everything carefully before you begin. Use blue or black pen[O.V.D.1] only. Avoid use of correction pen so you may present your complaints. Work individually and in silence.

I PART. Subjective Items.
A.MATCHING (6 points)

In COLUMN A, there are formal and informal types of greetings, leave takings and introductions. In COLUMN B, there are the names of these expressions. Relate the expressions to their names.
COLUMN A COLUMN B

See you later . . . . . . . . ( ) a. Formal greeting
What´s up ? . . . . . . . . . . . . .( ) b.Informal greeting

I´d like to introduce you to Mr. Clark . ( ) c. Formal introduction
Good morning. May I help you? . . . . . . ( ) d. Informal introduction
Hey, friends, this is Annie. . . . . . ( ) e.Formal leave taking
Good night, Mr.Brown. . . . . . . . ( ) f.Informal leave taking[O.V.D.2]

B. IDENTIFICATION[O.V.D.3] (13 points)
*Identify the correct personal pronoun. Write it on the space. (6points)

My name is John . ___________ am John.
Their names are Nancy and Maria. __________ are Nancy and Maria.
His name is Victor. ___________ is Victor.
Our names are Henry and Luis. ___________ are Henry and Luis.
Her name is Annie. ___________ is Annie.
Its name is Fifi. ________ is a dog.

*Identify[O.V.D.4] the correct possessive adjective. Write it on the space. (7 points)
1. We are Sara and Greg. _______ names are Sara nad Greg.
2. They are Jose and Kate. _______ names are Jose and Kate.
3. It is a cat. _______ name is Misingo.________ name is Misingo[O.V.D.5] .
4. She is Kenisha. ______ name is Kenisha.
5. He is Kevin. __________ name is Kevin.
6. You are Hellen. ______ name is Hellen.
7. I am Guadalupe. _______ name is Guadalupe.

C. SHORT ANSWER[O.V.D.6] .(12 POINTS)
*Find the words that complete the dialogue in the box, and fill in the blanks


BOX Hello-Bye- thanks -Pleased -about-study-How-friend –
meet – Henry – you-too



Ana: _______________.
Henry: Hi, Anna. ________ are you?
Ana: I´m O.K. How ______________ you?
Henry: I´m fine, ______________.
Ana: Henry, this is my ___________, Alonso.
Henry: Glad to ___________ you, Alonso.
Alonso: _______________ to meet you, too.
Ana: Where do _______ study?
Alonso: I ___________ at Liceo Nuevo. How about you?
Arturo: We both study at Liceo Nuevo, _______!
Alonso: Great! We will meet there soon.
Ana: See you later, _______________. Bye.
Arturo: ____________, Ana.

II PART SUBJECTIVE ITEMS.
Answer the questions with personal information.
(6 points)(1 point/correct answer, 1 point/correct structure)(8 points[O.V.D.7] )

1. What is your name?

2. Where do you live?

3. Where are you from?

4. How old are you?


Write an informal conversation. Use greetings,
leave takings and introductions. (12 points)(1 point/ correct structure, 1
point / correct expression)

*

*

*

*

[O.V.D.1]Ink


[O.V.D.2]Add
more options in column B.

[O.V.D.3]This
is not identification. It is a
completion exercise.

[O.V.D.4]This
is a copletion exercise, not identification.


[O.V.D.5]This
is repeated.

[O.V.D.6]This
is an identification exercise, not “short answers.”

[O.V.D.7]The
value of points is not clear. Is it worth 6 points or 8 points?

TESTING TECHNIQUES

Challenges may be found in elaborating test items and tasks because test items are the foundation of tests and other assessment instruments. Therefore, we will start by explaining a classification of test items according to the type of response from the test taker. There are selection items and supply items. Selection items are the items in which the student selects the correct answer from a number of presented options. These may be true/false, Multiple Choice, Matching, Numbering Sequence. On the other hand, in supply items, the test takers must supply or construct the correct answer. These items may be Cloze or Gap-Fill (No response provided), or Essay questions.

Test items may be also classified according to the scoring scoring procedure, into objective and subjective items. Thus, objective items are scored following an answer key, which does not requirethe marker (the person who corrects the tests) to have any knowledge of the subject being evaluated. These are "short answer-closed response items" (p.17) of recognition that are corrected easily and rapidly, reliable when there are plenty items, but difficult to properly construct. They can be used to test global and detailed understanding. Set in opposition, subjective items make imperative to have a marker knowledgeable of the subject being evaluated where much human judgment is required, although they are easy to construct. In addition, these items demand from the testers to produce open-ended responses and from the markers time and difficulty in checking.

MULTIPLE CHOICE QUESTIONS

Multiple choice questions (MCQ) are conventionally used in textbooks and in English language proficiency tests at the recall and comprehension levels. Their basic structure is a stem and response. The stem is usually written as a question or an incomplete statement and one response, or key, among two or three other choices or distractors. The test taker identifies the correct or most appropriate response, called the key. When testing reading, vocabulary and grammar, there are four choices, but only three for listening, not to rely much on the student´s memory. The advantages of MCQ are: their reliability, their usefulness in various levels, the test taker´s abilities in writing are irrelevant to these, they can be scored by computer, and students everywhere are familiarized with them. Teir disadvantages are that they encourage guessing, they are time consuming to write, they do not test productive language skills, and frequently, teachers forget MCQs can be used to asses higher-order thinking skills.

There are common MCQ item violations that can be mended. These violations are:

-grammatical inconsistency

-extraneous cues or clues

-three for one split (Three distractors are parallel and one is not.)

-impure items (They test more than one concept.)

-apples and oranges (Two response options have no relation to the other two.)

-subsuming response options (The key and one of the distractors could both be correct.)

-unparallel options (Key and distractors are not parallel in length or in grammatical consistency.)

-gender bias in language

-sensitivity (negative emotional impact material should be avoided.)

-double answer or key (More than one response option is correct.)

-no answer (There is no key among the response options.)

-Giveaway distractors

True/False Format

True/false questions are a specialized form of the MCQ format, written as statements, with only two possible alternatives. The two response categories are: True/False, Yes/No, Correct/Incorrect, Right/Wrong or Fact/Opinion. Their advantages include being able to test large amounts of content, they require less time to be responded and more items can be incorporates into the tests, which increases reliability, and scoring is rapid, reliable, and accurate. Their disadvantages are the guessing factor of 50% and that you need to include ample number for them to be reliable.

Matching Format

Matching format is an extended form of MCQ. It brings forth the test taker´s ability to make connections among ideas, vocabulary, and structure. It is presented in two columns of information where items in the left-hand column are called premises or stems, and the items in the right-hand column are called options. Some advantaages of this format are that test takers have more distractors per item, and writing these are easier than writing MCQs.

ASSESSMENT DEVELOPMENT PROCESS

When teachers are involved in the assessment development process, we can benefit by understanding all of its phases. This is of value for most of the times, we are involved in one or more of these phases as part of our activities.These phases are planning, development, administration, analysis, feedback, and reflection.We have to consider that "assessment is an integral part of the entire curriculum cycle,"

Planning

As we start planning, we must consider why we are assessing what type of assessment fits our needs. At this point, the target language use domain, TLU, which is delineated as the tasks that the test taker is likely to encounter outside of the test itself, and to which we want our inferences about language ability to generalize (Bachman and Palmer), is consideredin the initial stages of our planning to choose assessment tasks that reflect TLU domains, be these real-life domains or language instruction domains. We also have to decide the best means of assessment for our objectives, envisioning the skills we want to asses, the time and resources, seeing what realistic options we have in our situation.

LANGUAGE ASSESSMENT

LANGUAGE ASSESSMENT

The following are comments based on the reading of the book, A practical guide to assessing English language learners, by Christine Coombe, Keith Folse, Nancy Hubley, and the remarkable dissertation from Dr. Gilbert Hernandez.

All tasks and activities that teachers perform to evaluate the improvement and production of our students, on a daily basis, are enfolded in the concept of assessment. These vary in the manner and diversification that our students need so that us, teachers, may be able to make decisions about the best way to help them in the teaching/ learning process, or instructional decision-making.

One step that must be considered is the initial diagnostic assessment.Through it, we may identify the language proficiency of the students, their strengths and weaknesses in the academic performance, their achievement and mastery considering course goals.

COPING WITH LEARNING DISABILITIES