ΚΑΒΟΥΡΑΣ ΣΠΥΡΙΔΩΝ

 

HELLENIC OPEN UNIVERSITY

1. Ed. In TESOL

COURSE DESIGN AND EVALUATION

 

 

ASSIGNMENT 3:

EVALUATION OF AN EFL COURSEBOOK:

at a GREEK PUBLIC PRIMARY SCHOOL

by

KAVOURAS SPYRIDON

082197

 

TUTOR: Dr ELENI GHERALIS-ROUSSOS

 

 

CHIOS

April 2018

 

 

TABLE OF CONTENTS

Introduction                                                                            3

 

• Review of materials evaluation                                              3
• Description of teaching context and students’ needs 4
• Description of textbook to be evaluated 5

• • Textbook evaluation criteria 6

2.2     Results of Evaluation                                                              6

2.3     Implications for Teaching                                                       7

Conclusion                                                                              8

 

Bibliographical References

APPENDIX i: EVALUATION CRITERIA BASED ON MCDONOUGH AND SHAW CHECKLIST

APPENDIX ii: E CLASS PUPIL’S BOOK INTRODUCTION

APPENDIX iii: E CLASS PUPIL’S BOOK TABLE OF CONTENTS

APPENDIX iv: E CLASS TEACHER’S BOOK INTRODUCTION

APPENDIX iv:

 

 

Introduction

The use of coursebooks, or any educational material should be chosen with the target group in mind and the local setting and constraints. Materials evaluation can help teachers implement the aims, goals and objectives of the curriculum, its syllabus, successfully to their context, by adapting them locally, following the implications of needs analysis of their program. At the same time, it may facilitate teachers’ self-development as well as curriculum development. This assignment aims to present some basic notions on EFL evaluation, coursebooks evaluation, and evaluate the coursebook material used for EFL in E class of Greek primary school, their appropriacy to local context needs, with a view to reaching implications for best adaptation and effective use.

PART 1

1.1 Review of materials evaluation:

 Evaluation has been essential in our era of economic development, when criteria of accountability, cost-production effectiveness prevail, even in educational contexts (Weir & Roberts, 1994, p. 4). Evaluation can take many forms and be applied on differing aspects/factors of educational context: programmes and projects, (ibid, p.3), institutions, curriculum-syllabus design, teachers/staff development, teaching/learning processes, learners, learning environment, materials, ( Richards, 2001, pp. 198-278, and ibid, p. 287) . Its essential characteristics proposed by Simons feature: impartiality, confidentiality, negotiation and collaboration among all participants and accountability at all levels of the hierarchy ( Simons, 1979, as cited in White, 1988, p. 149). Karavas refers to Beretta and the Joint committee on Standards for Educational Evaluation for the importance of utility, feasibility, propriety and accuracy (Karavas, 2004, p.188).

Also there are various purposes of evaluation; according to Rea-Dickins & Germaine, 1992, (as cited in Karavas, 2004, p. 163): for accountability purposes, for curriculum development, and for self-development of teachers. Furthermore, according to Karavas (Karavas K. 2004, p. 163, 168) another purpose is for determining a project’s feasibility and impact. For Weir & Roberts, these can serve formative -before or during a programme-illuminative and summative evaluation- usually at the end of programmes (Weir, 1994, p.5; Richards, 2001, pp. 288-293). Formative evaluations are developmental in nature to determine the content of language programmes, summative  ones are connected mostly with accountability uses to assess the end product of learning, while illuminative ones provide insights into the processes of the programmes ( ibid, pp.288-293).

Models of evaluation can be classified under the terms: product oriented, static oriented, process oriented and decision facilitation approaches (Karavas, 2004, p.157, based on O’Brien, 1999). These are related both to the aims and the time of evaluation. Stages may include identification of needs, setting objectives, choice of content and methods, implementation of the programme (White, 1988, p. 148). Furthering the process, Karavas refers to seven stages: initiation and planning, designing, processing and interpreting, reporting, decision making, taking action, returning to step 1 (Karavas, 2004, p. 172; Ellis, 1997, p.38). Other important notions relevant to the time of coursebook evaluation are pre-use, in-use, or post-use evaluation (Cunningsworth, 1995, p.14).

Textbook evaluation is one of the most common processes for teachers for selecting appropriate materials for their classes. Yet since there can never be perfect material for any situation, evaluating as a means to assessing strength and weaknesses for successful adaptation to the local educational context appears as the only solution (McDonough et al. 2002, p. 51 and Karavas, 2004, p. 192-193). A number of theoretical evaluative frameworks have been published, some well known and extensively used: (Mcgrath, 2002; McDonough&Shaw 2003; Cunningsworth, 1995; Sheldon, 1988; Williams, 1983; Grant, 1987). Cunningsworth   proposed a list of guidelines, and 45 questions on areas aims, design, language content, skills, topic, methodology, teachers’ book practical considerations (Cunningsworth, 1995). Sheldon emphasized the locality of criteria and the need for their cultural and contextual adaptation (Sheldon, 1988, p. 242). Grant identified two stages in the process, the external evaluation with criteria forming the word CATALYST, and internal consisting of three questionnaires.(Grant, 1987). Finally, McDonough and Shaw advocated two levels of evaluation, based on external or internal coursebook features, ‘macro–level’ and ‘micro–level’ evaluation, complemented by some ‘overall evaluation factors’ (McDonough, 2013, pp.54-61).

1.2 Description of teaching context and students’ needs:

 As has been analysed in detail in the two previous assignment of ours, the official curriculum is the new Integrated Curriculum for Foreign Languages ( ΕΠΣΞΓ, 2016); it combines the TYPE A-product based- and TYPE B –process based-syllabi, ( Kavouras, uned. 2018, p. 4) aiming not only towards integration of the two approaches, but it offers freedom to teachers to adapt materials to their teaching context.

The coursebook is imposed by the Greek Ministry of Education, introduced at schools in 2009, designed under the aegis of the Pedagogical Institute (PI) (Tsagari, 2014,p.213). It comprises the only resource material at school, since the school lacks a school library, readers, grammar books, or even dictionaries for reference. Most children consider it as identical to the syllabus. It aims to get offer knowledge at A2 level of CEFL, which is to be further covered at year 6 six.

Learners are around the age of eleven, supposedly at A2- level of CEFL, (Tsagari, 2014, p. 213). They have been learning the language since year 3 three. In the needs analysis that preceded ( Kavouras, 2018, uned., see Appendix IV) their main reasons, motivation, for learning the language appeared to be getting a language certificate (16/17 answers), and to get good marks at school (17/17 answers!). The next most popular choices also were to travel to foreign countries, and to understand song lyrics. As for learning styles, the most popular appeared to be auditory learning (16/17 answers), accompanied by kineasthetic mode of learning-three quarters declared preference towards writing exercises, copying vocabulary, and writing letters/e-mails. Analytic-synthetic mode also prevailed with translation tasks, and grammar practice exercises. Almost half of the pupils opted for visual learning style. In social modes, group work seemed more popular than individual work-3/4 three quarters preferred group work to individual mode. Similarly, interaction with other kids and tourists attracted a vast majority of choices, along with interactive computer games. (on learning styles related to materials evaluation, see Bokyung Lee, 2015).

The teacher is a non-native speaker of English, university graduate, with 18 year of teaching experience.

Length of programme: School schedule determines English classes to be held for three 45’ sessions per week, along a time of approximately 28 school weeks.

Constraints: There is not a language laboratory at school; however there is a computer lab, though poorly equipped with a small number of just six obsolete computers, most of which without speakers. However, all classrooms are equipped with computers, projectors and internet access, which can be exploited in class.

The school library lacks EFL learning material; there are single copies of a dictionary, a grammar reference book, and of the school books audio CDs.  Literature books, or readers for English practice are non-existent.

The school timetable is fixed with a considerable number of missed sessions due to the school general cross-curricular projects, national holidays, or school excursions (ΕΠΣ-ΞΓ, 2016, p.26). Unfortunately, the missed classes cannot be substituted.

1.3 Description of textbook to be evaluated:

It is addressed to learners of English as a foreign language, mainly native speakers of Greek, as well as immigrant children living in Greece, a heterogenous group of mixed-ability learners. It consists of a Pupil’s book, an activity book, a teacher’s book, and audio CDs. The material is downloadable and accessible on the internet, in conventional and multimodal format at http://ebooks.edu.gr/new/course-main.php?course=DSDIM-E103; Also the material is supplemented by material in electronic format, audio or video on an educational web platform prganised by the ministry of education photodentro http://photodentro.edu.gr/lor/ (see Appendix ii).

The 5^th grade coursebook is issued by the Ministry of Education, distributed to learners  for free, yet being compulsory for schools to adopt. It was published following the directives of the previous curriculum, DEPPS-APS, 2003, with emphasis on the principles of literacy, multiculturalism, project–based learning in accordance with the cross–thematic approach of DEPPS-APS (see Appendix iv). It features thematic units, incorporating cross-thematic activities which set English as a global language for communication among nations (Kolovou E. & Kraniotou A., 2009, p.5), setting thus a target-speech community for learners (Seedhouse, 1995, p.61).

The pupil’s book is comprised of ten units divided in three lessons each (see Appendix iii). All skills are practiced within and across lessons. Units are thematically organized with topics from everyday-life or of global interest. (See Appendix ii). At the end of each unit, there is Self-assessment section for testing and materials consolidation. At the end of the book, an appendix offers differentiated activities for students to choose as supplement or for students to work at different pace of learning, for weaker students or for enrichment of topics. A concise grammar reference section follows with grammar theory in English, and a list of irregular verbs. Finally, some maps of Europe, the UK, the globe, and the London tube are appended for exploitation (see Appendix ii).

The activity book supplements the pupil’s book for further practice of the topics, functions, skills, vocabulary, or grammar presented there.

The teacher’s book gives key answers to the tasks in the other books, instructions on lesson plans, listening tapescripts, and suggestions for further activities in class.

PART B

2.1 Textbook evaluation criteria:

Having in mind the checklists for evaluating materials (see part A, section 1.1) and the basic principles for materials evaluation, we have opted for using McDonough & Shaw’s checklist, as it was adapted in the newer version of their book( McDonough et al., 2013, pp. 50-62). At the analysis of the results, we adapted the criteria to the school local context; comparing the questionnaires proposed by Grant (Grant, 1987), and Sheldon (Sheldon, 1988, pp. 242-245), we consider that McDonough & Shaw’s checklist can contain almost all the points raised by the others; for practical reasons, the criteria have been organized in three columns for easier reference (see Appendix i).

2.2 Results of Evaluation:

 External evaluation or  Macro-evaluation. It is based on the introduction on the pupil’s book, and teacher’s book. The intended audience is the one determined by the ministry, though the books’ proficiency level, at A2 level of CEFL, may be too difficult for E class learners the way it is presented (Tsagari, 2014, pp. 215-216). The context for use is learning English for general purpose, though Sifakis calls it ‘purposeless’’, based on the need for language certification (ibid, p. 213). Language presentation into units is unstable, since there is neither a clear layout nor exploitation of all skills in the same place across lessons or units, as the table of contents proves (see Appendix iii). Authors view methodology in congruity with the curriculum specifications about multiculturalism, plurilingualism, learner autonomy, multiple intelligencies (see Appendix iv). The book comprises the core of the programme. Teacher’s book is available through the internet, though in scarcity in most school libraries. Vocabulary lists or index does not exist either in the pupil’s book, the activity, or the teacher’s book, though in the needs analysis learners expressed preference for one and even for a companion with words translated in Greek. The table of contents is clear and concise ( see Appendix iii). Visual material is insufficient. Most texts are supplemented by a single image, chart/picture/diagram related to the text, though not always integrated or complementary to its meaning, whereas the illustration is unattractive and not appealing to learners’ interest (Μαλίτσα, 2012, p.9). The book layout is not clear for learners, cluttered with lengthy complicated instructions in English, hard for Ls to follow. As for cultural specificity, the book must be appraised for its multiculturalism, with reference to using English globally, eg, in unit 2, the text talks about pupils’ daily routine in India. Genders are neutrally depicted, for example the heroes mother is a working bank clerk, their father is mentioned doing household chores, etc. Minorities are fairly represented, eg, in unit 2 lesson 2 a professional basketball player is depicted as Greek of black skin colour. Audio/video material is scarce; only some listening tasks appear in each unit, unequally distributed across lessons (Tsagari, 2014, p. 218; see Appendix iii, skills column). Computer software for interactive games, or videos is non-existent, though it is supposedly supplemented by material at photodentro http://photodentro.edu.gr/lor/ , along with the audio files at http://ebooks.edu.gr/new/course-main.php?course=DSDIM-E103 for free. Finally diagnostic tests do not appear, end-of-unit tests do not revise the material presented within the unit, are too short, and do not comply with the KPG format, as the IFLC has entailed. Also they do not contain listening or speaking tasks (Tsagari, 2014, p. 218).

Internal evaluation or Micro-evaluation:  Treatment of skills is unequal (ibid, p.216). Sequencing seems to be linked to three heroes talking about their everyday life. Grading is loaded with too much material, demanding for learners (ibid, 215). Reading tasks appear at a variety of texts, with follow up activities promoting self reflection. Listening tapescripts, though, feature incomprehensibility an lac of authenticity, unnatural accent (ibid, p. 218). Tests do not revise all skills, and contain material not presented within the units. Learning styles are not adequately cared for, since , contrary to learners’ needs, visual and audio material are scarce in the books. Analytic-synthetic ability is not practiced either, since grammar exercises do not practice grammar inductively, whereas grammar is not practiced in context or texts, but only at the sentence level. Self-study provisions are supposedly made, according to the authors’ claims, yet the unilingual presentation which is inadequately escorted by icons or visuals to supplement their meaning, make learner autonomy an unjustified claim. Finally, it  motivates neither teachers or learners.

2.3 Implications for teaching:

Comparing the authors’ claims and students’ needs to what the material really features, a clash of objectives, methods and means arises; as a result, steps have to be taken by all stakeholders for maximum potential of the programme.

The material needs to be simplified, adapted or supplemented (McDonough et al. 2013, pp.63-76), since it cannot be replaced (Tsagari, 2014, p.220). Grammar reference sections may be simplified or translated; vocabulary lists with Greek translation may be created by teachers or downloaded from the internet. Vocabulary initiation and presentation activities can prepare learners for the reading or listening tasks; listening tapescripts can be printed from the teacher’s book, for reading, listening and pronunciation practice. Reading texts may be recorded by teachers or books by private publishers with audio bought. Extra tasks to fit pupils’ learning styles may increase its attractiveness; visual materials through the internet, or learners’ creative skills can be added; songs, videos, or extra audio material can be exploited as an extension to the coursebook tasks. Activities from past paper tests for the KPG exam could increase learners’ engagement as well as their performance.

Teachers should exploit the freedom offered by the curriculum and take responsibility for its implementation. Material for language certification exams may increase students’ motivation. The computer lab exploited may offer opportunities for learner autonomy-interactive games for pairs or groups, or even material for project work. Should teachers show and allow learners how to use electronic dictionaries on the classroom computer, problems of translation or lack of vocabulary companion might get surpassed.

School counsellors should press the ministry for review of the published material in accordance to the new curriculum specifications and research findings on methodology and evaluation of the existing books

Conclusion

The books evaluation is a means to teachers’ self-development, to increase lesson effectiveness, students’ engagement and programme accountability. Suggestions for modifications or adaptations are meant as indications for every teacher using the specific books to think in terms of their local context. Finally, pupils and parents must get involved in every evaluation and adaptation process, since their choices should be in congruity with teachers’ mind. In this way, every book can maximize its potential for the betterment of EFL learning.

WORD COUNT: 2616

 

 

Bibliographical References

Cunningsworth  A. (1995).  Choosing your Coursebook. Oxford: Heineman Karavas K.(2004). Chapter 5: The Evaluation of Courses, Textbooks and Tasks,  in  Ayakli C. & Karavas K. , Course design and Evaluation, volume 2, Patras, HOU.

Bokyung L. (2015). EFL Learners’ Perspectives on ELT Materials Evaluation Relative to Learning Styles. RELC Journal, 1-17

Ellis R. (1997). The empirical evaluation of language teaching materials. ELT Journal, 51/1:36-42

Ενιαίο Πρόγραμμα Σπουδών των Ξένων Γλωσσών, 2011, Αθήνα, Παιδαγωγικό Ινστιτούτο, retrieved on 23-3-2018 from http://rcel.enl.uoa.gr/xenesglosses/sps.htm.

Ενιαίο Πρόγραμμα Σπουδών για τις Ξένες Γλώσσες, Εφημερίς της Κυβερνήσεως της Ελληνικής Δημοκρατίας,  τ. Β, αρ. φυλ. 2871 /9-9-2016, Αθήνα, Εθνικό Τυπογραφείο.

Grant N. (1987). Making the Most of your Textbook. London:Longman

Κολοβού Ε. & Κρανιώτου Α. ( 2009 ). ΑΓΓΛΙΚΑ Ε ΔΗΜΟΤΙΚΟΥ, βιβλίο καθηγητή, Αθήνα,  ΥΠ.Ε.Π.Θ., Παιδαγωγικό Ινστιτούτο, ΟΕΔΒ. Retrieved February 2018 from http://ebooks.edu.gr/new/

Κολοβού Ε. & Κρανιώτου Α. ( 2009 ). ΑΓΓΛΙΚΑ Ε ΔΗΜΟΤΙΚΟΥ, βιβλίο μαθητή, Αθήνα,  ΥΠ.Ε.Π.Θ., Παιδαγωγικό Ινστιτούτο, ΟΕΔΒ. Retrieved February 2018 from http://ebooks.edu.gr/new/

Μαλίτσα Μ. (2012). Η καταλληλότητα της εικονογράφησης ως ένα από τα κριτήρια αξιολόγησης των υλικών διδασκαλίας/μάθησης. ASPECTS. 33:6-13.

McDonough J. & Shaw Chr. (2003). Materials and Methods in ELT, Oxford:Blackwell.

McDonough J., Shaw Chr. & Masuhara H. (2013). Materials and Methods in ELT, Wiley-Blackwell

McGrath I. (2002). Materials Evaluation and Design for Language Teaching. Edinburgh: Edinburgh University Press.

Rea-Dickins P. & Germaine K. (1992). Evaluation. Oxford: Oxford University Press.

Richards J. C. (2001). Curriculum Development in Language Teaching, Cambridge: Cambridge University Press.

Tsagari D. & Sifakis N. (2014) EFL coursebook evaluation in Greek primary schools. SYSTEM, 45: 211-226

O’Brien T. (1999). Course Design and Evaluation. Volume 3. Patras,  HOU.

Sheldon L. (1988). Evaluating ELT textbooks and materials. ELT Journal, 42/4:237-246.

Weir C. & Roberts J. (1996). Evaluation in ELT, Oxford: Basil Blackwell.

White Ronald V., (1988). The ELT Curriculum-Design, Innovation and Management, Oxford, Blackwell Publishers Ltd.

Williams D. (1983). Developing criteria for textbook evaluation. ELT Journal, 37/3:251-255.https://blogs.sch.gr/spkavour/files/2019/02/KAVOURAS-53-3-FULL-ASSIGNMENT.pdf

Καλωσήρθατε στο Blogs.sch.gr. Αυτό είναι το πρώτο σας άρθρο. Αλλάξτε το ή διαγράψτε το και αρχίστε το “Ιστολογείν”!