Revista Andina de Educación 5(2) (2022) 005211

EF English Proficiency Index and English in Ecuador: Uncertain assumptions of the international ranking

EF English Proficiency Index e inglés en Ecuador: Suposiciones inciertas del ranking internacional

Adeline De Angelisa

a SIT Graduate Institute. International and Global Education. 1 Kipling Road, 05301, Brattleboro, VT, Estados Unidos.

Received June 22, 2022. Accepted September 06, 2022. Published October 03, 2022

© 2022 De Angelis. CC BY-NC 4.0

https://doi.org/10.32719/26312816.2022.5.2.11

---

Abstract

The abysmal level of English proficiency in Ecuador according to the EF English Proficiency Index has become a habitual reference point when discussing English learning in Ecuador. This essay questions three assumptions implied by this international ranking: that it represents Ecuador’s language proficiency; that it offers a meaningful comparison between countries; and that concern for language proficiency is equivalent to concern for equity. Inspired by critical studies of international large-scale assessments, the principle that quantitative data is not neutral, and the concept that the index represents a discourse, I argue that this index is not as objective or trustworthy as it may appear and that the authority it is granted is worrisome for Ecuadorian education.

Keywords: English, language proficiency, international assessments, Ecuador

Resumen

El bajo nivel de las y los ecuatorianos en inglés, según el EF English Proficiency Index, se ha vuelto un punto de referencia habitual al tratar el aprendizaje del idioma en Ecuador. Este ensayo cuestiona tres suposiciones implícitas del ranking internacional: que representa el dominio de inglés en Ecuador; que presenta una comparación entre países significativa; y que preocuparse por las habilidades lingüísticas es preocuparse por la equidad. Inspirándome en estudios críticos de las evaluaciones internacionales a gran escala, el principio de parcialidad de los datos cuantitativos, y el concepto del índice como discurso, argumento que este no es tan objetivo ni confiable como puede parecer y que la autoridad que se le presta es preocupante para la educación ecuatoriana.

Palabras clave: Inglés, dominio de idioma, evaluaciones internacionales, Ecuador

---

Introduction

Ecuadorians periodically encounter headlines in the country’s major newspapers about their inadequate English: “Ecuador, the worst country in Latin America in English proficiency, according to report” (El Universo, 2019); “Ecuador, the worst country in the region in English proficiency, according to study” (El Universo, 2020); “Ecuador maintains a low level of proficiency in the English language” (Castillo, 2021). These headlines refer to the international raking published by EF Education First (EF), a global for-profit educational business that offers language learning programs (EF Education First, 2021a).

Media attention to the EF English Proficiency Index (EF EPI) arises from the apparent need that Ecuador (like many countries) perceives to ensure that its citizens speak English, and the lack of data on the subject. A report by the institute The Interamerican Dialogue explains that “near the top of the list of 21st century skills for most countries in the region is English language proficiency” (Cronquist & Fiszbein, 2017, p. 3). Similarly, the English curriculum of the Ministry of Education describes the need to prepare Ecuadorians for a globalized world (Ministerio de Educación, Ecuador, 2016). However, despite an apparent consensus on the importance of English and the world-wide tendency to use quantitative indicators to assure educational quality (Addey & Sellar, 2018; Grek, 2009), there is no standardized or large-scale assessment of the language in the Ecuadorian education system. In this context, the EF EPI has become a habitual reference point in publications and conversations about English in Ecuador.

However, as teachers, educational administrators, and the general public, we should ask ourselves what this ranking is based on and what importance it has. While it is important to read any data point critically to identify how it was constructed, a critical reading is especially needed with an index like the EF EPI, which does not follow methodological practices typically found in educational studies and which is widely disseminated without any notable public scrutiny.

The aim of this article is therefore to question the objectivity of the EF EPI and the interests and narratives that we support when we share it as an objective and trustworthy measure. First, I present some background concepts that provide a foundation for this discussion. Then, I examine three assumptions about the EF EPI that are at the root of its authority as a rating index: that it represents Ecuador’s English proficiency; that it offers a meaningful, evident and impartial comparison of countries; and that concern for language proficiency is equivalent to concern for equity. Finally, I suggest how we can refer to the EF EPI and the country’s English proficiency in a more cautious and critical manner.

Background concepts: international large-scale assessments and the subjectivity of numbers

International large-scale assessments (ILSAs) are standardized tests that are implemented in various countries to measure students’ performance and compare the quality of educational systems between countries and against international standards. ILSAs present a growing phenomenon over the last two decades and have an important role in how we conceptualize educational policy (Cardoso, 2020; Pizmony-Levy & Bjorklund, 2018). The EF EPI is different from traditional ILSAs in that it is not administered systematically in coordination with national governments, nor does it attempt to represent the national student population of the countries it includes. Nonetheless, it is a standardized assessment that generates international comparisons, and its results are publicized in the media in a manner similar to other ILSAs (Pizmony-Levy & Bjorklund, 2018). Several considerations regarding this type of assessment are therefore relevant to the EF EPI.

Authors like Bray et al. (2020), Cardoso (2020) y Schuelka (2013) have described how ILSAs’ methodological decisions produce biases in their representations of educational quality. One important factor influencing methodology in assessment is the doctrine of the organization creating the assessment—the ideological position of the institution regarding the purpose of education in society (Cardoso, 2020). For example, the OECD, which frames education as a tool for attaining economic prosperity, assesses young people according to their age, rather than their school year, on essential labor-market skills. In contrast, UNESCO conceptualizes education as a human right with intrinsic value, and its evaluations are administered to students in the same year of school, on school subjects. Cardoso (2020) demonstrates that these methodological differences produce contradictory evidence on the effect of school-year repetition on student performance. While the results of ILSAs reaffirm the perspectives of the organizations that produce them, they also shape the educational policies of these international bodies and influence countries’ decisions (Cardoso, 2020).

This aligns with the QuantCrit perspective, which holds that the biases and inequities that dominate society tend to shape quantitative data and its presentation (Gillborn et al., 2018). Thus, even though numbers are commonly considered the best evidence because we we perceive them as neutral and conclusive (Gillborn et al., 2018), numeric data is subject to the biases and interests of the individuals and institutions that contruct it. Those who collect, analyze and report quantiative data make various decisions in the process that, even with good intentions, can have the effect of distorting critical factors or patterns (Gillborn et al., 2018). The authors of QuantCrit therefore recommend an ‘ambivalent’ attitude to quantitative data, vigilant to assumptions implicit in the data that may justify or perpetuate exisiting power relations (Gillborn et al., 2018). This ‘ambivalent’ position guides my discussion of the EF EPI in that it leads me to consider that, like other ILSAs, this assessment implies an ideological perspective and influences educational policy with evidence that may reflect more its own position than objective fact.

Methodologically, I take the position that scores and rankings, though they are numbers, hold meaning due to the words that accompany them. The EF EPI reports are texts that not only transmit data, but also construct a discourse with a point of view, assumptions and intentions. Thus, according to the principles of critical discourse analysis, we should question what perpective is being spoken from and what alternative perspectives are possible (Cohen et al., 2018).

It is also important to recognize my own positionality, which shapes the perspective of this analysis. I approach the topic as a language teacher from the United States who considers Ecuador her home after nearly a decade of residence. In that time, I have worked in English teaching, teacher training, and design and administration of academic programs at a national scale, and so include myself among those promoting English learning in Ecuador and trying to contribute to its improvement. This article expresses a critique that I have been able to articulate since beginning my doctoral studies, a critique that pertains to me as well, as I have also cited the EF EPI and its arguments in my work.

With these background concepts established, I would like to call into question three assumptions about the language proficiency index.

First assumption: the EF EPI ranking represents Ecuador’s English proficiency

Headlines like “Ecuador, the worst country in Latin America in English proficiency, according to report” (El Universo, 2019) present Ecuador’s English proficiency according to the EF EPI as a fact revealed by a scientific study. Additionally, there are academic articles about English learning in Ecuador (such as, Argudo Serrano et al., 2021; Macías Mosquera & Villafuerte Holguin, 2020; Orosz et al., 2021; Sevy-Biloon et al., 2020) that cite the EF EPI to demonstrate the country’s low level of English. These references contain the implicit assumption that the EF EPI tells us something ‘real’ about Ecuadorians’ English level. Various aspects of the index undermine that assumption.

Lack of a representative sample and other methodological problems

It is well-known that, in order to suppose that data obtained from a subset of a group describes the group as a whole or its typical characteristics well, a representative sample is needed (Cohen et al., 2018). Even when studies obtain what is considered a representative sample, one may question who is included in or excluded from that sample. For instance, Schuelka (2013) questions how well ILSAs like PISA and TIMSS evaluate countries’ educational quality given that they exclude students with disabilities.

Unlike traditional ILSAs, the EF EPI does not seek a representative sample and its data is not the product of academic research. Rather, it is obtained through for-profit activities. As EF’s reports and website describe, the results come from two sources: placement tests taken by people who enroll in the organization’s English courses, and placement tests that are available online for free (in exchange for your email address). EF recognizes the limitations of this method of data collection on its website:

The test-taking population represented in this Index is self-selected and not guaranteed to be representative. Only those who want to learn English or are curious about their English skills will participate in one of these tests. This could skew scores lower or higher than those of the general population. (EF Education First, 2022, Sampling Biases section, paragraph 1)

EF also recognizes that this method of evaluation tends to exclude low-income individuals and those without internet access, which could likely produce artificially high scores (EF Education First, 2022). At the same time, one might presume that people who have mastered English do not have a reason to take EF’s placement tests, a factor that could lower the average scores. Whatever the case, without a representative sample there is no basis to claim a relationship between these results and the country overall.

Assuming that the EF EPI reveals something ‘real’ about Ecuadorians’ skills, conversations surrounding these rankings (such as in the media and in academic or public policy events) suggest that they represent the development of those skills over the years. That would be credible if the rankings were calculated in the same way every year, which is not the case. Though the specific calculations used to produce the rankings are not public, a comparison of the methodology sections of the annual reports reveals potentially important variations. From 2011 to 2015, the tests included in the index evaluated grammar, vocabulary, reading comprehension, and listening comprehension, but since 2016 they have only evaluated reading and listening comprehension (EF Education First, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021b). In another variation, the 2011 index combined results obtained from 2007 through 2009 and the 2012 index presented results from 2009 through 2011 (EF Education First, 2011, 2012), while from 2013 to 2015 each index reported just one year (EF Education First, 2013, 2014, 2015). Then, since 2016, scores have been calculated based not only on the current year’s results but also on those of the previous year (EF Education First, 2016, 2017, 2018, 2019, 2020, 2021). Additionally, depending on the year, one, two, three, or four versions of EF’s placement tests are used; for instance, in 2018, only the results from the online EF Set were included (EF Education First, 2018). Without access to more detailed information on the instruments, data collected, and method of analysis, it is not clear how those variations may impact the results, but we can certainly doubt the validity of comparing them across time.

In any case, the scores reported in the EF EPI would not make much sense without the proficiency categories that EF attributes to them. The definition of these categories has also varied between reports, with possible consequences for the way that Ecuador’s performance is represented.

The subjectivity of the language proficiency categories

To make the numeric results comprehensible, EF assigns them descriptive categories according to score ranges. There are five possible categorizations: ‘very low proficiency’, ‘low proficiency’, ‘moderate proficiency’, ‘high proficiency’ and ‘very high proficiency’. From 2011 to 2013 and again from 2019 to 2021, Ecuador was categorized as having ‘very low proficiency’ (see Table 1).

Table 1. Scores, categories and positions of Ecuador in the EF EPI rankings from 2011 to 2021

Source: EF (2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 y 2021b).

The EF EPI offers an explanation of the equivalence of its categories to the Common European Framework of Reference (CEFR), which provides the terminology most commonly used in the region to describe language abilities. The framework describes six levels of competence: “Basic user (A1 & A2), Independent user (B1 & B2) and Proficient user (C1 & C2)” (Council of Europe, 2021, p. 34). According to the CEFR, mastering a language implies having communicative competence—made up of linguistic, sociolinguistic, and pragmatic competence (Council of Europe, 2021, p. 130). That is to say, mastery means being able to interact by means of the language in sociocultural contexts. The EF EPI reduces language proficiency to just some of the elements of linguistic competence, limiting the coherence of seeing these scores as indicators of overall English proficiency, especially as equivalent to the CEFR.

In the EF EPI from 2011 to 2019, the category ‘very low’ corresponds to the CEFR A2 level, the ‘low’, ‘moderate’ and ‘high’ categories correspond to subdivisions of the CEFR B1 level, and ‘very high’ corresponds to the CEFR B2. This means that the lowest level of the CEFR (A1) is not even represented in the EF EPI categorization. It is worth mentioning that A1 is not a zero level, since “the learner can interact in a simple way, ask and answer simple questions… initiate and respond to simple statements in areas of immediate need or on very familiar topics” (Council of Europe, 2021, p. 35). Below the A1 level, one can even describe a pre-A1 level where the speaker can use greetings and simple expressions (Council of Europe, 2021). At the A2 level, in contrast, the speaker can now “understand sentences and frequently used expressions… communicate in simple and routine tasks… describe in simple terms aspects of his/her background, immediate environment and matters in areas of immediate need” (Council of Europe, 2001, p. 1): language skills that merit the categorization of ‘very low proficiency’ on the EF EPI. To summarize, the EF EPI does not distinguish between a total lack of language skill in English, the ability to use some basic expressions in the language, and the ability to adequately communicate about concrete topics in English, since all of these would appear as ‘very low proficiency’. It is worth noting that the distinctions between pre-A1, A1, and A2 can be very significant in contexts like Ecuador.

The tendency of the EF EPI to erase the distinctions between the initial stages of English learning was exacerbated in 2020 and 2021, as the score system changed from a scale of 0-100 to 0-800 and the equivalency of the categories was adjusted. ‘Very high proficiency’ was raised to represent a CEFR C1 level instead of B2, and the standards for the other categories were raised as well. Now ‘very low proficiency’ not only included A2, but also part of the level B1, in which individuals know how to “deal with most situations likely to arise whilst travelling… produce simple connected text… describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans” (Council of Europe, 2001, p. 1). It is surprising, to say the least, for that level of proficiency to be included in the worst category of results on the EF EPI. In Ecuador, the Ministry of Education curriculum aims for high school graduates to reach the CEFR B1 level (Ministerio de Educación, Ecuador, 2016), and it is also the standard for minimum proficiency at Ecuadorian universities (Cajas, 2017). If this goal were being reached at scale, which would be laudable (as B1-level speakers can do a lot with English even without perfect mastery of the language), Ecuador would not escape the category of ‘very low proficiency’ according to the EF EPI.

Additionally, comparing the highest and lowest scores in each category over the years reveals that the cut-off points between categories have not always been consistent. Some scores that were considered ‘very high proficiency’ in 2011 were ‘moderate proficiency’ in 2012, some previously considered ‘moderate proficiency’ were categorized as ‘low proficiency’, and so on, with the cut-off point of each category rising (EF Education First, 2011, 2012). In 2013, Vietnam had a score of 52.27 and was categorized as ‘moderate proficiency’, despite the fact that in the previous year five countries with higher scores were categorized as ‘low proficiency’ (EF Education First, 2012, 2013). After this drop in the lower limit of the ‘moderate proficiency’ category in 2013, the cut-off point apparently went up again as of 2015, when other countries with scores above 52.27, like Peru (52.46), were evaluated as having ‘low proficiency’ (EF Education First, 2015). The cut-off points between categories appear stable apart from those inconsistencies, but the observed fluctuations suggest that the categorization does not depend on inherent differences in quality, but rather on the opaque judgment of EF’s team.

In summary: although the EF EPI offers annual evaluations that supposedly describe the development of countries’ English proficiency over the last ten years, but it would be difficult for them to reflect progress because the method of measurement is inconsistent.

There are alternatives for describing Ecuadorians’ English proficiency

The critical discourse analysis approach suggests that we consider alternative ways of addressing this topic, alternatives that may illuminate the biases of the discourse in question. To take an alternative view on Ecuadorians’ performance on English assessments, I chose to examine TOEFL (Test of English as a Foreign Language) score reports from ETS, a private non-profit organization that provides educational testing services (ETS, 2022a)1. The TOEFL exam is similar to the EF EPI assessments in that it is a standardized test used around the world that also lacks representative sampling, as it is taken only by people who have some reason to evaluate their English. It is different in that the TOEFL is a paid test, it awards a certification that is recognized for academic and professional purposes, etc., and it evaluates speaking and writing skills in addition to receptive skills. Additionally, each ETS report represents the results of exams taken during the same year of the report, while the EF EPI indexes are based on evaluations taken during years previous to the publication date. It is worth noting that ETS shares raw data from the TOEFL with external researchers (ETS, 2022b), which EF does not permit (EF Education First, 2022), and many studies (such as, Fleckenstein et al., 2020; Harsch et al., 2017; Kyle et al., 2016) address the validity of the TOEFL as an assessment instrument.

ETS publishes the average score of the individuals who took the test in Ecuador each year, and also indicates equivalencies between TOEFL scores and CEFR levels on its website (ETS, 2022c, 2022d). The reports do not offer any interpretation of the scores but, based on the equivalencies provided on the ETS site, I have included CEFR levels in Table 2, which summarizes Ecuador’s results from 2013 to 2020.

Table 2. Ecuador’s average TOEFL scores and CEFR equivalencies, 2013 to 2020

* The maximum score changes from 677, until 2018, to a maximum score of 120 since 2019.

Source: ETS (2020)

In contrast to the EF EPI results visible in Table 1, the TOEFL results in Table 2 describe English proficiency in Ecuador as consistently intermediate, in the range of CEFR B1 to B2. There are various possible explanations for the discrepancy between these representations. It could be due to differences in sampling—it is probable that individuals take the TOEFL when they believe they have already reached an adequate level of English, and that its cost attracts individuals with greater resources and academic preparation. It is also possible that this paid assessment is taken more seriously than a free test. The fact that the TOEFL evaluates productive skills that are not part of the EF EPI evaluations could also have an impact.

Having said that, these differences between the samples and instruments should be similar in all the countries where the two assessments are taken and should therefore be less important when we consider Ecuadorians’ performance in comparison to other countries. (I will address the validity of these comparisons in Section 4.) According to the EF EPI, Ecuador’s comparative level is discouraging, with the exceptions of the 2014 and 2015 reports (see Table 1). The reports from 2019 and 2020 (which represent results from the years 2018 and 2019) are especially negative, as Ecuador has last place in Latin America. Meanwhile, Ecuadorians’ average scores in TOEFL results (see Table 3) are very similar to those of the region overall. Even in the EF EPI’s worst years, the TOEFL results suggest that Ecuador’s English is absolutely typical of the Americas. I should note that the EF EPI and the TOEFL define the region with which Ecuador is compared differently: EF’s index includes only Spanish- and Portuguese-speaking countries of the Americas, while ETS groups countries geographically in the region ‘Americas’ and includes French-speaking countries, like Haiti, and English-speaking countries, like the United States, where presumably test takers are primarily ‘non-native’ speakers. ETS also reports average TOEFL scores by native language (see column 4 of Table 3), and Ecuador’s results are equally typical of Spanish speakers.2

Table 3. Regional comparison of Ecuador’s TOEFL results, 2011 to 2021

Source: ETS (2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021

My argument is not that TOEFL results represent the country’s linguistic skills better than the EF EPI, but rather that, due to differences in methodology and presentation of the data, the lessons drawn from numerical evidence can vary dramatically. The version told by the EF EPI—that the country has an alarmingly low level of English that resists efforts for improvement—is not the only way to describe Ecuadorians’ performance in this area; and for the reasons described above, there are abundant reasons to take this version with reservations.

Second assumption: the comparison of countries is meaningful, evident and impartial

The second assumption regarding the EF EPI that I would like to call into question is that the comparisons it makes between countries are meaningful—that there are real differences in language proficiency when a country has a higher or lower score than another—and that publishing this type of comparison is a natural and disinterested use of the data. In this section, I will discuss the lack of support for these international comparisons, the interest in making them, and the possible consequences.

Lack of support for international comparisons

As a public, we have become accustomed over the last twenty years to the rankings and comparative results of international large-scale assessments. In discussions of educational policy, PISA rankings, for instance, are cited as if they indicated precisely how relatively effective one country’s educational system is compared to another, regardless of contextual differences (Gillborn et al., 2018). Despite many calls to contextualize these comparative measures and to interpret them with caution, they tend to be ‘taken for granted’ (Grek, 2009).

But the problem with the EF EPI’s comparisons goes beyond the general skepticism that may be applied to ILSAs in general. In Section 3.1, I addressed the importance that large-scale studies be based on a representative sample that takes into account the size and heterogeneity of the overall population (Cohen et al., 2018). Nonetheless, the EF EPI simply includes all the countries with at least 400 tests taken and does not report the number of observations available by country, though it asserts that, in many cases, the number is above the minimum. Despite this limitation, it claims that the value of its index lies in its capacity to make comparisons (EF Education First, 2022).

We should therefore understand the EF EPI ranking as the product of a deliberate decision to compare results and make classifications. Again, the example of the TOEFL allows us to consider what alternatives exist for representing national assessment data when there is no sampling procedure that would support making comparisons and there are various contextual factors to account for. ETS provides tables with the average scores for each country where at least 30 exams have been taken; the countries are organized by region and, within the region, alphabetically, not by score. The same type of table provides average scores by native language. Each report specifies that the purpose of these tables is to be able to compare an individual’s results with the average of his or her peers, and includes the following warning:

ETS, creator of the TOEFL test, does not endorse the practice of ranking countries on the basis of TOEFL scores, as this is a misuse of data. … The differences in the number of students taking the test in each country, how early English is introduced into the curriculum, how many hours per week are devoted to learning English, and the fact that those taking the test are not representative of all English speakers in each country or any defined population make ranking by test score meaningless. (ETS, 2020, p. 19)

All of the same reasons for caution that ETS mentions are also relevant in the case of the EF EPI. However, rather than cautioning against international comparisons, EF promotes and publicizes them.

Publicity and the interest in producing a ranking

In contrast to the simple tables of TOEFL scores, the EF EPI reports are visually attractive and easy to read and interpret. The index has its own website, where the rankings appear together with the banner “EF EPI in the press”, highlighting newspapers and magazines that have published articles about these results (EF Education First, 2022). In these ways, the EF EPI is designed to be attention-getting and publicized. Producing an international ranking generates news value, creating content that spreads ‘organically’ (autonomously, beyond paid publicity).

EF’s mission is “to open the world through education” (EF Education First, 2021a), but it is also a private for-profit company, which implies that it will necessarily act in accordance with its interest in generating profit as well as educating. Let us take the first paragraph of the most recent EF EPI report as an example: “In a globalized world, the adoption of a common language is inevitable. Communication enables connection, accelerates innovation and facilitates understanding… People are learning English because it is useful to them” (EF Education First, 2021b, p. 4). This sounds like an advertisement for English study, and it is. Publishing an English proficiency index creates a platform for EF to generate interest in its services, especially in countries that have a low classification in the rankings. The fact that the categorizations EF constructs tend to present proficiency levels in a negative light (classifying as ‘very low’ what is described by another framework as ‘independent user’) also coincides with EF’s interests. If our English is poor, we should improve it through study—and, among the options for study, EF has a competitive advantage thanks to the reputation it builds with its own ranking.

While the narrative that our English is poor works in favor of the business interests of companies like EF that sell language study programs, it may have worrisome consequences for the educational system. In countries where performance on ILSAs is comparatively low, these results tend to be presented in the media as a failure of the national education system and public confidence in the education system declines (Pizmony-Levy & Bjorklund, 2018). This in turn strengthens initiatives to reduce public funding in education and motivates those with the means to withdraw from the public system, which further weakens it. This is a familiar and worrying pattern in Ecuador, where socioeconomic status is highly predictive of academic performance, where (INEVAL, 2018) the quality of English learning is especially associated with the difference between public and private schooling. We should consider that when Ecuadorians’ English level is negatively evaluated, we tend to blame the public education system.

It is also in EF’s interests to establish itself as an educational authority, educational policy actor, and trusted provider. For example, in 2019, EF collaborated with the Municipality of Quito to assess students at municipal high schools for free. The Municipality’s blog illustrates how EF inserts itself into educational policy, commenting that:

The idea is, looking ahead, to start a conversation and make decisions to improve teaching... EF is a world leader in language teaching. “We have experts in linguistics who can give advice and contribute to this process of improvement, not only in student evaluation but also, on another level, in teacher training and education”, Iemmolo [Administrative Director of EF] reiterated. (Municipio de Quito, 2019, paragraph 7)

This is a perfect example of how edu-business, a growing business model in the last decades, operates (Ball, 2012). A for-profit educational company provides data that is useful for evaluating and guiding education policy, and also sells solutions to the problems diagnosed by its data. Ball (2012) explains that, in this way, the edu-business creates a “virtuous circle” that produces new profit opportunities (p. 99). To the extent that private companies provide needed resources and information, the intersection of business and educational goals may seem convenient. However, these instances of corporate participation in education policy legitimize the participation of corporations in decision making that was previously the purview of the State, despite their having no responsibility to the public or to the common good (Robertson & Verger, 2012; Zakharia & Menashy, 2020).

To sum up, I invite us to consider that when we share the EF EPI rankings, we are not necessarily contributing to a meaningful exploration of the quality of English language education, but supporting the interests of a for-profit company to promote itself as well as corporate authority in education—a worrisome trend with implications for equity in the quality of that education.

Third assumption: concern for language proficiency is equivalent to concern for equity

The last assumption that I would like to address is the idea that concern for who acquires language proficiency is in itself a way of addressing issues of equity. This assumption is illustrated in the executive summary of the 2021 EF EPI: “English has the potential to be a powerful driver of diversity and inclusion over the coming decade, if only everyone gets an equal shot at learning it” (EF Education First, 2021b, p. 5). There are two components to this assumption: the idea that learning English necessarily brings certain benefits, and the idea that access to effective language learning is sufficient to access those benefits.

The assumption that English is a driver of economic and social development

It is undeniable that many jobs and academic programs require good English proficiency: in Ecuador, many of us can identify moments when we obtained or missed certain opportunities due to our ability to speak English. This experience leads us to believe that, at a societal level, English learning generates opportunities and eliminates barriers. The EF EPI claims as much, asserting that “English increases economic competitiveness” (EF Education First, 2021b, p. 4) and that “because English opens up international opportunities, it can expand socioeconomic mobility beyond that which a country can provide on its own” (EF Education First, 2021b, p. 16). The claim that English can improve socioeconomic condition is usually based on the correlation between English proficiency (according to unreliable indicators) and GDP or per capita income (EF Education First, 2021b; McCormick, 2013). But such correlations do not necessarily imply that skill in English produces prosperity. We could also interpret good English as a product of the additional resources found in countries with greater economic prosperity.

Various authors have explored and questioned the instrumental value of English as a driver of economic and social development (Grin, 2001; Kubota, 2011; Mackenzie, 2021). The study most relevant to this analysis is Mackenzie’s exploration of the relationship between English and development in Colombia. Using various indicators, he examines the possible benefits of English for Colombians in terms of income and employment, trade and economic growth, international mobility, schooling and higher education, and foreign aid. His conclusion is that the benefits of learning English in the country go principally to the most socioeconomically advantaged, and that the presence of English in Colombia actually exacerbates existing inequities. This conclusion matches previous studies in other parts of the world that suggest that English proficiency is a positional good whose value depends largely on context (Grin, 2001) and that an individual’s capability to take advantage of his or her linguistic skills depends on gender, race, education level, and other factors related to social status (Mackenzie, 2021).

When EF proposes English as “driver of diversity and inclusion”, it also warns that “if English skills are distributed unequally, they will only compound the problem” (EF Education First, 2021b, p. 16). This is doubtless true, but framing it as a question of access to the language implies that, if we improve English for everyone, we will be addressing inequality and exclusion. Mackenzie’s (2021) analysis cautions that it is not only a question of the distribution of skill, but also of the social structures that create circumstances where the most advantaged members of society can reap the benefits of English, while those facing social and economic barriers have little chance to access socioeconomic mobility and social inclusion with or without English. That is not to say that teaching English or improving access to English learning is useless, but rather that those efforts do not fix problems of equity. We should not pretend that English is in itself a solution, particularly when it may even reinforce inequality.

Of course, EF is far from the only actor in the English language teaching field that promotes the assumption that English learning implies prosperity and equity. This is the rhetoric of English language educational policies of many countries, including Ecuador. It is based in human capital theory, the most influential educational theory of the last century (Tonini, 2021), which proposes that education has an economic value because investing in education is profitable for individuals and societies. However, this theory presupposes that the economic and social structure can provide better-paying jobs to people with greater educational attainment, which is not the case in all contexts (Tonini, 2021). Even though this perspective on English goes beyond the EF EPI, it should be an important consideration in the way that we talk about this index.

The power of large-scale assessments to legitimize educational doctrines

While EF’s discourse on the instrumental value of English in society is the product of the dominant global narrative about English and education, the EF EPI is also a powerful force in the legitimation of that narrative. As Cardoso (2020) explains (see Section 2.2), international large-scale assessments have the power to produce evidence that confirms their own perspectives on education and thus construct an educational landscape where their results are cited and respected. Such is the case with the EF EPI: EF’s Senior Vice-president for Academic Affairs uses the EF EPI data to assert in the Harvard Business Review that the interaction between English proficiency and gross national income per capita is a virtuous cycle, with improving English skills driving up salaries, which in turn give governments and individuals more money to invest in language training. (McCormick, 2013, párrafo 4)

This assertion is based on the fallacy that correlation proves causation. McCormick’s publication is then cited by publications about English teaching in Ecuador (Orosz et al., 2021) and in the region (Hernández-Fernández & Rojas, 2018) as evidence of the importance of English. In this way, the narrative that English is a vehicle for prosperity and inclusion to the extent that acceptable levels of proficiency are attained—a narrative written by EF and confirmed using its own data—becomes a natural element of how we in Ecuador understand the role of English.

The assumption that concern for linguistic skill equals concern for equity is influential in educational policy and practice. Cardona-Escobar et al. (2021) demonstrate how, in the case of Colombia’s Plan Nacional del Bilingüismo, English learning is framed as a solution to problems of equity in Colombian society. At the same time, the tools that are put forth to support that learning are standardized curriculum and testing that enable accountability to international standards. The problem with this strategy is that it does not represent a path to equity but rather “a desire for equality of inputs and resources and a hope for an equality of outcomes (laudable, but not equitable, goals)” (Cardona-Escobar et al. 2021, p. 8-9). Standardizing methods, materials and expectations does not mean that all students will have the same opportunity to learn English because, as Cardona-Escobar et al. explain, equity requires differentiating the educational approach according to specific populations and needs. The discourse of English as a vehicle for inclusion and equity, though it pays lip service to the goal of equity, does not actually help create equitable educational conditions.

Nonetheless, according to the perspective found in the EF EPI and in much public discussion about English, the important thing is that Ecuador’s average level is inadequate and must be raised to improve our participation in the global labor market. When one perspective becomes so generalized that it is almost invisible, it tends to make other possibilities unthinkable. What other ways of thinking about English teaching might we be missing when we perpetuate EF’s narrative? Of course, the country’s level of English certainly can and should be improved; but what other results, aside from listening and reading scores, could also be important in English classrooms? For instance, what about the socioemotional wellbeing of students and teachers, or the valuing of diverse cultural perspectives that languages, English and others, including local languages, can bring with them? These aspects of education are, in fact, mentioned in the national curriculum, but since they are not evaluated by international rankings, successes or failures, do not have much weight in English language educational policy. And beyond the classroom, what changes are needed so that learning English really makes sense for our least advantaged students? This question is even less addressed but is fundamental to ensuring that our efforts to improve English teaching really contribute to the common good and not only the benefit of a few.

6. Conclusion

In the EF EPI, we find a convenient source of data to describe English language proficiency in Ecuador and contrast it with other countries; yet we should think twice before taking this data at face value and citing it as fact. Like any type of evidence, the indicators this instrument provides are the product of human decisions and they imply an ideological position and a set of interests that do not necessarily coincide with the interests of Ecuadorian society. We see here that the assumptions supporting the importance and authority of the EF EPI are undermined by limitations in sampling, methodological inconsistencies, and misguided or deceptive use of correlated indicators, together with a discourse that confuses equity with the application of uniform standards. By calling attention to Ecuador as deficient in English, the EF EPI feeds a narrative according to which Ecuadorians are bad at learning English or the Ecuadorian educational system is bad at teaching it. Neither version of this narrative contributes to improving the conditions for learning English in Ecuador or for equity in education, but they do serve EF’s commercial interests. Though it is not unlikely that Ecuador’s English is in fact quite low, as the EF EPI asserts, I believe there are reasons to think that, rather than being the embarrassment of the region, our English is quite typically mediocre and has improved in the last ten years. In any case, the undisputed dissemination of a discourse that sells superficial and profitable solutions to complex social problems is worrisome.

With the arguments of this essay, I invite those of us who discuss English learning in Ecuador to approach the EF EPI with caution and to be deliberately skeptical when we refer to it. My recommendation would be for us to mention the specific methodological limitations of the EF EPI when we cite it, or at least clarify that its methodology is problematic. We can also avoid presenting it as the only possible source of information and mention other types of data on Ecuadorians’ English performance, such as the TOEFL. It would be good practice to note that the organization that produces the EF EPI, a for-profit company, has a specific perspective and motivations related to English learning.

Longer term, we should analyze what information regarding the state of English learning would be most valuable in the Ecuadorian context and generate alternative forms of evaluation to those currently available that could provide information that would guide us towards both quality and equity. Finally, when we talk about the role English may play in opening up opportunities for Ecuadorians, let’s also talk about the need for educational policies that are equitable and differentiated for those students who may encounter barriers when trying to learn the language or benefit from its use. In their critique of the way quantitative indicators can be idealized, Gillborn et al. (2018) note that, too often, those who use numerical data lack awareness of or interest in issues of social inequality. Let’s be the exception.

References

Addey, C., & Sellar, S. (2018). Why do countries participate in PISA? Understanding the role of international large-scale assessments in global education policy. In A. Verger, M. Novelli, & H. K. Altinyelten (Eds.), Global education policy and international development: New agendas, issues, and policies (2nd ed., pp. 97–117). Bloomsbury.

Argudo, J., Fajardo, T., Abad, M., & Cabrera, H. (2021). Students’ Perceptions on Their EFL Teacher Efficacy: A Study on EFL Teachers’ Language Proficiency and Their Self-Efficacy. MEXTESOL Journal, 45(1), 1-12.

Ball, S. J. (2012). Global Education Inc.: New policy networks and the neoliberal imaginary. Routledge.

Bray, M., Kobakhidze, M. N., & Suter, L. E. (2020). The Challenges of Measuring Outside-School-Time Educational Activities: Experiences and Lessons from the Programme for International Student Assessment (PISA). Comparative Education Review, 64(1), 87–106. https://doi.org/10.1086/706776

Cajas, D. (2017). Characterising the challenges and responses of Ecuadorian universities to recent EFL language policy changes: A mixed methods study [PhD Thesis]. University of Warwick.

Cardona-Escobar, D., Pruyn, M., & Barnes, M. (2021). Colombian National Bilingual Plan: A vehicle for equity or an instrument for accountability? Journal of Multilingual and Multicultural Development, 1–14. https://doi.org/10.1080/01434632.2021.1980574

Cardoso, M. E. (2020). Policy evidence by design: International large-scale assessments and grade repetition. Comparative Education Review, 64(4), 598–618. https://doi.org/10.1086/710777

Castillo, L. (2021, November 27). Ecuador mantiene un bajo nivel de dominio del idioma inglés. El Comercio.

Cohen, L., Manion, L., & Morrison, K. (2018). Research methods in education (8th ed.). Routledge.

Council of Europe. (2021). Common European Framework of Reference for languages: Learning, teaching, assessment. Companion volume with new descriptors.

Cronquist, K., & Fiszbein, A. (2017). English Language Learning in Latin America. The Dialogue.

EF Education First. (2011). EF English Proficiency Index.

EF Education First. (2012). EF English Proficiency Index.

EF Education First. (2013). EF English Proficiency Index.

EF Education First. (2014). EF English Proficiency Index.

EF Education First. (2015). EF English Proficiency Index.

EF Education First. (2016). EF English Proficiency Index.

EF Education First. (2017). EF English Proficiency Index.

EF Education First. (2018). EF English Proficiency Index.

EF Education First. (2019). EF English Proficiency Index.

EF Education First. (2020). EF English Proficiency Index.

EF Education First. (2021a). Company Fact Sheet 2021.

EF Education First. (2021b). EF English Proficiency Index.

EF Education First. (2022). EF EPI 2021 – EF English Proficiency Index – Sobre EF EPI.

El Universo. (2019, December 30). Ecuador, peor país de Latinoamérica en dominio de inglés, según informe. El Universo.

El Universo. (2020, November 26). Ecuador, el peor país de la región en dominio del idioma inglés, según análisis; educadores dan sus observaciones. El Universo.

ETS (2013). Test and Score Data Summary for TOEFL iBT® Tests.

ETS (2014). Test and Score Data Summary for TOEFL iBT® Tests.

ETS (2015). Test and Score Data Summary for TOEFL iBT® Tests.

ETS (2016). Test and Score Data Summary for TOEFL iBT® Tests.

ETS (2017). Test and Score Data Summary for TOEFL iBT® Tests.

ETS (2018). Test and Score Data Summary for TOEFL iBT® Tests.

ETS (2019). Test and Score Data Summary for TOEFL iBT® Tests.

ETS. (2020). TOEFL iBT®Test and Score Data Summary 2019.

ETS (2021). TOEFL iBT®Test and Score Data Summary 2020.

ETS. (2022a). About ETS.

ETS. (2022b). ETS Research: Data Requests.

ETS. (2022c). Interpreting TOEFL ITP® Scores.

ETS. (2022d). TOEFL Essentials Score Setting (For Score Users).

Fleckenstein, J., Keller, S., Krüger, M., Tannenbaum, R. J., & Köller, O. (2020). Linking TOEFL iBT® writing rubrics to CEFR levels: Cut scores and validity evidence from a standard setting study. Assessing Writing, 43, 100420. https://doi.org/10.1016/j.asw.2019.100420

Gillborn, D., Warmington, P., & Demack, S. (2018). QuantCrit: Education, policy, ‘Big Data’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. https://doi.org/10.1080/13613324.2017.1377417

Grek, S. (2009). Governing by numbers: The PISA ‘effect’ in Europe. Journal of Education Policy, 24(1), 23–37. https://doi.org/10.1080/02680930802412669

Grin, F. (2001). English as economic value: Facts and fallacies. World Englishes, 20(1), 65–78. https://doi.org/10.1111/1467-971X.00196

Harsch, C., Ushioda, E., & Ladroue, C. (2017). Investigating the predictive validity of TOEFL iBT® test scores and their use in informing policy in a United Kingdom university setting. ETS Research Report Series, 2017(1), 1–80. https://doi.org/10.1002/ets2.12167

Hernández-Fernández, J., & Rojas, J. (Eds.). (2018). English public policies in Latin America: Looking for innovation and systemic improvement in quality English language teaching. British Council Mexico.

IELTS Partners. (2022). IELTS Demographic data 2019.

INEVAL (2018). Educación en el Ecuador: Resultados de PISA para el desarrollo.

Kubota, R. (2011). Questioning linguistic instrumentalism: English, neoliberalism, and language tests in Japan. Linguistics and Education, 22(3), 248–260. https://doi.org/10.1016/j.linged.2011.02.002

Kyle, K., Crossley, S. A., & McNamara, D. S. (2016). Construct validity in TOEFL iBT speaking tasks: Insights from natural language processing. Language Testing, 33(3), 319–340. https://doi.org/10.1177/0265532215587391

Macías, K., & Villafuerte, J. (2020). Teaching English Language in Ecuador: A Review from the Inclusive Educational Approach. Journal of Arts and Humanities, 9(2), 75-90.

Mackenzie, L. (2021). Linguistic imperialism, English, and development: Implications for Colombia. Current Issues in Language Planning, 1–20. https://doi.org/10.1080/14664208.2021.1939977

McCormick, C. (2013, November 15). Countries with Better English Have Better Economies. Harvard Business Review.

Ministerio de Educación, Ecuador. (2016). Introduction: English as a Foreign Language.

Municipio de Quito. (2019, November 19). Más de 5000 estudiantes municipales serán evaluados en su nivel de inglés. Quito Informa.

Orosz, A., Monzón, M., & Velasco, P. (2021). Ecuadorian teachers’ perceptions of teaching English: Challenges in the public education sector. International Journal of Learning, Teaching and Educational Research, 20(3), 229–249. https://doi.org/10.26803/ijlter.20.3.14

Pizmony-Levy, O., & Bjorklund, P. (2018). International assessments of student achievement and public confidence in education: Evidence from a cross-national study. Oxford Review of Education, 44(2), 239–257. https://doi.org/10.1080/03054985.2017.1389714

Robertson, S. L., & Verger, A. (2012). Governing education through public private partnerships. In S. L. Robertson, K. Mundy, A. Verger, & F. Menashy (Eds.), Public private partnerships in education: New actors and modes of governance in a globalizing world (pp. 21–41). Edward Elgar Publishing Limited.

Schuelka, M. (2013). Excluding Students with Disabilities from the Culture of Achievement: The Case of the TIMSS, PIRLS, and PISA. Journal of Education Policy, 28(2), 216-230. https://doi.org/10.1080/02680939.2012.708789

Sevy-Biloon, J., Recino, U., & Muñoz, C. (2020). Factors Affecting English Language Teaching in Public Schools in Ecuador. International Journal of Learning, Teaching and Educational Research, 19(3), 276-294. https://doi.org/10.26803/ijlter.19.3.15

Tonini, D. C. (2021). Human capital theory in comparative and international education: Development, application, and problematics. In T. D. Jules, R. Shields, & M. A. M. Thomas (Eds.), The Bloomsbury handbook of theory in comparative and international education (pp. 69–86). Bloomsbury Academic.

Zakharia, Z., & Menashy, F. (2020). The emerging role of corporate actors as policymakers in education in emergencies: Evidence from the Syria refugee crisis. Journal on Education in Emergencies, 5(2), 40–70. https://doi.org/10.33682/pcbg-2fu2

1 EF asserts that its index is reliable in part because of the correlation of its results with TOEFL and IELTS tests (EF Education First, 2022). I use only the TOEFL as an example because IELTS does not report results for Ecuador (IELTS Partners, 2022).

2 While ETS provides averages by native language, it does not provide regional averages, so I have calculated the regional averages based on the national averages provided, without weighting these by population.

---

De Angelis, A. (2022). EF English Proficiency Index and English in Ecuador: Uncertain assumptions of the international ranking. Revista Andina de Educación 5(2), 0005211. Publicado bajo licencia  CC BY-NC 4.0