THE ICT LITERACY FRAMEWORK

A.Introduction

What does it mean to be a literate member of society? The growing acceptance of lifelong learning has expanded the views and demands of literacy. Literacy is no longer seen as a condition that one either has or is lacking. Rather, it is seen as a continuum of knowledge, skills, and strategies that individuals acquire over the course of their lives in various contexts and through interactions with their peers and with the larger communities in which they participate. As historians remind us, literacy in its earliest form consisted of little more than being able to sign one’s name on a legal document. It was not until later that fluent oral reading became important and not until the 20th century that reading to gain information was given primary emphasis. As we move into the 21st century, our conception of literacy is evolving once again. The prevalence of technology in the everyday lives of the world’s citizens has grown at a rate that many would have found hard to imagine 25 or even 10 years ago. Policy makers, business leaders, and educators have come to expand their notion of a literate populace to include the skills and abilities that will enable citizens to function in an increasingly technological world.

B. Developing a Framework

The task of the International ICT Literacy Panel was to develop a framework that would define ICT literacy and provide the foundation for the design and conduct of large-scale assessments and diagnostic tests. While the chief benefit of developing a framework for ICT literacy is improved measurement, a number of other potential benefits are also seen as important. Namely,

• A framework provides a common language and a vehicle for discussing the definition and assumptions surrounding the domain.

• Such a discussion provides a mechanism for building consensus around the framework and measurement goals that grow from it.

• We construct a better understanding of what is being measured through the process of developing the framework and linking it to evidence collected from assessment tasks.

• This understanding and its connection to what we say about learners provides an important link between public policy, assessment, and research which furthers the utility of the data that are collected. To accomplish this task, the panel chose to adopt the process used to develop frameworks for the International Adult Literacy Survey (OECD & STATCAN 1995; OECD & HRDC 1997; OECD & STATCAN 2000) and for the Reading Literacy Survey conducted as part of PISA, the Programmer for International Student Assessment (OECD, 1999). This process consists of six steps, shown in the following diagram and explained more fully below (Kirsch 2001).

1. The first step is to develop a working definition of the domain including the assumptions underlying it. Before the definition is developed, the domain and the skills and abilities it encompasses are wide open. It is the definition that sets the boundaries for what will be measured and what will not.

2. Once the definition is developed, it is important to think about the kinds of tasks that represent the skills and abilities included under that definition. Those tasks must then be categorized, or organized, to inform test design and result in meaningful score reporting. Step 2 allows one to move beyond a laundry list of tasks or skills to a coherent representation of the domain that will permit policy makers and others to summarize and report information in more useful ways.

3. Step 3 involves identifying a set of key characteristics that will be used in constructing tasks for the assessment. This may include characteristics of the stimulus materials to be used as well as characteristics of the tasks presented to examinees.

4. In step 4, the variables associated with each task characteristic are specified.

5. In step 5, research is conducted to show which variables account for large percentages of the variance in the distribution of tasks and thereby contribute most towards understanding task difficulty and predicting performance.

6. Finally in step 6, an interpretative scheme is built that uses the validated variables to explain task difficulty and examinee performance.

The work of this panel involved the first two steps: defining ICT literacy and organizing the domain.

C. Defining ICT Literacy

The International ICT Literacy Panel was comprised of educators, technology experts, scholars and industry and labor representatives from Australia, Brazil, Canada, France, and the United States. Our deliberations resulted in the following definition:

ICT literacy is using digital technology, communications tools, and/or networks to access, manage, integrate, evaluate and create information in order to function in a knowledge society. This definition carries several assumptions made by the panel and therefore it is important to consider each part of the definition in turn. “ICT...” Information Technology (IT) has been used for many years, particularly in the United States, and refers to the electronic display, processing, and storage of information, but not necessarily the transmission of the information. The term carries strong historical associations with enterprise data processing and centralized computer services. However, Information and Communication Technology (ICT) represents the set of activities and technologies that fall into the union of IT and communication technologies. Global industry, international media, and academics increasingly now use ICT to describe this union. The real benefit of adding “communication” doesn’t derive from including specific technologies, such as routers or servers, but from the dynamism implicit in interconnected social, economic, and information networks. ICT is characterized by unprecedented global flows in information, products, people, capital, and ideas. These flows are enabled by ICT: their sheer scale and pace would not be possible without the ability to connect vast networks of individuals across geographic boundaries at negligible marginal cost.“...literacy is...” The panel selected the term literacy over other terms such as competency, ability, or fluency that have been used in earlier frameworks (Committee on Information Technology Literacy, 1999). To some “literacy” connotes functional literacy and implies basic or fundamental skills. To the panel, the term literacy implies a universal need, a condition that must be met to enable full and equitable economic and social participation. We view literacy as a tool that may be applied to simple or more complicated contexts — like a hammer that can be used to build a shelf, or a house. In its broadest sense, literacy is a dynamic tool that allows individuals to continuously learn and grow. The increasing role of technology in our lives requires us to expand our notion of literacy. It is obvious that to function fully and effectively in society, individuals must be literate in terms of traditional domains such as reading and numeracy. But today it is becoming increasingly clear that ICT literacy joins the ranks of essential and fundamental requirements. Perhaps as important is the panel’s belief that those who fail to acquire this new kind of literacy, like the more traditional literacy skills, will find themselves falling further behind as economies and societies grow and change over the years ahead.

“...using digital technology, communications tools, and/or networks...”

The description of digital technology, communication tools, and/or networks reflects the same thinking that stimulated the panel’s use of information and communication technology (ICT) versus information technology (IT). Digital technology reflects hardware and software products, communication tools reflect those products and services used to transmit information, and networks themselves are the pathways for this transmission. The words are meant to be as inclusive as possible to reflect the breadth of hardware, software, and infrastructures that makeup ICT.

“...to access, manage, integrate, evaluate and create information...”

Technology is used for an ever-increasing range of purposes to accomplish many different kinds of tasks. This phrase is meant to reflect that range as well as to define five critical components of ICT literacy. The five components represent a continuum of skills and knowledge and are presented in a sequence suggesting increasing cognitive complexity. After discussions regarding the kinds of tasks represented by each component, the panel agreed to the following definitions:

• Access - knowing about and knowing how to collect and/or retrieve information.

• Manage - applying an existing organizational or classification scheme.

• Integrate - interpreting and representing information. It involves summarizing, comparing and contrasting.

• Evaluate - making judgments about the quality, relevance, usefulness, or efficiency of information.

• Create - generating information by adapting, applying, designing, inventing, or authoring information.

“...in order to function in a knowledge society.”

This phrase reflects the range of contexts in which individuals will be able to apply their ICT literacy —from defined ones such as graduating from school or functioning on a job to those which are less defined and less concrete but which can extend and enrich one’s personal life. The phrase “in order to function” is meant to acknowledge the fact that ICT literacy will provide individuals with a means of contributing to and benefiting from economically developed or developing societies. We believe that ICT literacy skills are becoming increasingly important not only for nations to maintain or improve their standard of living but for the well being of individuals as well. The phrase “in a knowledge society” refers to the changing nature of cultures in the 21st century — an age in which ideas and information are increasingly the drivers of progress. The expanding roles of technology and access to information on a global scale have the potential to change, and hopefully improve, the way we live, learn and work.

D. Organizing the Domain

Once we had defined what was meant by ICT literacy and laid out the assumptions underlying that definition, the next step was to develop an organizing framework for ICT literacy. This is an important step because the way in which the domain is organized affects test design and the kinds of tasks that will be developed to provide evidence about the status of ICT literacy in a population of interest. The panel’s task was to define the critical organizing categories for the domain of ICT literacy and how they were related. In our definition of ICT literacy, we identified five components we view as essential for functioning in a knowledge society: accessing, managing, integrating, evaluating and creating information in a technology context. 

These components, represented in Figure 1, formed the initial organizational scheme   for the domain of tasks that make up ICT literacy.

Upon further consideration, however, we chose to expand this unidimensional model to more fully represent the complexity of ICT literacy. This organizational scheme, shown below in Figure 2, illustrates the foundational set of skills and knowledge that underlie ICT literacy: cognitive and technical proficiency.

The three proficiencies are defined as follows.

• Cognitive Proficiency — the desired foundational skills of everyday life at school, at home, and at work. Literacy, numeracy, problem solving, and spatial/visual literacy demonstrate these proficiencies.

• Technical Proficiency — the basic components of digital literacy. It includes a foundational knowledge of hardware, software applications, networks, and elements of digital technology.

• ICT Proficiency — the integration and application of cognitive and technical skills. ICT proficiencies are seen as enablers; that is, they allow individuals to maximize the capabilities of technology. At the highest level, ICT proficiencies result in innovation, individual transformation, and societal change.

As conceived in this framework, ICT literacy includes both cognitive and technical proficiency. For example, in order to successfully perform an ICT task such as searching the Internet to find and compare treatment options for a medical condition, an individual must apply reading and problem solving skills (cognitive) and be able to access information on the Internet using a search engine (technical). While cognitive and technical proficiencies are both necessary components of ICT literacy, each is a distinct domain. Cognitive and technical proficiency each represent independent domains in which the associated knowledge and skills interact to influence ICT literacy. An example is shown below in Figure 3.

One would expect that individuals with low cognitive proficiency but high technical proficiency (cell ‘D’) would be able to perform particular technical tasks in which they had been trained. However, they would probably not possess the kind of generalizable skills or knowledge that could help them work with new applications or perform novel tasks and they would most likely not be able to acquire such skills independently. People with high cognitive proficiency but low technical proficiency (cell ‘A’) would require technical training (and possibly motivation or practice time) in order to develop ICT proficiency but would be expected to do so and once engaged with ICT would be able to acquire new skills and knowledge quickly and independently. The representation of ICT literacy shown in Figure 2 provides an organizational scheme for both developing large-scale assessments or individual tests and evaluating existing measures. The framework leaves open the possibility that different constituencies could develop different assessments or individual tests for different purposes by focusing on various parts of the framework itself and by defining the kinds of evidence that might be associated with each. For the purposes of discussion, we present three types of assessments that might be developed using the ICT literacy framework: holistic, component and diagnostic.

A holistic assessment would be of most interest to constituencies who wished to focus on how well a test taker completed a given set of tasks rather than on the component skills that make up those tasks. An example would be a task that required test takers to create a flyer for a neighborhood clean-up day. Specific task requirements as well as information such as when and where the event was to be held would be provided and test takers would be scored on how well they completed the final product. Such an assessment would allow one to rank order groups or individuals and to make general statements about their ICT knowledge and skills. Alternatively, one might choose to build an assessment that independently measured the knowledge and skills associated with each of the five components of ICT Proficiency (access, manage, evaluate, integrate and create).

This component assessment would result in a measure that could provide general information about the kinds of generative ICT tasks a population, or an individual, could perform. An example of one such task and its associated proficiencies is presented below.

But if an individual or a group of individuals performed poorly on this measure, one would be hard pressed to understand or explain why. Were there underlying reading or language problems? Did test takers have sufficient technical knowledge to complete the tasks presented in the ICT measures? To understand what role these other domains contributed one would have to include cognitive and technical tasks in the assessment or test. Alternatively, one might want to focus on particular ICT proficiencies (for example, how well a person can access and manage information) and their underlying cognitive and technical components.

This would involve creating tasks that measured these types of skills and knowledge across the three proficiency domains. These measures would provide evidence separating literacy and technology proficiencies from ICT proficiency. Such information would be useful for constituencies such as adult basic education centers interested in diagnosing and remediating problems students are having accessing information on the Internet. A series of tasks that might be appropriate in this context are presented below.

Students having trouble with this basic ICT task could be presented with related cognitive and technical tasks to help diagnose what was causing their difficulty. For example, students might be presented with multiple-choice questions asking them to select the best word or phrase to use when searching for some specified information. Included among the choices might be terms that are overly general or specific. Students having difficulty with this type of task might need practice in defining categories and efficient search strategies. In addition, very basic computer tasks, such as opening a search engine, clicking on sites, and navigating back to the search engine from those sites, might uncover technical skills requiring review or training. Currently, there are various measures of literacy, numeracy and problem solving being used in large-scale assessments of school age and adult populations. There is also a measure of technical knowledge and understanding that is being used with school-aged populations. These are traditional paper and pencil measures. No attempt has been made, however, to build computer-based tasks to measure the integration of these cognitive and technical domains or to separate out the role each plays in the development of these more generative ICT proficiencies. The panel believes that the measurement of ICT literacy using paper and pencil will limit the ability to assess the full domain of knowledge and skills. Valuable information will be lost if assessment tasks are not embedded in real-world settings incorporating technology. For example, the measurement of an individual’s ability to search for and access information would be hindered if the measurement did not provide an opportunity to log onto the Internet or a similar type of environment.

E. Next Steps

As the panel began its deliberations about ICT literacy and how should it be defined and operationalized, we soon recognized that many of our discussions focused around the issue of the digital divide. This divide is commonly defined in terms of connectivity and the inequalities of access that exist both within and across countries. The more important issue the panel recognized was that the true potential of ICT — that is, the ability to transform individuals and societies — came not just from being wired together but also from having the knowledge and skills to use technology and to understand the roles it can play in our lives. As the president of Morris Brown College recently stated, “Merely having access to a box — an information box — does not necessarily mean that you have improved, or that you’re more literate, or that you’re better able to solve problems in the community” (Young, 2001).

This perspective led the panel to determine what they saw as the important issue facing us as society continues to invest in technologies and as technology continues to alter the way we work and live our lives. Then we wanted to use this storyline as a lead in to the definition of ICT literacy and how it should be operationalized into a framework. This report has taken the initial steps in building a framework by providing a consensus definition of ICT literacy and a model that can be used to further operationalize this construct for a variety of purposes.

The next steps will involve defining the kinds of evidence that should be gathered with respect to each level of the model — ICT, cognitive and technical proficiencies — and the kinds of activities that would elicit that evidence.

This evidence and related activities will vary depending on the purpose of the planned assessment or test. The framework begun with this paper, along with a prototype of online tasks, will allow ETS to discuss the potential for large-scale assessments or individualized tests with potential sponsors. The major stakeholders who will be interested in this framework and its resulting assessments are international and diverse, and therefore create a unique challenge as well as opportunity. They include government policy makers, corporate leaders, industry associations, unions, workforce groups, educators (K-12, higher education, national educational associations, researchers), consumer and public interest groups, and relevant international associations. The buy-in, cooperation, and support of these groups will be essential in the achievement of global ICT literacy. ICT has become a permanent part of everyday life. It fundamentally changes how we live, learn, and work. Because ICT is considered an emerging and fundamental literacy, significant attention must be paid to insuring that all citizens have access and opportunity to gain the needed skills to function effectively in a knowledge society.

Reference

Panel, I. L. (2002). Digital transformation: A framework for ICT literacy.Educational Testing Service.‏