Organizing Data for Education Assessment

"I have buckets of information all over the place. What should I do with it?" If you have this question read on....

The best place to start is with an understanding of how data supports your goals. A clear description of your short, medium, and long-term organizational objectives should be supported by data. Documents like strategic plans, institutional reports, and budget justifications should already exist. These documents should be used to help define the relationship between goals and data.

The next step is to take inventory. Just how many "buckets of information" do you have? Where are they? And, what is the condition of the data in those buckets? Once again, documentation is needed to validate and share your findings. Check with institutional research, the provost's office, admissions, information technology services, as well as administrative staff within the unit, to find data sources.

Let's say you have a good idea of where you are headed in terms of goals. You know where data are located and its condition. The next step is to examine the data for “normalization” rules. Normalization rules are a product of the information technology profession. Few educators are familiar with these rules. The application of normalization rules do, however, enable data to be processed logically. It is also useful to know about the complexities of data as it relates to the education profession. The Center for Education Assessment has extensive experience organizing data for education assessment and reporting.

The data must be used consistently from one place in the system to the next. Processing logic dictates that there will be no duplicate data elements. Other technical rules for organizing data will also be applied. Applying organizational rules to groups of data within an information system enables the data to be aggregated and disaggregated.

Data also needs to be aligned with the computer system that will be used to process data. There are two basic options. You can modify an existing system or build one from scratch.

Many SCDEs have chosen to build an education assessment system from scratch. A "home-grown" system eliminates the need for model system customization. On the other hand, it adds to the skill sets, time, and money required to implement an acceptable assessment system.

A careful selection process is needed if you acquire an existing system. You will be limited by the data and logical relations within the system. For example, if you adopt a system that primarily deals with candidate portfolios, it will probably fall short when supporting broader goals that require data on faculty, productivity, and resources. On the other hand, if you decide to use an institutional system as a model, your education goals may not be adequately supported by data.

The more comprehensive and appropriate the model system the less modification will be needed. The CEA has a comprehensive model education assessment system that you should consider.

Let's recap. You now have a clear understanding of the purpose of the system. Available data has been inventoried. The processing logic is set through a decision on adopting system or investing in system analysis and design as part of a custom system. Now you are in a position to organize the data. Start with the reports that the system must produce. Work backwards to the data needed for the reports. If past reports do not exist for reference, mock up some fictitious reports. The key is to be specific about your reporting needs.

Trace the contents of the reports back to supporting data. If the specific data elements you need are in your inventory, you have a data bucket that you can use. If there are two or three data buckets with the same or similar information, you will need to either consolidate data from multiple sources or select a single data source. If critical data elements are missing, you will need to add data collection and file maintenance processes to your system.

You have identified the need for specific data as part of establishing a comprehensive information system. Now you need to implement Once data gas been identified, organized, and aligned with a well designed computer information system this assessment system can be built to stand the test of time. The completed computer system can utilize the logical relationships that tie data elements to the reports. These reports should serve as a support tool for education improvement.