Considerations for Program Configuration
This page is intended to help implementation teams determine the best way to set up programs if there are multiple categories of assessments.
Audience: Pearson delivery and customer lead project teams
See also:
This page explains:
- What is a Program?
- Importance of pre-planning
- Prerequisites
- Determining how many programs to create
- What is editable?
- FAQ
- Sample illustrations
What Are Programs in ADAM?
Programs group tests that are delivered over multiple testing windows for reporting purposes. A test must be assigned to a program if you want to access the Performance Report. Programs are designed to work for a single academic year, not multiple years. In most cases, the Pearson program delivery team will configure programs after working with customers to determine their reporting requirements.
What Should I do Before Creating a Program?
Before building out a program, take an inventory of all types of tests that will use ADAM in the upcoming school year (benchmarks, unit tests, diagnostics, etc.)
For each test type, determine the following:
- How many times will it be administered this year?
- What are the test windows called? (BOY, Unit 1 Fall, etc.)
- What grade levels are eligible (include off-grade testers)?
- If it includes high school grades, what content areas (e,g. Algebra I, HS ELA I, etc.)
- Will any tests have a separate Spanish version? If yes, do they need to be reported separately?
Why is it Important to Pre-Plan?
Each test in a program must have a unique combination of testing window, grade and content area. In the sample program setup shown below, this means:
- ‘Fall + 8 + Math’ could be used for ‘Grade 8 Math,’ but not also a separate test for ‘Grade 8 Math Honors’, because that combination has already been used; one solution to get around this would be to plan for a Content Area called Math Honors
- If Earth Science gets added as a test after other tests are assigned to this program, there is no way to add it as a content area
- If you want to report on a Spanish version of a test separately, build as separate tests rather than add an accommodated form in one test. If you want to have Math 5 and Math 5 Spanish, for example, you’ll need to plan for content areas of Math and Math (Spanish). Because you can multi-select in reports, you will be able to view the results next to each other (although not aggregated).
Sample Program Configuration: Each test must be tagged to a unique combination from these three fields.
How Many Different Programs Do We Need?
You must create a separate program for any test/collection of tests which use and require:
- Different set of score types (e.g., scale score, percent correct, points achieved, performance level)
- Different number of performance levels (differing cut scores are fine)
- Differing restrictions on what is visible in reporting (item content, item performance, performance by standard, etc.)
- Content created in both ADAM and external tests
What are the Advantages of More versus Fewer Programs?
The main advantage of having more programs is fewer options in report filter menus. Consolidating test categories into a single program allows for easier comparison of grades, subjects, and test windows in ADAM reports, including the Individual Student Report. Although reporting in ADAM is grouped by program, it is easy in reports to switch views from one program to another. Cross-program reporting is not currently available in ADAM
Note: The actual student's grade does not determine which grade ‘bucket’ they are reported into. Every student taking a specific test will be reported into the lowest grade tagged in the Program configuration. How students are eligible to take the test can be based on class enrollment or test assignment tags.
What is Editable?
Everything is editable in a program until the first test is linked. Make sure to plan and configure the program carefully before assigning tests.
After assigning a test to a program:
- Can be changed:
- Program Name
- Anything in the left column with a checkbox (all around report and program visibility)
- Program Code (is not required and not currently used or displayed on reports)
- Cannot be changed:
- Testing windows, grade levels, content areas, school year
- Anything about score types
Note: Cut scores, performance level labels and the option to display label only can be overridden at the test level before students start testing
FAQ
Q: Can I begin creating items and forms before program configurations are set?
A: Yes – configurations do not need to be set until you are ready to create tests
Q: Can test window dates overlap? (e.g., fall, spring, BOY)
A: Yes, these are just tags and have no bearing on testing dates
Q: For a site that needs score groups, how do I set up performance levels?
A: Set the performance level objective as the Primary Objective, using the Percent Correct as the Input Variable. Set Percent Correct as the Secondary Objective. This will optimize reporting. For cut scores, use the most commonly used ones (can override at the test level).
Q: How many score groups/performance levels should I have?
A: Consider the number of items - more than 5-6 is probably not very statistically useful
Q: When setting up performance levels, do I have to use whole numbers?
A: Yes
Q: Can I reorder the Content Areas if no tests have been linked?
A: No, but can remove a content area and add back in the desired order
Sample Illustrations
Figure 1: All tests are in one program. Can filter by any combination of subjects, grades and test windows. A menu at the top of this page provides access to other programs.
Figure 2: If the program contains multiple sets of Testing Windows, the filter on the Performance report can be quite long. For example, Spring/Winter/Fall, BOY/MOY/EOY, Pre-Test/Post-Test, Unit 1/2/3/4, etc.
Figure 3: In the Program report, the same filters are available as for the Performance report, but results are grouped as shown. If you had multiple test categories, the user could filter on the test windows that apply to limit the view (e.g., include unit 1/2, but not BOY/MOY/EOY). Multi-grade tests are displayed under the lowest tagged grade level.
Figure 4: Both the Performance and Program reports provide a menu to switch programs.
Figure 5: In Program Configuration all of these options are editable, even after students start testing.
Figure 6: For the score groups, only the cut scores, labels for the performance levels and the ability to show only the labels can be overridden at the test level. Nothing else can be changed once tests are linked.