Reporting Overview
Navigation: Quality Management> Reporting
OSCAR offers a variety of reports that are primarily concerned with response score data, though not exclusively. Each report is designed to offer a unique view of user-generated data across the tenant. All reports must be enabled in the Role Permissions first for the report's page to be available. Additionally, use the "Auditing to Assigned Sections" permission to limit a user's view in Reporting to only the teams they have been assigned.
All reports include a variety of filters that can be used to narrow down your data search. After selecting any filters, select Run Report to apply your filters in a search.
You can create downloadable files of your filtered reports:
Export (CSV)
Print (PDF)
While most reports are real-time data, there are a few reports that update nightly due to the size of the data available. In the descriptions of each Reporting page below, there is a note if the report runs once a day. Otherwise, each report has real-time data.
Completion
This report is designed to show a real-time status of every response loaded into the system. A scoring supervisor or admin can quickly see the state of the responses and how close an item/project is to completion. Each value in an item's row on the report is a clickable link to the Auditing page that will display all responses that match the column header. The clipboard button next the item name will take the user the Burndown report where item completion time can be estimated.
Found within Quality Management | Reporting | Completion.
Scorers
The Scorer Report is designed to allow scoring supervisors access to scorer progress for a project, item, or for all scoring results. The report displays all QC responses the users scored while in live scoring as well as the number of resolutions and their agreement against resolution scoring. The date filter keys of the score dates.
Found within Quality Management | Reporting | Scorers.
Payroll
The Payroll Report displays the number and type of scores as well as scoring time that were created by each user. There are filters for Project, Section, Item, Team and User as well as customizable date filters. The report is designed to display all payable rater data for a project over any timeframe.
Found within Quality Management | Reporting | Payroll.
Daily
The Daily Report is a tool that scoring supervisors use to identify drift, scorer issues and overall trait level scoring performance by comparing an individual scorer’s score distribution to the score distribution of the entire scorer pool for an item. The Daily Report provides real-time scoring frequency distribution and inter-rater reliability (IRR) results by trait. Data can be filtered by day or by any defined time over the course of a project. The date filter is applied based on the score dates and the IRR is calculated for all responses within the date filter range.
Found within Quality Management | Reporting | Daily.
Daily Scoring Management
The Daily Scoring Management report is a comprehensive item report that shows frequency distribution at the trait level across all items in the project as well item completion statistics and Validity agreement. The data can be filtered down to the Section level and also exported as a csv file. This report is run once a day, and the associated role permission must be enabled for the report to run.
Found within Quality Management | Reporting | Daily Scoring Mgmt.
Quotas
On the Quotas Report page, a table list of each type of item quota is displayed, showing each user's progress against the configured quotas. A specific item must be chosen in the filters as each quota is item specific and the results can be sorted by high or low values in each category. This report will display "No Quota" instead of data if an item quota has not been configured (Item Settings).
Found within Quality Management | Reporting | Quotas.
User Summary
The User Summary report provides a comprehensive “snapshot” view of all user activity associated with a section/user/item. It includes time on task, IRR/Validity/Resolution overview, QC overview, item summary overview and recent activity.
Found within Quality Management | Reporting | User Summary.
Item Summary
The Item Summary report provides a matrix view of IRR and the ability to see a specific scorer’s trends by trait. The report shows the number of responses when the individual scorer agreed with the same exact score assigned by another scorer, and the number of responses when the scorer was adjacent high or low and non-adjacent high or low.
Found within Quality Management | Reporting | Item Summary.
User Agreement
The User Agreement Report provides IRR and Validity exact and adjacent percentages for an item in aggregate and by trait as well as for each scorer. The Resolution required is the number of responses the user scored that went to Resolution and the Resolution agreement percentages are the users' IRR against the Resolution score.
Found within Quality Management | Reporting | User Agreement.
Backread Agreement
The Backread Agreement Report provides exact and adjacent agreement statistics for each user by item against Backreads. Trait-level details can be displayed using the plus button (Show Details) found to the right of each user.
Found within Quality Management | Reporting | Backread Agreement.
Project Agreement
The Project Agreement Report provides IRR and Validity exact or exact/adjacent percentage reporting for a project in aggregate and by trait for each scorer. The date filter keys of the creation date of the responses and the IRR only considers responses in the complete state.
Found within Quality Management | Reporting | Project Agreement.
Training Completion
The Training Completion Report is a comprehensive Training Course report that displays user progress on each available Training Course. User time and completion status is displayed, and this report is also where users can be manually passed when needed. The report can be exported as a pdf or csv file.
Found within Quality Management | Reporting | Training Completion.
SLA Report
The SLA (Service-level Agreement) Report shows how many incomplete responses there are for each item that have been in OSCAR for 24 hours or more. A different 24 hour increment can be used as a filter up to 120 hours. Each response total only includes responses that are not in a Complete state, and the total includes all responses at the given 24 hour increment or older. This report only calculates once a day, and the role permission must be enabled for the report to calculate.
Found within Quality Management | Reporting | Project Agreement.