Continuous adaptive testing with IRT-based scoring, per-competency tracking, and real-time xAPI analytics - no rounds, unlimited questions
How our assessment system is grounded in psychometric theory, Item Response Theory, and measurement validity frameworks
Assessment leverages polytomous IRT models (Partial Credit, Graded Response) for precise ability estimation. SATA questions with partial credit scoring capture nuanced competency levels, reducing measurement error by 30-40% compared to binary scoring.
Continuous flow assessment adapts in real-time based on learner performance. Per-competency tracking enables targeted remediation while rolling window scoring provides stable proficiency estimates with 30% fewer items than traditional tests.
xAPI tracking captures every learner interaction, building comprehensive learning records. Response patterns reveal misconceptions, partial knowledge states, and learning trajectories - supporting formative assessment and adaptive instruction.
Three-level context hierarchy (Class → DSC → Competency) ensures construct-relevant variance. Rolling window scoring balances stability and sensitivity, achieving reliability α = 0.85+ while detecting genuine competency changes.
From question generation to proficiency determination - understanding the complete workflow
LLM-powered scenario-based questions with competency targeting
SATA partial credit with rolling window proficiency calculation
xAPI tracking with per-competency performance insights
Deep dive into assessment creation, scoring algorithms, and analytics
All assessment questions generated by LLM through universal content gateway. Each question includes realistic scenario, competency-aligned stem, 4-6 options with 1-3 correct answers, and detailed distractors based on common misconceptions.
Questions cycle through all competencies in the DSC (Domain-Specific Competency). When reaching the end, automatically loops back to the first competency - ensuring balanced, comprehensive assessment across all learning objectives.
Select-All-That-Apply scoring algorithm: Score = max(0, correct - incorrect). If correct answers = [0,1,2] and user selects [0,1,3]: 2 correct, 1 incorrect → score = 1/3 (33%). Success requires perfect score only.
Overall score calculated from most recent N questions (default: 2 × competencies × questions per item). Balances sensitivity to recent performance with stability against random variation. Auto-adjusts as more questions answered.
Individual tracking for each competency: question count, total score, max score, percentage, proficiency level. Proficient (≥70%), Developing (50-69%), Needs Focus (<50%). Enables gap analysis and personalized remediation.
Three-tier proficiency model based on scaled scores: Proficient (70%+) indicates readiness for independent application; Developing (50-69%) shows emerging competency with support needed; Needs Focus (<50%) signals remediation required.
Every question tracked with standardized xAPI statement: Actor (learner), Verb ("answered"), Object (question ID + /sata suffix for SATA), Result (scores, success), Context (Class → DSC → Competency hierarchy), Timestamp.
xAPI context hierarchy: Parent (Class), Grouping (DSC), Category (Competency). Enables powerful LRS queries - filter by class, aggregate by DSC, drill down to competency. Supports class-wide analytics and individual learner reports.
All questions saved to browser localStorage (key: assessment_questions_${classId}). Includes question data, user answers, timestamps. Enables stop/resume anytime, offline progress calculation, and non-blocking xAPI (assessment continues if LRS fails).
"View Progress" button available anytime during assessment. Shows: overall rolling window score, per-competency breakdown with proficiency levels, best/worst competencies, total questions answered, proficient/developing/needs-focus counts. Close modal to resume.
Standard xAPI structure enables rich queries: All SATA questions (object.id contains /sata), Specific competency (category.id filter), Class performance (parent.id filter), Student history (actor.mbox filter). Supports report cards, gap analysis, longitudinal tracking.
When user clicks "Next", system immediately: (1) Saves question to localStorage, (2) Sends xAPI statement to LRS (non-blocking), (3) Generates next question. Even if browser crashes or LRS fails, progress preserved locally.
Every option includes detailed AI-generated feedback (2-3 sentences, 40+ words minimum). Correct options explain WHAT concept it demonstrates, HOW it applies, and WHY it's valid. Incorrect options explain misconceptions, why students might choose it, and correct understanding. Feedback stored in LRS options-analytics extension and displayed in review modal.
Review modal includes "Learn More" buttons linking to Knowledge Explorer (KE) and Socratic Playground (SPL) for each competency. URL format: /learning-session.html?classId=X&DSC=[competency]&LEVEL=Y&Mode=[ke/spl]. Enables seamless transition from assessment to deeper learning.
LLM prompts enforce MANDATORY SATA format with MINIMUM 2 correct options (typically 2-3 out of 4). Distribution: either 2 correct + 2 incorrect OR 3 correct + 1 incorrect. Each correct option represents different valid approach or aspect of competency mastery. NEVER generates single-answer multiple choice questions.
Three simple steps to begin your proficiency assessment
Navigate to /start-assessment.html or click "Start Assessment" from your dashboard. Select a Domain-Specific Competency (DSC) to assess - each DSC contains multiple competencies you'll be tested on.
Read each scenario-based question carefully. For SATA questions, select ALL correct answers - partial credit awarded. Click "Next" to save and continue. Stop anytime - progress auto-saves to browser.
Click "View Progress" anytime to see overall score, per-competency breakdown, and proficiency levels. Resume assessment anytime by returning to same DSC - picks up where you left off.
/start-assessment.html
/assessment.html?dscId=<dsc-id>
/docs/ASSESSMENT_SYSTEM.md
Continuous adaptive testing with IRT-based scoring and real-time analytics