SkillsGuide.in
Lab 08
25 mins

AI Evaluation Casebook

Mission Briefing

Assess 24 authored responses against reference facts. Label supported vs unsupported answers, calculate precision/recall, and decide release.

Success Criteria
  • Complete scenario tasks and verify feedback.
Interactive Sandbox Guardrails
Annotation GridGround Truth ViewerConfusion Matrix MatrixMetric Calculator

Deterministic browser simulation with server-authoritative scoring. Zero setup required.

Lab 08 • Responsible AI & LLM Evaluation

AI Evaluation Casebook & Faithfulness Metrics

Verify 12 authored model responses against ground-truth facts. Correctly label hallucinations to achieve TP 8, FP 2, FN 2 (Precision 0.8, Recall 0.8).

True Positives (TP)
8

Target benchmark: 8 cases

False Positives (FP)
2

Hallucinations mislabeled as supported (Target: 2)

Precision
80.0%

Formula: TP / (TP + FP) [Target: 80.0%]

Recall
100.0%

Formula: TP / (TP + FN) [Target: 80.0%]

Evaluation Queue (12 Samples)

Target Model:
C1What is the return window for electronics?

Customers can return electronics within 30 days of delivery if they have the receipt.

C2Are refurbished laptops covered by warranty?

Yes, certified refurbished laptops come with a 1-year limited warranty.

C3Does the enterprise plan include 24/7 phone support?

Yes, enterprise includes 24/7 phone and dedicated Slack support.

C4Can users export data to BigQuery?

BigQuery export is supported automatically every night via pipeline.

C5Where are EU customer databases hosted?

All EU customer records are stored securely in Frankfurt.

C6What is the maximum file upload size?

Pro users can upload files up to 50MB, while Enterprise tier supports 500MB.

C7Is HIPAA compliance BAA provided?

BAAs are available upon request for Healthcare tier Enterprise contracts.

C8How is user billing calculated during seat expansion?

Added team members are prorated based on remaining days in the cycle.

C9Can free accounts create custom subdomains?

Yes, any free account can claim a unique custom subdomain immediately.

C10Is multi-factor authentication mandatory?

MFA is completely optional across all subscription tiers.

C11What is the rate limit for REST APIs?

The API has zero rate limits with burst allowance up to 10k rps.

C12Can users restore deleted workspaces?

Deleted workspaces are instantly destroyed and cannot ever be recovered.

Inspection: Case C1Current: SUPPORTED

What is the return window for electronics?
Items can be returned within 30 days of delivery with receipt.
Customers can return electronics within 30 days of delivery if they have the receipt.

Model Governance & Deployment Gate

Intelligent Adaptive Discovery

Indian Career Compass & Transition Blueprint

Personalized specifically for your domain background (CA, Law, Pharmacy, Teaching, Tech, Engineering & Commerce). Questions dynamically adapt so you are evaluated on relevant skills — never forced into mismatched generic questions.

Adaptive 30-Signal EngineDomain-Specific Competency MatchingZero Unwanted Generic Coding for Non-Tech5 Adaptive Steps
STEP 1 OF 50 / 30 signals captured

Your Starting Point

Every educational background has specific high-leverage advantages. Let’s map yours.

💡 Your background unlocks specific accelerated career pathways without starting from scratch.

💡 Optional. Used to prevent suggesting career pathways with a negative pay delta.

💡 Used to benchmark local hiring clusters in your regional market.

🔒 Complete client-side privacy. Answers stay in your browser and are never uploaded without permission. Export your blueprint below to save a copy.