plan-module · supabase_json
KnowledgeVault AI plan-module M-05
plan-module artifact · for KnowledgeVault AI · phase M-05 · status draft
tasks
| id | role | brief | title | completion | depends on |
|---|---|---|---|---|---|
| 1 | developer | Create apps/knowledgevault/supabase/migrations/005_evaluation_knowledge_publication.sql. Runs after M-01–M-04. Assumes: users, capture_sessions, products tables exist; kv_set_updated_at() trigger function defined in M-01. All operations must be idempotent (CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, ADD COLUMN IF NOT EXISTS). Enable pgvector: CREATE EXTENSION IF NOT EXISTS vector; Create knowledge_chunks: id uuid PK DEFAULT gen_random_uuid(), session_id uuid NOT NULL REFERENCES capture_sessions(id) ON DELETE CASCADE, product_id uuid NOT NULL REFERENCES products(id), expert_id uui | Add evaluation and knowledge publication DB migration | - type: file - evaluate: Migration applies via psql -f with zero errors; knowledge_chunks has 12 columns including embedding vector(1536); payments.session_id UNIQUE constraint exists; virtual_experts.expert_id UNIQUE constraint exists; idx_kc_embedding IVFFlat index created; RLS enabled on all three tables with correct policies; running migration twice produces no error (idempotent); complies with POL-ARCH-001, POL-SEC-001, POL-PERF-001, POL-DATA-001. - min length: 200 | |
| 2 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/o3-scorer.ts. This TypeScript module is imported by the evaluate-session orchestrator. Export: scoreSession(context: SharedExpertContext, apiKey: string): Promise<ScoreResult> SharedExpertContext: { session_id: string, expert_id: string, product_id: string, questions: { question_text: string, answer_text: string, deepgram_confidence?: number }[] } ScoreResult: { dimension_scores: { technical_accuracy: number, completeness: number, practicality_tacit_insight: number, safety_compliance: number, clarity_structure: number, novelty: | Implement o3 scorer for 7-dimension quality scoring | - type: file - evaluate: Unit test asserts model='o3' in mocked request body; system prompt contains 'respond with valid JSON only'; all dimension scores=80 yields composite_score=80.00; mocked 300,001ms delay throws EvaluationTimeoutError; valid mocked response produces ScoreResult with no missing keys; complies with POL-STYLE-001. - min length: 200 | |
| 3 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/ground-truth-gate.ts. Runs in parallel with the o3 scorer inside the evaluate-session orchestrator. Export: runGroundTruthGate(input: GateInput, env: GateEnv): Promise<GateResult> Types: CandidateChunk = { chunk_type: string, content: string, novelty_score: number, deepgram_confidence?: number }. GateInput = { candidate_chunks: CandidateChunk[], novel_terms_flagged: string[], safety_claims_flagged: string[] }. GateEnv = { openaiApiKey: string, supabaseUrl: string, supabaseServiceKey: string, resendApiKey: string, sessionId: strin | Implement ground-truth verification gate | - type: file - evaluate: Unit test: 5-chunk set (1 safety_warning, 1 novelty=0.8, 1 psi numeric, 2 plain) triggers exactly 3 authoritative lookups; safety_warning contradiction sets payment_hold_triggered=true and Resend email called to justin@dreamborn.ai; deepgram_confidence<0.75 chunk skipped with no lookup; failing novel term sets novelty_score=0.0 and ground_truth_verified=false; complies with POL-STYLE-001, POL-STYLE-002. - min length: 200 | |
| 4 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/chunk-publisher.ts. Called by the evaluate-session orchestrator ONLY after BOTH o3-scorer AND ground-truth-gate resolve. Export: publishChunks(input: PublisherInput, env: PublisherEnv): Promise<void> Import ScoreResult from ./o3-scorer and VerifiedChunk from ./ground-truth-gate. PublisherInput: { score_object: ScoreResult, verified_chunks: VerifiedChunk[], payment_hold_triggered: boolean, payment_hold_reason: string | null, session: { id: string, expert_id: string, product_id: string, questions_answered: number }, product: { has | Implement chunk publisher, payment record, virtual expert upsert | - type: file - evaluate: Unit test: 5 chunks (3 verified, 2 not) yields 3 is_published=true with vector(1536) embeddings and 2 is_published=false with null embedding; payment test: questions=5, composite=80, 2 novel verified, no surge, no completion → total_amount=23.00; avg_accuracy_score([70,80,90])=80.00; virtual_experts.is_published=true after 3rd qualifying session; complies with POL-DATA-001, POL-STYLE-001. - min length: 200 | - 1 - 2 - 3 |
| 5 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/index.ts. Supabase Edge Function (Deno runtime) invoked fire-and-forget by POST /api/sessions/:id/complete from M-04. Request: HTTP POST body { session_id: string }. Orchestration flow: 1. Parse request body. Record start = Date.now(). Init Supabase admin client using Deno.env.get('SUPABASE_SERVICE_ROLE_KEY'). 2. Idempotency guard: SELECT status FROM capture_sessions WHERE id=session_id. If status is 'evaluation_complete' or 'evaluation_failed': return JSON({ skipped: true, reason: 'already_processed' }, 200) with zero DB writes. | Implement evaluate-session Edge Function orchestrator | - type: file - evaluate: supabase functions deploy evaluate-session exits 0; second invocation for same session_id returns {skipped:true, reason:'already_processed'} with no DB writes; EvaluationTimeoutError triggers one retry then sets status=evaluation_failed; structured JSON {event,session_id,status,duration_ms} logged on every invocation; complies with POL-STYLE-001, POL-STYLE-002, POL-SEC-004. - min length: 200 | - 2 - 3 - 4 |
| 6 | developer | Create apps/knowledgevault/apps/web/src/app/api/sessions/[id]/score/route.ts. Next.js 15 App Router API route polled every 10s by the EvaluationPendingScreen mobile client. Export GET handler. Auth flow: call supabase.auth.getUser() before any DB access. - Missing or invalid Bearer token → 401: { error: 'Unauthorized', code: 'unauthorized', status: 401 } - Fetch capture_sessions WHERE id=params.id. Not found → 404: { error: 'Session not found', code: 'session_not_found', status: 404 } - For expert JWT: confirm session.expert_id === auth.user.id → 403 on mismatch: { error: 'Forbidden', code: | Add GET /api/sessions/:id/score polling route | - type: file - evaluate: Route calls supabase.auth.getUser() before any DB query; all four response shapes (evaluating, complete, failed, payment_hold) returned correctly based on session.status and payments.status; 401 for missing auth, 404 for missing session, 403 for JWT expert_id mismatch; SUPABASE_SERVICE_ROLE_KEY used only in server context; complies with POL-SEC-002, POL-ARCH-003, POL-STYLE-001. - min length: 200 | - 1 |
| 7 | developer | Create three files: (1) apps/knowledgevault/apps/mobile/src/screens/EvaluationPendingScreen.tsx, (2) apps/knowledgevault/apps/mobile/src/screens/ScoreResultsScreen.tsx, (3) apps/knowledgevault/apps/mobile/src/__tests__/EvaluationScreens.test.tsx. EvaluationPendingScreen (S-04-a): Navigation param: session_id (string). On mount: call GET /api/sessions/:id/score immediately (zero delay). Poll every exactly 10,000ms. Three render states: (1) evaluating → headline 'Scoring your session', body 'AI is reviewing your answers. Usually ready in 1-2 minutes.', earnings text 'Estimated: $<min>-$<max>'; | Build EvaluationPendingScreen and ScoreResultsScreen | - type: file - evaluate: All 5 snapshot tests pass; polling test confirms exactly 3 API calls within 30,001ms fake-timer window; S-04-a status card has aria-live='polite'; auto-navigation to ScoreResultsScreen fires within 1,000ms of status=complete; S-04-b accordion collapsed on mount with 6 labeled items; both screens render skeleton before data loads; complies with POL-STYLE-001. - min length: 200 | - 6 |
Agent Handoff
Start Here
| id | role | brief | title | completion | depends on |
|---|---|---|---|---|---|
| 1 | developer | Create apps/knowledgevault/supabase/migrations/005_evaluation_knowledge_publication.sql. Runs after M-01–M-04. Assumes: users, capture_sessions, products tables exist; kv_set_updated_at() trigger function defined in M-01. All operations must be idempotent (CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, ADD COLUMN IF NOT EXISTS). Enable pgvector: CREATE EXTENSION IF NOT EXISTS vector; Create knowledge_chunks: id uuid PK DEFAULT gen_random_uuid(), session_id uuid NOT NULL REFERENCES capture_sessions(id) ON DELETE CASCADE, product_id uuid NOT NULL REFERENCES products(id), expert_id uui | Add evaluation and knowledge publication DB migration | - type: file - evaluate: Migration applies via psql -f with zero errors; knowledge_chunks has 12 columns including embedding vector(1536); payments.session_id UNIQUE constraint exists; virtual_experts.expert_id UNIQUE constraint exists; idx_kc_embedding IVFFlat index created; RLS enabled on all three tables with correct policies; running migration twice produces no error (idempotent); complies with POL-ARCH-001, POL-SEC-001, POL-PERF-001, POL-DATA-001. - min length: 200 | |
| 2 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/o3-scorer.ts. This TypeScript module is imported by the evaluate-session orchestrator. Export: scoreSession(context: SharedExpertContext, apiKey: string): Promise<ScoreResult> SharedExpertContext: { session_id: string, expert_id: string, product_id: string, questions: { question_text: string, answer_text: string, deepgram_confidence?: number }[] } ScoreResult: { dimension_scores: { technical_accuracy: number, completeness: number, practicality_tacit_insight: number, safety_compliance: number, clarity_structure: number, novelty: | Implement o3 scorer for 7-dimension quality scoring | - type: file - evaluate: Unit test asserts model='o3' in mocked request body; system prompt contains 'respond with valid JSON only'; all dimension scores=80 yields composite_score=80.00; mocked 300,001ms delay throws EvaluationTimeoutError; valid mocked response produces ScoreResult with no missing keys; complies with POL-STYLE-001. - min length: 200 | |
| 3 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/ground-truth-gate.ts. Runs in parallel with the o3 scorer inside the evaluate-session orchestrator. Export: runGroundTruthGate(input: GateInput, env: GateEnv): Promise<GateResult> Types: CandidateChunk = { chunk_type: string, content: string, novelty_score: number, deepgram_confidence?: number }. GateInput = { candidate_chunks: CandidateChunk[], novel_terms_flagged: string[], safety_claims_flagged: string[] }. GateEnv = { openaiApiKey: string, supabaseUrl: string, supabaseServiceKey: string, resendApiKey: string, sessionId: strin | Implement ground-truth verification gate | - type: file - evaluate: Unit test: 5-chunk set (1 safety_warning, 1 novelty=0.8, 1 psi numeric, 2 plain) triggers exactly 3 authoritative lookups; safety_warning contradiction sets payment_hold_triggered=true and Resend email called to justin@dreamborn.ai; deepgram_confidence<0.75 chunk skipped with no lookup; failing novel term sets novelty_score=0.0 and ground_truth_verified=false; complies with POL-STYLE-001, POL-STYLE-002. - min length: 200 | |
| 4 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/chunk-publisher.ts. Called by the evaluate-session orchestrator ONLY after BOTH o3-scorer AND ground-truth-gate resolve. Export: publishChunks(input: PublisherInput, env: PublisherEnv): Promise<void> Import ScoreResult from ./o3-scorer and VerifiedChunk from ./ground-truth-gate. PublisherInput: { score_object: ScoreResult, verified_chunks: VerifiedChunk[], payment_hold_triggered: boolean, payment_hold_reason: string | null, session: { id: string, expert_id: string, product_id: string, questions_answered: number }, product: { has | Implement chunk publisher, payment record, virtual expert upsert | - type: file - evaluate: Unit test: 5 chunks (3 verified, 2 not) yields 3 is_published=true with vector(1536) embeddings and 2 is_published=false with null embedding; payment test: questions=5, composite=80, 2 novel verified, no surge, no completion → total_amount=23.00; avg_accuracy_score([70,80,90])=80.00; virtual_experts.is_published=true after 3rd qualifying session; complies with POL-DATA-001, POL-STYLE-001. - min length: 200 | - 1 - 2 - 3 |
| 5 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/index.ts. Supabase Edge Function (Deno runtime) invoked fire-and-forget by POST /api/sessions/:id/complete from M-04. Request: HTTP POST body { session_id: string }. Orchestration flow: 1. Parse request body. Record start = Date.now(). Init Supabase admin client using Deno.env.get('SUPABASE_SERVICE_ROLE_KEY'). 2. Idempotency guard: SELECT status FROM capture_sessions WHERE id=session_id. If status is 'evaluation_complete' or 'evaluation_failed': return JSON({ skipped: true, reason: 'already_processed' }, 200) with zero DB writes. | Implement evaluate-session Edge Function orchestrator | - type: file - evaluate: supabase functions deploy evaluate-session exits 0; second invocation for same session_id returns {skipped:true, reason:'already_processed'} with no DB writes; EvaluationTimeoutError triggers one retry then sets status=evaluation_failed; structured JSON {event,session_id,status,duration_ms} logged on every invocation; complies with POL-STYLE-001, POL-STYLE-002, POL-SEC-004. - min length: 200 | - 2 - 3 - 4 |
| 6 | developer | Create apps/knowledgevault/apps/web/src/app/api/sessions/[id]/score/route.ts. Next.js 15 App Router API route polled every 10s by the EvaluationPendingScreen mobile client. Export GET handler. Auth flow: call supabase.auth.getUser() before any DB access. - Missing or invalid Bearer token → 401: { error: 'Unauthorized', code: 'unauthorized', status: 401 } - Fetch capture_sessions WHERE id=params.id. Not found → 404: { error: 'Session not found', code: 'session_not_found', status: 404 } - For expert JWT: confirm session.expert_id === auth.user.id → 403 on mismatch: { error: 'Forbidden', code: | Add GET /api/sessions/:id/score polling route | - type: file - evaluate: Route calls supabase.auth.getUser() before any DB query; all four response shapes (evaluating, complete, failed, payment_hold) returned correctly based on session.status and payments.status; 401 for missing auth, 404 for missing session, 403 for JWT expert_id mismatch; SUPABASE_SERVICE_ROLE_KEY used only in server context; complies with POL-SEC-002, POL-ARCH-003, POL-STYLE-001. - min length: 200 | - 1 |
| 7 | developer | Create three files: (1) apps/knowledgevault/apps/mobile/src/screens/EvaluationPendingScreen.tsx, (2) apps/knowledgevault/apps/mobile/src/screens/ScoreResultsScreen.tsx, (3) apps/knowledgevault/apps/mobile/src/__tests__/EvaluationScreens.test.tsx. EvaluationPendingScreen (S-04-a): Navigation param: session_id (string). On mount: call GET /api/sessions/:id/score immediately (zero delay). Poll every exactly 10,000ms. Three render states: (1) evaluating → headline 'Scoring your session', body 'AI is reviewing your answers. Usually ready in 1-2 minutes.', earnings text 'Estimated: $<min>-$<max>'; | Build EvaluationPendingScreen and ScoreResultsScreen | - type: file - evaluate: All 5 snapshot tests pass; polling test confirms exactly 3 API calls within 30,001ms fake-timer window; S-04-a status card has aria-live='polite'; auto-navigation to ScoreResultsScreen fires within 1,000ms of status=complete; S-04-b accordion collapsed on mount with 6 labeled items; both screens render skeleton before data loads; complies with POL-STYLE-001. - min length: 200 | - 6 |
Completion Evidence
No explicit evidence field yet. Require tests, screenshots, linked PRs, or reviewed outputs before marking complete.
Artifact Shape
- tasks: 7 items
- module id: M-05
- module name: Evaluation & Knowledge Publication
- schema version: 1.0
- depends on modules: 1 item
Structured Payload
Machine-readable source fields
tasks
| id | role | brief | title | completion | depends on |
|---|---|---|---|---|---|
| 1 | developer | Create apps/knowledgevault/supabase/migrations/005_evaluation_knowledge_publication.sql. Runs after M-01–M-04. Assumes: users, capture_sessions, products tables exist; kv_set_updated_at() trigger function defined in M-01. All operations must be idempotent (CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS, ADD COLUMN IF NOT EXISTS). Enable pgvector: CREATE EXTENSION IF NOT EXISTS vector; Create knowledge_chunks: id uuid PK DEFAULT gen_random_uuid(), session_id uuid NOT NULL REFERENCES capture_sessions(id) ON DELETE CASCADE, product_id uuid NOT NULL REFERENCES products(id), expert_id uui | Add evaluation and knowledge publication DB migration | - type: file - evaluate: Migration applies via psql -f with zero errors; knowledge_chunks has 12 columns including embedding vector(1536); payments.session_id UNIQUE constraint exists; virtual_experts.expert_id UNIQUE constraint exists; idx_kc_embedding IVFFlat index created; RLS enabled on all three tables with correct policies; running migration twice produces no error (idempotent); complies with POL-ARCH-001, POL-SEC-001, POL-PERF-001, POL-DATA-001. - min length: 200 | |
| 2 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/o3-scorer.ts. This TypeScript module is imported by the evaluate-session orchestrator. Export: scoreSession(context: SharedExpertContext, apiKey: string): Promise<ScoreResult> SharedExpertContext: { session_id: string, expert_id: string, product_id: string, questions: { question_text: string, answer_text: string, deepgram_confidence?: number }[] } ScoreResult: { dimension_scores: { technical_accuracy: number, completeness: number, practicality_tacit_insight: number, safety_compliance: number, clarity_structure: number, novelty: | Implement o3 scorer for 7-dimension quality scoring | - type: file - evaluate: Unit test asserts model='o3' in mocked request body; system prompt contains 'respond with valid JSON only'; all dimension scores=80 yields composite_score=80.00; mocked 300,001ms delay throws EvaluationTimeoutError; valid mocked response produces ScoreResult with no missing keys; complies with POL-STYLE-001. - min length: 200 | |
| 3 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/ground-truth-gate.ts. Runs in parallel with the o3 scorer inside the evaluate-session orchestrator. Export: runGroundTruthGate(input: GateInput, env: GateEnv): Promise<GateResult> Types: CandidateChunk = { chunk_type: string, content: string, novelty_score: number, deepgram_confidence?: number }. GateInput = { candidate_chunks: CandidateChunk[], novel_terms_flagged: string[], safety_claims_flagged: string[] }. GateEnv = { openaiApiKey: string, supabaseUrl: string, supabaseServiceKey: string, resendApiKey: string, sessionId: strin | Implement ground-truth verification gate | - type: file - evaluate: Unit test: 5-chunk set (1 safety_warning, 1 novelty=0.8, 1 psi numeric, 2 plain) triggers exactly 3 authoritative lookups; safety_warning contradiction sets payment_hold_triggered=true and Resend email called to justin@dreamborn.ai; deepgram_confidence<0.75 chunk skipped with no lookup; failing novel term sets novelty_score=0.0 and ground_truth_verified=false; complies with POL-STYLE-001, POL-STYLE-002. - min length: 200 | |
| 4 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/chunk-publisher.ts. Called by the evaluate-session orchestrator ONLY after BOTH o3-scorer AND ground-truth-gate resolve. Export: publishChunks(input: PublisherInput, env: PublisherEnv): Promise<void> Import ScoreResult from ./o3-scorer and VerifiedChunk from ./ground-truth-gate. PublisherInput: { score_object: ScoreResult, verified_chunks: VerifiedChunk[], payment_hold_triggered: boolean, payment_hold_reason: string | null, session: { id: string, expert_id: string, product_id: string, questions_answered: number }, product: { has | Implement chunk publisher, payment record, virtual expert upsert | - type: file - evaluate: Unit test: 5 chunks (3 verified, 2 not) yields 3 is_published=true with vector(1536) embeddings and 2 is_published=false with null embedding; payment test: questions=5, composite=80, 2 novel verified, no surge, no completion → total_amount=23.00; avg_accuracy_score([70,80,90])=80.00; virtual_experts.is_published=true after 3rd qualifying session; complies with POL-DATA-001, POL-STYLE-001. - min length: 200 | - 1 - 2 - 3 |
| 5 | developer | Create apps/knowledgevault/supabase/functions/evaluate-session/index.ts. Supabase Edge Function (Deno runtime) invoked fire-and-forget by POST /api/sessions/:id/complete from M-04. Request: HTTP POST body { session_id: string }. Orchestration flow: 1. Parse request body. Record start = Date.now(). Init Supabase admin client using Deno.env.get('SUPABASE_SERVICE_ROLE_KEY'). 2. Idempotency guard: SELECT status FROM capture_sessions WHERE id=session_id. If status is 'evaluation_complete' or 'evaluation_failed': return JSON({ skipped: true, reason: 'already_processed' }, 200) with zero DB writes. | Implement evaluate-session Edge Function orchestrator | - type: file - evaluate: supabase functions deploy evaluate-session exits 0; second invocation for same session_id returns {skipped:true, reason:'already_processed'} with no DB writes; EvaluationTimeoutError triggers one retry then sets status=evaluation_failed; structured JSON {event,session_id,status,duration_ms} logged on every invocation; complies with POL-STYLE-001, POL-STYLE-002, POL-SEC-004. - min length: 200 | - 2 - 3 - 4 |
| 6 | developer | Create apps/knowledgevault/apps/web/src/app/api/sessions/[id]/score/route.ts. Next.js 15 App Router API route polled every 10s by the EvaluationPendingScreen mobile client. Export GET handler. Auth flow: call supabase.auth.getUser() before any DB access. - Missing or invalid Bearer token → 401: { error: 'Unauthorized', code: 'unauthorized', status: 401 } - Fetch capture_sessions WHERE id=params.id. Not found → 404: { error: 'Session not found', code: 'session_not_found', status: 404 } - For expert JWT: confirm session.expert_id === auth.user.id → 403 on mismatch: { error: 'Forbidden', code: | Add GET /api/sessions/:id/score polling route | - type: file - evaluate: Route calls supabase.auth.getUser() before any DB query; all four response shapes (evaluating, complete, failed, payment_hold) returned correctly based on session.status and payments.status; 401 for missing auth, 404 for missing session, 403 for JWT expert_id mismatch; SUPABASE_SERVICE_ROLE_KEY used only in server context; complies with POL-SEC-002, POL-ARCH-003, POL-STYLE-001. - min length: 200 | - 1 |
| 7 | developer | Create three files: (1) apps/knowledgevault/apps/mobile/src/screens/EvaluationPendingScreen.tsx, (2) apps/knowledgevault/apps/mobile/src/screens/ScoreResultsScreen.tsx, (3) apps/knowledgevault/apps/mobile/src/__tests__/EvaluationScreens.test.tsx. EvaluationPendingScreen (S-04-a): Navigation param: session_id (string). On mount: call GET /api/sessions/:id/score immediately (zero delay). Poll every exactly 10,000ms. Three render states: (1) evaluating → headline 'Scoring your session', body 'AI is reviewing your answers. Usually ready in 1-2 minutes.', earnings text 'Estimated: $<min>-$<max>'; | Build EvaluationPendingScreen and ScoreResultsScreen | - type: file - evaluate: All 5 snapshot tests pass; polling test confirms exactly 3 API calls within 30,001ms fake-timer window; S-04-a status card has aria-live='polite'; auto-navigation to ScoreResultsScreen fires within 1,000ms of status=complete; S-04-b accordion collapsed on mount with 6 labeled items; both screens render skeleton before data loads; complies with POL-STYLE-001. - min length: 200 | - 6 |
module id
M-05
module name
Evaluation & Knowledge Publication
schema version
1.0
depends on modules
- M-04