OCDIT Documentation99 Venture

Pilot Verification Evidence

Updated 9 September 2026

Verified on 7 September 2026. Local checks and live evidence are distinguished below.

Real Provider Test — Passed

An initial card authorization rejection exposed a configuration error: card actions supplied the Teams 19: team ID without an Entra group ID. The configured team identifier and its stored scope were corrected to the independently verified ID for the same team, after a consistent database backup. No authorization checks were removed.

The installed manifest uses the real bot ID. Private phone numbers, message IDs, credentials and tenant/host identifiers are excluded from this shareable evidence. Meta API sending used v25.0; message webhook subscription used the portal's v26.0. The app remained unpublished and only test traffic was verified.

Automated Checks

npm test: 7 passed, 0 failed on Windows / Node 24.13.0. The earlier multi-user configuration passed in the Linux Docker image with external networking disabled. The current single-user update was verified on Windows.

  1. Shared pool: inbound/action deduplication, claim collision, private note, handover, non-owner denial, queued reply and out-of-order delivery receipts. Cards hide handover for a single configured staff member.
  2. Reply-window expiry at enqueue and send; late webhook timestamps do not reopen it; invalid handover rolls back ownership changes.
  3. Raw HMAC signature, wrong business-number routing, malformed webhook and media placeholder.
  4. Teams tenant/team/channel/Entra employee allowlist, including team group-ID mapping.
  5. Meta send request shape and timeout/5xx ambiguity; no blind retry; receipt reconciliation.
  6. File-backed restart retains ownership/history and quarantines an in-flight send.
  7. Single-user live-mode HTTP server: health works, demo routes absent, unauthenticated Teams request rejected by SDK, unsigned Meta callback rejected, valid challenge and signed empty event accepted.

These automated checks alone do not establish provider behavior or an SLA. The separate real-provider evidence above establishes the observed test result only.

Browser Walkthrough

Build and Package

Remaining Validation

Mobile Teams cards, live multi-user handover, real rate limits, token revocation/rotation recovery, load, backup restore and migration to Azure remain unverified. The Microsoft demo secret expires 14 September; the extended Meta user token expires 6 November, 11:28 AM SGT. Production should use an appropriately scoped service credential and a rotation process.

Tenant Isolation

The tester approved a custom app upload policy assigned only to their account. No extra staff account, license purchase or global policy change was made. Only the new OCDIT Azure resource group/bot and isolated private team were provisioned. The existing host runs a separate non-root service bound to loopback.

Account-Free HTTP Rehearsal

npm run rehearse passed on Windows on 7 September 2026. A separate demo-mode process received a signed non-empty webhook over HTTP; an unsigned request was rejected and replaying the signed message created no duplicate. HTTP actions verified ownership, note isolation, queued simulated sending, a signed simulated read receipt, handover and resolution. Its temporary database was removed. This exercises the HTTP-to-store-to-worker path with simulated providers; it does not exercise Microsoft authentication or outbound provider transport.

Live Support Inbox — 7 September 2026

Live multi-user handover, mobile Teams and sustained load remain unverified.

Live LINE Inbox — 7 September 2026

This is a one-employee, text-only technical demonstration. LINE media, unsend redaction, friendship/block state and production retention remain out of scope; address them before real customer rollout. No live multi-user handover claim.

Live WeChat Inbox — 7 September 2026

Hardening and Continuity Verification — 7 September

Knowledge Library — 7 September

Staff and Administrator Guides — 7 September

Inbox Clarity and Responsive Cut — 7 September 2026

Structured Escalation and Supervisor Recovery — 7 September

Live Teams Check

Customer Profiles and Verified Context — 7 September

Live Customer Profile Verification

On 7 September 2026, the deployed Inbox passed a native Teams check using the single authorized pilot tester: create and explicitly link a fictional training profile, record a service reference as Unverified, then mark it Verified with evidence. Stored verification retains its author, time and source case. One explicit link and three profile events were confirmed. Conversation count and all outbound message IDs were unchanged from the pre-deployment backup. Cross-channel linking, unlinking, conflicting edits and lost-response recovery were exercised locally; they are not multi-user live test claims.

The public profile guide returned HTTP 200; unauthenticated profile API access returned 401. Backup and isolated restore verification passed. The service environment and all eight unrelated nginx configurations were unchanged.

Staff Feedback and Period Reporting - 7 September 2026

23 application tests pass. New checks cover case-bound source evidence, invalid staff/category rejection, supervisor-only review, reason validation, version conflicts, unchanged customer delivery jobs, immutable original suggestions and idempotent retry after restart. Date checks cover Singapore midnight boundaries, inclusive end dates, invalid/reversed/overlong ranges, distinct resolved-case actions, excluded uncertain replies, first-contact samples and unchanged current backlog for an empty period.

Local Inbox checks passed for staff submission, supervisor review, stale-decision draft retention, latest-decision comparison and lost-response reconciliation without duplicate review. Desktop and mobile captures cover feedback forms, review history and period reporting. Invalid date ranges hide old results; empty periods show No Sample. The public guides now contain 14 pages, 12 fictional UI screenshots and the data-flow diagram. These local checks do not claim live multi-user or production readiness.

Live Staff Feedback Check

On 7 September 2026, the existing authorized tester submitted a clearly fictional process suggestion from the WeChat sandbox case in native Teams, selected the original enquiry as evidence, and recorded a Planned supervisor decision with a reason. Review History displayed the decision. Database checks confirmed one suggestion, one review and its case-bound source; conversation state/ownership, outbound message IDs and delivery jobs matched the pre-deployment snapshot. Distinct-person review, Implemented transitions, stale edits and uncertain save recovery remain local test evidence.

The default seven-day Singapore report loaded in native Teams, separately showing current backlog and period activity. Unauthenticated feedback/report API probes returned 401 and the public feedback guide returned 200. The deployment preserved the environment and all eight unrelated nginx configurations; verified consistent backups and the prior release were retained.

A native reload exposed an existing initialization race between the Inbox and deferred action scripts. Startup now waits for DOMContentLoaded. A browser check deliberately delayed escalation.js: the workspace stayed hidden until the scripts were ready, then opened without a page exception. A subsequent native Teams reload was clean. All 23 application tests still pass.

Platform Administration and Data Use — 8 September 2026

Pilot Evidence and Workload — 8 September 2026

Stage 1 Release — 8 September 2026

Stage 2–4 Release — 8 September 2026

Application Review — 8 September 2026

Knowledge Lifecycle and Builder Alignment — 8 September 2026

57 application tests plus three UI/layout checks pass. Clock-controlled regressions cover review and expiry boundaries, legacy approvals, in-flight generation, support insertion/delivery and renewal invalidation. Local browser verification covered the review queue, owner/date approval, process reapproval and agent republication. Copilot alignment uses the supplied screenshot and current official Microsoft documentation; it does not add Microsoft product integration or parity.

Builder Review and Test Workflow — 8 September 2026

60 application tests and four UI/layout checks pass. Working-draft tests keep immutable snapshots, share the bounded model budget, and are rejected at every support delivery boundary. Incomplete evaluation attempts retain their scenario identity and checks from reservation. Comparison checks cover missing results, changed conditions and latest-attempt selection.

Local browser checks verified saved-versus-published changes, unsaved draft testing, trace return, selected-agent/tab preservation, invalidated-result masking and evaluation comparisons. Build, Preview, Evaluate and Monitor fit a 390-pixel viewport without page overflow. Three additional fictional screenshots document these paths. No model requests or customer sends were made for this release.

Integrated Support Workflow — 9 September 2026

129 application tests and eight UI scripts pass for the integrated local workflow. Coverage includes shared assignment, immutable source tickets and successor transfers, reviewed summaries, customer history permission, central AI controls, routing/signatures, support tools, retention and central improvement review.

A migration rehearsal against the private live snapshot preserved five cases and 25 messages. Both support and Studio databases passed integrity checks. It did not replace the live databases. Local browser checks used fictional identities and included reviewed Transfer/Resolve, desktop/mobile layout and persona definition preview. The preview check did not import or publish an agent or call a model. Actual central feedback was empty in that browser check; positive source pagination is covered by the mocked UI regression, not that empty-list observation.

Live remains fc7c021. No Graph permissions were added. Optional Jason-only Activity installation and AI eligibility for the exact throwaway number's successor tickets require approval. Deployment checks and a real Teams rehearsal remain outstanding; these local results are not a production acceptance claim.

Support Workflow Deployment — 9 September 2026

Release b14e484 deployed 192 files and 28 runtime modules. Both databases passed integrity and foreign-key checks. Five cases and 25 messages were preserved, with outbound status counts unchanged: Accepted 2, Failed 1, Read 4. Environment and eight unrelated nginx configuration hashes were unchanged.

Teams web SSO showed Connected and the new Starred, Available, Assigned, Transfer and composer controls. Personal Admin OIDC and Support Settings authorization passed. Routing is enabled for Support Team with Jason only, availability capped at eight hours and capacity 20. Signature configuration was still in progress when this evidence was recorded. No new Microsoft permissions or AI scope were added.

One live summary request on the original approved throwaway case reached the provider. Its output failed schema_or_evidence validation; the uncertain reservation remains held. Manual fallback and cancellation passed, with no transfer or customer send. A validation/accounting fix and bounded retest are pending. This is not successful live AI-summary acceptance.

A separate local check showed AI Feedback as Planned after a history grant and hid it after withdrawal. The 129 application tests and eight UI scripts remain local evidence. Optional Jason-only Activity installation, successor-ticket AI eligibility and the approved live workflow rehearsal remain acceptance gates.

Live Workflow Acceptance — Release 3b994ec, 9 September 2026

The summary schema/accounting fix is deployed; 131 application tests and eight UI checks pass. The corrected model summary succeeded on the original approved throwaway case. Jason reviewed and edited it, then transferred through Support Team (Jason only). The source resolved and the successor displayed the transferred transcript and internal reviewed summary. A signed canned reply reached WhatsApp, and its inbound response returned to the successor. Manual Resolve and case-study nomination passed on that successor without a provider call.

A new enquiry after resolution created a distinct seventh case. Single-person round-robin assigned Jason; the two previous tickets stayed resolved. Past Conversations began at Permission Required. The WhatsApp permission prompt named Jason, specified exact YES/NO and 24 hours, and carried the Permissions Agent signature. After the tester replied exact YES, Teams displayed Permission Granted, expiry and both prior summaries. View Conversation opened a read-only resolved transcript with disabled status/assignment controls and no composer.

The personal Teams bot received the new-ticket notification. Open in Inbox opened that exact active ticket with its granted history status. This verifies the existing personal-bot fallback, not the optional Activity-feed installation.

Optional signatures are saved; Billing and Technical categories and two canned replies are configured. No Graph permissions or AI scope were expanded. AI remains original-case-only, with no provider calls on the successor or new ticket. The initial rejected summary run remains historical evidence with its held reservation.

Optional Jason-only Activity installation and later-ticket AI eligibility require approval. Distinct-person behavior remains local test evidence; this live rehearsal used one real staff member and does not establish customer acceptance or model quality.

Final reconciliation found seven tickets and 31 messages. The four unrelated tickets and all 19 of their messages retained identical stored content and delivery states; all six added messages belong to the designated test customer. Both current databases and consistent post-rehearsal backups passed integrity and foreign-key checks. Environment and all eight unrelated nginx configuration hashes match the pre-release baseline. No outbound send or model generation remains pending.

Provider-reported key usage is USD0.0053788 against the unchanged USD3 cap. The successful summary recorded 682 input and 133 output tokens, cost USD0.0004856. The earlier uncertain summary retains its conservative USD0.016 reservation.

Tenant-Wide AI Update — Local Verification, 9 September 2026

The user subsequently authorized tenant-wide AI. Commit 42af769 removes the support-case allowlist and per-ticket AI blocking. The local suite passes 132 application tests and eight UI checks. Capture exclusions, history permission, knowledge reuse decisions, feature/provider checks, quotas, publication and stop controls remain. The 100-request/USD3 budget is unchanged.

Deployment is pending. Earlier original-case-only observations above remain accurate for their test dates; they are no longer the requested scope or a pending approval requirement. Optional Teams Activity RSC installation is still unapproved. No Graph permission is added by the tenant-AI update.

Tenant-Wide AI Deployment — 9 September 2026

Release 6b1875f is deployed. Real Teams Reload App showed the simplified name, Ticket Details and Supervisor menus and Draft with AI on a fresh ticket. The first insufficient-context result was a zero-cost deterministic fallback. A subsequent inline completion from a staff-written opening was model-backed, usable and inserted, then cleared without sending. A model-backed transfer-summary preview was cancelled. No transfer, resolution or customer send occurred during these checks.

The completion used 626 input/243 output tokens, costing USD0.0006392. The summary used 511 input/110 output tokens, costing USD0.0003804. Both provider runs completed and were usable. All seven case records and 31 messages remained unchanged. Both databases and backups passed integrity and foreign-key checks. Environment and eight unrelated nginx hashes were unchanged.

The suite passes 132 application tests and eight UI checks. The 4fd1b18 supervisor-menu fix passed targeted Edge checks at 1212×575 and 900×575, with all menu labels inside the ticket pane. The 100-request/USD3 budget is unchanged; optional Teams Activity RSC installation remains unapproved.

Microsoft 365 Interface Release — 9 September 2026

Release 9fa2be7553b0 is deployed. 132 application tests, nine UI scripts and focused browser workflows pass; the production dependency audit reports zero known vulnerabilities. Live Teams SSO, scoped search and queue restoration passed. Administration, Support Settings and Agent Builder share navigation and appearance; real sign-in, saved light mode and restoration to Auto passed. The guide set has 31 refreshed fictional screenshots, with 21-page link and public-boundary checks. The reviewer scored the requested label, contrast, note-border and graph fixes resolved. Real multi-staff acceptance remains separate.

Seven cases, 31 messages and 75 Studio entities match the pre-release snapshot. Both databases passed integrity and foreign-key checks. Environment and eight unrelated nginx hashes are unchanged; all 208 installed release files match the manifest. Anonymous application APIs return 401. No customer send, model request, policy save or Microsoft permission change was needed for this UI release.

Back to Top

99 Venture · OCDIT. Print from your browser for a printable copy. Screenshots use fictional local data.