Phase 0 · Foundation Assessment — BEA-02 Ratings & Reviews Tracking
Establish the foundation before any analysis. The permanent base layer (FAD) for all future work.
Retroactive note: BEA-02 is already live (phase 6-execute). This foundation is being reconstructed after the fact to establish the analytical backing the build never received. Several criteria below are therefore written as ratification targets the live system must now be measured against — not as pre-build hypotheses.
1. Success criteria (measurable, time-bound)
- [ ] Vendor displacement / cost: Fully retire the Yogi contract and run BEA-02 in production at a fully-loaded annual run cost below $45K (compute + maintenance + ~0.25 FTE), i.e. ≥70% net savings against the ~$150K/yr Yogi spend, verified across one full renewal cycle (12 months from cutover).
- [ ] Coverage & freshness parity: Match or exceed Yogi's coverage — ≥95% of tracked Beats ASINs ingested on every scheduled run, with daily cron completing and publishing before 09:00 PT ≥95% of business days, measured over a rolling 90-day window.
- [ ] Decision adoption: ≥3 named stakeholder workflows (e.g. product/quality, merchandising/PDP, exec competitive readout) consume the daily/weekly/monthly outputs at least weekly, evidenced by tracked report opens or pulled metrics, within 90 days of cutover.
- [ ] Signal fidelity: Sentiment/theme classification agreement with a human-labeled audit sample ≥85% (Cohen's κ ≥0.7) on a quarterly 200-review sample — i.e. the analysis is trustworthy enough to act on, not just cheaper.
- [ ] Operational resilience: Mean time to detect a failed/stale ingestion run ≤24h and mean time to recovery ≤48h, with zero silent data gaps >2 days, over a rolling 90-day window.
2. Decision-maker profile (Three Ledgers)
- Public ledger (what we pitch): "Replace an expensive third-party review-analytics vendor (Yogi) with an in-house Amazon ratings & reviews aggregator that delivers the same daily/weekly/monthly competitive and quality intelligence at a fraction of the cost, with data we fully own."
- Shadow ledger (what the decision-maker actually optimizes for): Eliminating a recurring SaaS line item under budget pressure; removing dependence on an external vendor's roadmap and data-access terms; demonstrating that the internal team can ship proprietary data-intelligence (org credibility / headcount justification); avoiding the procurement and security-review friction of renewing an outside vendor.
- True ledger (what will actually get built): A scheduled scraping/ingestion pipeline against Amazon review data for Beats ASINs, an NLP/LLM sentiment-and-theme analysis layer, and cron-driven daily/weekly/monthly rollups — scoped to Beats products only, owned and operated internally, with reporting consumed by a handful of internal stakeholders rather than a polished multi-tenant product.
- Owner / sponsor: Patrick O'Brien (per manifest
owner). Acts as both build owner and budget sponsor; the cost-savings narrative is his to defend to finance/procurement.
3. Constraint inventory
| Constraint | Type (hard / soft / assumed) | Notes |
|---|---|---|
| Amazon review data access must not breach Amazon ToS / anti-scraping enforcement | Hard | Existential legal/operational risk for an Apple-owned property; sustainable access (compliant API, licensed feed, or resilient scraping) is the load-bearing assumption of the whole build. |
| Beats-proprietary; not reusable beyond Beats | Hard | README states "Reusable beyond Beats? No." Scope and data are locked to Beats assets; no multi-tenant or external productization. |
| Must reach functional parity with Yogi before the Yogi contract can be cancelled | Hard | Savings are not realized until cutover; running both in parallel temporarily erodes the ROI window. |
| Total run cost must stay well under ~$150K/yr to justify replacement | Soft | The economic thesis; some overrun tolerable if data ownership + control are valued, but large overrun kills the rationale. |
| Daily/weekly/monthly cron cadence and pre-09:00 PT delivery | Soft | Cadence is set by stakeholder reporting rhythm; negotiable in detail but the "daily intelligence" promise anchors adoption. |
| Apple internal security / data-handling review for any external data ingestion | Assumed | Assumed cleared or low-friction because internal; if InfoSec restricts external scraping or PII handling of reviewer data, scope changes materially. |
| Sentiment/theme model quality is "good enough" to replace Yogi's human-or-vendor-tuned analysis | Assumed | Replacing a specialized review-analytics vendor assumes an in-house model matches its taxonomy/accuracy — unvalidated at foundation. |
4. Scope boundary
- In scope: Amazon ratings & reviews ingestion for Beats ASINs; sentiment + theme/topic classification; daily/weekly/monthly aggregation via cron; internal reporting/readout of trends, anomalies, and competitive signal; full retirement of the Yogi vendor dependency.
- Out of scope: Non-Amazon retail channels (Best Buy, Target, carrier stores, Apple Store); non-Beats Apple products; social-media or support-ticket sentiment; multi-tenant / external productization; real-time (sub-daily) streaming; automated responses or actions taken on reviews (BEA-02 informs decisions; humans act).
- Recursion budget acknowledged: max 3 per phase.
5. Assumption log
| # | Assumption | Confidence (0–1) | Validated? |
|---|---|---|---|
| 1 | Sustainable, ToS-acceptable access to Amazon review data can be maintained at the required cadence and coverage | 0.45 | No — single largest unvalidated risk; must be proven, not assumed |
| 2 | An in-house NLP/LLM analysis layer reaches sentiment/theme fidelity comparable to what Yogi delivered | 0.55 | No — no benchmark against Yogi outputs documented at foundation |
| 3 | Fully-loaded annual run cost lands materially below $150K (target <$45K) | 0.6 | Partially — plausible for cron + compute + LLM calls, but maintenance/firefighting labor is typically undercounted |
| 4 | The displaced Yogi value was primarily the data + analysis, not relationship/SLAs/coverage breadth BEA-02 can't match | 0.5 | No — Yogi's full scope vs. BEA-02's Amazon-only scope not formally compared |
| 5 | ≥3 internal stakeholder workflows will actually adopt and act on BEA-02 outputs | 0.55 | No — adoption assumed from the vendor's prior usage, not re-confirmed for the new tool/UX |
| 6 | Internal security/data-handling review permits external review-data ingestion without blocking scope | 0.6 | No — treated as low-friction because internal; not confirmed |
| 7 | The daily/weekly/monthly cron cadence matches how stakeholders actually consume competitive/quality signal | 0.7 | Partially — inherited from Yogi's cadence, reasonable but not re-tested for the new workflow |
Gate: success criteria defined, constraints mapped, decision-maker identified → set phaseGates.0-foundation = passed in manifest.json.