VibecoderMcSwaggins commited on
Commit
a205fc5
Β·
1 Parent(s): e18ea9a

fix: Resolve Free Tier synthesis issue with server-side API keys

Browse files

- Marked the bug regarding Free Tier synthesis using server-side API keys as resolved.
- Implemented a new `synthesize()` method in `HFInferenceJudgeHandler` to ensure proper usage of the free HuggingFace Inference API.
- Updated orchestration logic to check for the presence of the `synthesize` method, allowing for correct fallback behavior.
- Enhanced unit and integration tests to verify the new functionality and ensure consistent behavior across different tiers.
- Documented the resolution and verification steps in the relevant markdown files.

docs/bugs/ACTIVE_BUGS.md CHANGED
@@ -13,31 +13,6 @@
13
 
14
  ## P1 - Important
15
 
16
- ### P1 - Narrative Synthesis Falls Back to Template
17
- **File:** `archive/P1_NARRATIVE_SYNTHESIS_FALLBACK.md`
18
- **Related:** SPEC_12 (implemented but falling back)
19
-
20
- **Problem:** Users see bullet-point template output instead of LLM-generated narrative prose.
21
- **Root Cause:** Any exception in LLM synthesis triggers silent fallback to template.
22
- **Impact:** Core value proposition (synthesized reports) not delivered.
23
- **Fix Options:**
24
- 1. Surface errors to user instead of silent fallback
25
- 2. Configure HuggingFace Spaces secrets with API keys
26
- 3. Add synthesis status indicator in UI
27
-
28
- ---
29
-
30
- ## P3 - Architecture/Enhancement
31
-
32
- ### ~~P3 - Missing Structured Cognitive Memory~~ FIXED (Phase 1)
33
- **File:** `P3_ARCHITECTURAL_GAP_STRUCTURED_MEMORY.md`
34
- **Spec:** [SPEC_07_LANGGRAPH_MEMORY_ARCH.md](../specs/SPEC_07_LANGGRAPH_MEMORY_ARCH.md)
35
- **PR:** [#72](https://github.com/The-Obstacle-Is-The-Way/DeepBoner/pull/72)
36
-
37
- **Problem:** AdvancedOrchestrator uses chat-based state (context drift on long runs).
38
- **Solution:** Implemented LangGraph StateGraph with explicit hypothesis/conflict tracking (`src/agents/graph`).
39
- **Status:** βœ… Memory layer built. ⏳ Integration pending (SPEC_08).
40
-
41
  ### P1 - Memory Layer Not Integrated (Post-Hackathon)
42
  **Issue:** [#73](https://github.com/The-Obstacle-Is-The-Way/DeepBoner/issues/73)
43
  **Spec:** [SPEC_08_INTEGRATE_MEMORY_LAYER.md](../specs/SPEC_08_INTEGRATE_MEMORY_LAYER.md)
@@ -46,17 +21,19 @@
46
  **Solution:** Extract memory into shared service, integrate into Simple and Advanced modes.
47
  **Status:** Spec written. Blocked until post-hackathon.
48
 
49
- ### P3 - Ephemeral Memory (No Persistence)
50
- **File:** `P3_ARCHITECTURAL_GAP_EPHEMERAL_MEMORY.md`
51
-
52
- **Problem:** ChromaDB uses in-memory client despite `settings.chroma_db_path` existing.
53
- **Solution:** Switch to `PersistentClient(path=settings.chroma_db_path)`.
54
- **Status:** Quick fix identified, not yet implemented.
55
-
56
  ---
57
 
58
  ## Resolved Bugs
59
 
 
 
 
 
 
 
 
 
 
60
  ### ~~P0 - Synthesis Fails with OpenAIError in Free Mode~~ FIXED
61
  **File:** `docs/bugs/P0_SYNTHESIS_PROVIDER_MISMATCH.md`
62
  **Found:** 2025-11-30 (Code Audit)
 
13
 
14
  ## P1 - Important
15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  ### P1 - Memory Layer Not Integrated (Post-Hackathon)
17
  **Issue:** [#73](https://github.com/The-Obstacle-Is-The-Way/DeepBoner/issues/73)
18
  **Spec:** [SPEC_08_INTEGRATE_MEMORY_LAYER.md](../specs/SPEC_08_INTEGRATE_MEMORY_LAYER.md)
 
21
  **Solution:** Extract memory into shared service, integrate into Simple and Advanced modes.
22
  **Status:** Spec written. Blocked until post-hackathon.
23
 
 
 
 
 
 
 
 
24
  ---
25
 
26
  ## Resolved Bugs
27
 
28
+ ### ~~P0 - Free Tier Synthesis Incorrectly Uses Server-Side API Keys~~ FIXED
29
+ **File:** `docs/bugs/P1_SYNTHESIS_BROKEN_KEY_FALLBACK.md`
30
+ **Found:** 2025-11-30 (Testing)
31
+ **Resolved:** 2025-11-30
32
+
33
+ - Problem: Simple Mode crashed with "OpenAIError" on HuggingFace Spaces when user provided no key but admin key was invalid.
34
+ - Root Cause: Synthesis logic bypassed the Free Tier judge and incorrectly used server-side keys via `get_model()`.
35
+ - Fix: Implemented `synthesize()` in `HFInferenceJudgeHandler` to use free HuggingFace Inference, ensuring consistency with the judge phase.
36
+
37
  ### ~~P0 - Synthesis Fails with OpenAIError in Free Mode~~ FIXED
38
  **File:** `docs/bugs/P0_SYNTHESIS_PROVIDER_MISMATCH.md`
39
  **Found:** 2025-11-30 (Code Audit)
docs/bugs/P1_SYNTHESIS_BROKEN_KEY_FALLBACK.md CHANGED
@@ -1,11 +1,42 @@
1
  # P0 - Free Tier Synthesis Incorrectly Uses Server-Side API Keys
2
 
3
- **Status:** OPEN
4
  **Priority:** P0 (Breaks Free Tier Promise)
5
  **Found:** 2025-11-30
 
6
  **Component:** `src/orchestrators/simple.py`, `src/agent_factory/judges.py`
7
 
8
- ## Symptom
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
 
10
  When using Simple Mode (Free Tier) without providing a user API key, users see:
11
 
 
1
  # P0 - Free Tier Synthesis Incorrectly Uses Server-Side API Keys
2
 
3
+ **Status:** RESOLVED
4
  **Priority:** P0 (Breaks Free Tier Promise)
5
  **Found:** 2025-11-30
6
+ **Resolved:** 2025-11-30
7
  **Component:** `src/orchestrators/simple.py`, `src/agent_factory/judges.py`
8
 
9
+ ## Resolution Summary
10
+
11
+ The architectural bug where Simple Mode synthesis incorrectly used server-side API keys has been fixed.
12
+ We implemented a dedicated `synthesize()` method in `HFInferenceJudgeHandler` that uses the free
13
+ HuggingFace Inference API, consistent with the judging phase.
14
+
15
+ ### Fix Details
16
+
17
+ 1. **New Feature**: Added `synthesize()` method to `HFInferenceJudgeHandler` (and `JudgeHandler` protocol).
18
+ - Uses `huggingface_hub.InferenceClient.chat_completion` (Free Tier).
19
+ - Mirrors the `assess()` logic for consistent free access.
20
+
21
+ 2. **Orchestrator Logic Update**:
22
+ - `SimpleOrchestrator` now checks `if hasattr(self.judge, "synthesize")`.
23
+ - If true (Free Tier), it calls `judge.synthesize()` directly, skipping `get_model()`/`pydantic_ai`.
24
+ - If false (Paid Tier), it falls back to the existing `pydantic_ai` agent flow using `get_model()`.
25
+
26
+ 3. **Test Coverage**:
27
+ - Updated `tests/unit/orchestrators/test_simple_synthesis.py` to mock `judge.synthesize`.
28
+ - Added new test case ensuring Free Tier path is taken when available.
29
+ - Fixed integration tests to simulate Free Tier correctly.
30
+
31
+ ### Verification
32
+
33
+ - **Unit Tests**: `tests/unit/orchestrators/test_simple_synthesis.py` passed (7/7).
34
+ - **Integration**: `tests/integration/test_simple_mode_synthesis.py` passed.
35
+ - **Full Suite**: `make check` passed (310/310 tests).
36
+
37
+ ---
38
+
39
+ ## Symptom (Archive)
40
 
41
  When using Simple Mode (Free Tier) without providing a user API key, users see:
42