tendril/TESTING_FINDINGS.md

# Testing Findings & Improvements

**Date**: November 10, 2024  
**Tester**: User testing in Zed IDE  
**Focus**: Transport modes and configuration validation

## Test Results Summary

### ✅ STDIO Mode - WORKING
- **Status**: Fully functional
- **Transport**: stdin/stdout direct communication
- **Configuration**: Minimal (token only)
- **Testing**: Confirmed working in production use
- **Recommendation**: Default and recommended mode

### ❌ SSE Mode - NOT WORKING (Removed)
- **Status**: Deprecated by gitea-mcp upstream
- **Issue**: MCP initialization timeout after 60 seconds
- **Root Cause**: gitea-mcp removed SSE mode in favor of HTTP Streaming
- **Action Taken**: Removed SSE support from Tendril v0.0.2+
- **Decision Rationale**: Dead code - no longer supported by server

## Key Findings

### 1. gitea-mcp Deprecated SSE Mode
- **Evidence**: Commit 88471b5de048ee35e929a5c3e5789f50ba57a845 on gitea-mcp
- **Reason**: MCP specification evolved to Streamable HTTP standard
- **Impact**: Any extension trying to use SSE mode will fail
- **Status**: Old MCP specification (pre-2025-03-26)

### 2. MCP Specification Update
- **Old Standard**: HTTP + SSE transport (Server-Sent Events)
- **New Standard**: Streamable HTTP transport (unified, bidirectional)
- **Effective Date**: MCP 2025-03-26 and later
- **Resource**: https://modelcontextprotocol.io/specification/2025-06-18/basic/transports

### 3. SSE Mode Error Signature
When attempting SSE mode, Zed produced:
```
ERROR [context_server::client] Unhandled JSON from context_server
ERROR [context_server::client] cancelled csp request task for "initialize" id 0 which took over 60s
ERROR [project::context_server_store] tendril-gitea-mcp context server failed to start: 
Context server request timeout
```

Server logs showed:
```
2025-11-10 14:57:59	INFO	operation/operation.go:61	Gitea MCP SSE server listening on :8987
(No further initialization - timeout occurs)
```

**Analysis**: 
- Server started listening on port
- Zed tried to initialize MCP protocol
- Handshake timed out after 60 seconds
- Root cause: gitea-mcp's SSE mode not properly implementing MCP protocol over SSE

## Improvements Made

### Code Cleanup
- Removed `use_sse` configuration option
- Removed `gitea_port` setting (was SSE-only)
- Removed SSE-specific command-line argument handling
- Simplified `GiteaContextServerSettings` struct from 5 to 3 fields
- Hardcoded STDIO transport (the only working mode)

### Configuration Simplification
**Before (v0.0.1)**:
```json
{
  "gitea_access_token": "token",
  "gitea_host": "optional",
  "gitea_port": "optional (for SSE)",
  "gitea_insecure": "optional",
  "use_sse": "optional (doesn't work)"
}
```

**After (v0.0.2+)**:
```json
{
  "gitea_access_token": "token",
  "gitea_host": "optional",
  "gitea_insecure": "optional"
}
```

### Documentation Updates
- Updated `default_settings.jsonc` - removed SSE and port options
- Updated `installation_instructions.md` - removed SSE documentation
- Updated code comments - clarified STDIO as only transport
- Added `SSE_MODE_ANALYSIS.md` - detailed investigation document

## User Impact

### For Existing Users
- **STDIO mode users**: No changes needed - everything continues to work
- **SSE mode users**: Update config to remove `use_sse` and `gitea_port` options
  - STDIO mode is now default (works the same way)
  - No functionality lost

### For New Users
- Simpler configuration (3 options instead of 5)
- No confusing SSE mode option that doesn't work
- Clearer documentation focused on working features
- Faster setup time

## Performance Implications

### STDIO Mode (Current)
- Direct process communication
- No network overhead
- Sub-millisecond latency
- Single connection (sufficient for IDE usage)
- Perfect for local development

### HTTP Streaming (Future Alternative)
- Network round-trip latency
- Supports multiple concurrent connections
- Better for remote/shared servers
- Not yet implemented but documented for future

## Lessons Learned

### 1. Keep Code Sync with Upstream
- SSE mode removal in gitea-mcp wasn't immediately apparent
- Testing revealed the issue
- Dead code maintenance becomes technical debt
- Regular dependency audits would help catch this earlier

### 2. Test Against Real Implementations
- Code looked correct for SSE mode
- But gitea-mcp implementation changed
- Live testing revealed the disconnect
- Upstream changes need monitoring

### 3. Follow Spec Evolution
- MCP specification evolved (HTTP+SSE → Streamable HTTP)
- gitea-mcp followed the new spec
- Tendril should stay current with both upstream and spec
- Document planned support for new standards

### 4. Configuration Matters
- More options = more confusion
- Non-working options hurt credibility
- Simpler is better when it works
- Users appreciate clarity over feature count

## Recommendations for Future Development

### Short-term (v0.1.0)
- Monitor gitea-mcp for HTTP Streaming support confirmation
- Add configurable binary path (high priority from original roadmap)
- Add Docker support option
- Keep STDIO as default and primary mode

### Medium-term (v0.2.0)
- When gitea-mcp has HTTP Streaming support, evaluate implementation
- Create HTTP client support in Tendril
- Allow users to choose between STDIO and HTTP Streaming
- Document use cases for each transport

### Long-term (v1.0+)
- Full HTTP Streaming implementation
- Support for remote gitea-mcp servers
- Connection pooling for multiple Zed instances
- Load balancing for large teams

## Testing Checklist for Future Contributors

When testing Tendril:
- [ ] Verify STDIO mode with local gitea-mcp binary
- [ ] Confirm token-based authentication works
- [ ] Test self-hosted Gitea instance
- [ ] Test self-signed certificate handling
- [ ] Verify error messages are helpful
- [ ] Check Zed logs for any warnings
- [ ] Monitor gitea-mcp logs during operation
- [ ] Test with different Gitea instances if possible

When updating gitea-mcp version:
- [ ] Review gitea-mcp release notes for transport changes
- [ ] Test with new version before updating docs
- [ ] Check if new transport modes became available
- [ ] Update dependencies and CHANGELOG
- [ ] Run full test suite

## References

- **gitea-mcp Repository**: https://gitea.com/gitea/gitea-mcp
- **Commit that removed SSE**: https://gitea.com/gitea/gitea-mcp/commit/88471b5de048ee35e929a5c3e5789f50ba57a845
- **MCP Specification (Current)**: https://modelcontextprotocol.io/specification/2025-06-18/basic/transports
- **Streamable HTTP Spec**: https://modelcontextprotocol.io/specification/2025-06-18/basic/transports#streamable-http
- **MCP Transport Overview**: https://docs.roocode.com/features/mcp/server-transports

## Conclusion

Testing revealed that SSE mode was already deprecated in gitea-mcp due to MCP specification evolution. This finding led to:

1. **Code Simplification**: Removed non-functional SSE code paths
2. **Configuration Improvement**: Reduced options from 5 to 3, removed confusing disabled mode
3. **Documentation Clarity**: Focused docs on what actually works
4. **Future Planning**: Identified HTTP Streaming as the next transport to support

The result is a cleaner, more maintainable extension with clearer user experience. STDIO mode remains robust and reliable, with a clear migration path documented for when HTTP Streaming support becomes desirable.

---

**Status**: v0.0.2 cleanup complete  
**Next Steps**: Focus on binary path configuration and Docker support (v0.1.0)  
**Verified By**: Live testing in Zed IDE, November 10, 2024