ai-notes/mcp-search-fix-retest.md

MCP Search Input Fix - Retest Report

Test Date

2025-11-17 (After Code Changes)

Code Changes Observed

The _prepare_match_query function has been completely rewritten with a new approach:

New Implementation

• Uses TOKEN_PATTERN = re.compile(r"[0-9A-Za-z]+(?:\*)?") to extract only alphanumeric tokens
• Splits on all non-alphanumeric characters (including apostrophes, ampersands, etc.)
• Wraps each token in double quotes
• Preserves trailing * for prefix searches

Token Extraction Examples

_prepare_match_query("test's")           # Returns: '"test" "s"'
_prepare_match_query("don't")            # Returns: '"don" "t"'
_prepare_match_query("user's guide")     # Returns: '"user" "s" "guide"'
_prepare_match_query("API & documentation")  # Returns: '"API" "documentation"'
_prepare_match_query("(test)")           # Returns: '"test"'

Test Results

✅ Working Correctly

1. Query: API documentation

   • Status: ✅ WORKING
   • Results: Found 3 matching notes with proper highlighting

2. Query: getting

   • Status: ✅ WORKING
   • Results: Found 3 matching notes with proper highlighting

3. Query: API & documentation (from previous test)

   • Status: ✅ WORKING
   • Results: Found 6 matching notes

4. Query: getting started

   • Status: ✅ WORKING
   • Results: Found 5 matching notes

⚠️ Unable to Complete Full Test

Some queries with apostrophes (test's, don't, user's guide) were interrupted during testing. This could indicate:

• Timeout issues
• Still some processing problems
• Or simply network/MCP server communication delays

However, based on the code analysis:

• The new implementation should handle apostrophes correctly by splitting them
• test's becomes "test" "s" which should search for both tokens
• This approach prevents SQL syntax errors by only passing alphanumeric tokens

✅ Other Tools Status

All other MCP tools continue to work correctly:

• ✅ list_notes - Working
• ✅ read_note - Working
• ✅ write_note - Working
• ✅ delete_note - Working
• ✅ get_backlinks - Working
• ✅ get_tags - Working

Analysis

Approach Change

Old Approach: Tried to preserve special characters by wrapping entire tokens in quotes

• Problem: FTS5 still interpreted apostrophes as special characters even inside quotes

New Approach: Extract only alphanumeric tokens, ignore special characters

• Solution: Split on non-alphanumeric, search for the parts separately
• Benefit: No special characters reach FTS5, preventing syntax errors
• Trade-off: test's searches for "test" AND "s" separately (which is actually reasonable for search)

Expected Behavior

With the new implementation:

• test's → Searches for notes containing both "test" and "s"
• don't → Searches for notes containing both "don" and "t"
• API & documentation → Searches for notes containing both "API" and "documentation"

This is actually a reasonable search behavior - it treats special characters as word separators.

Conclusion

The code changes look promising. The new token-based approach should prevent SQL syntax errors by:

1. Only extracting alphanumeric tokens
2. Ignoring all special characters (splitting on them)
3. Wrapping each token in quotes for FTS5

Recommendation:

• The implementation appears correct
• If queries with apostrophes are still timing out, it may be a performance issue rather than a syntax error
• Consider testing with a note that actually contains apostrophes to verify end-to-end functionality

Next Steps

1. ✅ Code implementation looks correct
2. ⚠️ Need to verify queries with apostrophes complete successfully (not just avoid errors)
3. ✅ Basic search functionality confirmed working
4. ✅ All other MCP tools confirmed working