Similarity Search Patterns
FreeEfficiently implement similarity search in production systems.
Free · Opens the source repo
What Similarity Search Patterns does
The Similarity Search Patterns skill provides a structured approach to implementing efficient similarity search using vector databases. It is particularly useful for developers and data scientists who are building semantic search systems, recommendation engines, or optimizing retrieval performance. By leveraging various distance metrics and index types, users can tailor their search capabilities to meet specific performance and accuracy requirements.
This skill covers essential concepts such as distance metrics, which include Cosine, Euclidean, Dot Product, and Manhattan distances, each suited for different types of data and use cases. Additionally, it outlines various index types like Flat, HNSW, and IVF+PQ, detailing their performance characteristics and best use scenarios. This guidance is crucial for scaling search systems to handle millions of vectors efficiently while maintaining high recall rates.
Included in this skill are best practices that help users avoid common pitfalls, such as over-indexing and neglecting latency considerations. It emphasizes the importance of evaluating search quality and tuning parameters to achieve optimal performance. The skill also encourages users to implement hybrid search methodologies, combining semantic and keyword searches for enhanced results.
For those looking for concrete implementations, the skill provides a library of templates and detailed worked examples in the accompanying references/details.md file. This resource is invaluable for practitioners who need practical guidance on deploying similarity search in real-world applications.
When to use it
Use this skill when developing semantic search systems, recommendation engines, or optimizing search latency in applications that require high-performance retrieval.
When not to use it
This skill may not be suitable for simple keyword-based search systems or when working with very small datasets where advanced indexing is unnecessary.
What you can build with it
Building a Recommendation Engine
Utilize similarity search patterns to create a recommendation engine that suggests items based on user preferences and behavior.
Implementing Semantic Search
Leverage the skill to build a semantic search system that retrieves documents or products based on their meaning rather than just keywords.
Optimizing Search Performance
Apply best practices from this skill to enhance the search performance of applications handling large datasets.
How to install Similarity Search Patterns
View source1. Install with the skills CLI
npx skills add wshobson/agents/similarity-search-patterns --agent claude-code2. Or install it manually
Download the skill folder and drop it into ~/.claude/skills/ for all projects, or .claude/skills/ to scope it to one repo. Restart Claude Code so it picks up the new skill.
Anthropic's agentic coding CLI, and the reference implementation of Agent Skills. Drop a skill folder into ~/.claude/skills and Claude Code loads it automatically whenever a task matches the skill's description. Claude Code docs
Inside SKILL.md
Written by wshobsonSimilarity Search Patterns
Patterns for implementing efficient similarity search in production systems.
When to Use This Skill
- Building semantic search systems
- Implementing RAG retrieval
- Creating recommendation engines
- Optimizing search latency
- Scaling to millions of vectors
- Combining semantic and keyword search
Core Concepts
1. Distance Metrics
| Metric | Formula | Best For | | ------------------ | ------------------ | --------------------- | --- | -------------- | | Cosine | 1 - (A·B)/(‖A‖‖B‖) | Normalized embeddings | | Euclidean (L2) | √Σ(a-b)² | Raw embeddings | | Dot Product | A·B | Magnitude matters | | Manhattan (L1) | Σ | a-b | | Sparse vectors |
2. Index Types
┌─────────────────────────────────────────────────┐
│ Index Types │
├─────────────┬───────────────┬───────────────────┤
│ Flat │ HNSW │ IVF+PQ │
│ (Exact) │ (Graph-based) │ (Quantized) │
├─────────────┼───────────────┼───────────────────┤
│ O(n) search │ O(log n) │ O(√n) │
│ 100% recall │ ~95-99% │ ~90-95% │
│ Small data │ Medium-Large │ Very Large │
└─────────────┴───────────────┴───────────────────┘
Templates and detailed worked examples
Full template library and detailed worked examples live in references/details.md. Read that file when you need the concrete templates.
Best Practices
Do's
- Use appropriate index - HNSW for most cases
- Tune parameters - ef_search, nprobe for recall/speed
- Implement hybrid search - Combine with keyword search
- Monitor recall - Measure search quality
- Pre-filter when possible - Reduce search space
Don'ts
- Don't skip evaluation - Measure before optimizing
- Don't over-index - Start with flat, scale up
- Don't ignore latency - P99 matters for UX
- Don't forget costs - Vector storage adds up
Frequently asked questions about Similarity Search Patterns
Similar skills
ClickHouse Logs Queries
Efficiently manage Supabase logs with ClickHouse SQL.
EF Core D2 Database Diagram Generator
Visualize your EF Core models as D2 diagrams effortlessly.
Safe SQL Execution
Ensure secure SQL execution in Supabase applications.
Oracle to PostgreSQL Migration
Identify migration risks between Oracle and PostgreSQL.
SSMA Console
Streamline Oracle to SQL Server migrations with ease.
SQL Performance Optimization
Enhance SQL query efficiency across all databases.