When to use it
Reach for full-text search when exact words, phrases, names, codes, or identifiers matter, not just semantic similarity. For semantic-only or vector-only workloads, an index with dense vectors (the Vectors API) is simpler. See Search overview to choose the right approach.Full-text search requires API version
2026-07: send X-Pinecone-Api-Version: 2026-07 on REST requests, or use the 2026-07 Python SDK (v10 or later).Capabilities
What full-text search does and doesn’t match:How it works
Pinecone’s Documents API stores typed fields you declare in a schema. End to end, the flow is short: create an index with a schema that declares your ranking fields, upsert your data as JSON documents, then search by choosing one ranking signal per search request withscore_by. The end-to-end example below stitches all three steps into one runnable script.
- You upsert data as JSON documents.
- You declare how each field should be indexed via a schema, as a
stringfield withfull_text_searchenabled (BM25 scoring), adense_vectorfield, or asparse_vectorfield. The schema is for ranking fields only; metadata fields are not declared. - Pinecone indexes each field’s content according to the type of the field declared in the schema. Any other fields on the upserted documents are automatically stored and indexed for filtering, no schema declaration required.
include_fields, and automatically indexed for filtering, see Metadata fields.
Newly upserted documents are indexed asynchronously, so they may not be searchable immediately. To confirm that an index holds the documents you expect, see Check data freshness.
Every search ranks by one scoring type. The score_by clause selects the scoring method for the request:
text, BM25 token matching over one or more FTS-enabledstringfields.query_string, Lucene query syntax across one or more FTS-enabledstringfields, including cross-field boolean queries.dense_vector, vector similarity against adense_vectorfield.sparse_vector, sparse-vector similarity against asparse_vectorfield.
score_by with a metadata filter, including the text-match operators $match_phrase, $match_all, and $match_any on FTS-enabled string fields, plus the standard logical and comparison operators ($and, $or, $not, $exists, etc.). The filter narrows what’s eligible, and the score_by ranks what remains. This is the most common hybrid pattern. The alternative is to run separate searches and merge the results client-side, for example with reciprocal rank fusion.
For example, on an index whose schema declares both a dense_vector field (review_embedding) and an FTS-enabled string field (review_text), this single request runs semantic search across the corpus but only over documents whose review_text contains the exact phrase “beautifully written”:
Python
text clause, or pass multiple text clauses, and Pinecone combines them into a single ranking. A query_string clause can target several fields too. Every field contributes equally in 2026-07, and there’s no per-field weight parameter.
End-to-end example
A complete run from index creation through search. Copy this into a single file, setPINECONE_API_KEY, and run.
Python
SchemaBuilder().add_string_field(..., full_text_search={"language": "en"})declares a BM25-indexed text field. Withoutfull_text_search, thestringfield would be rejected at index creation — schemas only declare ranking fields.index.documents.upsert(...)writes plain JSON documents. Schema fields are validated; non-schema fields (category,yearhere) are stored and auto-indexed for filtering. For large datasets, use Import instead.score_by=[{"type": "text", ...}]picks BM25 as the scoring type. One scoring type per request; combine scoring with text matching viafilterrather than mixing scoring methods.filternarrows candidates before ranking. Standard operators ($eq,$gte, etc.) apply to any metadata field; the text-match operators ($match_phrase,$match_all,$match_any) only apply to FTS-enabledstringfields._scoreis the system-owned relevance score. A user metadata field namedscorewould be returned alongside, untouched.
Filters vs. scoring
Filters are deterministic — each document either matches or it doesn’t — and they apply before scoring. Scoring methods (text/BM25, query_string/Lucene, dense_vector, sparse_vector) order whatever remains after filtering, and only the top top_k hits are returned (max 10,000).
When you’re combining text matching with vector ranking, start with the hard yes/no constraints as filters (including the text-match operators $match_phrase, $match_all, $match_any on FTS-enabled string fields), then pick a score_by method to rank whatever remains. Use BM25 (score_by text or query_string) when keyword and phrase ranking order matters, not just inclusion.
Schema definition
The schema is required at index creation and declares the fields that drive ranking or vector search. Filterable metadata is not declared in the schema: any field you upsert that isn’t in the schema is automatically stored and indexed for filtering. A schema can declare up to 100string fields with full_text_search enabled, but at most one dense_vector field and at most one sparse_vector field per index.
Schema field types
Schemas can only declare ranking fields. Declaring a metadata-only field (a
string field without full_text_search, or a string_list, float, or boolean field) is rejected at index creation with a 400 error. Metadata fields are auto-indexed at upsert time. See Metadata fields._ or $. The _ prefix is reserved for system-managed fields (for example, _id, _score); $ is reserved for filter operators. Field names are also limited to 64 bytes. Every document has a required _id field, which carries its unique identifier. A user metadata field named score is allowed, and match scores are returned as _score to avoid collisions.
Indexes with document schemas do not support integrated inference fields such as
semantic_text. To use dense or sparse vector ranking in an index with a document schema, declare a dense_vector or sparse_vector field and provide vector values at upsert time.A
string field with full_text_search isn’t metadata and doesn’t count toward the 40 KB metadata limit for documents. Use these FTS-enabled string fields for searchable chunk text. Indexes with document schemas do not support combining integrated inference fields, such as semantic_text fields, with full-text-search fields. To combine semantic ranking with full-text search, declare a dense_vector field alongside one or more FTS-enabled string fields and provide dense vector values when you upsert documents.Example schemas
A text-only schema. The minimal{} config enables FTS with all defaults; sub-fields like language, stemming, and stop_words are optional overrides:
Including
full_text_search, even an empty object {}, is what turns full-text search on for a string field. Without it, the field is rejected at index creation, because schemas only declare ranking fields.category (string), tags (array of strings), year (number), or in_stock (boolean). These fields are stored on the document, returned via include_fields, and automatically indexed for filtering. They do not need to be declared in the schema.
Metadata fields
Metadata fields are not declared in the schema. Any field you include on an upserted document that is not declared in the schema is treated as metadata: it is stored on the document, returned viainclude_fields, and automatically indexed for filtering with the standard operators ($eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $exists, $and, $or, $not).
Metadata field types are inferred from the values you upsert: strings, numbers (stored as floating point), booleans, and arrays of strings are all supported. You can mix metadata field types across documents in the same index.
Because metadata isn’t declared anywhere, track the field names and types you use in your application.
Schema validation
Documents are validated against the index schema on upsert. If any document is invalid, the entire upsert fails and nothing is written. For the validation rules, see Schema validation.Filter operators
Filters are applied before the search runs, so the search only considers documents that match. On document indexes, a filter can use the comparison and set operators ($eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, $exists), the logical operators $and, $or, and $not, and the text-match operators ($match_phrase, $match_all, $match_any) on FTS-enabled string fields. Multiple fields at the top level of a filter object combine with implicit AND. For operator details and examples, see Filter by metadata.
Search examples
These examples combinescore_by scoring with filter narrowing on a document index.
Token matching with a filter
Cross-field boolean query (query_string)
Dense ranking with a phrase-match filter
BM25 ranking with a text-match filter
body contains both “federal” and “reserve”, then ranks those candidates by BM25 score against “monetary policy impact”.
Phrase filter with negation
Troubleshooting
Document not appearing in search results
Document not appearing in search results
- Check indexing latency: new documents may take up to 1 minute to become searchable; schemas with multiple indexed fields may take slightly longer.
- Verify the upsert response shows the expected
upserted_count. - Confirm you’re searching the same namespace where you upserted.
- With
type: "text", multi-word queries use token OR matching — documents need not contain the full phrase. Try a single-term query first to confirm the document is searchable. - If using filters, ensure the document’s field values match your filter conditions. Metadata fields are auto-indexed at upsert time, so any field present on a document can be filtered on; filtering on a field that no document contains returns no results.
Unexpected search results
Unexpected search results
type: "text"uses OR across terms.machine learningmatches documents that contain “machine”, “learning”, or both (BM25 ranking). For an exact phrase, usetype: "query_string"withbody:("machine learning")or a$match_phrasefilter.type: "query_string"defaults to OR for unquoted terms.body:(machine learning)matches documents containing either term. UseANDor+for required terms.- Operators like
AND,OR,NOT,*,~, and^only work withtype: "query_string". Withtype: "text", they are treated as literal words.
Query syntax errors
Query syntax errors
Query syntax errors only apply to
type: "query_string". With type: "text", any input is valid as a literal string to be tokenized.- Unmatched quotes (
"machine learning): Close all quotes. - Empty query: Provide at least one search term.
- Invalid boolean syntax (
AND machine): Operators need terms on both sides. - Unbalanced parentheses: Match all opening and closing parens.
- Unknown field name: Field names in the query must match text-searchable fields in the schema.
API errors
API errors
401 Unauthorized: Check theApi-Keyheader.400 Bad Request: Check JSON syntax and required fields. Examples:fieldsarray with more than one element fordense_vector/sparse_vector; missing mutually-exclusive field for Fetch/Delete.404 Not Found: Verify the index name and host URL.- Missing API version: Add
X-Pinecone-Api-Version: 2026-07.
Upsert errors
Upsert errors
- Type mismatch: Ensure values match declared schema types.
- Invalid
_id: Every document must have a non-empty_idstring. - Reserved names: Field names cannot start with
_(reserved for system-managed fields like_idand_score) or$(reserved for filter operators), and must be at most 64 bytes.
Slow search performance
Slow search performance
- Reduce query complexity: Boolean operators and large phrase slop are more expensive than simple term queries.
- Simplify filters: Filters are applied before scoring, so broad filters increase the search space.
- For cost-sensitive workloads, use
read_capacity.mode: "Dedicated"to get predictable latency.
Common request-shape pitfalls
Common request-shape pitfalls
When a request is rejected with a 4xx that doesn’t seem to match your intent, the cause is usually one of these:
-
Sparse-vector
score_byclauses usesparse_values, notvalues. Thevalueskey is fordense_vector. A sparse clause needs the full object:"sparse_values": { "indices": [...], "values": [...] }. -
Every
score_byclause must includetype. It’s the discriminator that selects the scoring method (text,query_string,dense_vector,sparse_vector). Omitting it returns a 400. -
Every document must have a non-empty
_idstring. There is no default; the upsert request fails if any document in the batch is missing_idor has an empty value. -
Wait for
status.ready: truebefore searching. A newly created index can briefly return empty results. ForDedicatedread capacity, also wait forread_capacity.status.state: "Ready". -
The match-score response field is
_score, notscore. A user metadata field namedscoreis allowed and is returned alongside the system-owned_score. -
Namespace is part of the URL path. Use
__default__(the literal string) if you don’t need partitioning. An empty path segment is rejected. -
dense_vectorqueries usevalues, notquery. Onlytextandquery_stringclauses usequery(a string).dense_vectorandsparse_vectorusevalues(a float array) andsparse_values(an{indices, values}object) respectively.
Requirements and limitations
Everything in this section applies to indexes with document schemas. For dense, sparse, and integrated-inference indexes, see Search overview.Requirements
- Entry points: Use the REST API, the Python SDK (v10 or later), or the Pinecone console. REST requests must send
X-Pinecone-Api-Version: 2026-07, and the SDK sends it for you. - Endpoint compatibility: Indexes with document schemas use the
/namespaces/{namespace}/documents/*endpoints. Dense, sparse, and integrated-inference indexes continue to use/vectors/*, and/records/*for integrated inference. The two endpoint families are index-type-specific and don’t cross over. - Deployment: Only managed (serverless) deployments are supported, with
read_capacity.modeset toOnDemandorDedicated.
Limits
- Document and schema limits: See Upsert limits for fields per schema, field and token size, document size, request size, and metadata size.
- Query limits: See Query limits for the
top_kand result-size caps. Full-text search is optimized for ranked retrieval rather than for aggregation or count-style queries. - Update limits: See Update limits for how many documents one update request can patch.
Supported operations
- Partial updates: Upsert replaces the whole document for a given
_id.POST /namespaces/{namespace}/documents/updatepatches individual fields, by ID or by metadata filter. See Update documents. - Bulk import from object storage, via JSONL files. See Prepare document-schema files (JSONL).
- Namespace operations: You can create, list, describe, and delete namespaces on an index with a document schema, and describing a namespace returns its
record_countandsize_bytes. describe_index_stats: In the REST response,totalVectorCountand each namespace’svectorCountcount documents, and both are accurate for every schema.dimensionandmetricdescribe the schema’sdense_vectorfield, andvectorTypeisdense. When a schema declares nodense_vectorfield,metricfalls back todotproduct, anddimensionis0on a text-only schema (vectorTypeistext) or is omitted when the schema declares asparse_vectorfield (vectorTypeissparse). For the full response schema, see Get index stats.
Not supported
- Schema changes after index creation aren’t yet supported, including backfilling a schema onto an index created before
2026-07. Create a new index with the schema you want and reindex your documents. - Backup and restore are not yet supported.
- Indexes can’t be created in projects with customer-managed encryption keys (CMEK) enabled.
- An index can’t be changed from dedicated read capacity back to on-demand. Create a new on-demand index and reingest your data.