Couchbase Server 7.6.1-3200 (EE)
Bucket = ‘chatui’, Collection = ‘default’
follow this link Run a Vector Search with a Couchbase SDK | Couchbase Docs
I have tried doing a vector search:
def get_retriever(self, query: str, k: int = 4) -> VectorStoreRetriever:
# Generate the query vector using the embeddings model
query_vector = self.embeddings.embed_query(query)
try:
# Construct the vector search request
search_req = search.SearchRequest.create(search.MatchNoneQuery()).with_vector_search(
VectorSearch.from_vector_query(VectorQuery('vectors', query_vector, num_candidates=k))
)
#scope = self.bucket.default_scope()
query_index_manager = self.cluster.query_indexes()
indexes = query_index_manager.get_all_indexes('chatui')
index_names = {index.name for index in indexes} # Set of all index names for quick lookup
for name in index_names:
print(name)
# Execute the vector search request
result = self.cluster.search("vectors_index", search_req, SearchOptions(limit=k, fields=["text", "metadata"]))
# Extract the documents and vectors from the search result
documents = []
vectors = []
for row in result.rows():
document_data = row.fields
if "text" in document_data and "metadata" in document_data:
for text_chunk in document_data["text"]:
documents.append(Document(page_content=text_chunk, metadata=document_data["metadata"]))
vectors.extend(document_data["vectors"])
if not documents:
raise ValueError("No relevant documents found")
# Create a vector store using the VectorFactory
vectorstore, _, _ = VectorFactory.create_vector_storage(vector_type="faiss", embedding_size=len(vectors[0]))
# Add the extracted vectors to the vector store's index
vectorstore.add_texts([doc.page_content for doc in documents], [doc.metadata for doc in documents])
# Create a VectorStoreRetriever with the documents and vector store
retriever = VectorStoreRetriever(documents=documents, vectorstore=vectorstore.vectorstore)
return retriever
except CouchbaseException as ex:
import traceback
traceback.print_exc()
raise ex
the structure document looks like that
{
“metadata”: {
“source”: “Mitologia Celta”,
“file_id”: “662f75f5596c6e6c64527484”,
“user_id”: “user123”,
“file_size”: 1201455
},
“text”: [“Reina de las hadas irlandesas. Hija de Manannan. Se la confunde a veces con Morrigan, a veces con Dana.\n\n\n\n\n\n\n\n\nLUNED. Es una hada dotada de poderes sobrenaturales. Imagen de una antigua divinidad de amor y magia.\n\n\n\nGIGANTES\n===\n\nLos gigantes aparecen en toda la tradición Celta. Aunque hay muchos, parece ser que existió un grupo muy importante que fueron los Fomore. Este era un pueblo misterioso de la tradición irlandesa. Los Fomore no invadieron Irlanda, pero la amenzaron constantemente. Son gigantes que vivían en las islas que rodean a Irlanda. Estos son algunos de los gigantes más famosos.”],
“text_length”: 2,
“vectors”: [
[-0.006697558332234621, -0.014219328761100769, -0.018976973369717598, -0.023895440623164177, -0.021067658439278603, 0.0023051127791404724, -0.024069664999842644, 0.013053370639681816, -0.011391544714570045, -0.013636349700391293, 0.009656010195612907, 0.03704262524843216, 0.008476649411022663, 0.0016727143665775657, -0.010031260550022125,
and there is a index for the field “vectors” called “vector_index”
CREATE INDEX vector_index ON chatui(vectors)
we can check that exits
[
{
“code”: 4300,
“msg”: “The index vector_index already exists.”,
“reason”: {
“name”: “vector_index”
}
}
]
Ok, but there is no way, it doesn’t works, all time and exception raises, even the index stay here
File “C:\Projects Python\evo-brains-backend-v2\core\vectorstorage\couchbase.py”, line 294, in get_retriever
raise ex
File “C:\Projects Python\evo-brains-backend-v2\core\vectorstorage\couchbase.py”, line 270, in get_retriever
for row in result.rows():
File “c:\Users\rgutierrez.mourente\AppData\Local\anaconda3\envs\evopy2\Lib\site-packages\couchbase\search.py”, line 136, in next
raise ex
File “c:\Users\rgutierrez.mourente\AppData\Local\anaconda3\envs\evopy2\Lib\site-packages\couchbase\search.py”, line 130, in next
return self._get_next_row()
^^^^^^^^^^^^^^^^^^^^
File “c:\Users\rgutierrez.mourente\AppData\Local\anaconda3\envs\evopy2\Lib\site-packages\couchbase\search.py”, line 121, in _get_next_row
raise ErrorMapper.build_exception(row)
couchbase.exceptions.QueryIndexNotFoundException: QueryIndexNotFoundException(<ec=17, category=couchbase.common, message=index_not_found (17), context=SearchErrorContext({‘last_dispatched_to’: ‘127.0.0.1:8094’, ‘last_dispatched_from’: ‘127.0.0.1:57172’, ‘retry_attempts’: 0, ‘client_context_id’: ‘b5c992-1ece-cb4a-9289-351455534f79d4’, ‘method’: ‘POST’, ‘path’: ‘/api/index/vectors_index/query’, ‘http_status’: 400, ‘http_body’: '{“error”:“rest_auth: preparePerms, err: index not found”,“request”:{“ctl”:{“timeout”:75000},“explain”:false,“fields”:[“text”,“metadata”],“knn”:[{“field”:“vectors”,“k”:2,“vector”:[0.0058336150482652645,-0.004799766902900294,0.0…
I will be very grateful if someone can help me
