Amazon OpenSearch Service

Visual search enables users to perform search queries using rich multimedia content, such as images. Its implementation is similar to semantic search: vector embeddings are created from images in the knowledge base, and the OpenSearch service is queried with a vector (encoding of the search query). The distinction from semantic search lies in the use of an external embedding model (for example, a Convolutional Neural Network (CNN) hosted on Amazon SageMaker or an Amazon Titan model) to convert images into vectors.

Amazon OpenSearch has introduced multimedia support for Neural Search, reducing the need for integration with external embedding models and favoring the use of multimodal APIs for text and images provided by Amazon Bedrock. With multimodal support, queries can be made on OpenSearch using images, text, or both. This functionality allows searching for images by describing their visual characteristics, discovering other similar images using a visual knowledge base, or combining hybrid text and image search to find matches from both semantic and visual perspectives.

Retrieval Augmented Generation (RAG) is a method of generative AI that leverages Large Language Models (LLMs) to develop conversational experiences for users. However, generative language models, when used in isolation, may lead to hallucinations, where the model generates credible yet factually incorrect responses. The vector database service provided by Amazon OpenSearch emerges as the ideal solution to address this challenge, acting as an external knowledge base. It provides LLMs with a knowledge source derived from transforming textual documents into embeddings, aiming to significantly enhance the accuracy and coherence of responses.