Rockset today added a hybrid search capability that promises to make it simpler to query data regardless of whether it resides in a vector database or in a large language model (LLM).
Most use cases involving LLMs first search data residing within that platform before reaching out to a database that is being used to provide access to additional data. The challenge that creates is the more relevant vector, text, geospatial and structured data often resides in the vector database, so crawling the LLM first can adversely impact query performance, noted Rockset CEO Venkat Venkataramani.
In effect, Rockset is now making it simpler to query data in parallel based on its relevancy rather than physical location, he adds. Rockset is also making use of a revamped design that uses compressed bitmaps and covering indexes to improve search performance.
Ultimately, any use case involving LLMs in the enterprise will require a hybrid search capability; LLMs that trained using data up to a specific data. In many cases, the LLM being used may not have been exposed to any data that is relevant to a specific use case. “They just don’t know anything,” says Venkataramani. “You need a data retrieval system to unlock the value.”
Organizations building AI applications will constantly need to incorporate new signals, models, indexes and ranking algorithms to improve relevance, he adds.
The Rockset approach also provides the ability to support both explicit and implicit queries, such as searches that are automatically generated any time an end user accesses an e-commerce site that lists a specific type of product, notes Venkataramani.
The speed at which new data can be correlated alongside older data in a LLM will prove critical as the need to surface the latest data available in near real-time becomes more critical. Historically, most enterprise applications have been updated in batches once a day or sometimes even fewer, resulting in search results that don’t always reflect the most current, for example, inventory. In the age of AI, the expectation is that LLMs will be surfacing insights on the latest data available, all of which needs to flow into some external platform capable of processing in near real time.
As organizations move to operationalize AI, they are all encountering the same data management challenges. The quality of the output on any LLM is directly tied to the quality of the data it has been exposed to both before and after it is deployed. The better the quality of the data provided, the more likely it becomes that the LLM will make a better recommendation versus a generative a response to a prompt that is either incomplete, or simply makes no sense whatsoever.
Of course, no matter how much data an LLM is exposed to, the output provided is always going to be probabilistic rather than deterministic. In fact, an LLM may not always provide the same answer in the exact same manner. As such, there is always going to be a need to make sure that the output being generated is accurate before organizations decide to act on any recommendations provided.
