Zilliz announces major contributions to Milvus 2.1, the leading open source vector database for structured and unstructured data

Zilliz announces major contributions to Milvus 2.1, the leading open source vector database for structured and unstructured data

SAN FRANCISCO–(BUSINESS WIRE)–Zilliz, whose founders created the Milvus open source project, today announced major contributions to the Milvus 2.1 release. The additional functionality bridges the gap between data pools, eliminates data silos, and provides performance and availability improvements to address top developer concerns. Milvus is one of the most advanced vector databases in the world, capable of managing massive amounts of both structured and unstructured data to accelerate next-generation data structure development.

Milvus is a phased open source project under the LF AI & Data Foundation and is built for scalable similarity search and deployed by a wide range of companies across all industries. It features a distributed architecture and scales easily as data volumes and workloads grow. Highly scalable, reliable and exceptionally fast, Milvus supports add-delete-update (DML) operations and near real-time search of trillions of bytes of vectors.

With this 2.1 update, Milvus sees a significant improvement in its performance, reducing search latency to five milliseconds for millions of records while further simplifying deployment and operations workflow.

Close gaps and improve performance

Machine learning produces huge pools of scalar and vector data every day. With the introduction of more scalar data types, Milvus 2.1 bridges this critical gap between data pools.

“Data silos can now be better integrated and linked, enabling companies to realize the full potential of their data,” said Milvus project manager Xiaofan James Luan, who also serves as director of engineering at Zilliz. “When it comes to unstructured data, the solutions offered by incumbent industry vendors are typically add-on features or tools in a legacy database management system, while Milvus is designed for unstructured data from day one and now offers more built-in features to unlock more powerful ones and integrated data processing.”

Zilliz’s contributions to version 2.1 include:

  • A general performance increase including reduced latency; greatly improved throughput for small NQ application scenarios, such as reverse image search and intelligent chatbot; Support for multiple storage replicas for small tables to increase throughput; and 2X increase in search performance.
  • Improved scalar data processing which adds varchar to supported data types and supports creating indexes on scalar data, bringing hybrid search to a more intuitive level.
  • Production level upgrades and increased availability, with clearer monitoring metrics for observability, simpler and more diverse deployment options including embedded Milvus for easy deployment and Ansible for offline deployment, integration supporting Kafka as log storage, and improved security supporting password protection and TLS connection.
  • A developer-friendly ecosystem in development, which includes more tutorials on building real-world applications and connects Milvus to the open-source ETL framework for vector data Towhee; and that adds Feder, an open-source tool that helps Milvus users choose the most appropriate index for their usage scenario by visualizing the vector similarity search process.

In addition to the integration and security features enumerated, Milvus will provide other features essential to modern security mechanisms, including ACL (Access Control Lists) and advanced encryption methods.

Commitment to open source ecosystems

“As a data infrastructure for unstructured data, Milvus is revolutionary because it handles vector embeddings and not just strings. Going forward, Zilliz, the company founded by the developers of Milvus, aims to build an ecosystem of solutions around Milvus, and some of the projects that will contribute to this have already surfaced, including Towhee, our open-source ETL framework for vector data , and Feder, an interactive visualization tool for unstructured data. With Milvus 2.1 and the new demos, users can see how these products can come together to solve a range of problems affecting unstructured data,” added Luan.

Zilliz is committed to the developer community and will continue to contribute to open source projects like Milvus. The company’s technology has a wide range of applications that includes drug discovery, computer vision, recommender engines, chatbots and more.

About Zilliz

Zilliz is a leading vector database company for production-ready AI. Built by the engineers who created Milvus, the world’s most popular open-source vector database, Zilliz is on a mission to unleash data insights with AI. The company develops next-generation database technologies to help companies rapidly build AI/ML applications and unlock the potential of unstructured data. By relieving its users of the burden of complex data infrastructure management, Zilliz aims to bring the power of AI to every business, organization and individual.

Headquartered in San Francisco, Zilliz is backed by a number of reputable investors including Hillhouse Capital, Aramco’s Prosperity7 Ventures, Temasek’s Pavilion Capital, 5Y Capital, Yunqi Partners, Trustbridge Partners and others. Zilliz technologies and products help over 1000 organizations worldwide to easily build AI applications in various scenarios including computer vision, image retrieval, video analytics, NLP, recommendation engines, targeted advertising, customized search, intelligent chatbots, fraud detection, network security, new discovery medicines, and much more. Find out more at zilliz.com or follow us @zilliz_universe.

Leave a Comment

Your email address will not be published.