bb-11.4-vec-vicentiu-hugo : lp:maria : Git : Code : MariaDB

Get this branch:: git clone -b bb-11.4-vec-vicentiu-hugo https://git.launchpad.net/maria

Branch merges

Branch information

Name:: bb-11.4-vec-vicentiu-hugo

Repository:: lp:maria

Recent commits

42227a7... by Vicențiu Ciorbaru on 2024-04-29

Baseline results

0: MariaDB(m=16, ef_construction=200, ef_search=10) 0.996 772.343

7829259... by Vicențiu Ciorbaru on 2024-04-18

Bug fixes - on top of Hugo's patch

ee2cc47... by Vicențiu Ciorbaru on 2024-04-15

Support files for ann-workspace

437e214... by Hugo Wen <email address hidden> on 2024-04-12

MDEV-33408 Alter HNSW graph storage and fix memory leak

This commit changes the way HNSW graph information is stored in the
second table. Instead of storing connections as separate records, it now
stores neighbors for each node, leading to significant performance
improvements and storage savings.

Comparing with the previous approach, the insert speed is 5 times faster,
search speed improves by 23%, and storage usage is reduced by 73%, based
on ann-benchmark tests with random-xs-20-euclidean and
random-s-100-euclidean datasets.

Additionally, in previous code, vector objects were not released after
use, resulting in excessive memory consumption (over 20GB for building
the index with 90,000 records), preventing tests with large datasets.
Now ensure that vectors are released appropriately during the insert and
search functions. Note there are still some vectors that need to be
cleaned up after search query completion. Needs to be addressed in a
future commit.

All new code of the whole pull request, including one or several files
that are either new files or modified ones, are contributed under the
BSD-new license. I am contributing on behalf of my employer Amazon Web
Services, Inc.

3d0e4ea... by Vicențiu Ciorbaru on 2024-02-23