Skip Navigation
Search
Back to Search Innovations
Multi-tier Caching for High-throughput Indexing Source Code
A method for maintaining an index in multi-tier data structure. Invention utilizes key-value storage, and index caching using key-value pairs.
Please note, header image is purely illustrative. Source: Tumisu, pixabay, CC0.

Background

Index machines comprise of a cluster or surplus of databases and or file systems. The volume of index data being generated by network connected devices is outpacing data storage technologies’ speed, capacity and abilities. Applications that create index data and generate large volumes of index data. Data deduplication systems is technology made in efforts of carrying out the elimination of redundant data. Successful and efficient data deduplication relies on indexing to maintain performance levels and functionality.

Technology

Combining key‑value storage and indexing, multi‑tier data structures are able to maintain an index, and maintain performance for databases, database clusters, media tagging and more. An index in a multi‑tier data structure includes providing storage devices to form the structure, caching a list of key‑value pairs, stored on one or more tiers of the multi‑tier data structure. Sub‑lists utilize a caching method with keys, or pointers that reference a block, the data structures currently in use. Interfacing allows keys to manage and control access within individual tiers of these data structures, and point to data stored in the blocks. To manage resources within a multi‑tier storage system and methods for inserting and retrieving key‑value pairs in a machine exist and communicate with multiple storage devices managed in a hierarchy of tiers. Insert and retrieving key values from the machine, transfers sorted lists from memory onto the machine beginning with highest storage tier and then each remaining tier.

Advantages

Inserting the key‑value pairs in the machine includes transferring a first list of sorted key‑value pairs from a first memory onto the machine to a higher storage tier of the machine, using a merging method. Transferring a second list from the higher storage tiers to the lower storage tiers within the machine creates space in the higher storage tiers. Performing a merging method to redistribute resources within the multi‑tier storage system is aided by the merging method as it is automatically tuned for a workload without disabling the machine.

Application

- Computer Science - Computer Architecture - Data & Memory Storage

Inventors

Erez Zadok, Assistant Professor, Computer Science
Richard Spillane, Student, Computer Science

Licensing Potential

Development partner,Commercial partner,Licensing

Licensing Status

Available for License.

Licensing Contact

Donna Tumminello, Assistant Director, Intellectual Property Partners, donna.tumminello@stonybrook.edu, 6316324163

Patent Status

Patented

Tech Id

8274