QDBM(3)

NAME
QDBM - Quick DataBase Manager

DESCRIPTION

QDBM is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Each key must be unique within a database, so it is impossible to store two or more records with a key overlaps.

The following access methods are provided to the database: storing a record with a key and a value, deleting a record by a key, retrieving a record by a key. Moreover, traversal access to every key are provided, although the order is arbitrary. These access methods are similar to ones of DBM (or its compatibles, NDBM and GDBM) library defined in the UNIX standard. QDBM is an alternative for DBM because of its higher performance.

QDBM is developed referring to GDBM for the purpose of the following three points: higher processing speed, smaller size of a database file, and simpler API. They have been achieved. Moreover, the following three restrictions of traditional DBM: a process can handle only one database, the size of a key and a value is bounded, a database file is sparse, are cleared.

QDBM uses hash algorithm to retrieve records. If a bucket array has sufficient number of elements, the time complexity of retrieval is `O(1)'. That is, time required for retrieving a record is constant, regardless of the scale of a database. It is also the same about storing and deleting. Collision of hash values is managed by separate chains. Data structure of the chains is binary search tree. Even if a bucket array has unusually scarce elements, the time complexity of retrieval is `O(log n)'.

QDBM attains improvement in retrieval by loading RAM with the whole of a bucket array. If a bucket array is on RAM, it is possible to access a region of a target record by about one pass of file operations. A bucket array saved in a file is not read into RAM with the `read' call but directly mapped to RAM with the `mmap' call. Therefore, preparation time on connecting to a database is very short, and two or more processes can share the same memory map.

If the number of elements of a bucket array is about half of records stored within a database, although it depends on characteristic of the input, the probability of collision of hash values are about 56.7% (36.8% if the same, 21.3% if twice, 11.5% if four times, 6.0% if eight times). In such case, it is possible to retrieve a record by two or less passes of file operations. If it is made into a performance index, in order to handle a database containing one million of records, a bucket array with half a million of elements is needed. The size of each element is 4 bytes. That is, if 2M bytes of RAM is available, a database containing one million records can be handled.

QDBM provides two kinds of methods to connect to a database. A `reader' can perform retrieving but neither storing nor deleting. A `writer' can perform all access methods. Exclusion control between processes is performed when connecting to a database by file locking. While a writer is connecting to a database, no other readers and no other writers can connect. While a reader is connecting to a database, other readers can connect, but no other writers. According to this mechanism, adjustment of simultaneous connections to a database is guaranteed in multitasking environment.

Traditional DBM provides two modes of the storing operation, `insert' and `replace'. In a case a key overlaps an existing record, the insert mode keeps the existing value, while the replace mode transposes it to the specified value. In addition to the two modes, QDBM provides `concatenate' mode. In the mode, the specified value is concatenated at the end of the existing value and stored. This feature is useful when adding a element to a value as an array. Moreover, although DBM has a method to fetch out a value from a database only by reading the whole of a region of a record, QDBM has a method to fetch out a part of a region of a value. When a value is treated as an array, this feature is also useful.

If data alignment is assigned to a database, each record will place in a file with vacating suitable padding bytes. When it is going to overwrite a value with larger size than the size of an existing value, it is necessary to remove the region of the existing record to another position of the file. However, if the increasing size is settled in the size of the padding of the existing value, removing the region is not necessary. The time complexity of removing a region of a record depends on the size of the region. However, average performance of updating database is robust from size of records because the probability of removing are made low by enlargement of paddings according to size of records. Generally speaking, while succession of updating, fragmentation of available regions occurs, and the size of a database grows rapidly. QDBM deal with this problem by coalescence of dispensable regions and reuse of them.

QDBM provides four kinds of APIs: the basic API, the extended API, the NDBM-compatible API, and the GDBM-compatible API. In the basic API, a database is treated as a file. In the extended API, a database is treated as a directory containing one or more database files. If it is going to store all data of a database in one file, the file size may exceed the restriction of a file system. In the extended API, a database can be divided into one or more files in a directory. Because the basic API and the extended API are resembled mutually, it is easy to porting an application between each API. The NDBM-compatible API and GDBM-compatible API are for porting an application from NDBM and GDBM to QDBM. Besides, QDBM has command line interfaces corresponding to the four APIs. They are useful for unit tests, debugging applications and so on. They will also support prototyping of database applications with script languages.

BUILD

For building a program using QDBM, the program should be linked with a library file `libqdbm.a' or `libqdbm.so'. For example, the following command is executed to build `sample' from `sample.c'.

    gcc -I/usr/local/include -o sample sample.c -L/usr/local/lib -lqdbm

AUTHOR

QDBM is written by Mikio Hirabayashi and distributed as a free software. You can redistribute it and/or modify it under the terms of the GNU General Public License Version 2. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

SEE ALSO
depot(3), curia(3), relic(3), hovel(3), ndbm(3), gdbm(3)