# Stopwords

`Stopwords` -- are most frequently used words, i.e. words which appear in almost every document searched. Stopwords are filtered out prior to index construction, what is allow to reduce the total size of the index without any significant loss in quality of search.

## `StopwordFile` command

Load stop words from the given text file. You may specify either absolute file name or a name relative to DataparkSearch `/etc` directory. You may use several `StopwordFile` commands.

    StopwordFile stopwords/en.sl

# Clones

`Clones` -- are documents having equal values of Hash32 on all document sections. Indentical copies of the same document always have equal values of Hash32. This allow to eliminate duplicate documents in a collection. However, if only `title` section is defined in `sections.conf`, all documents with different bodies but with identical titles will be considered as clones.

## `DetectClones` command

    DetectClones yes/no

Allow/disallow clone detection and eliminating. If allowed, indexer will detect the same documents under different location, such as mirrors, and will index only one document from the group of such equal documents. "DetectClones yes" also allows to reduce space usage. Default value is "yes".

    DetectClones no