### Home Classes Methods

### In Files

- ext/r_analysis.c

### Parent

Ferret::Analysis::TokenStream

### Methods

- ::new

### Files

- MIT-LICENSE
- README
- TODO
- TUTORIAL

### Class/Module Index

Quicksearch

- Ferret
- Ferret::Analysis
- Ferret::Analysis::Analyzer
- Ferret::Analysis::AsciiLetterAnalyzer
- Ferret::Analysis::AsciiLetterTokenizer
- Ferret::Analysis::AsciiLowerCaseFilter
- Ferret::Analysis::AsciiStandardAnalyzer
- Ferret::Analysis::AsciiStandardTokenizer
- Ferret::Analysis::AsciiWhiteSpaceAnalyzer
- Ferret::Analysis::AsciiWhiteSpaceTokenizer
- Ferret::Analysis::HyphenFilter
- Ferret::Analysis::LetterAnalyzer
- Ferret::Analysis::LetterTokenizer
- Ferret::Analysis::LowerCaseFilter
- Ferret::Analysis::MappingFilter
- Ferret::Analysis::PerFieldAnalyzer
- Ferret::Analysis::RegExpAnalyzer
- Ferret::Analysis::RegExpTokenizer
- Ferret::Analysis::StandardAnalyzer
- Ferret::Analysis::StandardTokenizer
- Ferret::Analysis::StemFilter
- Ferret::Analysis::StopFilter
- Ferret::Analysis::Token
- Ferret::Analysis::TokenStream
- Ferret::Analysis::WhiteSpaceAnalyzer
- Ferret::Analysis::WhiteSpaceTokenizer
- Ferret::BlankSlate
- Ferret::BoostMixin
- Ferret::Browser
- Ferret::Browser::Controller
- Ferret::Browser::Delegator
- Ferret::Browser::DocumentController
- Ferret::Browser::ErrorController
- Ferret::Browser::HelpController
- Ferret::Browser::HomeController
- Ferret::Browser::TermController
- Ferret::Browser::TermVectorController
- Ferret::Browser::ViewHelper
- Ferret::Document
- Ferret::Field
- Ferret::FieldSymbol
- Ferret::FieldSymbolMethods
- Ferret::FileNotFoundError
- Ferret::Index
- Ferret::Index::FieldInfo
- Ferret::Index::FieldInfos
- Ferret::Index::Index
- Ferret::Index::IndexReader
- Ferret::Index::IndexWriter
- Ferret::Index::LazyDoc
- Ferret::Index::LazyDoc::LazyDocData
- Ferret::Index::TermDocEnum
- Ferret::Index::TermEnum
- Ferret::Index::TermVector
- Ferret::Index::TermVector::TVOffsets
- Ferret::Index::TermVector::TVTerm
- Ferret::ParseError
- Ferret::QueryParser
- Ferret::QueryParser::QueryParseException
- Ferret::Search
- Ferret::Search::BooleanQuery
- Ferret::Search::BooleanQuery::BooleanClause
- Ferret::Search::ConstantScoreQuery
- Ferret::Search::Explanation
- Ferret::Search::Filter
- Ferret::Search::FilteredQuery
- Ferret::Search::FuzzyQuery
- Ferret::Search::Hit
- Ferret::Search::MatchAllQuery
- Ferret::Search::MultiSearcher
- Ferret::Search::MultiTermQuery
- Ferret::Search::PhraseQuery
- Ferret::Search::PrefixQuery
- Ferret::Search::Query
- Ferret::Search::QueryFilter
- Ferret::Search::RangeFilter
- Ferret::Search::RangeQuery
- Ferret::Search::Searcher
- Ferret::Search::Sort
- Ferret::Search::SortField
- Ferret::Search::Spans
- Ferret::Search::Spans::SpanFirstQuery
- Ferret::Search::Spans::SpanMultiTermQuery
- Ferret::Search::Spans::SpanNearQuery
- Ferret::Search::Spans::SpanNotQuery
- Ferret::Search::Spans::SpanOrQuery
- Ferret::Search::Spans::SpanPrefixQuery
- Ferret::Search::Spans::SpanTermQuery
- Ferret::Search::TermQuery
- Ferret::Search::TopDocs
- Ferret::Search::TypedRangeFilter
- Ferret::Search::TypedRangeQuery
- Ferret::Search::WildcardQuery
- Ferret::StateError
- Ferret::Store
- Ferret::Store::Directory
- Ferret::Store::FSDirectory
- Ferret::Store::Lock
- Ferret::Store::Lock::LockError
- Ferret::Store::RAMDirectory
- Ferret::Utils
- Ferret::Utils::BitVector
- Ferret::Utils::MultiMapper
- Ferret::Utils::PriorityQueue
- WEBrick
- WEBrick::FerretBrowserHandler
- Date
- DateTime
- FileUtils
- Float
- Integer
- String
- Symbol
- Time

No matching classes.

# Ferret::Analysis::MappingFilter

## Summary

A MappingFilter maps strings in tokens. This is usually used to map UTF-8 characters to ASCII characters for easier searching and better search recall. The mapping is compiled into a Deterministic Finite Automata so it is super fast. This Filter can therefor be used for indexing very large datasets. Currently regular expressions are not supported. If you are really interested in the feature, please contact me at dbalmain@gmail.com.

## Example

    mapping = {
      ['à','á','â','ã','ä','å','ā','ă']         => 'a',
      'æ'                                       => 'ae',
      ['ď','đ']                                 => 'd',
      ['ç','ć','č','ĉ','ċ']                     => 'c',
      ['è','é','ê','ë','ē','ę','ě','ĕ','ė',]    => 'e',
      ['ƒ']                                     => 'f',
      ['ĝ','ğ','ġ','ģ']                         => 'g',
      ['ĥ','ħ']                                 => 'h',
      ['ì','ì','í','î','ï','ī','ĩ','ĭ']         => 'i',
      ['į','ı','ĳ','ĵ']                         => 'j',
      ['ķ','ĸ']                                 => 'k',
      ['ł','ľ','ĺ','ļ','ŀ']                     => 'l',
      ['ñ','ń','ň','ņ','ŉ','ŋ']                 => 'n',
      ['ò','ó','ô','õ','ö','ø','ō','ő','ŏ','ŏ'] => 'o',
      ['œ']                                     => 'oek',
      ['ą']                                     => 'q',
      ['ŕ','ř','ŗ']                             => 'r',
      ['ś','š','ş','ŝ','ș']                     => 's',
      ['ť','ţ','ŧ','ț']                         => 't',
      ['ù','ú','û','ü','ū','ů','ű','ŭ','ũ','ų'] => 'u',
      ['ŵ']                                     => 'w',
      ['ý','ÿ','ŷ']                             => 'y',
      ['ž','ż','ź']                             => 'z'
    }
    filt = MappingFilter.new(token_stream, mapping)

### Public Class Methods

new(token_stream, mapping) → token_stream click to toggle source

Create an MappingFilter which maps strings in tokens. This is usually used to map UTF-8 characters to ASCII characters for easier searching and better search recall. The mapping is compiled into a Deterministic Finite Automata so it is super fast. This Filter can therefor be used for indexing very large datasets. Currently regular expressions are not supported. If you are really interested in the feature, please contact me at dbalmain@gmail.com.

token_stream | TokenStream to be filtered
mapping | Hash of mappings to apply to tokens. The key can be a String or an Array of Strings. The value must be a String

## Example

    filt = MappingFilter.new(token_stream,
                             {
                               ['à','á','â','ã','ä','å'] => 'a',
                               ['è','é','ê','ë','ē','ę'] => 'e'
                             })

    static VALUE
    frb_mapping_filter_init(VALUE self, VALUE rsub_ts, VALUE mapping)
    {
        TokenStream *ts;
        ts = frb_get_cwrapped_rts(rsub_ts);
        ts = mapping_filter_new(ts);
        rb_hash_foreach(mapping, frb_add_mappings_i, (VALUE)ts);
        mulmap_compile(((MappingFilter *)ts)->mapper);
        object_add(&(TkFilt(ts)->sub_ts), rsub_ts);

        Frt_Wrap_Struct(self, &frb_tf_mark, &frb_tf_free, ts);
        object_add(ts, self);
        return self;
    }

[Validate]

Generated with the Darkfish Rdoc Generator 2.