# Indexing binaries output (exec: and cgi: virtual URL schemes)

DataparkSearch supports exec: and cgi: virtual URL schemes. They allows running an external program. This program must return a result to it's stdout. Result must be in HTTP standard, i.e. HTTP response header followed by document's content.

For example, when indexing both `cgi:/usr/local/bin/myprog` and `exec:/usr/local/bin/myprog`, indexer will execute the `/usr/local/bin/myprog` program.

## Passing parameters to cgi: virtual scheme

When executing a program given in cgi: virtual scheme, indexer emulates that program is running under HTTP server. It creates REQUEST_METHOD environment variable with "GET" value and QUERY_STRING variable according to HTTP standards. For example, if `cgi:/usr/local/apache/cgi-bin/test-cgi?a=b&d=e` is being indexed, indexer creates QUERY_STRING with `a=b&d=e` value. cgi: virtual URL scheme allows indexing your site without having to invoke web servers even if you want to index CGI scripts. For example, you have a web site with static documents under `/usr/local/apache/htdocs/` and with CGI scripts under `/usr/local/apache/cgi-bin/`. Use the following configuration:

    Server http://localhost/
    Alias  http://localhost/cgi-bin/	cgi:/usr/local/apache/cgi-bin/
    Alias  http://localhost/		file:/usr/local/apache/htdocs/

## Passing parameters to exec: virtual scheme

indexer does not create QUERY_STRING variable like in cgi: scheme. It creates a command line with argument given in URL after ? sign. For example, when indexing `exec:/usr/local/bin/myprog?a=b&d=e`, this command will be executed:

    /usr/local/bin/myprog "a=b&d=e"

## Using exec: virtual scheme as an external retrieval system

exec: virtual scheme allow using it as an external retrieval system. It allows using protocols which are not supported natively by DataparkSearch. For example, you can use curl program which is available from http://curl.haxx.se/ to index HTTPS sites.

Put this short script to `/usr/local/dpsearch/bin/` under `curl.sh` name.

    #!/bin/sh
    /usr/local/bin/curl -i $1 2>/dev/null

This script takes an URL given in command line argument and executes curl program to download it. -i argument says curl to output result together with HTTP headers.

Now use these commands in your `indexer.conf`:

    Server https://some.https.site/
    Alias  https://  exec:/usr/local/dpsearch/etc/curl.sh?https://

When indexing `https://some.https.site/path/to/page.html`, indexer will translate this URL to

    exec:/usr/local/dpsearch/etc/curl.sh?https://some.https.site/path/to/page.html

execute the `curl.sh` script:

    /usr/local/dpsearch/etc/curl.sh "https://some.https.site/path/to/page.html"

and take it's output.