RWPMAPBUILD(1)

NAME
rwpmapbuild - Create a binary prefix map from a text file

SYNOPSIS
  rwpmapbuild [--input-file=FILENAME] [--output-file=FILENAME]
        [--note-add=TEXT] [--note-file-add=FILENAME]

  rwpmapbuild --help

  rwpmapbuild --version

DESCRIPTION
Prefix maps provide a way to map field values (specifically either IPv4 addresses or protocol-port pairs) to string labels based on a user-defined map file. Using the prefix map plug-in, pmapfilter(3), you can you can partition, count, sort and display records based on arbitrary labels. rwpmapbuild reads textual input to create a binary prefix map file that the plug-in can use; the syntax of this input is described below.

The textual input is read from the specified input file, or from the standard input when the --input-file switch is not provided. The binary output is written to the named output file, or to the standard output when the --output-file switch is not provided and the standard output is not connected to a terminal.

OPTIONS
Option names may be abbreviated if the abbreviation is unique or is an exact match for an option. A parameter to an option may be specified as --arg=param or --arg param, though the first form is required for options that take optional parameters.

--input-file=FILENAME
    Read the textual input from FILENAME. You may use "stdin" or "-" to represent the standard input. When this switch is not provided, the input is read from the standard input. The input file format is described below.

--output-file=FILENAME
    Write the binary prefix map to FILENAME. You may use "stdout" or "-" to represent the standard output. When this switch is not provided, the prefix map is written to the standard output unless stdout is connected to a terminal.

--note-add=TEXT
    Add the specified TEXT to the header of the output file as an annotation. This switch may be repeated to add multiple annotations to a file. To view the annotations, use the rwfileinfo(1) tool.

--note-file-add=FILENAME
    Open FILENAME and add the contents of that file to the header of the output file as an annotation. This switch may be repeated to add multiple annotations. Currently the application makes no effort to ensure that FILENAME contains text; be careful that you do not attempt to add a SiLK data file as an annotation.

--help
    Print the available options and exit.

--version
    Print the version number and information about how SiLK was configured, then exit the application.

INPUT FILE FORMAT
The input file format consists of any number of input lines of the forms described below. Note that there is not a form that accepts a single IP address and a label; therefore, to provide a label for a single IP address you must append "/32" to the IPv4 address.

Blank lines in the input file are ignored, as are comments. Comments begin with the first "#" character on a line and extend to the end of the line.

In the following, the label-value represents either a numerical label identifier that was created with the label command or label-text.

label-text is a textual string that begins at the first non-whitespace character and extends to the final non-whitespace character on that line that does not appear in a comment. The label-text may include embedded whitespace and non-alphanumeric characters. When the parser reads label-text, it checks to see if it has seen the text before. If the label-text exists, the numeric identifier for that text is used. If the label-text has not been seen before, a new label identifier is created; if a label statement occurring later in the file attempts to use this label-text or automatically generated identifier, an error message is printed.

The following statements are supported:

map-name simple-string
    Creates a name for the data in this prefix map file. The simple-string cannot contain whitespace, a comma, or a colon. When the prefix map file is used by rwfilter(1), the simple-string is used to generate the filtering switch names. When the prefix map file is used by rwcut(1), rwgroup(1), rwsort(1), rwstats(1), or rwuniq(1), the simple-string is used to generate the field names. See pmapfilter(3) for details.

default label-value
    Make the given label identifier or label text the default value for any ranges not explicitly mentioned in this input file. You may not provide a numeric label that has not yet been defined.

label num label-text
    Associate the numeric identifier with the given label text. Note that any label text used in the remainder of the file will be automatically assigned a number. It is an error to give the same label text to multiple names, or the same text to multiple label identifiers.

mode { ip | proto-port }
    Process this file in IPv4 address or protocol/port mode. By default, processing is done in IPv4 address mode. (IPv6 addresses are not yet supported).

  IPv4 Address Mode

cidr-block label-value
    Associate the given label identifier or label text with this CIDR block. The CIDR block is composed of an IPv4 address in dotted-decimal notation, a slash "/", and the number of significant bits.

low-ip high-ip label-value
low-int high-int label-value
    Associate the given label identifier or label text with this IPv4 range. Both IPs must be specified in the same format: as either dotted-decimal IPv4 addresses or as integers.

  Protocol/Port Mode

proto/port proto/port label-value
    Associate the given label identifier or label text with all protocols and port numbers between these two values inclusive. Note that while port is not meaningful for all protocols (specifically, it is meaningful for TCP and UDP and may contain type/code information for ICMP), this file allows port numbers to be given for any protocol.

proto proto label-value
    Associate the given label identifier or label text with all protcols between these two values.

NOTES
The IP Adddress input file can contain nested CIDR blocks. They should be ordered with the more general blocks first, and the more specific blocks last. That is, use:

  10.0.0.0/8     My-space
  10.1.0.0/16    Special-Subnet-1
  10.1.2.0/24    Special-Subnet-2

Likewise, the protocol/port data can be nested:

  6 6            TCP
  6/0  6/1024    TCP/Generic reserved
  6/22 6/22      TCP/SSH
  6/25 6/25      TCP/SMTP
  6/80 6/80      TCP/HTTP

EXAMPLE
Reading and writing to a file:

  $ echo "10.1.2.3/32 my favorite host" > fav.txt
  $ rwpmapbuild -i fav.txt -o fav.pmap

Reading and writing to stdin and stdout:

  $ echo "10.9.8.128/27 suspicious subnet" \
      | rwpmapbuild --input-file=stdin --output-file=stdout > suspicious.pmap

  Complex IP File
  #    Numerical mappings of labels

  label 0             non-routable
  label 1             internal
  label 2             external

  #    Default to "external" for all un-defined ranges.

  default             external

  #    Force IP-mode

  mode                ip

  #    Create a name
  #
  #        This will add --pmap-src-network and --pmap-dst-network
  #        switches to rwfilter, and src-network and dst-network
  #        fields to rwcut, rwgroup, rwsort, rwstats, and rwuniq

  map-name            network

  ## Reserved and non-routable blocks ###########################

  #    Addresses in this block refer to source hosts on "this"
  #    network.  Address 0.0.0.0/32 may be used as a source
  #    address for this host on this network; other addresses
  #    within 0.0.0.0/8 may be used to refer to specified hosts
  #    on this network [RFC1700, page 4].

  0.0.0.0/8           non-routable

  #    This block is set aside for use in private networks.  Its
  #    intended use is documented in [RFC1918].  Addresses within
  #    this block should not appear on the public Internet.

  10.0.0.0/8          non-routable

  #    This block is assigned for use as the Internet host
  #    loopback address.  A datagram sent by a higher level
  #    protocol to an address anywhere within this block should
  #    loop back inside the host.  This is ordinarily
  #    implemented using only 127.0.0.1/32 for loopback, but no
  #    addresses within this block should ever appear on any
  #    network anywhere [RFC1700, page 5].

  127.0.0.0/8         non-routable

  #    This is the "link local" block.  It is allocated for
  #    communication between hosts on a single link.  Hosts
  #    obtain these addresses by auto-configuration, such as when
  #    a DHCP server may not be found.

  169.254.0.0/16      non-routable

  #    This block is set aside for use in private networks.  Its
  #    intended use is documented in [RFC1918].  Addresses within
  #    this block should not appear on the public Internet.

  172.16.0.0/12       non-routable

  #    This block is assigned as "TEST-NET" for use in
  #    documentation and example code.  It is often used in
  #    conjunction with domain names example.com or example.net
  #    in vendor and protocol documentation.  Addresses within
  #    this block should not appear on the public Internet.

  192.0.2.0/24        non-routable

  #    This block is set aside for use in private networks.
  #    Its intended use is documented in [RFC1918].  Addresses
  #    within this block should not appear on the public Internet.

  192.168.0.0/16      non-routable

  #    240.0.0.0/4 - This block, formerly known as the Class E
  #    address space, is reserved.  The "limited broadcast"
  #    destination address 255.255.255.255 should never be
  #    forwarded outside the (sub-)net of the source.  The
  #    remainder of this space is reserved for future use.
  #    [RFC1700, page 4]

  255.255.255.255/32  non-routable

  # -- Below this line, would add any mappings appropriate to
  # -- the local network.

  Complex Protocol/Port File
  #    Default to a hyphen ("-") for all un-defined ranges.

  default             -

  #    Force Protocol/Port-mode
  #
  #        This MUST be present, since IP mode is the default.

  mode                proto-port

  #    Protocol Overview

   1  1               ICMP
   6  6               TCP
  17 17               UDP
  50 50               ESP
  58 58               ICMPv6

  #    TCP -- Specific Ports

  6/0 6/1024          TCP/Generic Reserved
  6/21 6/21           TCP/ftp
  6/22 6/22           TCP/ssh
  6/25 6/25           TCP/smtp
  6/53 6/53           TCP/dns
  6/80 6/80           TCP/http
  6/6000 6/6063       TCP/X11

  #    UDP -- Specific Ports

  17/53 17/53         UDP/dns

  #    ICMP -- Specific Type/Code
  #
  #    To convert a type/code to a "port" value as stored in SiLK:
  #        (type << 8) | code     OR    (type * 256) + code
  #    so 3/3 (Destination Unreachable/Port Unreachable) becomes:

  1/771 1/771         ICMP/Destination Unreachable/Port Unreachable

SEE ALSO
pmapfilter(3), rwfilter(1), rwpmapcat(1). rwcut(1), rwgroup(1), rwsort(1), rwstats(1), rwuniq(1)

BUGS
Prefix map files do not support IPv6 addresses. Any IPv6 addresses in the input will be ignored.