\input texinfo  @c -*-texinfo-*-
@input texiplus

@setfilename gnatcoll.info
@include gnatcoll_include.texi
@set gnatcoll GNATColl
@set Title   @value{gnatcoll}: GNAT Reusable Components

@c --------------------------- macro
@c This macro can be used to insert code sample. Each line should end with
@c @NL{}, since otherwise texinfo doesn't guarantee that newlines will be
@c preserved
@macro CODESAMPLE{TXT}
@ifhtml
@smallexample
@group
\TXT\
@end group
@end smallexample
@end ifhtml
@ifnothtml
@cartouche
@example
@group
\TXT\
@end group
@end example
@end cartouche
@end ifnothtml
@end macro

@c simulates a newline when in a @CODESAMPLE
@macro NL{}
@end macro

@c --------------------------- macro
@c This macro can be used to insert a "Tip" or "Did you know" box. In HTML,
@c this box is displayed in the right margin of the text, and thus doesn't
@c disturbs the normal reading flow, but provides additional useful
@c information. A small icon is displayed on the left of the box.
@c Calls to this macro should be followed by @noindent for proper rendering
@c in pdf
@macro TIP{TXT}
@ifhtml
@html
<div class="tip">
@end html
\TXT\
@html
</div>
@end html
@end ifhtml
@ifnothtml
@quotation
@noindent
@image{tip,15pt}  \TXT\
@end quotation
@end ifnothtml
@end macro

@c -------------------------- macro
@c This macro can be used to insert an "Important" box in the text.
@c This box is displayed inline in the main body of the text, but is
@c surrounded by a frame, and has a small icon on the left.
@c Calls to this macro should be followed by @noindent for proper rendering
@c in pdf
@macro IMPORTANT{TXT}
@ifhtml
@html
<div class="important">
@end html
\TXT\
@html
</div>
@end html
@end ifhtml
@ifnothtml
@quotation
@noindent
@image{important,15pt}  \TXT\
@end quotation
@end ifnothtml
@end macro

@c ------------------------- macro
@c This macro can be used to insert a "Note" box in the text. It is
@c displayed inline in the main body of the text, but has a different
@c icon that the "Important" box
@c Calls to this macro should be followed by @noindent for proper rendering
@c in pdf
@macro NOTE{TXT}
@ifhtml
@html
<div class="note">
@end html
\TXT\
@html
</div>
@end html
@end ifhtml
@ifnothtml
@quotation
@noindent
@image{note,15pt}  \TXT\
@end quotation
@end ifnothtml
@end macro
@c ---------------------------

@settitle @value{Title}
@setchapternewpage odd
@syncodeindex fn cp
@syncodeindex vr cp

@titlepage
@flushleft
@title @value{Title}
@end flushleft
@subtitle Version @value{Version}
@author AdaCore
@page
@vskip 0pt plus 1filll

Copyright @copyright{} 2007-2010, AdaCore

This document may be copied, in whole or in part, in any form or by any
means, as is or with alterations, provided that (1) alterations are clearly
marked as alterations and (2) this copyright notice is included
unmodified in any copy.
@end titlepage

@ifnottex
@node Top,, (dir), (dir)
@top @value{Title}

Version @value{Version}

Copyright @copyright{} 2007-2010, AdaCore

This document may be copied, in whole or in part, in any form or by any
means, as is or with alterations, provided that (1) alterations are clearly
marked as alterations and (2) this copyright notice is included
unmodified in any copy.

@menu
* Introduction::
* Building the GNAT Reusable Components::
* Embedding script languages:: The scripting module.
* Logging information:: The traces module.
* Monitoring memory:: The memory module.
* Reading and Writing Files:: The mmap module.
* Searching strings:: The Boyer-Moore module.
* The templates module::
* Managing Email:: The email module.
* Ravenscar Patterns::
* Managing Memory:: The storage pools.
* Manipulating Files::
* Three state logic:: The tribooleans.
* Geometry::
* Projects::
* Database interface::
* Index::

@ifnothtml
@detailmenu
 --- The Detailed Node Listing ---

Introduction

Building the GNAT Reusable Components

Embedding script languages
* Supported languages::
* Scripts API::

Supported languages
* The Shell language::
* The Python language::
* Classes exported to all languages::

Scripts API
* Initializing the scripting module::
* Creating interactive consoles::
* Exporting classes and methods::
* Executing startup scripts::
* Debugging scripts::

Logging information
* Configuring traces::
* Using the traces module::
* Log decorators::
* Defining custom trace streams::
* Logging to syslog::
* Dynamically disabling features::

Monitoring memory

Reading and Writing Files

Searching strings

The templates module

Managing Email
* Message formats::
* Parsing messages::
* Parsing mailboxes::
* Creating messages::

Ravenscar Patterns
* Tasks::
* Servers::
* Timers::

Managing Memory

Manipulating Files
* Filesystems::
* Remote filesystems::
* Virtual files::
* GtkAda support for virtual files::

Three state logic

Geometry

Project

Database interface
* Supported database systems::
* Database schema monitoring::
* Writing queries::
* Executing queries::
* Getting results::
* Writing your own cursors::
* Creating your own SQL types::
* Query logs::
* Tasks and databases::

@end detailmenu
@end ifnothtml
@end menu
@end ifnottex

@iftex
@contents
@end iftex
@ifhtml
@contents
@end ifhtml

@c -----------------------------------------------------------------------
@node Introduction
@chapter Introduction
@c -----------------------------------------------------------------------

@noindent
The @value{gnatcoll} library provides a number of modules that can be reused in
your own applications to add extra features or help implementation.

The modules that are currently provided are:
@table @asis
@item Script languages
This module allows you to embed one or more scripting languages in your
application, thus providing extensibility to users (@pxref{Embedding script
languages})

@end table

@c -----------------------------------------------------------------------
@node Building the GNAT Reusable Components
@chapter Building the GNAT Reusable Components
@c -----------------------------------------------------------------------

@noindent
The compilation process tries to be as flexible as possible. You can choose
what modules to build, what features they should have,@dots{} This
flexibility comes at the cost of a certain complexity in the build
architecture, but that should be mostly transparent to you.

@IMPORTANT{@value{gnatcoll} requires a fairly recent Ada05 compatible compiler.
If you do not have such a compiler, please contact @email{sales@@adacore.com}}

@noindent
Since you are reading this documentation, it is assumed you have been able
to unpack the package in a temporary directory. In the following instructions,
we will assume the following: @var{prefix} is the directory in which you
would like to install @value{gnatcoll}.

@section Configuring the build environment

The first step is to configure the build environment. This is done by
running the @code{configure} command in the root directory of the
@value{gnatcoll} tree.

@cindex GNATCOLL.Projects
@cindex projects
@cindex gnat sources
If you have access to the sources of GNAT, you should create a link called
@file{gnat_src} that points to the directory containing those sources. They
are used to add support for @code{GNATCOLL.Projects}, which reuses the same
parser as the GNAT tools. If the link @file{gnat_src} does not exist, the
support for projects will not be compiled.

@code{configure} accepts lots of arguments, among which
the following ones are most useful:

@table @code
@item --prefix=@var{prefix}
This specifies the directory in which @value{gnatcoll} should be installed.

@item --enable-shared
@itemx --disable-shared
If none of these switches is specified, @value{gnatcoll} will try to build
both static and shared libraries (if the latter are supported on your
system). The compilation needs to be done twice, since the compilation options
might not be the same in both cases.

If you only intend to ever use static libraries, you can use
@code{--disable-shared} to only build static libraries.

When you link @value{gnatcoll} with your own application, the default is
to link with the static libraries. You can change this default, which becomes
the shared libraries if you explicitly specify @code{--enable-shared}.
However, even if the default are the static libraries, you can still override
that (see below the @code{LIBRARY_TYPE} variable).

@item --with-python=@var{directory}
@itemx --without-python
This specifies where @value{gnatcoll} should find python. If for instance
the python executable is in @file{/usr/bin}, the @var{directory} to
specify is @file{/usr}. In most cases, however, @code{configure} will be
able to detect this automatically, so this is only useful if python is
installed in unusual directories. If you specify the second option,
support for python will not be build in.

@item --enable-shared-python
This specifies that the python library should be searched directly
in @var{directory}/lib, and thus will in general by the shared library.
By default, configure will search in a different directory of the python
installation, and is more likely to find the static library instead (which
makes distributing your application easier). There is no guarantee though
that either the shared or the static will be used, since it depends on how
python was installed on your system.

@item --disable-gtk
If this switch is specified, then no package depending on the gtk+ graphical
toolkit will be built.

@item --disable-pygtk
If this switch is specified, then support for pygtk (@pxref{The Python
language}) will not be build. The support for this python module will also
be automatically disabled if python was not found or if you configured with
@code{--without-python}.

@item --disable-syslog
If this switch is specified, then support for syslog (@pxref{Logging to syslog})
will not be build. This support allows sending the traces from all or part of
your application to the system logger, rather than to files or stdout.

@item --with-postgresql=<dir>
@itemx --without-postgresql
@value{gnatcoll} embeds a set of packages to query a database engine.
@code{configure} attempts to find which systems are installed on your
system, and build support for those. But you can also explicitly disable
for those if you need.

If the directory in which PostgreSQL is installed contains spaces, you
should use a syntax like

@smallexample
./configure --with-postgres="/Program Files/PostgreSQL/8.4"
@end smallexample

Generally speaking, we do not recommend using paths with spaces since there
are often more difficulties in such a setup.

@end table

Special support exists in @value{gnatcoll} for the gtk+ graphical toolkit.
@code{configure} will attempt to find the installation directory for this
toolkit by using the @code{pkg-config} command, which must therefore be
available through your @var{PATH} environment variable. It also needs to
find the @file{gtkada.gpr} project file either because it is part of the
implicit search path for project files, or because you have put the
corresponding directory in the environment variable @code{GPR_PROJECT_PATH}.
If either of these two requirements fail, the modules of @value{gnatcoll}
that depend on GtkAda will not be built.

@smallexample
./configure --prefix=/usr/local/gnatcoll --without-python
@end smallexample

If all goes well (i.e. all required dependencies are found on the system),
configure will generate a number of files, including @file{Makefile},
@file{Makefile.conf} and @file{gnatcoll_shared.gpr}.

@section Building @value{gnatcoll}

If @code{configure} has run successfully, it generates a @code{Makefile}
to allow you to build the rest of @value{gnatcoll}.
This is done by simply typing the following command:

@smallexample
make
@end smallexample

Depending on the switches passed to @code{configure}, this will either
build both static and shared libraries, or static only (see the
@code{--disable-shared} configure switch).

Optionally, you can also build the examples and/or the automatic test suite,
with the following commands:

@smallexample
make examples
make test
@end smallexample

The latter will do a local installation of gnatcoll in a subdirectory called
@file{local_install}, and use this to run the tests. This ensures that the
installation process of gnatcoll works properly.

@section Installing @value{gnatcoll}

Installing the library is done with the following command:

@smallexample
make install
@end smallexample

Note that this makefile target does not try to recompile @value{gnatcoll},
so you must build it first.
This will install both the shared and the static libraries if both were
build.

As mentioned in the description of the @code{configure} switches, your
application will by default be linked with the static library, unless
you specified the @code{--enable-shared} switch.

However, you can always choose later on which kind of library to use for
@value{gnatcoll} by setting the environment variable @code{LIBRARY_TYPE}
to either @code{"relocatable"} or @code{"static"}.

Your application can now use the @value{gnatcoll} code through a project
file, by adding a with clause
to @file{gnatcoll.gpr}, @file{gnatcoll_gtk.gpr} or @file{gnatcoll_python.gpr}.
The second one will also force your application to be linked with the
gtk+ libraries, but provides additional capabilities as documented in each
of the modules.

@c ------------------------------------------------------------------------
@node Embedding script languages
@chapter Embedding script languages
@c ------------------------------------------------------------------------

@noindent
In a lot of contexts, you want to give the possibility to users to extend
your application. This can be done in several ways: define an Ada API from
which they can build dynamically loadable modules, provide the whole source
code to your application and let users recompile it, interface with a simpler
scripting languages,@dots{}

Dynamically loadable modules can be loaded on demand, as their name indicate.
However, they generally require a relatively complex environment to build,
and are somewhat less portable. But when your users are familiar with Ada,
they provide a programming environment in which they are comfortable.
As usual, changing the module requires recompilation, re-installation,...

Providing the source code to your application is generally even more
complex for users. This requires an even more complex setup, your application
is generally too big for users to dive into, and modifications done by one
users are hard to provide to other users, or will be lost when you
distribute a new version of your application.

The third solution is to embed one or more scripting languages in your
application, and export some functions to it. This often requires your users
to learn a new language, but these languages are generally relatively simple,
and since they are interpreted they are easier to learn in an interactive
console. The resulting scripts can easily be redistributed to other users or
even distributed with future versions of your application.

The module in @value{gnatcoll} helps you implement the third solution. It was
used extensively in the GPS programming environment for its python interface.

@TIP{Each of the scripting language is optional}
This module can be compiled with any of these languages as an optional
dependency (except for the shell language, which is always built-in, but is
extremely minimal, and doesn't have to be loaded at run time anyway).
If the necessary libraries are found on the system, @value{gnatcoll} will
be build with support for the corresponding language, but your application
can chose at run time whether or not to activate the support for a specific
language.

@TIP{Optional support is provided for the @emph{gtk+} library}.
Likewise, extensions are provided if the gtk+ libraries were found on your
system. These provide a number of Ada subprograms that help interface with
code using this library, and help export the corresponding classes.
This support for gtk+ is also optional, and you can still build
@value{gnatcoll} even if gtk+ wasn't installed on your system (or if your
application is text-only, in which case you likely do not want to depend
at link time on graphical libraries).

@cindex test driver
@cindex testing your application
@TIP{Use a scripting language to provide an automatic testing framework for
your application.}
@noindent
The GPS environment uses python command for its @emph{automatic test suite},
including graphical tests such as pressing on a button, selecting a
menu,@dots{}

@menu
* Supported languages::
* Scripts API::
@end menu

@c -----------------------------------------------------------------------
@node Supported languages
@section Supported languages
@c -----------------------------------------------------------------------

@noindent
The module provides built-in support for several scripting languages, and
other languages can "easily" be added. Your application does not change
when new languages are added, since the interface to export subprograms
and classes to the scripting languages is language-neutral, and will
automatically export to all known scripting languages.

Support is provided for the following languages:

@table @b
@item Shell

This is a very simple-minded scripting language, which doesn't provide
flow-control instructions (@pxref{The Shell language}).

@item Python

Python (@url{http://www.python.org}) is an advanced scripting language
that comes with an extensive library. It is fully object-oriented
(@pxref{The Python language}).

@end table

@menu
* The Shell language::
* The Python language::
* Classes exported to all languages::
@end menu

@c ----------------------------------------------------------------------
@node The Shell language
@subsection The Shell language
@c ----------------------------------------------------------------------

@noindent
The shell language was initially developed in the context of the GPS
programming environment, as a way to embed scripting commands in XML
configuration files.

In this language, you can execute any of the commands exported by the
application, passing any number of arguments they need. Arguments to function
calls can, but need not, be quoted. Quoting is only mandatory when they
contain spaces, newline characters, or double-quotes ('"'). To quote an
argument, surround it by double-quotes, and precede each double-quote it
contains by a backslash character. Another way of quoting is similar to
what python provides, which is to triple-quote the argument, i.e. surround it
by '"""' on each side. In such a case, any special character (in particular
other double-quotes or backslashes) lose their special meaning and are just
taken as part of the argument. This is in particular useful when you do not
know in advance the contents of the argument you are quoting.

@CODESAMPLE{
Shell> function_name arg1 "arg 2" """arg 3"""
}

Commands are executed as if on a stack machine: the result of a command is
pushed on the stack, and later commands can reference it using @code{%}
following by a number. By default, the number of previous results that are
kept is set to 9, and this can only be changed by modifying the source code
for @value{gnatcoll}. The return values are also modified by commands executed
internally by your application, and that might have no visible output from
the user's point of view. As a result, you should never assume you know
what @code{%1},@dots{} contain unless you just executed a command in the
same script.

@CODESAMPLE{
Shell> function_name arg1
Shell> function2_name %1
}

In particular, the @var{%1} syntax is used when emulating object-oriented
programming in the shell. A method of a class is just a particular function
that contains a '.' in its name, and whose first implicit argument is the
instance on which it applies. This instance is generally the result of
calling a constructor in an earlier call. Assuming, for instance, that we
have exported a class "Base" to the shell from our Ada core, we could use
the following code:
@CODESAMPLE{
Shell> Base arg1 arg2
Shell> Base.method %1 arg1 arg2
}
to create an instance and call one of its methods.
Of course, the shell is not the best language for object-oriented programming,
and better languages should be used instead.

Some commands are automatically added to the shell when this scripting
language is added to the application. These are

@deffn Function load file
Loads the content of @var{file} from the disk, and execute each of its lines as
a Shell command. This can for instance be used to load scripts when your
application is loaded
@end deffn

@deffn Function echo arg@dots{}
This function takes any number of argument, and prints them in the console
associated with the language. By default, when in an interactive console, the
output of commands is automatically printed to the console. But when you
execute a script through @code{load} above, you need to explicitly call
@code{echo} to make some output visible.
@end deffn

@deffn Function clear_cache
This frees the memory used to store the output of previous commands. Calling
@var{%1} afterward will not make sense until further commands are executed.
@end deffn

@c ----------------------------------------------------------------------
@node The Python language
@subsection The Python language
@c ----------------------------------------------------------------------

@cindex Python 
@noindent
Python is an interpreted, object-oriented language. See
@url{http://www.python.org} for more information, including tutorials, on
this language.

@NOTE{
Python support is optional in @value{gnatcoll}. If it hasn't been installed
on your system, @value{gnatcoll} will be compiled without it, but that
will not impact applications using @value{gnatcoll}, since the same packages
(and the same API therein) are provided in both cases. Of course, if python
support wasn't compiled in, these packages will do nothing.
}

@noindent
@cindex GNATCOLL.Python
@cindex gnatcoll-python.ads
In addition to the API common to all languages (@pxref{Scripts API}),
@value{gnatcoll} also comes with a low-level interface to the python
library. This interface is available in the @file{GNATCOLL.Python} package.
In general, it is much simpler to use the common API rather than this
specialized one, though, since otherwise you will need to take care of lots
of details like memory management, conversion to and from python types,@dots{}

@TIP{All functions exported to python are available in a specific namespace}
@noindent
All functions exported to python through @value{gnatcoll} are available in
a single python module, whose name you must specify when adding support
for python. This is done to avoid namespace pollution. You can further
organize the subprograms through python classes to provide more logical
namespaces.

As in Ada, python lets you use named parameters in subprogram calls,
and thus let's you change the order of arguments on the command line.
This is fully supported by @value{gnatcoll}, although your callbacks will
need to specify the name of the parameters for this to work fine.
@CODESAMPLE{
>>> func_name (arg1, arg2)
>>> func_name (arg2=arg2, arg1=arg1)`
}

Some commands and types are always exported by @value{gnatcoll}, since they
are needed by most application, or even internally by @value{gnatcoll}
itself.

@deffn  Exception Unexpected_Exception
@deffnx Exception Exception
@deffnx Exception Missing_Arguments
@deffnx Exception Invalid_Argument
A number of exceptions are added automatically, so that the internal
state of your application is reflected in python. These are raised on
unexpected uncaught Ada exceptions, when your callbacks return explicit
errors, or when a function call is missing some arguments.
@end deffn

@deffn Function exec_in_console command
This function can be used in your script when you need to modify the
contents of the python interpreter itself.

When you run a python script, all its commands (including the global
variables) are within the context of the script. Therefore, you cannot
affect variables which are used for instance in the rest of your
application or in the python console. With this function, @var{command}
will be executed as if it had been typed in the python console.

@example
exec_in_console ("sys.ps1 = 'foo'")
	@result{} foo>  # Prompt was changed in the console
@end example
@end deffn

@cindex pygtk
PyGtk is a python extension that provides an interface to the popular
gtk+ library. It gives access to a host of functions for writing graphical
interfaces from python. @value{gnatcoll} interfaces nicely with this extension
if it is found.

@NOTE{PyGtk support is also optional. It will be activated in your application
if the four following conditions are met: Python was detected on your system,
PyGtk was also detected when @value{gnatcoll} is built, PyGtk is detected
dynamically when your application is launched and your code is calling the
@code{Init_PyGtk_Support} function
@c @end itemize
}

When PyGtk is detected, you can add the following method to any of the
classes you export to python:

@defmethod AnyClass pywidget
This function returns an instance of a PyGtk class corresponding to the
graphical object represented by @var{AnyClass}. In general, it makes sense when
@var{AnyClass} is bound, in your Ada code, to a GtkAda object. As a result, the
same graphical element visible to the user on the screen is available from
three different programming languages: C, Ada and Python. All three can
manipulate it in the same way
@end defmethod

@c ----------------------------------------------------------------------
@node Classes exported to all languages
@subsection Classes exported to all languages
@c ----------------------------------------------------------------------

@noindent
In addition to the functions exported by each specific scripting language,
as described above, @value{gnatcoll} exports the following to all the
scripting languages. These are exported when your Ada code calls the
Ada procedure @code{GNATCOLL.Scripts.Register_Standard_Classes}, which should
done after you have loaded all the scripting languages.

@deftp Class Console
@code{Console} is a name that you can chose yourself when you call the
above Ada procedure. It will be assumed to be @code{Console} in the rest
of this document.

This class provides an interface to consoles. A console is an input/output
area in your application (whether it is a text area in a graphical
application, or simply standard text I/O in text mode). In particular,
the python standard output streams @code{sys.stdin}, @code{sys.stdout}
and @code{sys.stderr} are redirected to an instance of that class. If you
want to see python's error messages or usual output in your application,
you must register that class, and define a default console for your
scripting language through calls to
@code{GNATCOLL.Scripts.Set_Default_Console}.

You can later add new methods to this class, which would be specific to your
application. Or you can derive this class into a new class to achieve a similar
goal.
@end deftp

@defmethod Console write text
This method writes @var{text} to the console associated with the class
instance. See the examples delivered with @value{gnatcoll} for examples on
how to create a graphical window and make it into a @code{Console}.
@end defmethod

@defmethod Console clear
Clears the contents of the console.
@end defmethod

@defmethod Console flush
Does nothing currently, but is needed for compatibility with python.
Output through @code{Console} instances is not buffered anyway.
@end defmethod

@deftypemethod Console Boolean isatty
Whether the console is a pseudo-terminal. This is always wrong in the
case of @value{gnatcoll}.
@end deftypemethod

@deftypemethod Console string read [size]
Reads at most @var{size} bytes from the console, and returns the resulting
string.
@end deftypemethod

@deftypemethod Console string readline [size]
Reads at most @var{size} lines from the console, and returns them as a single
string.
@end deftypemethod

@c -----------------------------------------------------------------------
@node Scripts API
@section Scripts API
@c -----------------------------------------------------------------------

@noindent
This section will give an overview of the API used in the scripts module.
The reference documentation for this API is in the source files themselves. In
particular, each @file{.ads} file fully documents all its public API.

As described above, @value{gnatcoll} contains several levels of API. In
particular, it provides a low-level interface to python, in the packages
@code{GNATCOLL.Python}. This interface is used by the rest of @value{gnatcoll},
but is likely too low-level to really be convenient in your applications,
since you need to take care of memory management and type conversions by
yourself.

Instead, @value{gnatcoll} provides a language-neutral Ada API. Using this
API, it is transparent for your application whether you are talking to the
Shell, to python, or to another language integrated in @value{gnatcoll}.
The code remains exactly the same, and new scripting languages can be added
in later releases of @value{gnatcoll} without requiring a change in your
application. This flexibility is central to the design of @value{gnatcoll}.

In exchange for that flexibility, however, there are language-specific
features that cannot be performed through the @value{gnatcoll} API. At
present, this includes for instance exporting functions that return hash
tables. But @value{gnatcoll} doesn't try to export the greatest set of
features common to all languages. On the contrary, it tries to fully
support all the languages, and provide reasonable fallback for languages
that do not support that feature. For instance, named parameters (which
are a part of the python language) are fully supported, although the
shell language doesn't support them. But that's an implementation detail
transparent to your own application.

Likewise, your application might decide to always load the python
scripting language. If @value{gnatcoll} wasn't compiled with python support,
the corresponding Ada function still exists (and thus your code still
compiles), although of course it does nothing. But since the rest of the
code is independent of python, this is totally transparent for your
application.

@TIP{@value{gnatcoll} comes with some examples, which you can use
  as a reference when building your own application.
  See the @file{scripts/examples} directory.}

@noindent
Interfacing your application with the scripting module is a multistep
process:

@itemize @bullet
@item You @emph{must} @b{initialize} @value{gnatcoll} and decide which features
  to load
@item You @emph{can} create an @b{interactive console} for the various
  languages, so that users can perform experiments interactively. This
  is optional, and you could decide to keep the scripting language has a
  hidden implementation detail (or just for automatic testing purposes
  for instance)
@item You @emph{can} @b{export} some classes and methods.
  This is optional, but it doesn't really make sense to just embed a
  scripting language and export nothing to it. In such a case, you might
  as well spawn a separate executable.
@item You @emph{can} load @b{start up scripts} or plug-ins that users have
  written to extend your application.
@end itemize

@menu
* Initializing the scripting module::
* Creating interactive consoles::
* Exporting classes and methods::
* Executing startup scripts::
* Debugging scripts::
@end menu

@c -----------------------------------------------------------------------
@node Initializing the scripting module
@subsection Initializing the scripting module
@c -----------------------------------------------------------------------

@noindent
@value{gnatcoll} must be initialized properly in order to provide added
value to your application. This cannot be done automatically simply by
depending on the library, since this initialization requires multiple-step
that must be done at specific moments in the initialization of your whole
application.

This initialization does not depend on whether you have build support
for python or for gtk+ in @value{gnatcoll}. The same packages and subprograms
are available in all cases, and therefore you do not need conditional
compilation in your application to support the various cases.

@c -----------------------------------------------------------------------
@subsubsection Create the scripts repository
@c -----------------------------------------------------------------------
@noindent
The type @code{GNATCOLL.Scripts.Scripts_Repository} will contain various
variables common to all the scripting languages, as well as a list of the
languages that were activated. This is the starting point for all other
types, since from there you have access to everything. You will have only
one variable of this type in your application, but it should generally be
available from all the code that interfaces with the scripting language.

Like the rest of @value{gnatcoll}, this is a tagged type, which you can
extend in your own code. For instance, the GPS programming environment is
organized as a kernel and several option modules. The kernel provides
the core functionality of GPS, and should be available from most functions
that interface with the scripting languages. Since these functions have
very specific profiles, we cannot pass additional arguments to them. One
way to work around this limitation is to store the additional arguments
(in this case a pointer to the kernel) in a class derived from
@code{Scripts_Repository_Data}.

As a result, the code would look like

@CODESAMPLE{@b{with} GNATCOLL.Scripts;@NL{}
Repo : Scripts_Repository := @b{new} Scripts_Repository_Data;}

@noindent
or, in the more complex case of GPS described above:

@CODESAMPLE{@b{type} Kernel_Scripts_Repository @b{is new}@NL{}
   Scripts_Repository_Data @b{with record}@NL{}
      Kernel : ...;@NL{}
@b{end record;}@NL{}
Repo : Scripts_Repository := new Kernel_Scripts_Repository'@NL{}
   (Scripts_Repository_Data @b{with} Kernel => ...);}

@c -----------------------------------------------------------------------
@subsubsection Loading the scripting language
@c -----------------------------------------------------------------------
@noindent
The next step is to decide which scripting languages should be made
available to users. This must be done before any function is exported,
since only functions exported after a language has been loaded will be
made available in that language.

@NOTE{If for instance python support was build into @value{gnatcoll}, and
if you decide not to make it available to users, your application will
still be linked with @file{libpython}. It is therefore recommended although
not mandatory to only build those languages that you will use}

This is done through a simple call to one or more subprograms. The following
example registers both the shell and python languages

@CODESAMPLE{@b{with} GNATCOLL.Scripts.Python;@NL{}
@b{with} GNATCOLL.Scripts.Shell;@NL{}
Register_Shell_Scripting (Repo);@NL{}
Register_Python_Scripting (Repo, "MyModule");}


@deffn Procedure Register_Shell_Scripting Repo
This adds support for the shell language. Any class or function that is
now exported through @value{gnatcoll} will be made available in the shell
@end deffn

@deffn Procedure Register_Python_Scripting Repo Module_Name
This adds support for the python language. Any class or function exported
from now on will be made available in python, in the module specified
by @var{Module_Name}
@end deffn

@c -----------------------------------------------------------------------
@subsubsection Exporting standard classes
@c -----------------------------------------------------------------------
@noindent
To be fully functional, @value{gnatcoll} requires some predefined classes
to be exported to all languages (@pxref{Classes exported to all languages}).
For instance, the @code{Console} class is needed for proper interactive with
the consoles associated with each language.

These classes are created with the following code:
@CODESAMPLE{Register_Standard_Classes (Repo, "Console");}

This must be done only after all the scripting languages were loaded in the
previous step, since otherwise the new classes would not be visible in the
other languages.

@deffn Procedure Register_Standard_Classes Repo Console_Class
The second parameter @var{Console_Class} is the name of the class that
is bound to a console, and thus provides input/output support. You can chose
this name so that it matches the classes you intend to export later on from
your application.
@end deffn

@c -----------------------------------------------------------------------
@node Creating interactive consoles
@subsection Creating interactive consoles
@c -----------------------------------------------------------------------
@noindent
The goal of the scripting module in @value{gnatcoll} is to work both in
text-only applications and graphical applications that use the gtk+ toolkit.
However, in both cases applications will need a way to capture the output
of scripting languages and display them to the user (at least for errors, to
help debugging scripts), and possibly emulate input when a script is waiting
for such input.

@value{gnatcoll} solved this problem by using an abstract class
@code{GNATCOLL.Scripts.Virtual_Console_Record} that defines an API for these
consoles. This API is used throughout @code{GNATCOLL.Scripts} whenever input or
output has to be performed.

@TIP{The @file{scripts/examples} directory in the @value{gnatcoll} package
shows how to implement a console in text mode and in graphical mode.}
@noindent
If you want to provide feedback or interact with users, you will need to
provide an actual implementation for these @code{Virtual_Console}, specific
to your application. This could be a graphical text window, or based on
@code{Ada.Text_IO}. The full API is fully documented in
@file{gnatcoll-scripts.ads}, but here is a list of the main subprograms that
need to be overriden.

@defmethod Virtual_Console  Insert_Text Txt
@defmethodx Virtual_Console Insert_Log Txt
@defmethodx Virtual_Console Insert_Error Txt
These are the various methods for doing output. Error messages could for
instance be printed in a different color. Log messages should in general
be directed elsewhere, and not be made visible to users unless in special
debugging modes.
@end defmethod

@defmethod Virtual_Console Insert_Prompt Txt
This method must display a prompt so that the user knows input is expected.
Graphical consoles will in general need to remember where the prompt ended
so that they also know where the user input starts
@end defmethod

@defmethod Virtual_Console Set_As_Default_Console Script
This method is called when the console becomes the default console for
a scripting language. They should in general keep a pointer on that
language, so that when the user presses @key{enter} they know which language
must execute the command
@end defmethod

@deftypemethod Virtual_Console String Read Size Whole_Line
Read either several characters or whole lines from the console. This is
called when the user scripts read from their stdin.
@end deftypemethod

@defmethod Virtual_Console Set_Data_Primitive Instance
@defmethodx Virtual_Console Get_Instance Console
These two methods are responsible for storing an instance of @var{Console}
into a @code{GNATCOLL.Scripts.Class_Instance}. Such an instance is
what the user
manipulates from his scripting language. But when he executes a method, the
Ada callback must know how to get the associated @code{Virtual_Console}
back to perform actual operations on it.

These methods are implemented using one of the @code{GNATCOLL.Scripts.Set_Data}
and @code{GNATCOLL.Scripts.Get_Data} operations when in text mode, or possibly
@code{GNATCOLL.Scripts.Gtkada.Set_Data} and
@code{GNATCOLL.Scripts.Gtkada.Get_Data}
when manipulating graphical GtkAda objects.
@end defmethod

There are lots of small details to take into account when writing a
graphical console. The example in @file{scripts/examples/gtkconsole.ads}
should provide a good starting point. However, it doesn't handle things
like history of commands, preventing the user from moving the cursor
to previous lines,@dots{} which are all small details that need to be right
for the user to feel comfortable with the console.

Once you have created one or more of these console, you can set them as
the default console for each of the scripting languages. This way, any
input/output done by scripts in this language will interact with that
console, instead of being discarded. This is done through code similar
to:

@CODESAMPLE{Console := GtkConsole.Create (...);@NL{}
Set_Default_Console@NL{}
  (Lookup_Scripting_Language (Repo, "python"),@NL{}
   Console);} 

@c -----------------------------------------------------------------------
@node Exporting classes and methods
@subsection Exporting classes and methods
@c -----------------------------------------------------------------------
@noindent
Once all scripting languages have been loaded, you can start exporting
new classes and functions to all the scripting languages. It is important
to realize that through a single Ada call, they are exported to all loaded
scripting languages, without further work required on your part.

@subsubsection Classes diagram
@noindent

The following diagram shows the dependencies between the major data types
defined in @file{GNATCOLL.Scripts}. Most of these are abstract classes that
are implemented by the various scripting languages. Here is a brief description
of the role of each type:

@cindex class diagram, script module
@image{classes,15cm}

@deftp Class Scripts_Repository
As we have seen before, this is a type of which there is a single instance
in your whole application, and whose main role is to give access to each
of the scripting languages (@code{Lookup_Scripting_Language} function), and
to make it possible to register each exported function only once (it then
takes care of exporting it to each scripting language).
@end deftp

@deftp Class Scripting_Language
Instances of this type represent a specific language. It provides various
operations to export subprograms, execute commands, create the other types
described below,... There should exists a single instance of this class per
supported language.

This class interacts with the script interpreter (for instance python), and
all code executed in python goes through this type, which then executes your
Ada callbacks to perform the actual operation.

It is also associated with a default console, as described above, so that
all input and output of the scripts can be made visible to the user.
@end deftp

@deftp Class Callback_Data
This type is an opaque tagged type that provides a language-independent
interface to the scripting language. It gives for instance access to the
various parameters passed to your subprogram (@code{Nth_Arg} functions),
allows you to set the return value (@code{Set_Return_Value} procedure),
or raise exceptions (@code{Set_Error_Msg} procedure),@dots{}
@end deftp

@deftp Record Class_Type
This type is not tagged, and cannot be extended. It basically represents a
class in any of the scripting languages, and is used to create new instances
of that class from Ada.
@end deftp

@deftp Class Class_Instance
A class instance represents a specific instance of a class. In general,
such an instance is strongly bound to an instance of an Ada type. For
instance, if you have a @code{Foo} type in your application that you wish
to export, you would create a @code{Class_Type} called "Foo", and then the
user can create as many instances as he wants of that class, each of which
is associated with different values of @code{Foo} in Ada.

Another more specific example is the predefined @code{Console} class. As
we have seen before, this is a @code{Virtual_Console} in Ada. You could
for instance have two graphical windows in your application, each of which
is a @code{Virtual_Console}. In the scripting language, this is exported
as a class named @code{Console}. The user can create two
instances of those, each of which is associated with one of your graphical
windows. This way, executing @code{Console.write} on these instances would
print the string on their respective graphical window.

Some scripting languages, in particular python, allow you to store any
data within the class instances. In the example above, the user could for
instance store the time stamp of the last output in each of the instances.
It is therefore important that, as much as possible, you always return the
same @code{Class_Instance} for a given Ada object. See the following
python example:

@CODESAMPLE{myconsole = Console ("title") # Create new console@NL{}
myconsole.mydata = "20060619"  # Any data, really@NL{}
myconsole = Console ("title2")  # Create another window@NL{}
myconsole = Console ("title") # Must be same as first, @NL{}
print myconsole.mydata  # so that this prints "20060619"}
@end deftp

@deftp Class Instance_Property
As we have seen above, a @code{Class_Instance} is associated in general with
an Ada object. This @code{Instance_Property} tagged type should be extended
for each Ada type you want to be able to store in a @code{Class_Instance}.
You can then use the @code{Set_Data} and @code{Get_Data} methods of the
@code{Class_Instance} to get and retrieve that associated Ada object.
@end deftp

@deftp Class Subprogram_Record
This class represents a callback in the scripting language, that is some
code that can be executed when some conditions are met.

The exact semantic here depends on each of the programming languages. For
instance, if you are programming in python, this is the name of a python
method to execute. If you are programming in shell, this is any shell code.

The idea here is to blend in as smoothly as possible with the usual constructs
of each language. For instance, in python one would prefer to write the
second line rather than the third:

@CODESAMPLE{def on_exit(): pass@NL{}
set_on_exit_callback (on_exit)   # Yes, python style@NL{}
set_on_exit_callback ("on_exit") # No}

The last line (using a string as a parameter) would be extremely unusual
in python, and would for instance force you to qualify the subprogram name
with the name of its namespace (there would be no implicit namespace
resolution).

To support this special type of parameters, the @code{Subprogram_Record}
type was created in Ada.
@end deftp

@noindent
Although the exact way they are all these types are created is largely
irrelevant to your specific application in general, it might be useful for you
to override part of the types to provide more advanced features. For instance,
GPS redefines its own Shell language, that has basically the same behavior as
the Shell language described above but whose @code{Subprogram_Record} in fact
execute internal GPS actions rather than any shell code.

@subsubsection Exporting functions
@noindent
All functions that you export to the scripting languages will result in a
call to an Ada subprogram from your own application. This subprogram must
have the following profile:

@CODESAMPLE{@b{procedure} Handler@NL{}
   (Data    : @b{in out} Callback_Data'@b{Class};@NL{}
    Command : String);}

The first parameter @var{Data} gives you access to the parameters of the
subprogram as passed from the scripting language, and the second parameter
@var{Command} is the name of the command to execute. The idea behind this
second parameter is that a single Ada procedure might handle several
different script function (for instance because they require common actions
to be performed).

@defun Register_Command Repo Command Min_Args Max_Args Handler
Each of the shell functions is then exported through a call to
@code{Register_Command}. In its simplest form, this procedure takes the
following arguments. @var{Repo} is the scripts repository, so that the
command is exported to all the scripting languages. @var{Command} is the
name of the command. @var{Min_Args} and @var{Max_Args} are the minimum and
maximum number of arguments. Most language allow option parameters, and
this is how you specify them. @var{Handler} is the Ada procedure to call
to execute the command.
@end defun

Here is a simple example. It implements a function called @code{Add}, which
takes two integers in parameter, and returns their sum.

@CODESAMPLE{Arg1_C : @b{aliased constant} String := "arg1";@NL{}
Arg2_C : @b{aliased constant} String := "arg2";@NL{}
@b{procedure} Sum@NL{}
   (Data : @b{in out} Callback_Data'@b{Class};@NL{}
    Command : String)@NL{}
@b{is}@NL{}
   Arg1, Arg2 : Integer;@NL{}
@b{begin}@NL{}
   Name_Parameters ((1 => Arg1_C'Access, 2 => Arg2_C'Access));@NL{}
   Arg1 := Nth_Arg (Data, 1);@NL{}
   Arg2 := Nth_Arg (Data, 2);@NL{}
   Set_Return_Value (Data, Arg1 + Arg2);@NL{}
@b{end} Sum;@NL{}
@NL{}
Register_Command (Repo, "sum", 2, 2, Sum'@b{Access});}

Not the most useful function to export! Still, it illustrates a number of
important concepts.

@TIP{Automatic parameters types}
@noindent
When the command is registered, the number of arguments is specified.
This means that @value{gnatcoll} will check on its own whether the right
number of arguments is provided. But the type of these arguments is not
specified. Instead, your callback should proceed as if they were correct,
and try to retrieve them through one of the numerous @code{Nth_Arg}
functions. In the example above, we assume they are integer. But if one of
them was passed as a string, an exception would be raised and sent back to
the scripting language to display a proper error message to the user. You
have nothing special to do here.

@TIP{Support for named parameters}
@noindent
Some languages (especially python) support named parameters, ie parameters
can be specified in any order on the command line, as long as they are
properly identified (very similar to Ada's own capabilities). In the example
above, the call to @code{Name_Parameters} is really optional, but adds this
support for your own functions as well. You just have to specify the name
of the parameters, and @value{gnatcoll} will then ensure that when you
call @code{Nth_Arg} the parameter number 1 is really "arg1".
For scripting languages that do not support named parameters, this has no
effect.

Your code can then perform as complex a code as needed, and finally
return a value (or not) to the scripting language, through a call to
@code{Set_Return_Value}.

After the above code has been executed, your users can go to the python
console and type for instance

@example
from MyModule import *    # MyModule is the name we declared above
print sum (1,2)
	@result{} 3
print sum ()
	@error{} Wrong number of parameters
print sum ("1", 2)
	@error{} Parameter 1 should be an integer
print sum (arg2=2, arg1=1)
	@result{} 3
@end example


@subsubsection Exporting classes
@noindent
Whenever you want to make an Ada type accessible through the scripting
languages, you should export it as a class. For object-oriented languages,
this would map to the appropriate concept. For other languages, this provides
a namespace, so that each method of the class now takes an additional first
parameter which is the instance of the class, and the name of the method is
prefixed by the class name.

Creating a new class is done through a call to @code{New_Class}, as shown
in the example below.

@CODESAMPLE{MyClass : Class_Type;@NL{}
MyClass := GNATCOLL.Scripts.New_Class (Repo, "MyClass");}

At this stage, nothing is visible in the scripting language, but all the
required setup has been done internally so that you can now add methods to
this class.

You can then register the class methods in the same way that you registered
functions. An additional parameter @var{Class} exists for
@code{Register_Command}. A method is really just a standard function that
has an implicit first parameter which is a @code{Class_Instance}. This
extra parameter should not be taken into account in @var{Min_Args} and
@var{Max_Args}. You can also declare the method as a static method, ie
one that doesn't take this extra implicit parameter, and basically just
uses the class as a namespace.

Some special method names are available. In particular,
@code{Constructor_Method} should be used for the constructor of a class.
It is a method that receives, as its first argument, a class instance that
has just been created. It should associate that instance with the Ada
object it represents.

Here is a simple example that exports a class. Each instance of this class
is associated with a string, passed in parameter to the constructor. The
class has a single method @code{print}, which prints its string parameter
prefixed by the instance's string. To start with, here is a python example
on what we want to achieve:

@example
c1 = MyClass ("prefix1")
c1.print ("foo")
	@result{} "prefix1 foo"
c2 = MyClass ()  # Using a default prefix
c2.print ("foo")
	@result{} "default foo"
@end example

Here is the corresponding Ada code.

@CODESAMPLE{@b{with} GNATCOLL.Scripts.Impl;@NL{}
@b{procedure} Handler@NL{}
   (Data : @b{in out} Callback_Data'Class; Command : String)@NL{}
@b{is}@NL{}
   Inst : Class_Instance := Nth_Arg (Data, 1, MyClass);@NL{}
@b{begin}@NL{}
   @b{if} Command = Constructor_Method @b{then}@NL{}
     Set_Data (Inst, MyClass, Nth_Arg (Data, 2, "default"));@NL{}
   @b{elsif} Command = "print" @b{then}@NL{}
     Insert_Text@NL{}
        (Get_Script (Data), null,@NL{}
         String'(Get_Data (Inst)) & " " & Nth_Arg (Data, 2));@NL{}
   @b{end if};@NL{}
@b{end} Handler;@NL{}
@NL{}
Register_Command@NL{}
  (Repo, Constructor_Method, 0, 1, Handler'@b{Access}, MyClass);@NL{}
Register_Command@NL{}
  (Repo, "print", 1, 1, Handler'@b{Access}, MyClass);@NL{}}

This example also demonstrates a few concepts: the constructor is declared
as a method that takes one optional argument. The default value is in
fact passed in the call to @code{Nth_Arg} and is set to "default".
In the handler, we know there is always a first argument which is the
instance on which the method applies. The implementation for the
constructor stores the prefix in the instance itself, so that several
instances can have different prefixes (we can't use global variables,
of course, since we don't know in advance how many instances will exist).
The implementation for @code{print} inserts code in the default console
for the script (we could of course use @code{Put_Line} or any other way
to output data), and computes the string to output by concatenating the
instance's prefix and the parameter to @code{print}.

Note that @code{Set_Data} and @code{Get_Data} take the class in parameter,
in addition to the class instance. This is needed for proper handling of
multiple inheritance: say we have a class @code{C} that extends two classes
@code{A} and @code{B}. The Ada code that deals with @code{A} associates an
integer with the class instance, whereas the code that deals with @code{B}
associates a string. Now, if you have an instance of @code{C} but call a
method inherited from @code{A}, and if @code{Get_Data} didn't specify the
class, there would be a risk that a string would be returned instead of the
expected integer. In fact, the proper solution here is that both @code{A}
and @code{B} store their preferred data at the same time in the instances,
but only fetch the one they actually need. Therefore instances of @code{C}
are associated with two datas.

Here is a more advanced example that shows how to export an Ada object. Let's
assume we have the following Ada type that we want to make available to
scripts:

@CODESAMPLE{@b{type} MyType @b{is record}@NL{}
  Field : Integer;@NL{}
@b{end record};}

As you can see, this is not a tagged type, but could certainly be. There is
of course no procedure @code{Set_Data} in @file{GNATCOLL.Scripts} that enables
us to store @code{MyType} in a @code{Class_Instance}. This example shows how
to write such a procedure. The rest of the code would be similar to the
first example, with a constructor that calls @code{Set_Data}, and methods
that call @code{Get_Data}.

@CODESAMPLE{@b{type} MyPropsR @b{is new} Instance_Property_Record @b{with record}@NL{}
   Val : MyType;@NL{}
@b{end record};@NL{}
@b{type} MyProps @b{is access all} MyPropsR'@b{Class};@NL{}
@NL{}
@b{procedure} Set_Data@NL{}
  (Inst : Class_Instance; Val : MyType)@NL{}
@b{is}@NL{}
@b{begin}@NL{}
  Set_Data (Inst, Get_Name (MyClass), MyPropsR'(Val => Val));@NL{}
@b{end} Set_Data;@NL{}
@NL{}
@b{function} Get_Data (Inst : Class_Instance) @b{return} MyType @b{is}@NL{}
   Data : MyProps := MyProps (Instance_Property'@NL{}
      (Get_Data (Inst, Get_Name (MyClass))));@NL{}
@b{begin}@NL{}
   @b{return} Data.Val;@NL{}
@b{end} Get_Data;}

Several aspects worth noting in this example. Each data is associated with
a name, not a class as in the previous example. That's in fact the same
thing, and mostly for historical reasons. We have to create our own
instance of @code{Instance_Property_Record} to store the data, but the
implementation presents no special difficulty. In fact, we don't absolutely
need to create @code{Set_Data} and @code{Get_Data} and could do everything
inline in the method implementation, but it is cleaner this way and easier
to reuse.

@value{gnatcoll} is fully responsible for managing the lifetime of the
data associated with the class instances and you can override the procedure
@code{Destroy} if you need special memory management.


@subsubsection Reusing class instances
@noindent
We mentioned above that it is more convenient for users of your exported
classes if you always return the same class instance for the same Ada
object (for instance a graphical window should always be associated with
the same class instance), so that users can associate their own internal
data with them.

@value{gnatcoll} provides a few types to facilitate this. In passing, it
is worth noting that in fact the Ada objects will be associated with a
single instance @emph{per scripting language}, but each language has its
own instance. Data is not magically transferred from python to shell!

There are two cases to distinguish here:

@itemize @bullet
@item The Ada object derives from a GtkAda object

In such a case, the package @file{GNATCOLL.Scripts.GtkAda} provides three
procedures that automatically associate the instance with the object,
and can return the class instance associated with any given GtkAda
object, or can return the GtkAda object stored in the instance. There is
nothing else to do that to call @code{Set_Data} as we have seen above.
See below for a brief discussion on the Factory design pattern. The internal
handling is complex, since python
for instance has ref-counted types, and so does gtk+. For the memory to
be correctly freed when no longer needed, @value{gnatcoll} must properly
takes care of these reference counting. The result is that the class
instance will never be destroyed while the gtk+ object exists, but the
gtk+ object might be destroyed while the class instance still exists (in
which case no further operation on that instance is possible).

@item The Ada object does not derive from a GtkAda object

In such a case, you should store the list of associated instances with
your object. The type @code{GNATCOLL.Scripts.Instance_List_Access} is meant for
that purpose, and provides two @code{Set} and @code{Get} primitives
to retrieve existing instances.

There is one catch however, related to memory management. The instances
must continue to exist as long as the Ada object exist (and not be
destroyed for instance when the python variables goes out of scope).
@value{gnatcoll} mostly takes care of that for you, but requires a little
bit of help still: when you implement a new @code{Instance_Property_Record}
as in the example above, you must also override its primitive
@code{Get_Instances} to return the @code{Instance_List_Access}. That's it.
@end itemize

The final aspect to consider here is how to return existing instances.
This cannot be done from the constructor method, since when it is called
it has already received the created instance (this is forced by python, and
was done the same for other languages for compatibility reasons).
There are two ways to work around that limitation:

@itemize @bullet
@item Static @code{get} methods

With each of your classes, you can export a static method generally called
@code{get} that takes in parameter a way to identify an existing instance,
and either return it or create a new one. It is also recommended to disable
the constructor, ie force it to raise an error. Let's examine the python
code as it would be used:

@example
ed = Editor ("file.adb")  # constructor
	@result{} Error, cannot construct instances
ed = Editor.get ("file.adb")
	@result{} Create a new instance
ed2 = Editor.get ("file.adb")
	@result{} Return existing instance
ed == ed2
	@result{} True	
@end example

The corresponding Ada code would be something like:

@CODESAMPLE{
@b{type} MyType @b{is record}@NL{}
   Val : Integer;@NL{}
   Inst : Instance_List_Access;@NL{}
@b{end record};@NL{}
@b{type} MyTypeAccess @b{is access all} MyType;@NL{}
@b{procedure} Handler@NL{}
  (Data : @b{in out} Callback_Data'Class; Cmd : String)@NL{}
@b{is}@NL{}
   Inst : Class_Instance;@NL{}
   Tmp  : MyTypeAccess;@NL{}
@b{begin}@NL{}
   @b{if} Cmd = Constructor_Method @b{then}@NL{}
     Set_Error_Msg (Data, "cannot construct instances");@NL{}
   @b{elsif} Cmd = "get" @b{then}@NL{}
     Tmp := check_if_exists (Nth_Arg (Data, 1));@NL{}
     @b{if} Tmp = null @b{then}@NL{}
        Tmp := create_new_mytype (Nth_Arg (Data, 1));@NL{}
        Tmp.Inst := @b{new} Instance_List;@NL{}
     @b{end if};@NL{}
     Inst := Get (Tmp.Inst.all, Get_Script (Data));@NL{}
     @b{if} Inst = null @b{then}@NL{}
        Inst := New_Instance (Get_Script (Data), MyClass);@NL{}
        Set (Tmp.Inst.all, Get_Script (Data), Inst);@NL{}
        Set_Data (Inst, Tmp);@NL{}
     @b{end if};
     @b{return} Inst;@NL{}
   @b{end if};@NL{}
@b{end} Handler;}


@item Factory classes

The standard way to do this in python, which applies to other languages
as well, is to use the Factory design pattern. For this, we need to
create one class (@code{MyClassImpl}) and one factory
function (@code{MyClass}).

The python code now looks like

@example
ed = MyClass ("file.adb")  # Create new instance
	@result{} ed is of type MyClassImpl
ed = MyClass ("file.adb")  # return same instance
ed.do_something() 
@end example

It is important to realize that in the call above, we are not calling
the constructor of a class, but a function. At the Ada level, the function
has basically the same implementation as the one we gave for @code{get}
above. But the python code looks nicer because we do not have these
additional @code{.get()} calls. The name of the class @code{MyClassImpl}
doesn't appear anywhere in the python code, so this is mostly transparent.

However, if you have more than one scripting language, in particular for
the shell, the code looks less nice in this case:

@example
MyClass "file.adb"
	@result{}  <MyClassImpl_Instance_0x12345>
MyClassImpl.do_something %1
@end example

and the new name of the class is visible in the method call.


@end itemize

@c -----------------------------------------------------------------------
@node Executing startup scripts
@subsection Executing startup scripts
@c -----------------------------------------------------------------------
@noindent
The final step in starting up your application is to load extensions or
plug-ins written in one of the scripting languages.

There is not much to be said here, except that you should use the
@code{GNATCOLL.Scripts.Execute_File} procedure to do so.

@c -----------------------------------------------------------------------
@node Debugging scripts
@subsection Debugging scripts
@c -----------------------------------------------------------------------
@noindent

@value{gnatcoll} provides a convenient hook to debug your script. By default,
a script (python for instance) will call your Ada callback, which might
raise errors. Most of the time, the error should indeed be reported to the
user, and you can thus raise a standard exception, or call
@code{Set_Error_Msg}.

BUt if you wish to know which script was executing the command, it is
generally not doable. You can however activate a trace (@pxref{Logging
information}) called @code{"PYTHON.TB"} (for "traceback"), which will
output the name of the command that is being executed, as well as the
full traceback within the python scripts. This will help you locate which
script is raising an exception.


@c -----------------------------------------------------------------------
@node Logging information
@chapter Logging information
@c -----------------------------------------------------------------------
@noindent

Most applications need to log various kinds of information: error messages,
information messages or debug messages among others. These logs can be
displayed and stored in a number of places: standard output, a file, the
system logger, an application-specific database table,@dots{}

The package @file{GNATCOLL.Traces} addresses the various needs, except for the
application-specific database, which of course is specific to your business
and needs various custom fields in any case, which cannot be easily provided
through a general interface.

This module is organized around two tagged types (used through access types,
in fact, so the latter are mentioned below as a shortcut):

@table @code
@item Trace_Handle
This type defines a handle (similar to a file descriptor in other contexts)
which is latter used to output messages. An application will generally
define several handles, which can be enabled or disabled separately, therefore
limiting the amount of logging.

@item Trace_Stream
Streams are the ultimate types responsible for the output of the messages.
One or more handles are associated with each stream. The latter can be a file,
the standard output, a graphical window, a socket,@dots{} New types of streams
can easily be defined in your application.

@end table

@menu
* Configuring traces::
* Using the traces module::
* Log decorators::
* Defining custom trace streams::
* Logging to syslog::
* Dynamically disabling features::
@end menu

@c ------------------------------------------------------------------------
@node Configuring traces
@section Configuring traces
@c ------------------------------------------------------------------------
@noindent

As mentioned above, an application will generally create several
@code{Trace_Handle} (typically one per module in the application). When
new features are added to the application, the developers will generally
need to add lots of traces to help investigate problems once the application
is installed at a customer's site. The problem here is that each module
might output a lot of information, thus confusing the logs; this also does
not help debugging.

The @code{GNATCOLL.Traces} package allows the user to configure which handles
should actually generate logs, and which should just be silent and not
generate anything. Depending on the part of the application that needs to
be investigated, one can therefore enable a set of handles or another, to
be able to concentrate on that part of the application.

This configuration is done at two levels:
@itemize @bullet
@item either in the source code itself, where some @code{trace_handle}
might be disabled or enabled by default. This will be described in more
details in later sections.

@item or in a configuration file which is read at runtime, and overrides
the defaults set in the source code.
@end itemize

The configuration file is found in one of three places, in the following
order:

@itemize @bullet
@item The file name is specified in the source code in the call to
@code{Parse_Config_File}.

@cindex ADA_DEBUG_FILE
@item If no file name was specified in that call, the environment variable
@code{ADA_DEBUG_FILE} might point to a configuration file.

@cindex .gnatdebug
@item If the above two attempts did not find a suitable configuration file,
the current directory is searched for a file called @code{.gnatdebug}.
Finally, the user's home directory will also be searched for that file.
@end itemize

In all cases, the format of the configuration file is the same. Its goal is
to associate the name of a @code{trace_handle} with the name of a
@code{trace_stream} on which it should be displayed.

Streams are identified by a name. You can provide additional streams by
creating a new tagged object (@pxref{Defining custom trace streams}). Here are
the various possibilities to reference a stream:

@table @code
@item "name"
where name is a string made of letters, digits and slash ('/') characters.
This is the name of a file to which the traces should be redirected. The
previous contents of the file is discarded. If the name of the file is a
relative path, it is relative to the location of the configuration file, not
necessarily to the current directory when the file is parsed. If you used
">>" instead of ">" to redirect to that stream, the initial content of the
file is not overridden, and new traces are appended to the file instead.

@item "&1"
This syntax is similar to the one used on Unix shells, and indicates that
the output should be displayed on the standard output for the application.
If the application is graphical, and in particular on Windows platforms, it
is possible that there is no standard output!

@item "&2"
Similar to the previous one, but the output is sent to standard error.

@item "&syslog"
@xref{Logging to syslog}.
@end table

Comments in a configuration file must be on a line of their own, and start
with @code{--}. Empty lines are ignored. The rest of the lines represent
configurations, as in:

@itemize @bullet
@item If a line contains the single character @code{"+"}, it activates all
@code{trace_handle} by default. This means the rest of the configuration
file should disable those handles that are not needed. The default is that
all handles are disabled by default, and the configuration file should
activate the ones it needs. The Ada source code can change the default
status of each handles, as well

@item If the line starts with the character @code{">"}, followed by a
stream name (as defined above), this becomes the default stream. All handles
will be displayed on that stream, unless otherwise specified. If the stream
does not exist, it defaults to standard output.

@item Otherwise, the first token on the line is the name of a handle.
If that is the only element on the line, the handle is activated, and will
be displayed on the default stream.

Otherwise, the next element on the line should be a @code{"="} sign,
followed by either @code{"yes"} or @code{"no"}, depending on whether the
handle should resp. be enabled or disabled.

Finally, the rest of the line can optionally contain the @code{">"}
character followed by the name of the stream to which the handle should
be directed.
@end itemize

Here is a short example of a configuration file. It activates all handles
by default, and defines four handles: two of them are directed to the
default stream (standard error), the third one to a file on the disk,
and the last one to the system logger syslog (if your system supports it,
otherwise to the default stream, ie standard error).

@CODESAMPLE{
+@NL{}
>&2@NL{}
MODULE1@NL{}
MODULE2=yes@NL{}
SYSLOG=yes >&syslog:local0:info@NL{}
FILE=yes >/tmp/file@NL{}
@NL{}
--  decorators (see below)@NL{}
DEBUG.COLORS=yes}

@c ------------------------------------------------------------------------
@node Using the traces module
@section Using the traces module
@c ------------------------------------------------------------------------
@noindent

If you need or want to parse an external configuration file as described
in the first section, the code that initializes your application should
contain a call to @code{GNATCOLL.Traces.Parse_Config_File}. As documented,
this takes in parameter the name of the configuration file to parse. When
none is specified, the algorithm specified in the previous section will be
used to find an appropriate configuration.

@CODESAMPLE{GNATCOLL.Traces.Parse_Config_File;}

The code, as written, will end up looking for a file @file{.gnatdebug} in
the current directory.

You then need to declare each of the @code{trace_handle} that your
application will use. The same handle can be declared several times, so
the recommended approach is to declare locally in each package body the
handles it will need, even if several bodies actually need the same
handle. That helps to know which traces to activate when debugging a
package, and limits the dependencies of packages on a shared package
somewhere that would contain the declaration of all shared handles.

@deftypefn Function Trace_Handle Create Name Default Stream Factory Finalize
This function creates (or return an existing) a @code{trace_handle} with
the specified @var{Name}. Its default activation status can also be
specified (through @var{Default}), although the default behavior is to
get it from the configuration file. If a handle is created several times,
only the first call that is executed can define the default activation
status, the following calls will have no effect.

@var{Stream} is the name of the stream to which it should be directed.
Here as well, it is generally better to leave things to the configuration
file, although in some cases you might want to force a specific behavior.

@var{Factory} is used to create your own child types of @code{trace_handle}
(@pxref{Log decorators}).

@end deftypefn

Here is an example with two package bodies that define their own handles,
which are later used for output.

@CODESAMPLE{
@b{package body} Pkg1 @b{is}@NL{}
   Me : @b{constant} Trace_Handle := Create ("PKG1");@NL{}
   Log : @b{constant} Trace_Handle := Create ("LOG", Stream => "@@syslog");@NL{}
@b{end} Pkg1;@NL{}
@b{package body} Pkg2 @b{is}@NL{}
   Me : @b{constant} Trace_Handle := Create ("PKG2");@NL{}
   Log : @b{constant} Trace_Handle := Create ("LOG", Stream => "@@syslog");@NL{}
@b{end} Pkg2;@NL{}}

Once the handles have been declared, output is a matter of calling the
@code{GNATCOLL.Traces.Trace} procedure, as in the following sample:

@CODESAMPLE{
   Trace (Me, "I am here");
}

@TIP{Check whether the handle is active}
@noindent
As we noted before, handles can be disabled. In that case, your application
should not spend time preparing the output string, since that would be
wasted time. In particular, using the standard Ada string concatenation
operator requires allocating temporary memory. It is therefore recommended,
when the string to display is complex, to first test whether the handle is
active. This is done with the following code:

@CODESAMPLE{
@b{if} Active (Me) @b{then}@NL{}
   Trace (Me, A & B & C & D & E);@NL{}
@b{end if};}

An additional subprogram can be used to test for assertions (pre-conditions
or post-conditions in your program), and output a message whether the
assertion is met or not.

@CODESAMPLE{
   Assert (Me, A = B, "A is not equal to B");
}

If the output of the stream is done in color, a failed assertion is
displayed with a red background to make it more obvious.

@c ------------------------------------------------------------------------
@node Log decorators
@section Log decorators
@cindex decorator, log
@c ------------------------------------------------------------------------
@noindent

Speaking of color, a number of decorators are defined by
@code{GNATCOLL.Traces}. Their goal is not to be used for outputting information,
but to configure what extra information should be output with all log
messages. They are activated through the same configuration file as the
traces, with the same syntax (i.e either @code{"=yes"} or @code{"=no"}).

Here is an exhaustive list:

@table @code
@item DEBUG.ABSOLUTE_TIME
If this decorator is activated in the configuration file, the absolute time
when Trace is called is automatically added to the output, when the
streams supports it (in particular, this has no effect for syslog, which
already does this on its own).

@item DEBUG.ELAPSED_TIME
If this decorator is activated, then the elapsed time since the last call to
Trace for the same handle is also displayed.

@item DEBUG.STACK_TRACE
If this decorator is activated, then the stack trace is also displayed. It can
be converted to a symbolic stack trace through the use of the external
application @code{addr2line}, but that would be too costly to do this
automatically for each message.

@item DEBUG.LOCATION
If this decorator is activated, the location of the call to Trace is
automatically displayed. This is a file:line:column information. This
works even when the executable wasn't compiled with debug information

@item DEBUG.ENCLOSING_ENTITY
Activate this decorator to automatically display the name of the subprogram
that contains the call to @code{Trace}.

@item DEBUG.COLORS
If this decorator is activated, the messages will use colors for the various
fields, if the stream supports it (syslog doesn't).

@item DEBUG.COUNT
This decorator displays two additional numbers on each line: the first is
the number of times this handle was used so far in the application, the second
is the total number of traces emitted so far. These numbers can for instance
be used to set conditional breakpoints on a specific trace (break on
@code{gnat.traces.log} or @code{gnat.traces.trace} and check the value of
@code{Handle.Count}. It can also be used to refer to a specific line in some
comment file.

@item DEBUG.FINALIZE_TRACES
This handle is activated by default, and indicates whether
@code{GNATCOLL.Traces.Finalize} should have any effect. This can be set to False
when debugging, to ensure that traces are available during the finalization
of your application.
@end table

Here is an example of output where several decorators were activated. In this
example, the output is folded on several lines, but in reality everything is
output on a single line.

@CODESAMPLE{
 [MODULE] 6/247 User Message (2007-07-03 13:12:53.46)@NL{}
    (elapsed: 2ms)(loc: gnatcoll-traces.adb:224)@NL{}
    (entity:GNATCOLL.Traces.Log)@NL{}
    (callstack: 40FD9902 082FCFDD 082FE8DF )
}

Depending on your application, there are lots of other possible decorators
that could be useful (for instance the current thread, or the name of the
executable when you have several of them,@dots{}). Since @code{GNATCOLL.Traces}
cannot provide all possible decorators, it provides support, through tagged
types, so that you can create your own decorators.

This needs you to override the @code{Trace_Handle_Record} tagged type. Since
this type is created through calls to @code{GNATCOLL.Traces.Create}. This is done
by providing an additional @var{Factory} parameter to @code{Create}; this is
a function that allocates and returns the new handle.

Then you can override either (or both) of the primitive operations
@code{Pre_Decorator} and @code{Post_Decorator}. The following example creates
a new type of handles, and prints a constant string just after the module
name:

@CODESAMPLE{
@b{type} My_Handle @b{is new} Trace_Handle_Record @b{with null record};@NL{}
@b{procedure}  Pre_Decorator@NL{}
  (Handle  : @b{in out} My_Handle;@NL{}
   Stream  : @b{in out} Trace_Stream_Record'@b{Class};@NL{}
   Message : String) @b{is}@NL{}
@b{begin}@NL{}
   Put (Stream, "TEST");@NL{}
   Pre_Decorator (Trace_Handle_Record (Handle), Stream, Message);@NL{}
@b{end};@NL{}
@NL{}
@b{function} Factory @b{return} Trace_Handle @b{is}@NL{}
@b{begin}@NL{}
   @b{return new} My_Handle;@NL{}
@b{end};@NL{}
@NL{}
Me : Trace_Handle := Create ("MODULE", Factory => Factory'Access);@NL{}
}

As we will see below (@pxref{Dynamically disabling features}), you can also
make all or part of your decorators conditional and configurable through
the same configuration file as the trace handles themselves.

@c ------------------------------------------------------------------------
@node Defining custom trace streams
@section Defining custom trace streams
@c ------------------------------------------------------------------------
@noindent

We noted above that several predefined streams exist, to output to a file,
to standard output or to standard error. Depending on your specific needs,
you might want to output to other media. For instance, in a graphical
application, you could have a window that shows the traces (perhaps in
addition to filing them in a file, since otherwise the window would
disappear along with its contents if the application crashes); or you could
write to a socket (or even a CORBA ORB) to communicate with another
application which is charge of monitoring your application.

@code{GNATCOLL.Traces} provides the type @code{Trace_Stream_Record}, which can
be overridden to redirect the traces to your own streams.

Let's assume for now that you have defined a new type of stream (called
@code{"mystream"}). To keep the example simple, we will assume this stream
also redirects to a file. For flexibility, however, you want to let the user
configure the file name from the traces configuration file. Here is an
example of a configuration file that sets the default stream to a file
called @file{foo}, and redirects a specific handle to another file called
@file{bar}. Note how the same syntax that was used for standard output and
standard error is also reused (ie the stream name starts with the @code{"&"}
symbol, to avoid confusion with standard file names).

@CODESAMPLE{
>&mystream:foo@NL{}
MODULE=yes >&mystream:bar}

You need of course to do a bit of coding in Ada to create the stream. This
is done by creating a new child of @code{Trace_Stream_Record}, and override
the two primitive operations @code{Put} and @code{Newline} (at least).
In this implementation, and because @code{GNATCOLL.Traces.Trace} takes care of
not outputting two messages at the same time, we can just output to the
file as characters are made available. In some other cases, however,
the implementation will need to buffer the characters until the end of
line is seen, and output the line with a single call. See for instance
the implementation of @code{GNATCOLL.Traces.Syslog}, which needs to do
exactly that.

@CODESAMPLE{
@b{type} My_Stream @b{is new} Trace_Stream_Record @b{with record}@NL{}
   File : @b{access} File_Type;@NL{}
@b{end record};@NL{}
@b{procedure} Put@NL{}
  (Stream : @b{in out} My_Stream; Str : String) @b{is}@NL{}
@b{begin}@NL{}
  Put (Stream.File.all, Str);@NL{}
@b{end} Put;@NL{}
@b{procedure} Newline (Stream : @b{in out} My_Stream) @b{is}@NL{}
@b{begin}@NL{}
  New_Line (Stream.File.all);@NL{}
@b{end} Newline;@NL{}}

The above code did not open the file itself, as you might have noticed,
nor did it register the name @code{"mystream"} so that it can be used in
the configuration file. All this is done by creating a factory, ie a
function in charge of creating the new stream. This function receives
in parameter the argument specified by the user in the configuration file
(after the @code{":"} character, if any), and must return a newly
allocated stream. This function is also never called twice with the
same argument, since @code{GNATCOLL.Traces} automatically reuses an existing
stream when one with the same name and arguments already exists.

@CODESAMPLE{
@b{function} Factory (Args : String) @b{return} Trace_Stream @b{is}@NL{}
   Str : access My_Stream := @b{new} My_Stream;@NL{}
@b{begin}@NL{}
   Str.File := @b{new} File_Type;@NL{} 
   Open (Str.File, Out_File, Args);@NL{}
   @b{return} Str;@NL{}
@b{end} Factory;@NL{}
@NL{}
Register_Stream_Factory ("mystream", Factory'Access);}

@c ------------------------------------------------------------------------
@node Logging to syslog
@section Logging to syslog
@cindex syslog
@c ------------------------------------------------------------------------
@noindent

@cindex gnat.traces.syslog
Among the predefined streams, @value{gnatcoll} gives access to the system
logger @code{syslog}. This is a standard utility on all Unix systems, but is
not available on other systems. When you compile @value{gnatcoll}, you should
specify the switch @code{--enable-syslog} to configure to activate the
support. If either this switch wasn't specified, or configure could not find
the relevant header files anyway, then support for @code{syslog} will not
be available. In this case, the package @code{GNATCOLL.Traces.Syslog} is still
available, but contains a single function that does nothing. If your
configuration files redirect some trace handles to @code{"syslog"}, they will
instead be redirect to the default stream or to standard output.

Activating support for syslog requires the following call in your application:

@CODESAMPLE{GNATCOLL.Traces.Syslog.Register_Syslog_Stream;}

This procedure is always available, whether your system supports or not
syslog, and will simply do nothing if it doesn't support syslog. This means
that you do not need to have conditional code in your application to handle
that, and you can let @value{gnatcoll} take care of this.

After the above call, trace handles can be redirected to a stream named
@code{"syslog"}.

The package @code{GNATCOLL.Traces.Syslog} also contains a low-level interface
to syslog, which, although fully functional, you should probably not use,
since that would make your code system-dependent.

Syslog itself dispatches its output based on two criteria: the
@code{facility}, which indicates what application emitted the message,
and where it should be filed, and the @code{level} which indicates the
urgency level of the message. Both of these criteria can be specified in
the @code{GNATCOLL.Traces} configuration file, as follows:

@CODESAMPLE{
  MODULE=yes >&syslog:user:error
}

The above configuration will redirect to a facility called @code{user},
with an urgency level @code{error}. See the enumeration types in
@file{gnatcoll-traces-syslog.ads} for more information on valid facilities
and levels.

@c ------------------------------------------------------------------------
@node Dynamically disabling features
@section Dynamically disabling features
@c ------------------------------------------------------------------------
@noindent

Although the trace handles are primarily meant for outputting messages,
they can be used in another context. The goal is to take advantage of
the external configuration file, without reimplementing a similar
feature in your application. Since the configuration file can be used to
activated or de-activated a handle dynamically, you can then have
conditional sections in your application that depends on that handle,
as in the following example:

@CODESAMPLE{
CONDITIONAL=yes
}
@noindent
and in the Ada code:

@CODESAMPLE{
@b{package} Pkg @b{is}@NL{}
   Me : @b{constant} Trace_Handle := Create ("CONDITIONAL");@NL{}
@b{begin}@NL{}
   @b{if} Active (Me) @b{then}@NL{}
      ... conditional code@NL{}
   @b{end if};@NL{}
@b{end} Pkg;@NL{}}

In particular, this can be used if you write your own decorators, as
explained above.

@c -----------------------------------------------------------------------
@node Monitoring memory
@chapter Monitoring memory
@c -----------------------------------------------------------------------
@noindent

The GNAT compiler allocates and deallocates all memory either through
type-specific debug pools that you have defined yourself, or defaults to
the standard malloc and free system calls. However, it calls those through
an Ada proxy, in the package @code{System.Memory} that you can also
replace in your own application if need be.

@code{gnatcoll} provides such a possible replacement. Its implementation
is also based on @code{malloc} and @code{free}, but if you so chose you
can activate extra monitoring capabilities to help you find out which parts
of your program is allocating the most memory, or where memory is allocated
at any moment in the life of your application.

This package is called @code{GNATCOLL.Memory}. To use it requires a bit of
preparation in your application:

@itemize @bullet
@item You need to create your own version of @file{s-memory.adb} with the
template below, and put it somewhere in your source path. This file should
contain the following bit of code

@CODESAMPLE{
@b{with} GNATCOLL.Memory;@NL{}
@b{package body} System.Memory @b{is}@NL{}
   @b{package} M @b{renames} GNATCOLL.Memory;@NL{}
@NL{}
   @b{function} Alloc (Size : size_t) @b{return} System.Address @b{is}@NL{}
   @b{begin}@NL{}
      @b{return} M.Alloc (M.size_t (Size));@NL{}
   @b{end} Alloc;@NL{}
@NL{}
   @b{procedure} Free (Ptr : System.Address)@NL{}
      @b{renames} M.Free;@NL{}
@NL{}
   @b{function} Realloc@NL{}
     (Ptr  : System.Address;@NL{}
      Size : size_t)@NL{}
      @b{return} System.Address is@NL{}
   @b{begin}@NL{}
      @b{return} M.Realloc (Ptr, M.size_t (Size));@NL{}
   @b{end} Realloc;@NL{}
@b{end} System.Memory;@NL{}
}

@item You then need to compile your application with the extra switch
@code{-a} passed to @code{gnatmake} or @code{gprbuild}, so that this
file is appropriately compiled and linked with your application

@item If you only do this, the monitor is disabled by default. This
basically has zero overhead for your application (apart from the initial
small allocation of some internal data). When you call the procedure
@code{GNATCOLL.Memory.Configure} to activate the monitor, each memory
allocation or deallocation will result in extra overhead that will slow
down your application a bit. But at that point you can then get access
to the information stored in the monitor

@end itemize

We actually recommend that the activation of the monitor be based on an
environment variable or command line switch of your application, so that
you can decide at any time to rerun your application with the monitor
activated, rather than have to go through an extra recompilation.

All allocations and deallocations are monitor automatically when this
module is activated. However, you can also manually call
@code{GNATCOLL.Memory.Mark_Traceback} to add a dummy entry in the
internal tables that matches the current stack trace. This is helpful
for instance if you want to monitor the calls to a specific subprogram,
and know both the number of calls, and which callers executed it how
many times. This can help find hotspots in your application to optimize
the code.

The information that is available through the monitor is the list of
all chunks of memory that were allocated in Ada (this does not include
allocations done in other languages like C). These chunks are grouped
based on the stack trace at the time of their invocation, and this
package knows how many times each stack trace executed each allocation.

As a result, you can call the function @code{GNATCOLL.Memory.Dump} to
dump on the standard output various types of data, sorted. To limit the
output to a somewhat usable format, @code{Dump} asks you to specify
how many blocks it should output.

@table @b
@item Memory usage
Blocks are sorted based on the amount of memory they have allocated and
is still allocated. This helps you find which part of your application
is currently using the most memory.

@item Allocations count
Blocks are sorted based on the number of allocation that are still
allocated. This helps you find which part of your application has done
the most number of allocations (since malloc is a rather slow system
call, it is in general a good idea to try and reduce the number of
allocations in an application).

@item Total number of allocations
This is similar to the above, but includes all allocations ever done
in this block, even if memory has been deallocated since then.

@item Marked blocks
These are the blocks that were created through your calls to
@code{GNATCOLL.Memory.Mark_Traceback}. They are sorted by the number
of allocation for that stacktrace, and also shows you the total number
of such allocations in marked blocks. This is useful to monitor and
analyze calls to specific places in your code

@end table

@c -----------------------------------------------------------------------
@node Reading and Writing Files
@chapter Reading and Writing Files
@cindex mmap
@c -----------------------------------------------------------------------
@noindent

Most applications need to efficiently read files from the disk. Some also
need in addition to modify them and write them back. The Ada run-time
profiles several high-level functions to do so, most notably in the
@file{Ada.Text_IO} package. However, these subprograms require a lot of
additional housekeeping in the run-time, and therefore tend to be slow.

GNAT provides a number of low-level functions in its @file{GNAT.OS_Lib}
package. These are direct import of the usual C system calls @code{read()},
@code{write()} and @code{open()}. These are much faster, and suitable for
most applications.

However, if you happen to manipulate big files (several megabytes and much
more), these functions are still slow. The reason is that to use @code{read}
you basically need a few other system calls: allocate some memory to
temporarily store the contents of the file, then read the whole contents of
the file (even if you are only going to read a small part of it, although
presumably you would use @code{lseek} in such a case).

On most Unix systems, there exists an additional system call @code{mmap()}
which basically replaces @code{open}, and makes the contents of the file
immediately accessible, in the order of a few micro-seconds. You do not
need to allocate memory specifically for that purpose. When you access
part of the file, the actual contents is temporarily mapped in memory
by the system. To modify the file, you just modify the contents of the
memory, and do not worry about writing the file back to the disk.

When your application does not need to read the whole contents of the file,
the speed up can be several orders of magnitude faster than @code{read()}.
Even when you need to read the whole contents, using @code{mmap()} is
still two or three times faster, which is especially interesting on big
files.

@value{gnatcoll}'s @code{GNATCOLL.Mmap} package provides a high-level abstraction
on top of the @code{mmap} system call. As for most other packages in
@value{gnatcoll}, it also nicely handles the case where your system does not
actually support @code{mmap}, and will in that case fallback on using
@code{read} and @code{write} transparently. In such a case, your application
will perform a little slower, but you do not have to modify your code to
adapt it to the new system.

Due to the low-level C API that is needed underneath, the various subprograms
in this package do not directly manipulate Ada strings with valid bounds.
Instead, a new type @code{Str_Access} was defined. It does not contain the
bounds of the string, and therefore you cannot use the usual
@code{'First} and @code{'Last} attributes on that string. But there are other
subprograms that provide those values.

Here is how to read a whole file at once. This is what your code will use
in most cases, unless you expect to read files bigger than @code{Integer'Last}
bytes long. In such cases you need to read chunks of the file separately.
The @code{mmap} system call is such that its performance does not depend on
the size of the file your are mapping. Of course, this could be a problem if
@code{GNATCOLL.Mmap} falls back on calling @code{read}, since in that case it
needs to allocate as much memory as your file. Therefore in some cases you
will also want to only read chunks of the file at once.

@CODESAMPLE{
@b{declare}@NL{}
   File : Mapped_File;@NL{}
   Str  : Str_Access;@NL{}
@b{begin}@NL{}
   File := Open_Read ("/tmp/file_on_disk");@NL{}
   Read (File);  @i{--  read the whole file}@NL{}
   Str := Data (File);@NL{}
   @b{for} S @b{in} 1 .. Last (File) @b{loop}@NL{}
       Put (Str (S));@NL{}
   @b{end loop};@NL{}
   Close (File);@NL{}
@b{end};@NL{}
}

To read only a chunk of the file, your code would look like the following.
At the low-level, the system call will always read chunks multiple of a
size called the page_size. Although @code{GNATCOLL.Mmap} takes care of rounding
the numbers appropriately, it is recommended that you pass parameters that
are multiples of that size. That optimizes the number of system calls you
will need to do, and therefore speeds up your application somewhat.

@CODESAMPLE{
@b{declare}@NL{}
   File   : Mapped_File;@NL{}
   Str    : Str_Access;@NL{}
   Offs   : Long_Integer := 0;@NL{}
   Page   : @b{constant} Integer := Get_Page_Size;@NL{}
@b{begin}@NL{}
   File := Open_Read ("/tmp/file_on_disk");@NL{}
   @b{while} Offs < Length (File) @b{loop}@NL{}
       Read (File, Offs, Length => Long_Integer (Page) * 4);@NL{}
       Str := Data (File);@NL{}
@NL{}
       @i{--  Print characters for this chunk:}@NL{}
       @b{for} S @b{in} Integer (Offs - Offset (File)) + 1 .. Last (File) @b{loop}@NL{}
          Put (Str (S));@NL{}
       @b{end loop};@NL{}
@NL{}
       Offs := Offs + Long_Integer (Last (File));@NL{}
   @b{end loop};@NL{}
   Close (File);@NL{}
}

There are a number of subtle details in the code above. Since the system call
only manipulates chunk of the file on boundaries multiple of the code size,
there is no guarantee that the part of the file we actually read really starts
exactly at @var{Offs}. If could in fact start before, for rounding issues.
Therefore when we loop over the contents of the buffer, we make sure to
actually start at the @var{Offs}-th character in the file.

In the particular case of this code, we make sure we only manipulate multiples
of the page_size, so we could in fact replace the loop with the simpler

@CODESAMPLE{
 @b{for} S @b{in} 1 .. Last (File) @b{loop} @NL{}
}

If you intend to modify the contents of the file, not that @code{GNATCOLL.Mmap}
currently gives you no way to change the size of the file. The only difference
compared to the code used for reading the file is the call to open the file,
which should be

@CODESAMPLE{
    File := Open_Write ("/tmp/file_on_disk");
}

Modifications to Str are automatically reflected in the file. However, there
is no guarantee this saving is done immediately. It could be done only when
you call @code{Close}. This is in particular always the case when your system
does not support @code{mmap} and @code{GNATCOLL.Mmap} had to fallback on calls to
@code{read}.

@c -----------------------------------------------------------------------
@node Searching strings
@chapter Searching strings
@cindex Boyer-Moore
@cindex search
@c -----------------------------------------------------------------------
@noindent

Although the Ada standard provides a number of string-searching subprograms
(most notably in the @code{Ada.Strings.Fixed}, @code{Ada.Strings.Unbounded}
and @code{Ada.Strings.Bounded} packages through the @code{Index} functions),
these subprograms do not in general provide the most efficient algorithms
for searching strings.

The package @code{GNATCOLL.Boyer_Moore} provides one such optimize algorithm,
although there exists several others which might be more efficient depending
on the pattern.

It deals with string searching, and does not handle regular expressions for
instance.

This algorithm needs to preprocess its key (the searched string), but does
not need to perform any specific analysis of the string to be searched.
Its execution time can be sub-linear: it doesn't need to actually check
every character of the string to be searched, and will skip over some of
them. The worst case for this algorithm has been proved to need approximately
3 * N comparisons, hence the algorithm has a complexity of O(n).

The longer the key, the faster the algorithm in general, since that provides
more context as to how many characters can be skipped when a non-matching
character is found..

We will not go into the details of the algorithm, although a general
description follows: when the pattern is being preprocessed, Boyer-Moore
computes how many characters can be skipped if an incorrect match is
found at that point, depending on which character was read.
In addition, this algorithm tries to match the key starting from its end,
which in general provides a greater number of characters to skip.

For instance, if you are looking for "ABC" in the string "ABDEFG" at the
first position, the algorithm will compare "C" and "D". Since "D" does not
appear in the key "ABC", it knows that it can immediately skip 3 characters
and start the search after "D".

Using this package is extremely easy, and it has only a limited API.

@CODESAMPLE{
@b{declare}@NL{}
  Str : @b{constant} String := "ABDEABCFGABC";@NL{}
  Key : Pattern;@NL{}
  Index : Integer;@NL{}
@b{begin}@NL{}
  Compile (Key, "ABC");@NL{}
  Index := Search (Key, Str);@NL{}
@b{end}@NL{}
}

@code{Search} will either return -1 when the pattern did not match, or
the index of the first match in the string. In the example above, it
will return 5.

If you want to find the next match, you have to pass a substring to
search, as in

@CODESAMPLE{
  Index := Search (Key, Str (6 .. Str'Last));@NL{}
}

@c -----------------------------------------------------------------------
@node The templates module
@chapter The templates module
@cindex templates
@c -----------------------------------------------------------------------
@noindent
This module provides convenient subprograms for replacing specific
substrings with other values. It is typically used to replace substrings
like "%@{version@}" in a longer string with the actual version, at run time.

This module is not the same as the templates parser provided in the context
of AWS, the Ada web server, where external files are parsed and processed
to generate other files. The latter provides advanced features like filters,
loops,@dots{}

The substrings to be replaced always start with a specific delimiter, which
is set to @code{%} by default, but can be overridden in your code. The name
of the substring to be replaced is then the identifier following that
delimiter, with the following rules:

@itemize @bullet
@item If the character following the delimiter is the delimiter itself,
 then the final string will contain a single instance of that delimiter, and
 no further substitution is done for that delimiter. An example of this is
 @code{"%%"}.

@item If the character immediately after the delimiter is a curly brace
 (@code{@{}), then the name of the identifier is the text until the next
 closing curly brace. It can then contain any character expect a closing
 curly brace. An example of this is @code{"%@{long name@}"}

@item If the first character after the delimiter is a digit, then the
 name of the identifier is the number after the delimiter. An example of
 this is @code{"%12"}. As a special case, if the first non-digit
 character is the symbol @code{-}, it is added as part of the name of the
 identifier, as in @code{"%1-"}. One use for this feature is to indicate
 you want to replace it with all the positional parameters %1%2%3%4. For
 instance, if you are writing the command line to spawn an external tool,
 to which the user can pass any number of parameter, you could specify that
 command line as @code{"tool -o %1 %2-"} to indicate that all parameters
 should be concatenated on the command line.

@item If the first character after the delimiter is a letter, the identifier
 follows the same rules as for Ada identifiers, and can contain any letter,
 digit, or underscore character. An example of this is @code{"%ab_12"}. For
 readability, it is recommended to use the curly brace notation when the
 name is complex, but that is not mandatory.

@item Otherwise the name of the identifier is the single character
 following the delimiter

@end itemize

For each substring matching the rules above, the @code{Substitute} subprogram
will look for possible replacement text in the following order:

@itemize @bullet
@item If the @code{Substrings} parameter contains an entry for that name,
 the corresponding value is used.

@item Otherwise, if a @code{callback} was specified, it is called with the
 name of the identifier, and should return the appropriate substitution (or
 raise an exception if no such substitution makes sense).

@item A default value provided in the substring itself

@item When no replacement string was found, the substring is kept unmodified

@end itemize

@c -----------------------------------------------------------------------
@node Managing Email
@chapter Managing Email
@cindex email
@c -----------------------------------------------------------------------
@noindent

@value{gnatcoll} provides a set of packages for managing and processing
email messages. Through this packages, you can extract the various messages
contained in an existing mailbox, extract the various components of a message,
editing previously parsed messages, or create new messages from scratch.

This module fully supports MIME-encoded messages, with attachments.

This module currently does not provide a way to send the message through the
SMTP protocol. Rather, it is used to create an in-memory representation of
the message, which you can then convert to a string, and pass this to a
socket. See for instance the AWS library
(@url{http://www.adacore.com/home/gnatpro/add-on_technologies/web_technologies})
which contains the necessary subprograms to connect with an SMTP server.

@menu
* Message formats::
* Parsing messages::
* Parsing mailboxes::
* Creating messages::
@end menu

@c -----------------------------------------------------------------------
@node Message formats
@section Message formats
@cindex GNATCOLL.Email.Utils
@c -----------------------------------------------------------------------
@noindent

The format of mail messages is defined through numerous RFC documents.
@value{gnatcoll} tries to conform to these as best as possible. Basically,
a message is made of two parts:

@table @bullet
@item The headers
These are various fields that indicate who sent the message, when, to whom,
and so on

@item The payload (aka body)
This is the actual contents of the message. It can either be a simple text,
or made of one or more attachments in various formats. These attachments can
be HTML text, images, or any binary file. Since email transfer is done through
various servers, the set of bytes that can be sent is generally limited to
7 bit characters. Therefore, the attachments are generally encoded through one
of the encoding defined in the various MIME RFCs, and they need to be decoded
before the original file can be manipulated again.

@end table

@value{gnatcoll} gives you access to these various components, as will be
seen in the section @pxref{Parsing messages}.

@cindex MIME
@cindex encoding
The package @file{GNATCOLL.Email.Utils} contains various subprograms to decode
MIME-encoded streams, which you can use independently from the rest of the
packages in the email module.

The headers part of the message contains various pieces of information about
the message. Most of the headers have a well-defined semantics and format.
However, a user is free to add new headers, which will generally start with
@code{X-} prefix. For those fields where the format is well-defined, they
contain various pieces of information:

@table @bullet
@item Email addresses
The @code{From}, @code{TO} or @code{CC} fields, among others, contain
list of recipients. These recipients are the usual email addresses. However,
the format is quite complex, because the full name of the recipient can also
be specified, along with comments. The package @file{GNATCOLL.Email.Utils}
provides various subprograms for parsing email addresses and list of
recipients.

@item Dates
The @code{Date} header indicates when the message was sent. The format of the
date is also precisely defined in the RFC, and the package
@file{GNATCOLL.Email.Utils} provides subprograms for parsing this date (or,
on the contrary, to create a string from an existing time).

@item Text
The @code{Subject} header provides a brief overview of the message. It is
a simple text header. However, one complication comes from the fact that the
user might want to use extended characters not in the ASCII subset. In such
cases, the Subject (or part of it) will be MIME-encoded. The package
@file{GNATCOLL.Email.Utils} provides subprograms to decode MIME-encoded strings,
with the various charsets.

@end table

@c -----------------------------------------------------------------------
@node Parsing messages
@section Parsing messages
@c -----------------------------------------------------------------------
@noindent

There are two ways a message is represented in memory: initially, it is
a free-form @code{String}. The usual Ada operations can be used on the string,
of course, but there is no way to extract the various components of the
message. For this, the message must first be parsed into an instance of the
@code{Message} type.

This type is controlled, which means that the memory will be freed
automatically when the message is no longer needed.

@cindex GNATCOLL.Email.Parser
The package @file{GNATCOLL.Email.Parser} provides various subprograms that
parse a message (passed as a string), and create a @code{Message} out of it.
Parsing a message might be costly in some cases, for instance if a big
attachment needs to be decoded first. In some cases, your application will
not need that information (for instance you might only be looking for a few
of the headers of the message, and not need any information from the body).
This efficiency concern is why there are multiple parsers. Some of them will
ignore parts of the message, and thus be more efficient if you can use them.

@cindex GNATCOLL.Email
Once a @code{Message} has been created, the subprograms in
@code{GNATCOLL.Email}
can be used to access its various parts.
The documentation for these subprograms is found in the file
@code{gnatcoll-email.ads} directly, and is not duplicated here.

@c -----------------------------------------------------------------------
@node Parsing mailboxes
@section Parsing mailboxes
@c -----------------------------------------------------------------------
@noindent

Most often, a message is not found on its own (unless you are for instance
writing a filter for incoming messages). Instead, the messages are stored
in what is called a mailbox. The latter can contain thousands of such
messages.

There are traditionally multiple formats that have been used for mailboxes.
At this stage, @value{gnatcoll} only supports one of them, the @code{mbox}
format. In this format, the messages are concatenated in a single file,
and separated by a newline.

@cindex GNATCOLL.Email.Mailboxes
The package @code{GNATCOLL.Email.Mailboxes} provides all the types and
subprograms
to manipulate mailboxes.
Tagged types are used, so that new formats of mailboxes can relatively easily
be added later on, or in your own application.

Here is a small code example that opens an mbox on the disk, and parses each
message it contains

@CODESAMPLE{
@b{declare}@NL{}
  Box  : Mbox;@NL{}
  Curs : Cursor;@NL{}
  Msg  : Message;@NL{}
@b{begin}@NL{}
  Open (Box, Filename => "my_mbox");@NL{}
  Curs := Mbox_Cursor (First (Box));@NL{}
  @b{while} Has_Element (Curs) @b{loop}@NL{}
     Get_Message (Curs, Box, Msg);@NL{}
     @b{if} Msg /= Null_Message @b{then}@NL{}
        ...@NL{}
     @b{end if};@NL{}
     Next (Curs, Box);
  @b{end loop};@NL{}
@b{end;}@NL{}
}

As you can see, the mailbox needs to be opened first. Then we get an
iterator (called a cursor, to match the Ada2005 containers naming scheme),
and we then parse each message. The @code{if} test is optional, but
recommended: the message that is returned might be null if the mailbox
was corrupted and the message could not be parsed. There are still chances
that the next message will be readable, so only the current message should
be ignored.

@c -----------------------------------------------------------------------
@node Creating messages
@section Creating messages
@c -----------------------------------------------------------------------
@noindent

The subprograms in @code{GNATCOLL.Email} can also be used to create a message
from scratch. Alternatively, if you have already parsed a message, you
can alter it, or easily generate a reply to it (using the @code{Reply_To}
subprogram. The latter will preset some headers, so that message threading
is preserved in the user's mailers.

@c ------------------------------------------------------------------------
@node Ravenscar Patterns
@chapter Ravenscar Patterns
@cindex ravenscar
@c ------------------------------------------------------------------------
@noindent

@value{gnatcoll} provides a set of patterns for concurrent programming using
Ravenscar-compliant semantics only. The core goal of the GNATCOLL.Ravenscar
(sub) packages is to ease the development of high-integrity multitasking
applications by factorizing common behavior into instantiable,
Ravenscar-compliant, generic packages. Instances of such generic packages
guarantee predictable timing behavior and thus permit the application of most
common timing analysis techniques.  

@menu
* Tasks::
* Servers::
* Timers::
@end menu

@node Tasks
@section Tasks
The @code{GNATCOLL.Ravenscar.Simple_Cyclic_Task} generic package lets
instantiate a cyclic tasks executing the same operation at regular time
intervals; on the other side, the
@code{GNATCOLL.Ravenscar.Simple_Sporadic_Task} task lets instantiate sporadic
tasks enforcing a minimum inter-release time.


@node Servers
@section Servers
Servers present a more sophisticated run-time semantics than tasks: for example, they 
can fulfill different kind of requests (see multiple queues servers). 
@code{Gnat.Ravenscar.Sporadic_Server_With_Callback} and
 @code{Gnat.Ravenscar.Timed_Out_Sporadic_Server} are particularly interesting. The former shows 
how synchronous inter-task communication can be faked in Ravenscar (the only form of communication
permitted by the profile is through shared resources): the server receives a request to fulfill, 
computes the result and returns it by invoking a call-back. The latter enforces both a minimum 
and a maximum inter-release time: the server automatically releases itself and invokes an appropriate
handler if a request is not posted within a given period of time. 


@node Timers
@section Timers
@code{Gnat.Ravenscar.Timers.One_Shot_Timer} is the Ravenscar implementation of time-triggered event through
Ada 2005 Timing Events.


@c -----------------------------------------------------------------------
@node Managing Memory
@chapter Managing Memory: The storage pools
@c -----------------------------------------------------------------------
@noindent

Ada gives full control to the user for memory management. That allows for
a number of optimization in your application. For instance, if you need to
allocate a lot of small chunks of memory, it is generally more efficient
to allocate a single large chunk, which is later divided into smaller
chunks. That results in a single system call, which speeds up your
application.

This can of course be done in most languages. However, that generally
means you have to remember not to use the standard memory allocations
like @code{malloc} or @code{new}, and instead call one of your
subprograms. If you ever decide to change the allocation strategy, or
want to experiment with several strategies, that means updating your
code in several places.

In Ada, when you declare the type of your data, you also specify through
a @code{'Storage_Pool} attribute how the memory for instances of that
type should be allocated. And that's it. You then use the usual
@code{new} keyword to allocate memory.

@value{gnatcoll} provides a number of examples for such storage pools,
with various goals. There is also one advanced such pool in the GNAT
run-time itself, called @code{GNAT.Debug_Pools}, which allows you to
control memory leaks and whether all accesses do reference valid memory
location (and not memory that has already been deallocated).

In @value{gnatcoll}, you will find the following storage pools:

@table @bullet
@item @code{GNATCOLL.Storage_Pools.Alignment}

This pool gives you full control over the alignment of your data. In
general, Ada will only allow you to specify alignments up to a limited
number of bytes, because the compiler must only accept alignments
that can be satisfied in all contexts, in particular on the stack.

This package overcomes that limitation, by allocating larger chunks
of memory than needed, and returning an address within that chunk which
is properly aligned.

@end table

@c -----------------------------------------------------------------------
@node Manipulating Files
@chapter Manipulating Files
@c -----------------------------------------------------------------------
@noindent

Ada was meant from the beginning to be a very portable language, across
architectures. As a result, most of the code you write on one machine has
good chances of working as is on other machines. There remains, however,
some areas that are somewhat system specific. The Ada run-time, the GNAT
specific run-time and @value{gnatcoll} all try to abstract some of those
operations to help you make your code more portable.

One of these areas is related to the way files are represented and
manipulated. Reading or writing to a file is system independent, and taken
care of by the standard run-time. Other differences between systems include
the way file names are represented (can a given file be accessed through
various casing or not, are directories separated with a backslash or a
forward slash, or some other mean, and a few others). The GNAT run-time does
a good job at providing subprograms that work on most types of filesystems,
but the relevant subprograms are split between several packages and not always
easy to locate. @value{gnatcoll} groups all these functions into a single
convenient tagged type hierarchy. In addition, it provides the framework for
transparently manipulating files on other machines.

Another difference is specific to the application code: sometimes, a
subprogram needs to manipulate the base name (no directory information) of
a file, whereas sometimes the full file name is needed. It is somewhat hard
to document this in the API, and certainly fills the code with lots of
conversion from full name to base name, and sometimes reverse (which, of
course, might be an expansive computation). To make this easier,
@value{gnatcoll} provides a type that encapsulates the notion of a file,
and removes the need for the application to indicate whether it needs a
full name, a base name, or any other part of the file name.

@menu
Manipulating Files
* Filesystems::
* Remote filesystems::
* Virtual files::
* GtkAda support for virtual files::
@end menu

@c -----------------------------------------------------------------------
@node Filesystems
@section Filesystems abstraction
@c -----------------------------------------------------------------------
@noindent

There exists lots of different filesystems on all machines. These include
such things as FAT, VFAT, NTFS, ext2, VMS,@dots{}. However, all these can
be grouped into three families of filesystems:

@itemize @bullet
@item windows-based filesystems

On such filesystems, the full name of a file is split into three parts: the
name of the drive (c:, d:,@dots{}), the directories which are separated by
a backslash, and the base name. Such filesystems are sometimes inaccurately
said to be case insensitive: by that, one means that the same file can be
accessed through various casing. However, a user is generally expecting a
specific casing when a file name is displayed, and the application should
strive to preserve that casing (as opposed to, for instance, systematically
convert the file name to lower cases).

A special case of a windows-based filesystems is that emulated by the
cygwin development environment. In this case, the filesystem is seen as if
it was unix-based (see below), with one special quirk to indicate the drive
letter (the file name starts with "/cygwin/c/").

@item unix-based filesystems

On such filesystems, directories are separated by forward slashed. File
names are case sensitive, that is a directory can contain both "foo" and
"Foo", which is not possible on windows-based filesystems.

@item vms filesystem

This filesystem represents path differently than the other two, using
brackets to indicate parent directories

@end itemize

A given machine can actually have several file systems in parallel, when
a remote disk is mounted through NFS or samba for instance. There is
generally no easy way to guess that information automatically, and it
generally does not matter since the system will convert from the native file
system to that of the remote host transparently (for instance, if you mount
a windows disk on a unix machine, you access its files through forward slash-
separated directory names).

@value{gnatcoll} abstracts the differences between these filesystems through
a set of tagged types in the @code{GNATCOLL.Filesystem} package and its
children. Such a type has primitive operations to manipulate the names of
files (retrieving the base name from a full name for instance), to check
various attributes of the file (is this a directory, a symbolic link, is the
file readable or writable), or to
manipulate the file itself (copying, deleting, reading and writing).
It provides similar operations for directories (creating or deleting paths,
reading the list of files in a directory,@dots{}).

It also provides information on the system itself (the list of available drives
on a windows machine for instance).

The root type @code{Filesystem_Record} is abstract, and is specialized in
various child types. A convenient factory is provided to return the filesystem
appropriate for the local machine (@code{Get_Local_Filesystem}), but you
might chose to create your own factory in your application if you have
specialized needs (@pxref{Remote filesystems}).

@subsection file names encoding

One delicate part when dealing with filesystems is handling files whose
name cannot be described in ASCII. This includes names in asian languages
for instance, or names with accented letters.

There is unfortunately no way, in general, to know what the encoding is for
a filesystem. In fact, there might not even be such an encoding (on linux,
for instance, one can happily create a file with a chinese name and another
one with a french name in the same directory). As a result, @value{gnatcoll}
always treats file names as a series of bytes, and does not try to assume
any specific encoding for them. This works fine as long as you are
interfacing the system (since the same series of bytes that was returned by
it is also used to access the file later on).

However, this becomes a problem when the time comes to display the name for
the user (for instance in a graphical interface). At that point, you need to
convert the file name to a specific encoding, generally UTF-8 but not
necessarily (it could be ISO-8859-1 in some cases for instance).

Since @value{gnatcoll} cannot guess whether the file names have a specific
encoding on the file system, or what encoding you might wish in the end, it
lets you take care of the conversion. To do so, you can use either of the
two subprograms @code{Locale_To_Display} and
@code{Set_Locale_To_Display_Encoder}

@c -----------------------------------------------------------------------
@node Remote filesystems
@section Remote filesystems
@c -----------------------------------------------------------------------
@noindent

Once the abstract for filesystems exists, it is tempting to use it to
access files on remote machines. There are of course lots of differences
with filesystems on the local machine: their names are manipulated
similarly (although you need to somehow indicate on which host they are
to be found), but any operation of the file itself needs to be done on the
remote host itself, as it can't be done through calls to the system's
standard C library.

Note that when we speak of disks on a remote machine, we indicate disks
that are not accessible locally, for instance through NFS mounts or samba.
In such cases, the files are accessed transparently as if they were local,
and all this is taken care of by the system itself, no special layer is
needed at the application level.

@value{gnatcoll} provides an extensive framework for manipulating such
remote files. It knows what commands need to be run on the remote host to
perform the operations ("cp" or "copy", "stat" or "dir /a-d",...) and
will happily perform these operations when you try to manipulate such
files.

There are however two operations that your own application needs to take
care of to take full advantage of remote files.

@subsection Filesystem factory

@value{gnatcoll} cannot know in advance what filesystem is running on the
remote host, so it does not try to guess it. As a result, your application
should have a factory that creates the proper instance of a
@code{Filesystem_Record} depending on the host. Something like:

@CODESAMPLE{
@b{type} Filesystem_Type @b{is} (Windows, Unix);@NL{}
@b{function} Filesystem_Factory@NL{}
  (Typ  : Filesystem_Type;@NL{}
   Host : String)@NL{}
  @b{return} Filesystem_Access@NL{}
@b{is}@NL{}
   FS : Filesystem_Access;@NL{}
@b{begin}@NL{}
   @b{if} Host = "" @b{then}@NL{}
     @b{case} Typ @b{is}@NL{}
       @b{when} Unix =>@NL{}
         FS := @b{new} Unix_Filesystem_Record;@NL{}
       @b{when} Windows =>@NL{}
         FS := @b{new} Windows_Filesystem_Record;@NL{}
     @b{end case};@NL{}
   @b{else}@NL{}
     @b{case} Typ @b{is}@NL{}
       @b{when} Unix =>@NL{}
         FS := @b{new} Remote_Unix_Filesystem_Record;@NL{}
         Setup (Remote_Unix_Filesystem_Record (FS.all),@NL{}
                Host      => Host,@NL{}
                Transport => ...); @i{--  see below}@NL{}
       @b{when} Windows =>@NL{}
         FS := new Remote_Windows_Filesystem_Record;@NL{}
         Setup (Remote_Windows_Filesystem_Record (FS.all),@NL{}
                Host      => Host,@NL{}
                Transport => ...);@NL{}
     @b{end case};@NL{}
   @b{end if};@NL{}
@NL{}
   Set_Locale_To_Display_Encoder@NL{}
     (FS.all, Encode_To_UTF8'Access);@NL{}
   @b{return} FS;@NL{}
@b{end} Filesystem_Factory;}


@subsection Transport layer

There exists lots of protocols to communicate with a remote machine, so as
to be able to perform operations on it. These include protocols such as
@code{rsh}, @code{ssh} or @code{telnet}. In most of these cases, a user
name and password is needed (and will likely be asked to the user).
Furthermore, you might not want to use the same protocol to connect to
different machines.

@value{gnatcoll} does not try to second guess your intention here. It
performs all its remote operations through a tagged type defined in
@code{GNATCOLL.Filesystem.Transport}. This type is abstract, and must be
overridden in your application. For instance, GPS has a full support for
choosing which protocol to use on which host, what kind of filesystem is
running on that host, to recognize password queries from the transport
protocol,@dots{}. All these can be encapsulated in the transport
protocol.

Once you have created one or more children of
@code{Filesystem_Transport_Record}, you associate them with your
instance of the filesystem through a call to the @code{Setup} primitive
operation of the filesystem. See the factory example above.

@c -----------------------------------------------------------------------
@node Virtual files
@section Virtual files
@c -----------------------------------------------------------------------
@noindent

As we have seen, the filesystem type abstracts all the operations for
manipulating files and their names. There is however another aspect when
dealing with file names in an application: it is often unclear whether a
full name (with directories) is expected, or whether the base name itself
is sufficient. There are also some aspects about a file that can be cached
to improve the efficiency.

For these reasons, @value{gnatcoll} provides a new type
@code{GNATCOLL.VFS.Virtual_File} which abstracts the notion of file. It
provides lots of primitive operations to manipulate such files (which
are of course implemented based on the filesystem abstract, so support
files on remote hosts among other advantages), and encapsulate the base
name and the full name of a file so that your API becomes clearer (you
are not expecting just any string, but really a file).

This type is reference counted: it takes care of memory management on
its own, and will free its internal data (file name and cached data)
automatically when the file is no longer needed. This has of course a
slight efficiency cost, due to controlled types, but we have found in
the context of GPS that the added flexibility was well worth it.

@c -----------------------------------------------------------------------
@node GtkAda support for virtual files
@section GtkAda support for virtual files
@c -----------------------------------------------------------------------
@noindent

If you are programming a graphical interface to your application, and the
latter is using the @code{Virtual_File} abstraction all other the place,
it might be a problem to convert back to a string when you store a file
name in a graphical element (for instance in a tree model if you display
an explorer-like interface in your application).

Thus, @value{gnatcoll} provides the @code{GNATCOLL.VFS.GtkAda} package,
which is only build if @code{GtkAda} was detected when @value{gnatcoll}
was compiled, which allows you to encapsulate a @code{Virtual_File}
into a @code{GValue}, and therefore to store it in a tree model.

@c -----------------------------------------------------------------------
@node Three state logic
@chapter Three state logic
@c -----------------------------------------------------------------------
@noindent

Through the package @code{GNATCOLL.Tribooleans}, @value{gnatcoll} provides
a type that extends the classical @code{Boolean} type with an
@code{Indeterminate} value.

There are various cases where such a type is useful. One example we have
is when a user is doing a search (on a database or any set of data), and
can specify some optional boolean criteria ("must the contact be french?").
He can choose to only see french people ("True"), to see no french people
at all ("False"), or to get all contacts ("Indeterminate"). With a classical
boolean, there is no way to cover all these cases.

Of course, there are more advanced use cases for such a type. To support
these cases, the @code{Tribooleans} package overrides the usual logical
operations @code{"and"}, @code{"or"}, @code{"xor"}, @code{"not"} and
provides an @code{Equal} function.

See the specs of the package to see the truth tables associated with those
operators.

@c -----------------------------------------------------------------------
@node Geometry
@chapter Geometry
@c -----------------------------------------------------------------------
@noindent

@value{gnatcoll} provides the package @code{GNATCOLL.Geometry}. This
package includes a number of primitive operations on geometric figures
like points, segments, lines, circles, rectangles and polygons.
In particular, you can compute their intersections, the distances,@dots{}

This package is generic, so that you can specify the type of coordinates
you wish to handle.

@CODESAMPLE{
 declare@NL{}
    package Float_Geometry is new GNATCOLL.Geometry (Float);@NL{}
    use Float_Geometry;@NL{}
@NL{}
    P1 : constant Point := (1.0, 1.0);@NL{}
    P2 : constant Point := (2.0, 3.0);@NL{}
 begin@NL{}
    Put_Line ("Distance P1-P2 is" & Distance (P1, P2)'Img);@NL{}
    --  Will print 2.23607@NL{}
 end;@NL{}
}

Or some operations involving a polygon:

@CODESAMPLE{
   declare@NL{}
      P3 : constant Point := (3.7, 2.0);@NL{}
      P  : constant Polygon :=@NL{}
         ((2.0, 1.3), (4.1, 3.0), (5.3, 2.6), (2.9, 0.7), (2.0, 1.3));@NL{}
   begin@NL{}
      Put_Line ("Area of polygon:" & Area (P));   --   3.015@NL{}
      Put_Line ("P3 inside polygon ? " & Inside (P3, P)'Img);  --  True@NL{}
   end;@NL{}
}

@c -----------------------------------------------------------------------
@node Projects
@chapter Projects
@c -----------------------------------------------------------------------
@noindent

The package @code{GNATCOLL.Projects} provides an extensive interface to
parse, manipulate and edit project files (@file{.gpr} files).

Although the interface is best used using the Ada05 notation, it is fully
compatible with Ada95.

Here is a quick example on how to use the interface, although the spec
file itself contains much more detailed information on all the subprograms
related to the manipulation of project files.


@CODESAMPLE{
@b{with} GNATCOLL.Projects; @b{use} GNATCOLL.Projects;@NL{}
@b{with} GNATCOLL.VFS;      @b{use} GNATCOLL.VFS;@NL{}
@NL{}
Tree  : Project_Tree;@NL{}
Files : File_Array_Access;@NL{}
@NL{}
Tree.Load (GNATCOLL.VFS.Create (+"path_to_project.gpr"));@NL{}
@NL{}
@i{--  List the source files for project and all imported projects}@NL{}
@NL{}
Files := Tree.Root_Project.Source_Files (Recursive => True);@NL{}
@b{for} F @b{in} Files'Range @b{loop}@NL{}
   Put_Line ("File is: " & Files (F).Display_Full_Name);@NL{}
@b{end loop};@NL{}
}

@c -----------------------------------------------------------------------
@node Database interface
@chapter Database interface
@c -----------------------------------------------------------------------
@noindent

@value{gnatcoll} provides an interface to various database systems.
Currently, only PostgreSQL and MySQL are supported, but adding a new
back-end is a matter of extending a tagged type and overriding the
appropriate subprograms.

This interface was designed with several goals in mind: type-safety,
integrity with regards to changes to the database schema, ease of
writing queries and performance. A paper was published at the
Ada-Europe conference in 2008 which describes the various steps
we went through in the design of this library. The rest of this
chapter describes the current status of the library, not its
history.

@menu
* Supported database systems::
* Database schema monitoring::
* Writing queries::
* Executing queries::
* Getting results::
* Writing your own cursors::
* Creating your own SQL types::
* Query logs::
* Tasks and databases::
@end menu

@c -----------------------------------------------------------------------
@node Supported database systems
@section Supported database systems
@c -----------------------------------------------------------------------
@noindent

This library abstracts the specifics of the various database engines
it supports. Ideally, a goal written for one database could be ported
almost transparently to another engine. This is not completely doable
in practice, since each system has its own SQL specifics, and unless
you are writing things very carefully, the interpretation of your queries
might be different from one system to the next.

However, the Ada code should remain untouched if you change the engine.
Various engines are supported out of the box (PostgreSQL and Sqlite),
although new ones can be added by overriding the appropriate SQL type
(@code{Database_Connection}). When you compile @value{gnatcoll}, the
build scripts will try and detect what systems are installed on your
machine, and only build support for those. It is possible, if no
database was installed on your machine at that time, that the database
interface API is available (and your application compiles), but no
connection can be done to database at run time.

In your code, you will need to create a connection to the database
system that you wish to interact with. One possible implementation
is

@CODESAMPLE{
@b{function} Connection_Factory@NL{}
  (Desc : GNATCOLL.SQL.Exec.Database_Description)@NL{}
  @b{return} GNATCOLL.SQL.Exec.Database_Connection@NL{}
@b{is}@NL{}
   DBMS : @b{constant} String := Get_DBMS (Desc);@NL{}
@b{begin}@NL{}
   @b{if} DBMS = DBMS_Postgresql @b{then}@NL{}
      @b{return} GNATCOLL.SQL.Postgres.Build_Postgres_Connection;@NL{}
   @b{elsif} DBMS = DBMS_Sqlite @b{then}@NL{}
      @b{return} GNATCOLL.SQL.Sqlite.Build_Sqlite;@NL{}
   @b{else}@NL{}
      @b{return null};@NL{}
   @b{end if};@NL{}
@b{end} Connection_Factory;@NL{}
@NL{}
@b{declare}@NL{}
   DB_Descr : GNATCOLL.SQL.Exec.Database_Description;@NL{}
   DB : GNATCOLL.SQL.Exec.Database_Connection;@NL{}
@b{begin}@NL{}@NL{}
   GNATCOLL.SQL.Exec.Setup_Database@NL{}
     (Description => DB_Descr,@NL{}
      Database    => "dbname");@NL{}
@NL{}
   DB := GNATCOLL.SQL.Exec.Get_Task_Connection@NL{}
        (Description  => DB_Descr,@NL{}
         Factory      => Connection_Factory'Access,@NL{}
         Username     => "myself");@NL{}
@b{end}@NL{}
}

The code acts in three steps:
@itemize @bullet
@item Describe connection parameters
   The call to @code{Setup_Database} provides the required parameters to
   establish a connection to a database server which might possibly be
   running on a remote host. In this call, one specifies the name of
   the database, the user login and password, and the type of database.
   At this point, no connection or exchange of information has been
   done, the @code{Database_Description} type stores this information
   for later.

@item Connect to the database
   The call to @code{Get_Task_Connection} establishes the actual
   connection. As will be seen later, the recommended practice when
   the database backend supports it is to establish one connection
   per thread in your application, and keep it alive even if the
   thread is reused for another reason later on. An example is a
   web server which has a pool of tasks, and uses the first
   available one to reply to a request. The function
   @code{Get_Task_Connection} will reuse the connection to the
   database that was already established in this thread, or create
   a new one through the @code{Factory} callback if none was
   created yet.

   This call is the only one that is needed in the various tasks
   of your application, you obviously do not need to repeat the
   call to @code{Setup_Database}.

@item Create the connection
   If the current task is not associated with a connection, one
   needs to be created. @value{gnatcoll} does this through a
   factory callback, instead of just doing it on its own, so that
   your application can create its own child types for a
   @code{Database_Connection}, and thus store additional data in
   that child type. This is also a convenient way to support
   multiple database backends without @code{with}-ing all the
   corresponding code in your application (in the example above,
   the application only ever supports PostgreSQL, and therefore
   does not need to elaborate the MySQL packages for instance).

@end itemize

@c -----------------------------------------------------------------------
@node Database schema monitoring
@section Database schema monitoring
@c -----------------------------------------------------------------------
@noindent

As stated in the introduction, one of the goals of this library is to
make sure the application's code follows changes in the schema of your
database. The schema is the list of tables and fields in your database,
and the relationship between those tables.

To reach this goal, an external tool, @file{gnatcoll_db2ada} is provided
with @value{gnatcoll}, and should be spawned as the first step of the
build process, or at least whenever the database schema changes. It
generates an Ada package (@code{Database}) which reflects the current
schema of the database.

This tool supports a number of command line parameters (the complete list
of which is available through the @file{-h} switch). The most important of
those switches are:

@table @code
@item -dbhost host
@itemx -dbname name
@itemx -dbuser user
@itemx -dbpasswd passwd
@itemx -dbtype type
These parameters specify the connection parameters for the database. To
find out the schema, @file{gnatcoll_db2ada} needs to connect to that
database and do a number of read-only queries. The user does not need to
have write permission on the database

@item -dbmodel file
This parameter can replace the above @code{-dbname},... It specifies the
name of a text file that contains the description of the database, therefore
avoiding the need for already having a database up-and-running to generate
the Ada interface.

The format of this text file is the same as generated by the @code{-text}
switch (note also that the parser doesn't try to check for all possible
types of errors, so if you have a syntax error in your file you might end
up with an exception).

This switch is not compatible with @code{-enum} and @code{-vars} that
really need an access to the database.

@item -enum table,id,name,prefix,base
This parameter can be repeated several times if needed. It identifies
one of the special tables of the database that acts as an enumeration
type. It is indeed often the case that one or more tables in the
database have a role similar to Ada's enumeration types, ie contains
a list of values for information like the list of possible priorities,
a list of countries,... Such lists are only manipulated by the
maintainer of the database, not interactively, and some of their
values have impact on the application's code (for instance, if a
ticket has an urgent priority, we need to send a reminder every day --
but the application needs to know what an urgent priority is).
In such a case, it is convenient to generate these values as
constants in the generated package. The output will be similar to:

@CODESAMPLE{
@b{subtype} Priority_Id is @b{Integer};@NL{}
Priority_High   : @b{constant} Priority_Id := 3;@NL{}
Priority_Medium : @b{constant} Priority_Id := 2;@NL{}
Priority_Low    : @b{constant} Priority_Id := 1;@NL{}
Priority_High_Internal : @b{constant} Priority_Id := 4;}

This code would be extracted from a database table called, for
instance, @code{ticket_priorities}, which contains the following:

@CODESAMPLE{
table ticket_priorities:@NL{}
name		| priority	| category@NL{}
high		| 3		| customer@NL{}
medium	| 2		| customer@NL{}
low		| 1		| customer@NL{}
high_internal	| 4		| internal@NL{}}

To generate the above Ada code, you need to pass the following
parameter to @file{gnatcoll_db2ada}:

@CODESAMPLE{-enum ticket_priorities,Priority,Priority,Integer}

where the second parameter is the name of the field in the table, and
the first is the prefix to add in front of the name to generate the
Ada constant's name. The last parameter should be either @code{Integer}
or @code{String}, which influences the way the value of the Ada constant
is generated (surrounded or not by quotes).

@item -var name,table,field,criteria,comment
This is similar to the @code{-enum} switch, but extracts a single value
from the database. Although applications should try and depend as little
as possible on such specific values, it is sometimes unavoidable.

For instance, if we have a table in the table with the following
contents:

@CODESAMPLE{
table staff@NL{}
staff_id	| login@NL{}
0		| unassigned@NL{}
1		| user1@NL{}}

We could extract the id that helps detect unassigned tickets with the
following command line:

@CODESAMPLE{-var no_assign_id,staff,staff_id,"login='unassigned'","help"}

which generates

@CODESAMPLE{No_Assigne_Id : @b{constant} := 0;@NL{}
@i{--  help}}

The application should use this constant rather than some hard-coded
string @code{"unassigned"} or a named constant with the same value.
The reason is that presumably the login will be made visible somewhere
to the user, and we could decide to change it (or translate it to
another language). In such a case, the application would break. On the
other hand, using the constant @code{0} which we just extracted will
remain valid, whatever the actual text we display for the user.

@item -text

Instead of creating Ada files to represent the database schema, this switch
will ask @code{gnatcoll_db2ada} to dump the schema as text. This is in a
form hopefully easy to parse automatically, in case you have tools that
need the schema information from your database in a DBMS-independent
manner. See below for a description of the format.

@item -createdb

Instead of the usual default output, @code{gnatcoll_db2ada} will output a
set of SQL commands that can be used to re-create the set of all tables in
your schema. This does not create the database itself (which might require
special rights depending on your DBMS), only the tables.

@end table

@subsection Textual description of database schema

@code{gnatcoll_db2ada} can either automatically extracts the database schema
from a running instance of your DBMS (see @code{-dbname}), or get the schema
from a text file (see @code{-dbmodel}). In the latter case, it can then create
the database for you.

This text file is better since you have more control over names (for instance
for foreign keys) and is slightly higher level.

This file is a collection of paragraphs, each of which relates to one table
or one SQL view in your database. The paragraphs start with a line containing:

@CODESAMPLE{
  table ::= '|' ('ABSTRACT')? ('TABLE'|'VIEW') ['(' supertable ')']@NL{}
            '|' <name> '|' <name_row> @NL{}
}

"name" is the name of the table. The third pipe and third column are optional,
and should be used to specify the name for the element represented by a single
row. For instance, if the table is called "books", the third column could
contain "book". This is mostly used when generating high-level Ada code, an
upcoming feature not yet available in GNATCOLL.

If the first line starts with the keyword @code{ABSTRACT}, then no instance
of that table actually exists in the database. This is used in the context
of table inheritance, so define shared fields only once among multiple tables.

The keyword @code{TABLE} can be followed by the name of a table from which it
inherits the fields. Currently, that supertable must be abstract, and the
fields declared in that table are simply duplicated in the new table. For
instance:

@CODESAMPLE{
| ABSTRACT TABLE | parent  | | |@NL{}
| field1         | INTEGER | | |@NL{}
@NL{}
| TABLE mytable(parent) |         |    | | |@NL{}
| id                    | INTEGER | PK | | |@NL{}
}

The table @code{mytable} in fact has two columns, @code{id} and @code{field1}.

Each line must then start with a pipe character ("|"), and contain a number
of pipe-separated fields. The order of the fields is always given by the
following grammar:

@CODESAMPLE{
  fields ::= '|' <name> '|' <type>@NL{}
      '|' ('PK'|''|'NULL'|'NOT NULL') '|' [default] '|' [doc] '|'@NL{}
}

The type of the fields is the SQL type ("INTEGER", "TEXT",...). The tool
will automatically convert these to Ada when generating Ada code. If the
field is a foreign key (that is a value that must correspond to a row in
another table), you can use the special syntax

@CODESAMPLE{
  fk_type ::= 'FK' <table_name> (<revert_name>)
}

As you can see, the type of the field is not specified explicitly, but will
always be that of the foreign table's primary key. With this syntax, the
foreign table must have a single field for its primary key.

In the future, gnatcoll will include a tool that generates an Ada API that
completly hides the underlying SQL commands. The "revert_name" argument is
used in the context of that tool. Although you are encouraged to provide one,
this is currently optional and will have no impact on the output of the
tool.

"revert_name" is the name that will be generated in the Ada code for the
reverse relationship. For instance: assume a book is always found in a library.
If we have two tables "books" and "libraries", we would have one foreign key
from "books" to "libraries", which, for each book, indicates to which library
it belongs. As a result, in the generated code, it would be easy to use
"mybook.Library" to get that library. But given a library, we also want to
be able to retrieve all its books with code similar to "mylibrary.Books".
The foreign key would be declared as such in the books table:

@CODESAMPLE{
table books
| library | FK libraries(books) | NOT NULL | | where the book is |
}

If the "revert_name" is empty (the parenthesis are shown), no revert
relationship is generated. If the parenthesis and the revert_name are both
omitted, a default name is generated.

The third column in the fields definition indicates whether we have a
primary key ("PK"), which must
never be null. In other cases, the column indicates whether the column can
have null values (the default, or explicitly "NULL"), or not ("NOT NULL").
These are similar to the corresponding SQL constraints.

The fourth column gives the default value for the field, and is given in SQL.

The fifth column contains documentation for the field (if any). This
documentation will be included in the generated code, so that IDEs can
provide useful tooltips when navigating your application's code.

After all the fields have been defined, you can specify extract constraints
on the table. In particular, if you have a foreign key to a table that uses
a tuple as its primary key, you can define that foreign key as

@CODESAMPLE{
  FK ::= '|' "FK:" '|' <table> '|' <field_names>* '|' <field_names>* '|'@NL{}
@NL{}
table tableA
| FK: | tableB | fieldA1, fieldA2 | fieldB1, fieldB2 |
}
@subsection Default output of gnatcoll_db2ada

From the command line arguments, @code{gnatcoll_db2ada} will generate
an Ada package, which contains one type per table in the database.
Each of these types has a similar structure. The implementation
details are not shown here, since they are mostly irrelevant and might
change. Currently, a lot of this code are types with discriminants. The
latter are @code{access-to-string}, to avoid duplicating strings in
memory and allocating and freeing memory for these. This provides a
better performance.

@CODESAMPLE{
@b{package} Database @b{is}@NL{}
   @b{type} T_Ticket_Priorities (...) @b{is new} SQL_Table (...) @b{with record}@NL{}
       Priority : SQL_Field_Integer;@NL{}
       Name     : SQL_Field_Text;@NL{}
   @b{end record};@NL{}
   @NL{}
   @b{overriding function} FK (Self : T_Ticket_Priorities; Foreign : SQL_Table'Class)@NL{}
      @b{return} SQL_Criteria;@NL{}
   @NL{}
   Ticket_Priorities : constant T_Ticket_Priorities (...);@NL{}
@b{end} Database;}

It provides a default instance of that type, which can be used to
write queries (see the next section). This type overrides one primitive
operation which is used to compute the foreign keys between that table
and any other table in the database (@pxref{Writing queries}).

Note that the fields which are generated for the table (our example reuses
the previously seen table @code{ticket_priorities}) are typed, which as we
will see provides a simple additional type safety for our SQL queries.

@c -----------------------------------------------------------------------
@node Writing queries
@section Writing queries
@c -----------------------------------------------------------------------
@noindent

The second part of the database support in @value{gnatcoll} is a set
of Ada subprograms which help write SQL queries. Traditional ways to
write such queries have been through embedded SQL (which requires a
pre-processing phase and complicate the editing of source files in
Ada-aware editors), or through simple strings that are passed as is
to the server. In the latter case, the compiler can not do any
verification on the string, and errors such a missing parenthesis or
misspelled table or field names will not be detected until the code
executes the query.

@value{gnatcoll} tries to make sure that code that compiles contains
syntactically correct SQL queries and only reference existing tables
and fields. This of course does not ensure that the query is
semantically correct, but helps detect trivial errors as early as
possible.

Such queries are thus written via calls to Ada subprograms, as in the
following example.

@CODESAMPLE{
@b{with} GNATCOLL.SQL;  @b{use} GNATCOLL.SQL;@NL{}
@b{with} Database; @b{use} Database;@NL{}
@b{declare}@NL{}
   Q : SQL_Query;@NL{}
@b{begin}@NL{}
   Q := SQL_Select@NL{}
     (Fields => Max (Ticket_Priorities.Priority)@NL{}
         & Ticket_Priorities.Category,@NL{}
      From   => Ticket_Priorities,@NL{}
      Where  => Ticket_Priorities.Name /= @i{"low"},@NL{}
      Group_By => Ticket_Priorities.Category);@NL{}
@b{end}}

The above example will return, for each type of priority (internal or
customer) the highest possible value. The interest of this query is
left to the user...

This is very similar to an actual SQL query. Field and table names come
from the package that was automatically generated by the
@code{gnatcoll_db2ada} tool, and therefore we know that our query is
only referencing existing fields. The syntactic correctness is ensured by
standard Ada rules. The @code{SQL_Select} accepts several parameters
corresponding to the usual SQL attributes like @code{GROUP BY},
@code{HAVING}, @code{ORDER BY} and @code{LIMIT}.

The @code{From} parameter could be a list of tables if we need to join
them in some ways. Such a list is created with the overridden @code{"&"}
operator, just as for fields which you can see in the above example.
@value{gnatcoll} also provides a @code{Left_Join} function to join two
tables when the second might have no matching field (see the SQL
documentation).

Similar functions exist for @code{SQL_Insert}, @code{SQL_Update} and
@code{SQL_Delete}. Each of those is extensively documented in the
@file{gnatcoll-sql.ads} file.

It is worth noting that we do not have to write the query all at once.
In fact, we could build it depending on some other criteria. For
instance, imagine we have a procedure that does the query above, and
omits the priority specified as a parameter, or shows all priorities
if the empty string is passed. Such a procedure could be written as

@CODESAMPLE{
@b{procedure} List_Priorities (Omit : String := "") @b{is}@NL{}
  Q : SQL_Query;@NL{}
  C : SQL_Criteria := No_Criteria;@NL{}
@b{begin}@NL{}
  @b{if} Omit /= "" @b{then}@NL{}
     C := Ticket_Priorities.Name /= Omit;@NL{}
  @b{end if};@NL{}
  Q := SQL_Select@NL{}
    (Fields => ..., @i{-- as before}@NL{}
     Where  => C);@NL{}
@b{end};}

With such a code, it becomes easier to create queries on the fly
than it would be with directly writing strings.

The above call has not sent anything to the database yet, only created
a data structure in memory (more precisely a tree). In fact, we could
be somewhat lazy when writing the query and rely on auto-completion,
as in the following example:

@CODESAMPLE{
Q := SQL_Select@NL{}
 (Fields => Max (Ticket_Priorities.Priority)@NL{}
     & Ticket_Priorities.Category,@NL{}
  Where  => Ticket_Priorities.Name /= @i{"low"});@NL{}
@NL{}
Auto_Complete (Q);}

This query is exactly the same as before. However, we did not have to
specify the list of tables (which @value{gnatcoll} can compute on its
own by looking at all the fields referenced in the query), nor the list
of fields in the @code{GROUP BY} clause, which once again can be computed
automatically by looking at those fields that are not used in a SQL
aggregate function. This auto-completion helps the maintenance of those
queries.

There is another case where @value{gnatcoll} makes it somewhat easier
to write the queries, and that is to handle joins between tables. If your
schema was build with foreign keys, @value{gnatcoll} can take advantage
of those.

For instance, imagine we have a table @code{ticket}. These tickets have
a priority, which is an integer id pointing to the @code{ticket_priorities}
table we saw previously. In SQL, this means that the @code{ticket} table
has a foreign key on the @code{ticket_priorities} table, which implies
that any priority set for a ticket must exist in the second table. Using
an id instead of the actual name of the priority in the @code{ticket}
table means that it is easy to change the name of the priority, without
impacting the rest of the database.

Imagine, now, that a query needs to list all tickets with their priorities.
Since we want to show the output to the user, we do not want to show the
internal id, but the actual name of the priority. This would be done with
a query similar to:

@CODESAMPLE{
   Q := SQL_Select@NL{}
     (Fields => Ticket.Number & Ticket_Priorities.Name,@NL{}
      From   => Ticket & Ticket_Priorities,@NL{}
      Where  => Ticket.Priority = Ticket_Priorities.Priority);}

In fact, with the auto-completion, we could write it as

@CODESAMPLE{
   Q := SQL_Select@NL{}
     (Fields => Ticket.Number & Ticket_Priorities.Name,@NL{}
      Where  => Ticket.Priority = Ticket_Priorities.Priority);}

The @code{WHERE} clause comes straight from the definition of the
foreign key. It can be shorten using the @code{FK} primitive operation
that we saw was generated in the @code{Database} package. The following
example uses the Ada05 dotted notation for the call to FK, but that is
not mandatory, of course.

@CODESAMPLE{
   Q := SQL_Select@NL{}
     (Fields => Ticket.Number & Ticket_Priorities.Name,@NL{}
      Where  => Ticket.FK (Ticket_Priorities));}

One advantage is that we avoid possible errors in writing the join,
and if at some point the foreign key between the two tables involves
more fields, for instance, the query remains valid and we do not have
to change our code.

@c -----------------------------------------------------------------------
@node Executing queries
@section Executing queries
@c -----------------------------------------------------------------------
@noindent

Once we have our query in memory, we need to pass it on to the database
server itself, and retrieve the results.

Executing is done through the @code{GNATCOLL.SQL.Exec} package, as in the
following example:

@CODESAMPLE{
@b{declare}@NL{}
   R : Forward_Cursor;@NL{}
@b{begin}@NL{}
   Execute (Connection => DB, Result => R, Query => Q);
@b{end};}

This reuses the connection we have established previously (@code{DB}),
and sends it the query. The result of that query is then stored in
@code{R}, to be used later.

There are several versions of @code{Execute}. In particular, there are
versions which do not have the @code{R} parameter. Some queries do not
return anything useful to our application (for instance a @code{INSERT}
query), and thus we can simplify the call.

@code{Execute} has an extra parameter @code{Use_Cache}, set to
@code{False} by default. If this parameter is true, and the exact same
query has already been executed before, its result will be reused
without even contacting the database server. The cache is automatically
invalidated every hour in any case. This cache is mostly useful for
tables that act like enumeration types, as we have seen before when
discussing the @code{-enum} parameter to @file{gnatcoll_db2ada}. In this
case, the contents of the table changes very rarely, and the cache can
provide important speedups, whether the server is local or distant.
However, we recommend that you do actual measurements to know whether
this is indeed beneficial for you. You can always invalidated the
current cache with a call to @code{Invalidate_Cache} to force the
query to be done on the database server.

If for some reason the connection to the database is no longer valid
(a transient network problem for instance), @value{gnatcoll} will
attempt to reconnect and re-execute your query transparently, so that
your application does not need to handle this case.

If your query produces an error (whether it is invalid, or any other
reason), a flag is toggled in the @code{Connection} parameter, which
you can query through the @code{Success} subprogram. As a result,
a possible continuation of the above code is:

@CODESAMPLE{
@b{if} Success (DB) @b{then}@NL{}
   ...@NL{}
@b{else}@NL{}
   ...  @i{an error occurred}@NL{}
@b{end if}@NL{}}

@value{gnatcoll} also tries to be helpful in the way it handles SQL
transactions. Such transactions are a way to execute your query in a
sandbox, ie without affecting the database itself until you decide to
@code{COMMIT} the query. Should you decide to abort it (or
@code{ROLLBACK} as they say for SQL), then it is just as if nothing
happened. As a result, it is in general recommended to do all your changes
to the database from within a transaction. If one of the queries fail
because of invalid parameters, you just rollback and report the error
to the user. The database is still left in a consistent state. As an
additional benefit, executing within a transaction is sometimes faster,
as is the case for PostgreSQL for instance.

To help with this, @value{gnatcoll} will automatically start a
transaction the first time you edit the database. It is then your
responsibility to either commit or rollback the transaction when you
are done modifying. A lot of database engines (among which PostgreSQL)
will not accept any further change to the database if one command in
the transaction has failed. To take advantage of this, @value{gnatcoll}
will therefore not even send the command to the server if it is in a
failure state.

Here is code sample that modifies the database:

@CODESAMPLE{
Execute (DB, SQL_Insert (...));@NL{}
@i{--  The code above starts a transaction and inserts a new row}@NL{}
@NL{}
Execute (DB, SQL_Insert (...));@NL{}
@i{--  Executed in the same transaction}@NL{}
@NL{}
Commit_Or_Rollback (DB);@NL{}
@i{--  Commit if both insertion succeeded, rollback otherwise}@NL{}
@i{--  You can still check Success(DB) afterward if needed}}

@c -----------------------------------------------------------------------
@node Getting results
@section Getting results
@c -----------------------------------------------------------------------
@noindent

One you have executed a @code{SELECT} query, you generally need to
examine the rows that were returned by the database server. This is done
in a loop, as in

@CODESAMPLE{
@b{while} Has_Row (R) @b{loop}@NL{}
    Put_Line (@i{"Max priority="} & Integer_Value (R, 0)'Img@NL{}
             & @i{" for category="} & Value (R, 1));@NL{}
    Next (R);@NL{}
@b{end loop};}

You can only read one row at a time, and as soon as you have moved to the
next row, there is no way to access a previously fetched row. This is the
greatest common denominator between the various database systems. In
particular, it proves efficient, since only one row needs to be kept in
memory at any point in time.

For each row, we then call one of the @code{Value} or @code{*Value}
functions which return the value in a specific row and a specific
column.

@c -----------------------------------------------------------------------
@node Writing your own cursors
@section Writing your own cursors
@c -----------------------------------------------------------------------
@noindent

The cursor interface we just saw is low-level, in that you get access to
each of the fields one by one. Often, when you design your own application,
it is better to abstract the database interface layer as much as possible.
As a result, it is often better to create record or other Ada types to
represent the contents of a row.

Fortunately, this can be done very easily based on the low-level interface
provided by GNATCOLL. Here is a code example that shows how this can be
done.

@CODESAMPLE{
  @b{type} My_Row @b{is record}@NL{}
     Id   : Integer;@NL{}
     Name : Unbounded_String;@NL{}
  @b{end record;}@NL{}
@NL{}
  @b{type} My_Cursor @b{is new} Forward_Cursor @b{with null record};@NL{}
  @b{function} Element (Self : My_Cursor) @b{return} My_Row;@NL{}
  @b{function} Do_Query @b{return} My_Cursor;@NL{}
}

The idea is that you create a function that does the query for you (based
on some parameters that are not show here), and then returns a cursor over
the resulting set of rows. For each row, you can use the @code{Element}
function to get an Ada record for easier manipulation.

Let's first see how these types would be used in practice:

@CODESAMPLE{
  @b{declare}@NL{}
    C : My_Cursor := Do_Query (...);@NL{}
  @b{begin}@NL{}
    @b{while} Has_Row (C) @b{loop}@NL{}
       Put_Line ("Id = " & Element (C).Id);@NL{}
       Next (C);@NL{}
    @b{end loop};@NL{}
  @b{end};}

So the loop itself is the same as before, except we no longer access each of
the individual fields directly. This means that if the query changes to
return more fields (or the same fields in a differente order for instance),
the code in your application does not need to change.

The specific implementation of the subprograms could be similar to the
following subprograms (we do not detail the writing of the SQL query itself,
which of course is specific to your application)

@CODESAMPLE{
  @b{function} Do_Query @b{return} My_Cursor @b{is}@NL{}
     Q : constant SQL_Query := ....;@NL{}
     R : My_Cursor;@NL{}
  @b{begin}@NL{}
     Execute (DB, R, Q);@NL{}
     return R;@NL{}
  @b{end} Do_Query;@NL{}
@NL{}
  @b{function} Element (Self : My_Cursor) @b{return} My_Row @b{is}@NL{}
  @b{begin}@NL{}
    @b{return} My_Row'@NL{}
       (Id => Integer_Value (Self, 0),@NL{}
        Name => To_Unbounded_String (Value (Self, 1)));@NL{}
  @b{end} Element;
}

There is one more complex case though. It might happen that an element
needs access to several rows to fill the Ada record. For instance, if we
are writing a CRM application and query the contacts and the companies they
work for, it is possible that a contact works for several companies. The
result of the SQL query would then look like this:

@CODESAMPLE{
   contact_id | company_id@NL{}
       1      |    100    @NL{}
       1      |    101    @NL{}
       2      |    100    @NL{}
}

The sample code shown above will not work in this case, since Element is
not allowed to modify the cursor. In such a case, we need to take a slightly
different approach.

@CODESAMPLE{
    @b{type} My_Cursor @b{is new} Forward_Cursor @b{with null record};@NL{}
    @b{function} Do_Query @b{return} My_Cursor; @i{--  as before}@NL{}
    @b{procedure} Element_And_Next@NL{}
       (Self : @b{in out} My_Cursor; Value : @b{out} My_Row);@NL{}
}

where @code{Element_And_Next} will fill Value and call Next as many times
as needed. On exit, the cursor is left on the next row to be processed. The
usage then becomes

@CODESAMPLE{
   @b{while} Has_Row (R) @b{loop}@NL{}
      Element_And_Next (R, Value);@NL{}
   @b{end loop};@NL{}
}

To prevent the user from using Next incorrectly, you should probably override
@code{Next} with a procedure that does nothing (or raises a Program_Error
maybe). Make sure that in @code{Element_And_Next} you are calling the
inherited function, not the one you have overridden, though.

There is still one more catch. The user might depend on the two subprograms
@code{Rows_Count} and @code{Processed_Rows} to find out how many rows there
were in the query. In practice, he will likely be interested in the number
of distinct contacts in the tables (2 in our example) rather than the number
of rows in the result (3 in the example). You thus need to also override
those two subprograms to return correct values.

@c -----------------------------------------------------------------------
@node Creating your own SQL types
@section Creating your own SQL types
@c -----------------------------------------------------------------------
@noindent

@value{gnatcoll} comes with a number of predefined types that you can use in
your queries. @file{gnatcoll_db2ada} will generate a file using any of these
predefined types, based on what is defined in your actual database.

But sometimes, it is convenient to define your own SQL types to better
represent the logic of your application. For instance, you might want to
define a type that would be for a @code{Character} field, rather than use
the general @code{SQL_Field_Text}, just so that you can write statements
like:

@CODESAMPLE{
  @b{declare}@NL{}
     C : Character := 'A';@NL{}
     Q : SQL_Query;@NL{}
  @b{begin}@NL{}
     Q := SQL_Select (.., Where => Table.Field = C);@NL{}
  @b{end}@NL{}
}

This is fortunately easily achieved by instantiating one generic package,
as such

@CODESAMPLE{
  @b{with} GNATCOLL.SQL_Impl; @b{use} GNATCOLL.SQL_Impl;@NL{}
@NL{}
  @b{function} To_SQL (C : Character) @b{return} String @b{is}@NL{}
  @b{begin}@NL{}
     @b{return} "'" & C & "'";@NL{}
  @b{end} To_SQL;@NL{}
@NL{}
  @b{package} Character_Fields @b{is new} Field_Types (Character, To_SQL);@NL{}
  @b{type} SQL_Field_Character @b{is new} Character_Fields.Field@NL{}
     @b{with null record};@NL{}
}

This automatically makes available both the field type (which you can use in
your database description, as @file{gnatcoll_db2ada} would do, but also
all comparison operators like @code{<}, @code{>}, @code{=}, and so on, both
to compare with another character field, or with @code{Character} Ada
variable. Likewise, this makes available the assignment operator @code{=}
so that you can create @code{INSERT} statements in the database.

Finally, the package @code{Character_Fields} contain other generic
packages which you can instantiate to bind SQL operators and functions that
are either predefined in SQL and have no equivalent in @value{gnatcoll} yet,
or that are functions that you have created yourself on your DBMS server.

See the specs of @code{GNATCOLL.SQL_Impl} for more details. This package
is only really useful when writing your own types, since otherwise you
just have to use @code{GNATCOLL.SQL} to write the actual queries.

@c -----------------------------------------------------------------------
@node Query logs
@section Query logs
@c -----------------------------------------------------------------------
@noindent

In @ref{Logging information} we discovered the logging module of
@value{gnatcoll}. The database interface uses this module to log the
queries that are sent to the server.

If you activate traces in your application, the user can then activate
one of the following trace handles to get more information on the
exchange that exists between the database and the application. As we saw
before, the output of these traces can be sent to the standard output, a
file, the system logs,...

The following handles are provided:

@itemize @bullet
@item SQL.ERROR
This stream is activated by default. Any error returned by the database
(connection issues, failed transactions,...) will be logged on this stream

@item SQL
This stream logs all queries that are not SELECT queries, ie mostly all
queries that actually modify the database

@item SQL.SELECT
This stream logs all select queries. It is separated from SQL because
very often you will be mostly interested in the queries that impact the
database, and logging all selects can generate a lot of output.
@end itemize

@c -----------------------------------------------------------------------
@node Tasks and databases
@section Tasks and databases
@c -----------------------------------------------------------------------
@noindent

As we saw before, the database interface can be used in multi-tasking
applications. In such a case, it is recommended that each thread has its
own connection to the database, since that is more efficient and you do
not have to handle locking.

However, this assumes that the database server itself is thread safe,
which most often is the case, but not for @code{sqlite} for instance.
In such a case, you can only connect one per application to the database,
and you will have to manage a queue of queries somehow.

@c -----------------------------------------------------------------------
@node Index
@unnumbered Index
@c -----------------------------------------------------------------------
@printindex cp
@bye

