Monte Carlo simulations of photoproduction reactions and the
detector response are an integral part of data analysis for
\hd.  Monte Carlo data sets an order of magnitude larger than
the real data for specific channels must be produced and analyzed
within a unified analysis framework.  The computer resources
needed for this task were discussed in the previous chapter.
This chapter describes how the simulation is to be carried out,
the specific software components that exist at present, and some
preliminary results regarding detector acceptance and resolution.

During the conceptual design phase of \hd\ two parallel paths
of Monte Carlo development have been followed.  The first has been
focused on simulating reconstructed events for acceptance and
resolution studies, and for tests of partial-wave analysis.  On
this path the simulation of particle interactions in the detector 
followed by track/cluster reconstruction is replaced by a model
which accounts for the smearing of the final particle momenta
according to detector resolution.  This so-called {\it fast}
Monte Carlo approach is computationally very efficient and
permits the exploration of large regions of detector parameter
space during design.  In fact, important parts of the design
evaluation can only be accomplished by this approach, before a
full event reconstruction package is available.

When the event reconstruction package arrives, a different sort of
simulation code will be needed.  This so-called {\it physics}
simulation relies on a detailed geometrical description of the
detector and a library of known particle-material interactions
to estimate the detector response to a given event as accurately
as possible. From this response it forms a simulated event record
that is analyzed by the reconstruction package in a similar way
as real events.  The physics simulation package should come first
in the order of software development  because it provides useful
test data for debugging the rest of the analysis chain.  The physics
simulation is also useful at the design stage for estimating
background rates in detector and trigger elements.  This is the
second path of Monte Carlo development being pursued by \hd. 

These Monte Carlo simulation programs are the first components in
what will grow to be a large body of code for the \hd\ experiment.
It is useful to consider at the outset what pieces of these codes
might be of broader use than strictly for simulation.  For example,
the reconstruction code will need access to the same alignment
data as is used by the simulator.  Some of the requirements for
simulation can be met by incorporating existing software packages
from other sources; however their use must be coordinated to avoid
conflicts and unwanted dependencies in the future.  Software
developed at this early stage of the experiment must undergo 
numerous stages of evolution if it is to be of lasting usefulness.
The incorporation of industry standards into the code wherever
possible lays the groundwork for a smooth evolution in the
foreseeable future.  All of these things come together in the
formulation of a software {\it framework} for the experiment.  

In the sections which follow are discussed, first, the software
framework, followed by a description of the individual components
of the simulation package.  The following three sections summarize
the results from early design studies carried out with the fast
Monte Carlo.  The final section describes the general method
how simulation results are incorporated into a partial-wave analysis.

\section{Monte Carlo framework}
\label{mc:framework}

In this context, a framework refers to a set of specified
interfaces through which the different software components in a
system interact and exchange information, together with a set of
common tools that facilitate access to information through these
interfaces by application programs .  Use of a framework
allows builders of individual components to have a relative
degree of independence in their implementation choices, knowing
what requirements they must satisfy in order to work successfully
with the other pieces.  Before proceeding to the specifics, it is
worthwhile to note two general principles that have been adopted
for \hd\ code development.

\newcounter{li}
\begin{list}{\arabic{li}.}{\usecounter{li}}
\item
All data within the framework must be viewable in a well-formed xml
document format that expresses the structure and relationships within
the data.
\item
All major interfaces should be implemented as web services, in addition
to the normal API.
\end{list}

Not specified in this list is any mandated set of languages, operating
systems, or disk file formats.  While prudence suggests a restricted
set of choices for each of these for developing new code, it was decided
that the benefits of the freedom to borrow existing programs from a
variety of sources outweighs the cost in complexity.  Where necessary,
legacy code can be wrapped in such a way that it provides its functionality
through a protected interface.  In any case, software technology is
changing too fast at present to allow a final decision on such questions 
at this point in the life-cycle of the experiment.

\begin{figure}[ht]\centering
\begin{tabular}{c}
\mbox{\epsfig{file=ch10_dataflow.eps,height=5in}
%,width=0.9\textwidth,height=0.75\textwidth}
} 
\end{tabular}
\caption[Data flow diagram]{\label{fig:dataflow}
Data flow diagram showing the major software components responsible
for data processing in \hd.}
\end{figure}

The overall data-flow scheme for \hd\ is shown in Fig.~\ref{fig:dataflow}.
Data flows from the top to the bottom of the figure.  On the right-hand
side, digitized events come from the detector, are converted to hits
by applying corrections derived from the calibration database, and are
passed to the reconstruction programs for further processing.  On the
left-hand column, events start off as lists of particles and their
momenta coming from a physics event generator and are converted to hits
in the simulator, after which they follow the same path as the real
events.

The remainder of this chapter is concerned mainly with what happens to
the left of the vertical line in Fig.~\ref{fig:dataflow}.  For clarity,
we will distinguish between event {\it generation} (first step) and
{\it simulation} (next step) in the Monte Carlo process.  As far as
the framework is concerned, the only thing of concern is how programs
(or people) can access what is inside the boxes.  The details of how
data is stored inside the boxes, or what happens inside the processing
steps is beyond the scope of the framework.  The generator needs to be
able to find out what kind and how many events to generate.  The
simulator needs to be able to get from the generator a sequence of
event specifications, and it must be able to provide its event hits
to the reconstruction code in a format that it understands.  Not shown
in the figure but also important are the detector geometry information
and the simulation control data which the simulator also needs.
Within the \hd\ framework, all of these data have the common property
that they are viewable in xml.  When the software components
are fully incorporated into the framework then each of the processing
steps will be available on the \hd\ grid as a web service.

The formal specification of all of these interfaces is incomplete
at present.  The most complete specification is that of the detector
geometry information, which has been published \cite{hdds} on
the web.  It is described in more detail in a later section.  A
draft specification for the event description has also been published.
\cite{hddm}.  Depending on the location along the data-flow pipeline,
different pieces of event information are available.  However it is
decided that access to all event data by application code within the
\hd\ framework is through a single interface.  That interface must
provide a mechanism for determining what kinds of information are
available in an event and for providing what is available in a
standard way to the client program.

This is all quite easy to do by specifying the interface in terms of
an xml schema.  However doing event input/output through xml libraries
is very expensive for large data sets, not only in terms of data
volume but also cpu overhead.  This is why the framework specifies
that all data should be {\em viewable} in xml, not necessarily 
{\em stored} in xml.  No restrictions
are placed on what data formats are actually used
internally by applications, or how events are stored in disk files.
In practice it proved convenient for the purposes of Monte Carlo
to create a self-describing event data format that is very close to
the underlying xml, called {\em hddm}.  An hddm event stream (or
file) begins with an event template in plain-text xml that
describes the information that is available for each event,
followed by the actual event data.  The tags have been suppressed
in the event data and the values written in binary format, so that
the event record size is roughly equivalent to other binary formats.
Framework tools exist which can automatically generate a miniature
c or c++ library that contains the calls needed by an application
program to access the event data, just by reading the first few
lines in an hddm event file.  Applications built with one of these
libraries automatically verify that the data they require are present
in the file before access is attempted.  Finally, a single pair of
translators called hddm-xml and xml-hddm exist which are capable
of converting any hddm data stream to and from xml.

Thus the interface to the data in each of the boxes in
Fig~\ref{fig:dataflow} is expressed in a xml specification that
serves as an event template.  The specification contains an
inheritance mechanism that makes it easy to extend the event
definition, so that producers and consumers of event data can
decide to exchange additional information through the extended
interface without interfering with the operation on the same data
by older programs that rely on the base interface.  All of this is
verified automatically by the framework API library without any
need for checks by application code.  Writers of application
code have the choice of accessing the data through the API
(currently provided in c and c++ only) or by reading and parsing
the xml.  Use of the API is more efficient in that it eliminates
the xml parsing step, but the choice of languages is restricted.
On the other hand, standard tools are available in all major
languages that make it easy to read and write xml.  The advantage
of this design is that anyone in any language that has the
capability of reading ascii text has access to the event data in 
a standard way.

The hddm scheme is effectively an efficient mechanism for
prototyping interfaces to event data.  Eventually the information
content of an event will stabilize to the point where the interface
can be frozen, at least for the early stages of the pipeline.  At
that point the choice of the format for event data decouples from
the interface.  Different event formats at various stages along
the data-flow path may be adopted based upon considerations of
efficiency and prevailing technology.  None of this has practical
consequences for application code, provided that the interface
remains everywhere the same.
