remove stale documentation

git-svn-id: https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2@4247 ce84e21b-d406-0410-9b95-82705330c041

1 files changed, 0 insertions, 556 deletions

diff --git a/doc/architecture.xml b/doc/architecture.xml
deleted file mode 100644
index ef9d6bca8..000000000
--- a/doc/architecture.xml
+++ /dev/null
@@ -1,556 +0,0 @@
-<chapter id='chap:architecture'>
-<title>Bcfg2 Architecture</title>
-
-  <para>
-    Bcfg2 is based on a client-server architecture. The client is
-    reponsible for interpreting (but not processing) the configuration
-    served by the server. This configuration is literal, so no local
-    process is required. After completion of the configuration
-    process, the client uploads a set of statistics to the
-    server. This section will describe the goals and then the
-    architecture motivated by it.
-  </para>
-
-  <section id='sec:goals'>
-    <title>Goals</title>
-
-    <itemizedlist>
-      <listitem>
-	<para>
-	  Model configurations using declarative
-	  semantics. Declarative semantics maximize the utility of
-	  configuration management tools; they provide the most
-	  flexibility for the tool to determine the right course of
-	  action in any given situation. This means that users can
-	  focus on the task of describing the desired configuration,
-	  while leaving the task of transitioning clients states to
-	  the tool. 
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Configuration descriptions should be comprehensive. This
-	  means that configurations served to the client should be
-	  sufficent to reproduce all desired functionality. This
-	  assumption allows the use of heuristics to detect extra
-	  configuration, aiding in reliable, comprehensive
-	  configuration definitions.
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Provide a flexible approach to user interactions. Most
-	  configuration management systems take a rigid approach to
-	  user interactions; that is, either the client system is
-	  always correct, or the central system is. This means that
-	  users are forced into an overly proscribed model where the
-	  system asserts where correct data is. Configuration data
-	  modification is frequently undertaken on both the
-	  configuration server and clients. Hence, the existance of a
-	  single canonical data location can easily pose a problem
-	  during normal tool use.
-	</para>
-	<para>
-	  Bcfg2 takes a different approach. The default assumption is
-	  that data on the server is correct, however, the client has
-	  option to run in another mode where local changes are
-	  catalogued for server-side integration. If the Bcfg2 client is
-	  run in dry run mode, it can help to reconcile differences
-	  between current client state and the configuration described
-	  on the server.
-	</para>
-	<para>
-	  The Bcfg2 client also searches for extra configuration; that
-	  is, configuration that is not specified by the configuration
-	  description. When extra configuration is found, either
-	  configuration has been removed from the configuration
-	  description on the server, or manual configuration has
-	  occurred on the client. Options related to two-way
-	  verification and removal are useful for configuration
-	  reconciliation when interactive access is used.
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Plugins and administrative applications.
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Imcremental operations.
-	</para>
-      </listitem>
-    </itemizedlist>
-  </section>
-
-  <section id='sec:client'>
-    <title>The Bcfg2 Client</title>
-
-    <para>The Bcfg2 client performs all client configuration or
-    reconfiguration operations. It renders a declarative configuration
-    specification, provided by the Bcfg2 server, into a set of
-    configuration operations which will, if executed, attempt to
-    change the client's state into that described by the configuration
-    specification. Conceptually, the Bcfg2 client serves to isolate
-    the Bcfg2 server and specification from the imperative operations
-    required to implement configuration changes. This isolation allows
-    declarative specifications to be manipulated symbolically on the
-    server, without needing to understand the properties of the
-    underlying system tools. In this way, the Bcfg2 client acts as a
-    sort of expert system that "knows" how to implement declarative
-    configuration changes.</para>
-
-    <para>The operation of the Bcfg2 client is intended to be as
-      simple as possible. The normal configuration process consists of
-      four main steps:</para>
-
-<!--     <procedure> -->
-<!--       <title>Bcfg2 Client Execution Process</title> -->
-
-<!--       <step>foo</step> -->
-<!--     </procedure> -->
-
-    <orderedlist>
-      <listitem>
-	<para>Probe Execution</para>
-	<para>During the probe execution stage, the client connects to
-	  the server and downloads a series of probes to execute. These
-	  probes reveal local facts to the Bcfg2 server. For example, a
-	  probe could discover the type of video card in a system. The
-	  Bcfg2 client returns this data to the server, where it can
-	  influence the client configuration generation
-	  process. 
-	</para>
-      </listitem>
-      <listitem>
-	<para>Configuration Download and Inventory</para>
-	<para>
-	  The Bcfg2 client now downloads a configuration
-	  specification from the Bcfg2 server. The configuration
-	  describes the complete target state of the machine. That is,
-	  all aspects of client configuration should be represented in
-	  this specification. For example, all software packages and
-	  services should be represented in the configuration
-	  specification. 
-	</para>
-	  
-	<para>
-	  The client now performs a local system inventory. This
-	  process consists of verifying each entry present in the
-	  configuration specification. After this check is completed,
-	  heuristic checks for configuration not included in the
-	  configuration specification. We refer to this inventory
-	  process as 2-way validation, as first we verify that the
-	  client contains all configuration that is included in the
-	  specification, then we check if the client has any extra
-	  configuration that isn't present. This provides a fairly
-	  rigorous notion of client configuration congruence.  
-	</para>
-
-	<para>
-	  Once the 2-way verification process has been
-	  performed, the client has built a list of all configuration
-	  entries that are out of spec. This list has two parts:
-	  specified configuration that is incorrect (or missing) and
-	  unspecified configuration that should be removed.
-	</para>
-      </listitem>
-      <listitem>
-	<para>Configuration Update</para>
-	<para>
-	  The client now attempts to update its configuration to
-	  match the specification. Depending on options, changes may
-	  not (or only partially) be performed. First, if extra
-	  configuration correction is enabled, extra configuration can
-	  be removed. Then the remaining changes are processed. The
-	  Bcfg2 client loops while progress is made in the correction
-	  of these incorrect configuration entries. This loop results
-	  in the client being able to accomplish all it will be able
-	  to during one execution. Once all entries are fixed, or no
-	  progress is being made, the loop terminates.
-	</para>
-	  
-	<para>
-	  Once all configuration changes that can be performed have
-	  been, bundle dependencies are handled. Bundle groupings
-	  result in two different behaviors. Contained entries are
-	  assumed to be inter-dependant. To address this, the client
-	  re-verifies each entry in any bundle containing an updates
-	  configuration entry. Also, services contained in modified
-	  bundles are restarted.
-	</para>
-      </listitem>
-      <listitem>
-	<para>Statistics Upload</para>
-	<para>
-	  Once the reconfiguration process has concluded, the
-	  client reports information back to the server about the
-	  actions it performed during the reconfiguration
-	  process. Statistics function as a detailed return code from
-	  the client. The server stores statistics
-	  information. Information included in this statistics update
-	  includes (but is not limited to):
-	</para>
-
-	<itemizedlist>
-	  <listitem>
-	    <para>Overall client status (clean/dirty)</para>
-	  </listitem>
-	  <listitem>
-	    <para>List of modified configuration entries</para>
-	  </listitem>
-	  <listitem>
-	    <para>List of uncorrectable configuration entries</para>
-	  </listitem>
-	</itemizedlist>
-      </listitem>
-    </orderedlist>
-
-    <section id='sec:arch-abstraction'>
-      <title>Architecture Abstraction</title>
-
-      <para>The Bcfg2 client internally supports the administrative
-	tools available on different architectures. For example, rpm and
-	apt-get are both supported, allowing operation of Debian,
-	Redhat, SUSE, and Mandriva systems. The client toolset is
-	specified in the configuration specification. The client merely
-	includes a series of libraries which describe how to interact
-	with the system tools on a particular platform.
-      </para>
-
-      <para>
-	Three of the libraries exist. There is a base set of
-	functions, which contain definitions describing how to perform
-	POSIX operations. Support for configuration files,
-	directories, and symlinks are included here. Two other
-	libraries subclass this one, providing support for Debian and
-	rpm-based systems.
-      </para>
-
-      <para>
-	The Debian toolset includes support for apt-get and
-	update-rc.d. These tools provide the ability to install and
-	remove packages, and to install and remove services.
-      </para>
-
-      <para>
-	The Redhat toolset includes support for rpm and chkconfig. Any
-	other platform that uses these tools can also use this
-	toolset. Hence, all of the other familiar rpm-based
-	distributions can use this toolset without issue. 
-      </para>
-
-      <para>
-	Other platforms can easily use the POSIX toolset, ignoring
-	support for packages or services. Alternatively, adding
-	support for new toolsets isn't difficult. Each toolset
-	consists of about 125 lines of python code.
-      </para>
-    </section>
-  </section>
-  <section id='sec:server'>
-    <title>The Bcfg2 Server</title>
-    <para>
-      The Bcfg2 server is responsible for taking a network description
-      and turning it into a series of configuration specifications for
-      particular clients. It also manages probed data and tracks
-      statistics for clients. 
-    </para>
-
-    <para>
-      The Bcfg2 server takes information from two sources when
-      generating client configuration specifications. The first is a
-      pool of metadata that describes clients as members of an
-      aspect-based classing system. That is, clients are defined in
-      terms of aspects of their behavior. The other is a file system
-      repository that contains mappings from metadata to literal
-      configuration. These are combined to form the literal
-      configuration specifications for clients.
-    </para>
-
-    <section id='sec:construction'>
-      <title>The Configuration Specification Construction Process</title>
-      <para>
-	As we described in the previous section, the client connects
-	to the server to request a configuration specification. The
-	server uses the client's metadata and the file system
-	repository to build a specification that is tailored for the
-	client. This process consists of the following steps:
-      </para>
-
-      <orderedlist>
-	<listitem>
-	  <para>Metadata Lookup</para>
-	  <para>
-	    The server uses the client's IP address to initiate the
-	    metadata lookup. This initial metadata consists of a
-	    (profile, image) tuple. If the client already has
-	    metadata registered, then it is used. If not, then
-	    default values are used and stored for future use.
-	  </para>
-	  
-	  <para>
-	    This metadata tuple is expanded using some profile and
-	    class definitions also included in the metadata. The end
-	    result of this process is metadata consisting of
-	    hostname, profile, image, a list of classes, a list of
-	    attributes and a list of bundles.
-	  </para>
-	</listitem>
-	<listitem>
-	  <para>Abstract Configuration Construction</para>
-	  <para>
-	    Once the server has the client metadata, it is used to
-	    create an abstract configuration. An abstract
-	    configuration contains all of the configuration elements
-	    that will exist in the final specification without any
-	    specifics. All entries will be typed (ie the tagname
-	    will be one of Package, ConfigurationFile, Service,
-	    Symlink, or Directory) and will include a name. These
-	    configuration entries are grouped into bundles, which
-	    document installation time interdependencies.
-	  </para>
-	</listitem>
-	<listitem>
-	  <para>Configuration Binding</para>
-	  <para>
-	    The abstract configuration determines the structure of
-	    the client configuration, however, it doesn't contain
-	    literal configuration information. After the abstract
-	    configuration is created, each configuration entry must
-	    be bound to a client-specific value. The Bcfg2 server
-	    uses plugins to provide these client-specific bindings.
-	  </para>
-
-	  <para>
-	    The Bcfg2 server core contains a dispatch table that
-	    describes which plugins can handle requests of a
-	    particular type. The responsible plugin is located
-	    for each entry. It is called, passing in the
-	    configuration entry and the client's metadata. The
-	    behavior of plugins is explicitly undefined, so as to
-	    allow maximum flexibility. The behaviors of the stock
-	    plugins are documented elsewhere in this manual.
-	  </para>
-
-	  <para>
-	    Once this binding process is completed, the server has a
-	    literal, client-specific configuration
-	    specification. This specification is complete and
-	    comprehensive; the client doesn't need to process it at
-	    all in order to use it. It also represents the totality
-	    of the configuration specified for the client.	   
-	  </para>
-	</listitem>
-      </orderedlist>
-    </section>
-  </section>
-
-  <section id='sec:literal'>
-    <title>The Literal Configuration Specification</title>
-    
-    <para>
-      Literal configuration specifications are served to clients by
-      the Bcfg2 server. This is a differentiating factor for Bcfg2;
-      all other major configuration management systems use a
-      non-literal configuration specification. That is, the clients
-      receive a symbolic configuration that they process to implement
-      target states. We took the literal approach for a few reasons:
-    </para>
-    
-    <itemizedlist>
-      <listitem>
-	<para>
-	  A small list of configuration element types can be defined,
-	  each of which can have a set of defined semantics. This
-	  allows the server to have a well-formed model of
-	  client-side operations. Without a static lexicon with
-	  defined semantics, this isn't possible. This allows the
-	  server, for example, to record the update of a package as a
-	  coherent event. 
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Literal configurations do not require client-side
-	  processing. Removing client-side processing reduces the
-	  critical footprint of the tool. That is, the Bcfg2 client
-	  (and the tools it calls) need to be functional, but the rest
-	  of the system can be in any state. Yet, the client will
-	  receive a correct configuration.
-	</para>
-      </listitem>
-      <listitem>
-	<para>
-	  Having static, defined element semantics also requires that all
-	  operations be defined and implemented in advance. The
-	  implementation can maximize reliability and robustness. In
-	  more ad-hoc setups, these operations aren't necessarily
-	  safely implemented.
-	</para>
-      </listitem>
-    </itemizedlist>
-
-    <section id='sec:spec-structure'>
-      <title>The Structure of Specifications</title>
-      <para>
-	Configuration specifications contain some number of
-	clauses. Two types of clauses exist. Bundles are groups of
-	inter-dependant configuration entities. The purpose of bundles
-	is to encode installation-time dependencies such that all new
-	configuration is properly activated during reconfiguration
-	operations. That is, if a daemon configuration file is
-	changed, its daemon should be restarted. Another example of
-	bundle usage is the reconfiguration of a software package. If
-	a package contains a default configuration file, but it gets
-	overwritten by an environment-specific one, then that updated
-	configuration file should survive package upgrade. The purpose
-	of bundles is to describe services, or reconfigured software
-	packages. Independent clauses contains groups of configuration
-	entities that aren't related in any way. This provides a
-	convenient mechanism that can be used for bulk installations
-	of software.
-      </para>
-
-      <para>
-	Each of these clauses contains some number of configuration
-	entities. Five types of configuration entities exist:
-	ConfigurationFile, Package, SymLink, Directory, and
-	Service. Each of these correspond to the obvious system item.
-	Configuration specifications can get quite large; many systems
-	have specifications that top one megabyte in size. An example
-	of one is included in an appendix. These configurations can be
-	written by hand, or generated by the server. The easiest way
-	to start using Bcfg2 is to write small static configurations for
-	clients. Once configurations get larger, this process gets
-	unwieldy; at this point, using the server makes more sense.
-      </para>
-    </section>
-  </section>
-
-  <section id='sec:design-considerations'>
-    <title>Design Considerations</title>
-
-    <para>
-      This section will discuss several aspects of the design of
-      bcfg2, and the particular use cases that motivated
-      them. Initially, this will consist of a discussion of the system
-      metadata, and the intended usage model for package indices as
-      well. 
-    </para>
-
-    <section id='sec:metadata'>
-      <title>System Metadata</title>
-
-      <para>
-	Bcfg2 system metadata describes the underlying patterns in
-	system configurations. It describes commonalities and
-	differences between these specifications in a rigorous way. The
-	groups used by bcfg2's metadata are responsible for
-	differentiating clients from one another, and building
-	collections of allocatable configuration.
-      </para>
-
-      <para>
-	The Bcfg2 metadata system has been designed with several
-	high-level goals in mind. Flexibility and precision are
-	paramount concerns; no configuration should be undescribable
-	using the constructs present in the bcfg2 repository. We have
-	found (generally the hard way) that any assumptions about the
-	inherent simplicity of configuration patterns tend to be
-	wrong, so obscenely complex configurations must be
-	representable, even if these requirements seem illogical
-	during the implementation. 
-      </para>
-
-      <para>
-	In particular, we wanted to streamline several operations that
-	commonly occurred in our environment. 
-      </para>
-      
-      <orderedlist>
-	<listitem>
-	  <para>Copying one node's profile to another
-	  node.</para>
-
-	  <para>
-	    In many environments, many nodes are instances of a common
-	    configuration specification. They all have similar roles
-	    and software. In our environment, desktop machines were
-	    the best example of this. Other than strictly per-host
-	    configuration like SSH keys, all desktop machines use a
-	    common configuration specification. This trivializes the
-	    process of creating a new desktop machine.
-	  </para>
-	</listitem>
-	<listitem>
-	  <para>Creating a specialized version of a
-	  currently existing profile. 
-	  </para>
-
-	  <para>
-	    In environments with highly varied configurations,
-	    departmental infrastructure being a good example, "another
-	    machine like X but with extra software" is a common
-	    requirement. For this reason, it must be trivially
-	    possible to inherit most of a configuration specification
-	    from some more generic source, while being able to
-	    describe overriding aspects in a convenient fashion.
-	  </para>
-	</listitem>
-	<listitem>
-	  <para>
-	    Compose several pre-existing configuration aspects to
-	    create a new profile.
-	  </para>
-
-	  <para>
-	    The ability to compose configuration aspects allows the
-	    easy creation of new profiles based on a series of
-	    predefined set of configuration specification
-	    fragments. The end result is more agility in environments
-	    where change is the norm.
-	  </para>
-	</listitem>
-      </orderedlist>
-
-      <para>
-	In order for a classing system to be comprehensive, it must be
-	usable in complex ways. The Bcfg2 metadata system has
-	constructs that map cleanly to first-order logic. This implies
-	that any complex configuration pattern can be represented (at
-	all) by the metadata, as first-order logic is provably
-	comprehensive. (There is a discussion later in the document
-	describing the metadata system in detail, and showing how it
-	corresponds to first-order logic)
-      </para>
-
-      <para>
-	These use cases motivate several of the design decisions that
-	we made:
-      </para>
-
-      <orderedlist>
-	<listitem>
-	  <para>
-	    There must be a many to one correspondence between clients
-	    and groups. Membership in a given profile group must imbue
-	    a client with all of its configuration properties. 
-	  </para>
-	  </listitem>
-      </orderedlist>
-    </section>
-
-    <section id='sec:pkg-management'>
-      <title>Package Management</title>
-
-      <para>
-	The interface provided in the bcfg2 repository for package
-	specification was designed with automation in mind. The goal
-	was to support an append only interface to the repository, so
-	that users do not need to continuously re-write already
-	existing bits of specification.
-      </para>
-    </section>
-  </section>
-</chapter>