diff options
author | Sol Jerome <solj@ices.utexas.edu> | 2010-02-08 19:24:43 +0000 |
---|---|---|
committer | Sol Jerome <solj@ices.utexas.edu> | 2010-02-08 19:24:43 +0000 |
commit | 29a28347e809c502fb359a3d6852faa9e89dff6c (patch) | |
tree | c4a1476308a2ddcaf68697bc730f55b34d917b8b /doc | |
parent | 42cffe6228369a75f27c14eea22ad13baf2c0854 (diff) | |
download | bcfg2-29a28347e809c502fb359a3d6852faa9e89dff6c.tar.gz bcfg2-29a28347e809c502fb359a3d6852faa9e89dff6c.tar.bz2 bcfg2-29a28347e809c502fb359a3d6852faa9e89dff6c.zip |
doc: Migrate some more docs over from the wiki
Signed-off-by: Sol Jerome <solj@ices.utexas.edu>
git-svn-id: https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2@5717 ce84e21b-d406-0410-9b95-82705330c041
Diffstat (limited to 'doc')
-rw-r--r-- | doc/client/paranoid.txt | 38 | ||||
-rw-r--r-- | doc/contents.txt | 1 | ||||
-rw-r--r-- | doc/development/index.txt | 304 | ||||
-rw-r--r-- | doc/server/plugins/grouping/grouppatterns.txt | 92 | ||||
-rw-r--r-- | doc/unsorted/bundler_examples.txt | 102 | ||||
-rw-r--r-- | doc/unsorted/index.txt | 17 |
6 files changed, 537 insertions, 17 deletions
diff --git a/doc/client/paranoid.txt b/doc/client/paranoid.txt new file mode 100644 index 000000000..1bf8c4fa5 --- /dev/null +++ b/doc/client/paranoid.txt @@ -0,0 +1,38 @@ +.. -*- mode: rst -*- + +.. _client-paranoid: + +============= +Paranoid Mode +============= + +What is paranoid mode? +====================== + +Paranoid mode creates a backup of a local configuration file before +Bcfg2 replaces the file. This allows for easier recovery by the local +administrator. + +How do I use it? +================ + +#. In the Bcfg2 repository, put `paranoid='true'` in the `info.xml` file. +#. On the client, create `/var/cache/bcfg2` (in 1.0, you can specify an alternate path in the [paranoid] section of /etc/bcfg2). +#. On the client, run `bcfg2` with the `-P` option. + +This will save a copy of the replaced file in `/var/cache/bcfg2`, but +it'll be named as the path to the file with /'s replaced by _'s. For +example, the old `/etc/hosts` will be named `/var/cache/bcfg2/etc_hosts`. + +Extra configuration +=================== + +Here is an example of how to use the extra paranoid features available +in 1.0. For the following section in bcfg2.conf:: + + [paranoid] + path = /my/custom/backup/path + max_copies = 5 + +You will have the file backups store in /my/custom/backup/path. This +will also keep the five most recent backups of files. diff --git a/doc/contents.txt b/doc/contents.txt index dd1c98423..1509cdd3a 100644 --- a/doc/contents.txt +++ b/doc/contents.txt @@ -17,6 +17,7 @@ Welcome to Bcfg2's documentation! server/index client/index gettinghelp + development/index unsorted/index diff --git a/doc/development/index.txt b/doc/development/index.txt new file mode 100644 index 000000000..4e2f29a54 --- /dev/null +++ b/doc/development/index.txt @@ -0,0 +1,304 @@ +.. -*- mode: rst -*- + +.. _development-index: + +Bcfg2 Development +================= + +There are many ways to get involved in Bcfg2 development. Here we will +outline some things that can help you get familiar with the various +areas of the Bcfg2 code. + +Tips for Bcfg2 Development +-------------------------- + +#. Focus on either the client or server code. This focuses the development process down to the precise pieces of code that matter for the task at hand. + + * If you are developing a client driver, then write up a small configuration specification that includes the needed characteristics. + * If you are working on the server, run bcfg2-info and use to assess the code + +#. Use the python interpreter. One of python's most appealing features is interactive use of the interpreter. + + * If you are developing for the client-side, run ``python -i /usr/sbin/bcfg2`` with the appropriate bcfg2 options. This will cause the python interpreter to continue running, leaving all variables intact. This can be used to examine data state in a convenient fashion. + * If you are developing for the server side, use bcfg2-info and the "debug" option. This will leave you at a python interpreter prompt, with the server core loaded in the variable "bcore". + +#. Use pylint obsessively. It raises a lot of style-related warnings which can be ignored, but most all of the errors are legitimate. +#. If you are doing anything with Regular Expressions, [http://kodos.sourceforge.net/ Kodos - The Python Regular Expression Debugger] and [http://re-try.appspot.com/ re-try] are your friends. + +Environment setup for development +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* Check out a copy of the code:: + + svn co https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2 + +* Create link to src/lib:: + + cd bcfg2 + ln -s src/lib Bcfg2 + +* Add ``bcfg2/src/sbin`` to your PATH environment variable +* Add ``bcfg2`` to your PYTHONPATH environment variable + +Writing A Client Tool Driver +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +This page describes the step-by-step process of writing a client tool +driver for a configuration element type. The included example describes +an existing driver, and the process that was used to create it. + +#. Pick a name for the driver. In this case, we picked the name RPM. +#. Add "RPM" to the __all__ list in src/lib/Client/Tools/__init__.py +#. Create a file in src/lib/Client/Tools with the same name (RPM.py) +#. Create a class in this file with the same name (class RPM) + + * If it handles Package entries, subclass Bcfg2.Client.Tools.PkgTool (from here referenced as branch [P]) + * If it handles Service entries, subclass Bcfg2.Client.Tools.SvcTool (from here referenced as branch [S]) + * Otherwise, subclass Bcfg2.Client.Tools.Tool (from here referenced as branch [T]) + +#. Set __name__ to "RPM" +#. Add any required executable programs to __execs__ +#. Set __handles__ to a list of (entry.tag, entry.get('type')) tuples. This determines which entries the Tool module can be used on. In this case, we set __handles__ = [('Package', 'rpm')]. +#. Add verification. This method should return True/False depending on current entry installation status. + + * [T] Add a Verify<entry.tag> method. + * [P] Add a VerifyPackage method. + * [S] Add a VerifyService method. + * In the failure path, the current state of failing entry attributes should be set in the entry, to aid in auditing. [[BR]] (For example, if a file should be mode 644, and is currently mode 600, then set attribute current_perms='600' in the input entry) + +#. Add installation support. This method should return True/False depending on the results of the installation process. + + * [T,S] Add an Install<entry.tag> method. + * [P] The PkgTool baseclass has a generic mechanism for performing all-at-once installations, followed, in the case of failures, by single installations. To enable this support, set the pkgtype attribute to the package type handled by this driver. Set the pkgtool to a tuple ("command string %s", ("per-package string format", [list of package entry fields])). For RPM, we have setup pkgtool = ("rpm --oldpackage --replacepkgs --quiet -U %s", ("%s", ["url"])) + +#. Implement entry removal + + * [T,S] Implement a Remove method that removes all specified entries (prototype Remove(self, entries)) + * [P] Implement a !RemovePackages that removes all specified entries (same prototype as Remove) + +#. Add a FindExtra method that locates entries not included in the configuration. This may or may not be required, certain drivers do not have the capability to find extra entries. +#. [P] Package drivers require a !RefreshPackages method that updates the internal representation of the package database. + +Writing Tool Driver Methods +""""""""""""""""""""""""""" + +#. Programs can be run using self.cmd.run. This function returns a (return code, stdout list) tuple. +#. The configuration is available as self.config +#. Runtime options are available in a dictionary as self.setup +#. Informational, error, and debug messages can be produced by running self.logger.info/error/debug. + +Bcfg2 Plugin development +------------------------ + +While the Bcfg2 server provides a good interface for representing +general system configurations, its plugin interface offers the ability +to implement configuration interfaces and representation tailored to +problems encountered by a particular site. This chapter describes what +plugins are good for, what they can do, and how to implement them. + +Bcfg2 Plugins +^^^^^^^^^^^^^ + +Bcfg2 plugins are loadable python modules that the Bcfg2 server loads at +initialization time. These plugins can contribute to the functions already +offered by the Bcfg2 server or can extend its functionality. In general, +plugins will provide some portion of the configuration for clients, with a +data representation that is tuned for a set of common tasks. Much of the +core functionality of Bcfg2 is implemented by several plugins, however, +they are not special in any way; new plugins could easily supplant one +or all of them. + +The following table describes the various functions of bcfg2 plugins. + ++--------------------+---------------------------------------------+ +| Name | Description | ++====================+=============================================+ +| Probes | Plugins can issue commands to collect | +| | client-side state (like hardware inventory) | +| | to include in client configurations | ++--------------------+---------------------------------------------+ +| ConfigurationEntry | Plugins can construct a list of per-client | +| List | configuration entry lists to include in | +| | client configurations. | ++--------------------+---------------------------------------------+ +| ConfigurationEntry | Literal values for configuration entries | +| contents | | ++--------------------+---------------------------------------------+ +| XML-RPC functions | Plugins can export function calls that | +| | expose internal functions. | ++--------------------+---------------------------------------------+ + +Writing Bcfg2 Plugins +^^^^^^^^^^^^^^^^^^^^^ + +Bcfg2 plugins are python classes that subclass from +Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must be set +in the new plugin. These values dictate how the new plugin will behave +with respect to the above four functions. The following table describes +all important member fields. + ++-----------------+-----------------------------------+--------------------------+ +| Name | Description | Format | ++=================+===================================+==========================+ +| __name__ | The name of the plugin | string | ++-----------------+-----------------------------------+--------------------------+ +| __version__ | The plugin version (generally | string | +| | tied to revctl keyword expansion) | | ++-----------------+-----------------------------------+--------------------------+ +| __author__ | The plugin author. | string | ++-----------------+-----------------------------------+--------------------------+ +| __author__ | The plugin author. | string | ++-----------------+-----------------------------------+--------------------------+ +| __rmi__ | Set of functions to be exposed as | List of function names | +| | XML-RPC functions | (strings) | ++-----------------+-----------------------------------+--------------------------+ +| Entries | Multidimentional dictionary of | Dictionary of | +| | keys that point to the function | ConfigurationEntityType, | +| | used to bind literal contents for | Name keys, and function | +| | a given configuration entity. | reference values | ++-----------------+-----------------------------------+--------------------------+ +| BuildStructures | Function that returns a list of | Member function | +| | the structures for a given client | | ++-----------------+-----------------------------------+--------------------------+ +| GetProbes | Function that returns a list of | Member function | +| | probes that a given client should | | +| | execute | | ++-----------------+-----------------------------------+--------------------------+ +| ReceiveData | Function that accepts the probe | Member function | +| | results for a given client. | | ++-----------------+-----------------------------------+--------------------------+ + +Example Plugin +^^^^^^^^^^^^^^ + +.. code-block:: python + + import Bcfg2.Server.Plugin + class MyPlugin(Bcfg2.Server.Plugin.Plugin): + '''An example plugin''' + # All plugins need to subclass Bcfg2.Server.Plugin.Plugin + __name__ = 'MyPlugin' + __version__ = '1' + __author__ = 'me@me.com' + __rmi__ = ['myfunction'] + # myfunction is not available remotely as MyPlugin.myfunction + + def __init__(self, core, datastore): + Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) + self.Entries = {'ConfigFile':{'/etc/foo.conf': self.buildFoo}} + + def myfunction(self): + '''function for xmlrpc rmi call''' + #do something + return True + + def buildFoo(self, entry, metadata): + '''Bind per-client information into entry based on metadata''' + entry.attrib.update({'owner':'root', 'group':'root', 'perms':'644'}) + entry.text = '''contents of foo.conf''' + +Example Connector +^^^^^^^^^^^^^^^^^ + +.. code-block:: python + + import Bcfg2.Server.Plugin + + class Foo(Bcfg2.Server.Plugin.Plugin, + Bcfg2.Server.Plugin.Connector): + '''The Foo plugin is here to illustrate a barebones connector''' + name = 'Foo' + version = '$Revision: $' + experimental = True + + def __init__(self, core, datastore): + Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) + Bcfg2.Server.Plugin.Connector.__init__(self) + self.store = XMLFileBacked(self.data, core.fam) + + def get_additional_data(self, metadata): + + mydata = {} + for data in self.store.entries['foo.xml'].data.get("foo", []): + + mydata[data] = "bar" + + return dict([('mydata', mydata)]) + + def get_additional_groups(self, meta): + return self.cgroups.get(meta.hostname, list()) + +Server Plugin Types +------------------- + +Generator +^^^^^^^^^ + +Generator plugins contribute to literal client configurations + +Structure +^^^^^^^^^ + +Structure Plugins contribute to abstract client configurations + +Metadata +^^^^^^^^ + +Signal metadata capabilities + +Connector +^^^^^^^^^ + +Connector Plugins augment client metadata instances + +Probing +^^^^^^^ + +Signal probe capability + +Statistics +^^^^^^^^^^ + +Signal statistics handling capability + +Decision +^^^^^^^^ + +Signal decision handling capability + +Version +^^^^^^^ + +Interact with various version control systems + +Writing Server Plugins +---------------------- + +Metadata +^^^^^^^^ + +If you would like to define your own Metadata plugin (to extend/change +functionality of the existing Metadata plugin), here are the steps to +do so. We will call our new plugin `MyMetadata`. + +#. Add MyMetadata.py + + .. code-block:: python + + __revision__ = '$Revision$' + + import Bcfg2.Server.Plugins.Metadata + + class MyMetadata(Bcfg2.Server.Plugins.Metadata.Metadata): + '''This class contains data for bcfg2 server metadata''' + __version__ = '$Id$' + __author__ = 'bcfg-dev@mcs.anl.gov' + + def __init__(self, core, datastore, watch_clients=True): + Bcfg2.Server.Plugins.Metadata.Metadata.__init__(self, core, datastore, watch_clients) + +#. Add MyMetadata to ``src/lib/Server/Plugins/__init__.py`` +#. Replace Metadata with MyMetadata in the plugins line of bcfg2.conf + +.. toctree:: + :maxdepth: 1 diff --git a/doc/server/plugins/grouping/grouppatterns.txt b/doc/server/plugins/grouping/grouppatterns.txt new file mode 100644 index 000000000..384b00c5e --- /dev/null +++ b/doc/server/plugins/grouping/grouppatterns.txt @@ -0,0 +1,92 @@ +.. -*- mode: rst -*- + +.. _server-plugins-grouping-grouppatterns: + +============= +GroupPatterns +============= + +The GroupPatterns plugin is a connector that can assign clients +group membership pased on patterns in client hostnames. Two basic +methods are supported: regular expressions (NamePatterns) and ranges +(NameRange). Hosts that match the specification are placed in the group +or groups specified by the pattern. + +Setup +===== + +#. Enable the GroupPatterns plugin +#. Create the GroupPatterns/config.xml file (similar to the example below). +#. Client groups will be augmented based on the specification + +Pattern Types +============= + +NamePatterns use regular expressions to match client hostnames. All +matching clients are placed in the resulting groups. NamePatterns also +have the ability to use regular expression matched groups to dynamically +create group names. The first two examples below are NamePatterns. The +first adds client hostname to both groups gp-test1 and gp-test2. The +second matches the hostname as a group and places the client in a group +called group-<hostname>. + +NameRange patterns allow the use of the application of numeric ranges +to host names. The final pattern below matches any of node1-node32 and +places them all into the rack1 group. Dynamically generated group names +are not supported with NameRange. + +Examples +======== + +.. code-block:: xml + + <GroupPatterns> + <GroupPattern> + <NamePattern>hostname</NamePattern> + <Group>gp-test1</Group> + <Group>gp-test2</Group> + </GroupPattern> + <GroupPattern> + <NamePattern>(.*)</NamePattern> + <Group>group-$1'</Group> + </GroupPattern> + <GroupPattern> + <NameRange>node[[1-32]]</NameRange> + <Group>rack1</Group> + </GroupPattern> + </GroupPatterns> + +Cluster Example +--------------- + +Functional aspects are extracted from hostname strings, and dynamic +groups are created. + +Expected hostname to group mapping:: + + xnfs1.example.com -> nfs-server + xnfs2.example.com -> nfs-server + xlogin1.example.com -> login-server + xlogin2.example.com -> login-server + xpvfs1.example.com -> pvfs-server + xpvfs2.example.com -> pvfs-server + xwww.example.com -> www-server + +GroupPatterns configuration: + +.. code-block:: xml + + <GroupPatterns> + <GroupPattern> + <NamePattern>^x(\w[^\d|\.]+)\d*\..*</NamePattern> + <Group>$1-server</Group> + </GroupPattern> + </GroupPatterns> + +Regex explanation: + +#. !^x Match any hostname that begins with "x" +#. (\w[!^\d|\.]+) followed by one or more word characters that are not a decimal digit or "." and save the string to $1 +#. \d* followed by 0 or more decimal digit(s) +#. \..* followed by a "." +#. .* followed by 1 or more of anything else. diff --git a/doc/unsorted/bundler_examples.txt b/doc/unsorted/bundler_examples.txt new file mode 100644 index 000000000..88db94f56 --- /dev/null +++ b/doc/unsorted/bundler_examples.txt @@ -0,0 +1,102 @@ +.. -*- mode: rst -*- + +.. _unsorted-bundler_examples: + +================ +Bundler Examples +================ + +Some simple examples of Bundles can be found in the example repository +at the locations in the following table: + ++--------------+---------------------+ +| Bundle Name | Description | ++==============+=====================+ +| `atxml`_ | At bundle | ++--------------+---------------------+ +| `bcfgxml`_ | Bcfg2 client bundle | ++--------------+---------------------+ +| `ntpxml`_ | NTP bundle | ++--------------+---------------------+ +| `sshxml`_ | OpenSSH bundle | ++--------------+---------------------+ +| `syslogxml`_ | syslog bundle | ++--------------+---------------------+ + +.. _atxml: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/repository/Bundler/at.xml +.. _bcfgxml: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/repository/Bundler/bcfg.xml +.. _ntpxml: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/repository/Bundler/ntp.xml +.. _sshxml: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/repository/Bundler/ssh.xml +.. _syslogxml: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/repository/Bundler/syslog.xml + +In addition to the example repository, the following is a list of some more complex example Bundles. To add your own example, create a wiki page called Plugins/Bundler/examples/<examplename> + +[[TitleIndex(Plugins/Bundler/examples/)]] + +=========================== +Fun and Profit using altsrc +=========================== + +Altsrc is a generic, bcfg2-server-side mechanism for performing configuration entry name remapping for the purpose of data binding. + +Use Cases +========= + +* Equivalent configuration entries on different architectures with different names +* Mapping entries with the same name to different bind results in a configuration (two packages with the same name but different types) +* A single configuration entry across multiple specifications (multi-plugin, or multi-repo) + +Examples +======== + +* Consider the case of /etc/hosts on linux and /etc/inet/hosts on solaris. These files contain the same data in the same format, and should typically be synchronized, however, exist in different locations. Classically, one would need to create one entry for each in Cfg or TCheetah and perform manual synchronization. Or, you could use symlinks and pray. Altsrc is driven from the bundle side. For example: + + .. code-block:: xml + + <Bundle name='netinfo'> + <Group name='solaris'> + <Path name='/etc/inet/hosts' altsrc='/etc/hosts'/> + </Group> + <Group name='linux'> + <Path name='/etc/hosts'/> + </Group> + </Bundle> + + In this case, when a solaris host gets the 'netinfo' bundle, it will get the first Path entry, which includes an altsrc parameter. This will cause the server to bind the entry as if it were a Path called '/etc/hosts'. This configuration entry is still called '/etc/inet/hosts', and is installed as such. + +* On encap systems, frequently multiple packages of the same name, but of different types will exist. For example, there might be an openssl encap package, and an openssl rpm package. This can be dealt with using a bundle like: + + .. code-block:: xml + + <Bundle name='openssl'> + <Package name='openssl' altsrc='openssl-encap'/> + <Package name='openssl' altsrc='openssl-rpm'/> + </Bundle> + + This bundle will bind data for the packages "openssl-encap" and "openssl-rpm", but will be delivered to the client with both packages named "openssl" with different types. + +* Finally, consider the case where there exist complicated, but completely independent specifications for the same configuration entry but different groups of clients. The following bundle will allow the use of two different TCheetah templates /etc/firewall-rules-external and /etc/firewall-rules-internal for different clients based on their group membership. + + .. code-block:: xml + + <Bundle name='firewall'> + ... + <Group name='conduit'> + <Path name='/etc/firewall-rules' altsrc='/etc/firewall-rules-external'/> + </Group> + <Group name='internal'> + <Path name='/etc/firewall-rules' altsrc='/etc/firewall-rules-internal'/> + </Group> + </Bundle> + +* Consider the case where a variety of files can be constructed by a single template (TCheetah or TGenshi). It would be possible to copy this template into the proper location for each file, but that requires proper synchronization upon modification and knowing up front what the files will all be called. Instead, the following bundle allows the use of a single template for all proper config file instances. + + .. code-block:: xml + + <Bundle name='netconfig'> + <Path name='/etc/sysconfig/network-scripts/ifcfg-eth0' altsrc='/etc/ifcfg-template'/> + <Path name='/etc/sysconfig/network-scripts/ifcfg-eth1' altsrc='/etc/ifcfg-template'/> + <Path name='/etc/sysconfig/network-scripts/ifcfg-eth2' altsrc='/etc/ifcfg-template'/> + </Bundle> + + altsrc can be used as a parameter for any entry type, and can be used in any structure, including Bundler, Base, and SGenshi. diff --git a/doc/unsorted/index.txt b/doc/unsorted/index.txt index 17feb13e6..364a10407 100644 --- a/doc/unsorted/index.txt +++ b/doc/unsorted/index.txt @@ -16,12 +16,7 @@ list below. * `ManualPages` * `NewDynamicReports` * `NewDynamicReportsInstallation` -* `News` -* `Overview` -* `ParanoidMode` -* `Plugins/Account` * `Plugins/Actions` -* `Plugins/Bundler/examples` * `Plugins/Bundler/examples/kernel.xml` * `Plugins/Bundler/examples/moab.xml` * `Plugins/Bundler/examples/nagios.xml` @@ -29,13 +24,7 @@ list below. * `Plugins/Bundler/examples/snmpd.xml` * `Plugins/Bundler/examples/torque.xml` * `Plugins/Bundler/examples/yp.xml` -* `Plugins/GroupPatterns` -* `Plugins/Hostbase` -* `Plugins/Metadata` -* `Plugins/NagiosGen` * `Plugins/Ohai` -* `Plugins/Probes` -* `Plugins/Probes/examples` * `Plugins/Probes/examples/current-kernel` * `Plugins/Probes/examples/group` * `Plugins/Probes/examples/grub-serial-order` @@ -43,10 +32,7 @@ list below. * `Plugins/Probes/examples/producttype` * `Plugins/Probes/examples/serial-console-speed` * `Plugins/Properties` -* `Plugins/Rules` -* `Plugins/SSHbase` * `Plugins/Snapshots` -* `Plugins/TGenshi/examples` * `Plugins/TGenshi/examples/bcfg2_cron` * `Plugins/TGenshi/examples/clients.xml` * `Plugins/TGenshi/examples/ganglia` @@ -62,8 +48,6 @@ list below. * `Publications` * `QuickStart2` * `QuickStart3` -* `RecentChanges` -* `RefactorClient` * `SchemaEvolution` * `SecurityDevPlan` * `ServerSideOverview` @@ -74,7 +58,6 @@ list below. * `UpgradeTesting` * `UsingRcache` * `VimSnippetSupport` -* `WritingClientToolDrivers` .. toctree:: :maxdepth: 2 |