diff options
105 files changed, 4539 insertions, 1098 deletions
diff --git a/doc/appendix/articles.txt b/doc/appendix/articles.txt new file mode 100644 index 000000000..4e31dd7ed --- /dev/null +++ b/doc/appendix/articles.txt @@ -0,0 +1,25 @@ +.. -*- mode: rst -*- + +.. _appendix-articles: + +======== +Articles +======== + +* Configuration and change management with Bcfg2: "The Dean" - The powerful Bcfg2 provides a sophisticated environment for centralized configuration management. + + * Marko Jung, Nils Magnus + * In the english 'Linux Magazine', 04/09, pages 30-35, April 2009 + * The `Bcfg2 code listings <ftp://ftp.linux-magazin.com/pub/listings/magazine/101/bcfg2/>`_ for the article are public. + +* `Konfigurations- und Change-Management in Bcfg2 <http://www.linux-magazin.de/Heft-Abo/Ausgaben/2008/10/Abstrakte-Avantgarde?category=0>`_ + + * Marko Jung, Nils Magnus + * In the german 'Linux Magazin', 10/08, pages 76-80, September 2008 + * The `code listings <http://www.linux-magazin.de/static/listings/magazin/2008/10/bcfg2/>`_ for the article are public. + +* `System Management Methodologies with Bcfg2 <ftp://ftp.mcs.anl.gov/pub/bcfg/papers/login-reports.pdf>`_ + + * Narayan Desai, Rick Bradshaw and Joey Hagedorn + * In ;login: Magazine, Volume 31, #1, pages 11-18, February 2006 + diff --git a/doc/appendix/books.txt b/doc/appendix/books.txt new file mode 100644 index 000000000..7cb864810 --- /dev/null +++ b/doc/appendix/books.txt @@ -0,0 +1,12 @@ +.. -*- mode: rst -*- + +.. _appendix-books: + +===== +Books +===== + +* `Configuration Management with Bcfg2 <http://www.sage.org/pubs/19_bcfg2/>`_ + + * Narayan Desai and Cory Lueninghoener + diff --git a/doc/appendix/configuration.txt b/doc/appendix/configuration.txt new file mode 100644 index 000000000..3c3258514 --- /dev/null +++ b/doc/appendix/configuration.txt @@ -0,0 +1,15 @@ +.. -*- mode: rst -*- + +.. _appendix-configuration: + +===================== +Example configuration +===================== + +This section contains useful configuration of additional tools. + +.. toctree:: + :maxdepth: 1 + :glob: + + configuration/* diff --git a/doc/unsorted/mrepo.txt b/doc/appendix/configuration/mrepo.txt index 61c3fd8c8..0633af98e 100644 --- a/doc/unsorted/mrepo.txt +++ b/doc/appendix/configuration/mrepo.txt @@ -1,23 +1,23 @@ .. -*- mode: rst -*- -.. _unsorted-mrepo: +.. _mrepo: http://dag.wieers.com/home-made/mrepo/ + +.. _appendix-configuration-mrepo: -===== mrepo ===== -This page describes how to setup an `mrepo`_ mirror. +This section describes how to setup an `mrepo`_ mirror. -.. _mrepo: http://dag.wieers.com/home-made/mrepo/ - -mrepo builds a local APT/Yum RPM repository from local ISO files, +`mrepo`_ builds a local APT/Yum RPM repository from local ISO files, downloaded updates, and extra packages from 3rd party repositories. It takes care of setting up the ISO files, downloading the RPMs, configuring HTTP access and providing PXE/TFTP resources for remote network installations. + Sample mrepo configuration -========================== +-------------------------- :: @@ -63,9 +63,9 @@ Sample mrepo configuration ### Examples can be found in the documentation at: ### /usr/share/doc/mrepo-0.8.6/dists/. -Run mrepo to update the repositories -==================================== +Update the repositories +----------------------- -:: +To update your local repository, just lauch the following command :: mrepo -ug diff --git a/doc/appendix/contributors.txt b/doc/appendix/contributors.txt new file mode 100644 index 000000000..88246b513 --- /dev/null +++ b/doc/appendix/contributors.txt @@ -0,0 +1,53 @@ +.. -*- mode: rst -*- + +.. _AUTHORS: http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/bcfg2/AUTHORS +.. _MCS: http://www.mcs.anl.gov/ + +.. _appendix-contributors: + +============ +Contributors +============ + +.. + This is list is no longer in chronological order like the + AUTHORS file because it's easier to maintain (for me). + Automatically sorted. + +In alphabetical order of the given name: + +- Brian Pellin and Andrew Lusk did substantial work on Bcfg1, some of which was used in the bcfg2 client. +- Chris Vuletich <vuletich@mcs.anl.gov> wrote some SSL code and the verification debugging code +- Cory Lueninghoener <cory@mcs.anl.gov> wrote the showentries function in ``bcfg2-info`` +- Daniel Clark <dclark@pobox.com> created encap packages for bcfg2 and deps, wrote fossil-scm dvcs support, and helps with Debian packaging +- Danny Clark enabled the Encap packaging +- David Dahl worked on Hostbase +- David Strauss worked on CentOS, RHEL, Yum, and Bazaar VCS support +- Ed Smith <esmith4@inf.ed.ac.uk> has done substantial hardening of the bcfg client and server and implemented a common logging infrastructure. +- Fabian Affolter <fabian@bernewireless.net> made some patches and worked on the manual +- Jason Pepas <cell@ices.utexas.edu> has written a RPM package list creator has contributed patches to the Red Hat toolset +- Joey Hagedorn <hagedorn@mcs.anl.gov> has written the reporting subsystem, including StatReports, GenerateHostinfo, and the xslt, css and javascript associated with it. +- Jos Catnook fixed bugs +- Ken Raffenetti <raffenet@mcs.anl.gov> and Rick Bradshaw have written the Hostbase plugin +- Michael Jinks <mjinks@uchicago.edu> wrote the Gentoo tool drivers +- Narayan Desai <desai@mcs.anl.gov> has written most of bcfg2, including all parts not explicitly mentioned in this file +- Patrick Ruckstuhl fixed bugs in the templating +- Pedro Flores made the Reporting system design help +- Rick Bradshaw <bradshaw@mcs.anl.gov> has written several of the tools included in the ``tools/`` subdirectory +- Sami Haahtinen <ressu@ressukka.net> has written Debian packaging logic. +- Scott Behrens <behrens@mcs.anl.gov> and Rick Bradshaw have written the VHost plugin +- Scott Matott +- Sol Jerome <solj@ices.utexas.edu> squashes bugs, helps manage the project roadmap, and implements various interesting features. +- Ti Leggett worked on ebuild packaging and bugfixes, RPM packaging +- Zach Lowry Solaris support and general hardening + + +The entire MCS_ systems team has provided invaluable help in the +design process and refinement of the user interface. In particular, +Gene Rackow and Sandra Bittner have provided great assistance +throughout this project. Philip Steinbachs provided detailed +feedback as an early external user. + +The most updated listing is available in the AUTHORS_ file in the +SVN :term:`repository` of Bcfg2. + diff --git a/doc/appendix/files.txt b/doc/appendix/files.txt new file mode 100644 index 000000000..e5217b684 --- /dev/null +++ b/doc/appendix/files.txt @@ -0,0 +1,16 @@ +.. -*- mode: rst -*- + +.. _appendix-files: + +============= +Example files +============= + +In this section are some examples for getting started with a more +indeep usage of Bcfg2. + +.. toctree:: + :maxdepth: 1 + :glob: + + files/* diff --git a/doc/appendix/files/mysql.txt b/doc/appendix/files/mysql.txt new file mode 100644 index 000000000..ae4a1450b --- /dev/null +++ b/doc/appendix/files/mysql.txt @@ -0,0 +1,63 @@ +.. -*- mode: rst -*- + +.. _getting_started-mysql: + +.. Author: Patrick Ruckstuhl + +Mysql example +============= + +I had some time ago to continue with putting my configuration into +Bcfg2 and maybe this helps someone else. + +I added a new bundle: + +.. code-block:: xml + + <Bundle name="mysql-server" version="3.0"> + <ConfigFile name="/root/bcfg2-install/mysql/users.sh"/> + <ConfigFile name="/root/bcfg2-install/mysql/users.sql"/> + <PostInstall name="/root/bcfg2-install/mysql/users.sh"/> + <Package name="mysql-server-4.1"/> + <Service name="mysql"/> + </Bundle> + +The ``users.sh`` script looks like this: + +.. code-block:: sh + + #!/bin/sh + + mysql --defaults-extra-file=/etc/mysql/debian.cnf mysql \ + < /root/bcfg2-install/mysql/users.sql + +On debian there is a user account in ``/etc/mysql/debian.cnf`` +automatically created, but you could also (manually) create a +user in the database that has enough permissions and add the +login information in a file yourself. This file looks like this: + +.. code-block:: sh + + [client] + host = localhost + user = debian-sys-maint + password = XXXXXXXXXX + +The ``users.sql`` looks like this: + +.. code-block:: sh + + DELETE FROM db; + INSERT INTO db VALUES ('localhost', 'phpmyadmin', 'pma', 'Y', 'Y', + 'Y', 'Y', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N'); + + DELETE FROM user WHERE User <> 'debian-sys-maint'; + INSERT INTO user VALUES ('localhost', 'root', 'XXXXXXXXXXX', 'Y', 'Y', + 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', 'Y', + 'Y', 'Y', 'Y', 'Y', 'Y', '', '', '', '', 0, 0, 0); + INSERT INTO user VALUES ('localhost', 'pma', '', 'N', 'N', 'N', 'N', + 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', 'N', + 'N', 'N', 'N', '', '', '', '', 0, 0, 0); + + FLUSH PRIVILEGES; + diff --git a/doc/appendix/files/ntp.txt b/doc/appendix/files/ntp.txt new file mode 100644 index 000000000..33cb3bfbb --- /dev/null +++ b/doc/appendix/files/ntp.txt @@ -0,0 +1,151 @@ +.. -*- mode: rst -*- + +.. _appendix-files-ntp: + +.. Author: Jason Pepas + +ntp example +=========== + +Here is a series of example configurations for Bcfg2, each introducing +another layer of functionality. + +* After each change, run ``bcfg-repo-validate -v`` +* Run the server with ``bcfg2-server -v`` +* Update the client with ``bcfg2 -v -d -n`` (will not actually make + client changes) + +Package only +------------ + +Our example starts with the bare minimum configuration setup. We have +a client, a profile group, a list of packages, and a base configuration. + +.. code-block:: sh + + # cat Metadata/clients.xml + <Clients version='3.0'> + <Client profile='fedora' pingable='N' pingtime='0' name='foo.bar.com'/> + </Clients> + +.. code-block:: sh + + # cat Metadata/groups.xml + <Groups version='3.0'> + <Group profile='true' name='fedora' toolset='rh'/> + </Groups> + +.. code-block:: sh + + # cat Base/base.xml + <Base> + <Group name='fedora'> + <Package name='ntp'/> + </Group> + </Base> + +.. code-block:: sh + + # cat Pkgmgr/packages.xml + <PackageList type='rpm' priority='0'> + <Package name='ntp' version='4.2.0.a.20050816-11.FC5'/> + </PackageList> + +Add service +----------- + +Configure the service, and add it to the base. + +.. code-block:: sh + + # cat Svcmgr/services.xml + <Services priority='0'> + <Service name='ntpd' status='on'/> + </Services> + +.. code-block:: sh + + # cat Base/base.xml + <Base> + <Group name='fedora'> + <Package name='ntp'/> + <Service name='ntpd'/> + </Group> + </Base> + +Add config file +--------------- + +Setup an ``etc/`` directory structure, and add it to the base. + +.. code-block:: sh + + # cat Cfg/etc/ntp.conf/ntp.conf + server ntp1.utexas.edu + +.. code-block:: sh + + # cat Base/base.xml + <Base> + <Group name='fedora'> + <Package name='ntp'/> + <Service name='ntpd'/> + <ConfigFile name='/etc/ntp.conf'/> + </Group> + </Base> + +Create a bundle +--------------- + +The above configuration layout works fine for a single service, but +that method of organization would quickly become a nightmare as you +approach the number of packages, services, and config files required +to represent a fully configured host. Bundles allow the grouping of +related configuration entries that are used to provide a single +service. This is done for several reasons: + +* Grouping related things in one place makes it easier to add those + entries for a multiple groups of clients +* Grouping entries into bundles makes their validation occur + collectively. This means that config files can override the + contents of packages. Also, config files are rechecked after + packages are upgraded, so that they can be repaired if the + package install clobbered them. +* Services associated with a bundle get restarted whenever any entity + in that bundle is modified. This ensures that new configuration + files and software are used after installation. + +The config file, package, and service are really all related +components describing the idea of an ntp client, so they should be +logically grouped together. We use a bundle to accomplish this. + +.. code-block:: sh + + # cat Bundler/ntp.xml + <Bundle name='ntp' version='2.0'> + <Package name='ntp'/> + <Service name='ntpd'/> + <ConfigFile name='/etc/ntp.conf'/> + </Bundle> + +After this bundle is created, it must be associated with a group +(or groups). Add a bundle child element to the group(s) which should +install this bundle. + +.. code-block:: sh + + # cat Metadata/groups.xml + <Groups> + ... + <Group name='fedora'> + <Bundle name='ntp'/> + </Group> + ... + </Groups> + +Once this bundle is created, a client reconfigure will install these +entries. If any are modified, then the ``ntpd`` service will be +restarted. If you only want ntp configurations to be updated +(and nothing else), the bcfg2 client can be run with a +``-b <bundle name>`` option that will only update entries in +the specified bundle. diff --git a/doc/appendix/guides.txt b/doc/appendix/guides.txt new file mode 100644 index 000000000..81e77cc1f --- /dev/null +++ b/doc/appendix/guides.txt @@ -0,0 +1,16 @@ +.. -*- mode: rst -*- + +.. _appendix-guides: + +====== +Guides +====== + +This section contains platform-specific quickstart guides and howtos +around Bcfg2. + +.. toctree:: + :maxdepth: 1 + :glob: + + guides/* diff --git a/doc/appendix/guides/authentication.txt b/doc/appendix/guides/authentication.txt new file mode 100644 index 000000000..b9efbb929 --- /dev/null +++ b/doc/appendix/guides/authentication.txt @@ -0,0 +1,143 @@ +.. -*- mode: rst -*- + +.. _guide-authentication: + +============== +Authentication +============== + +Scenarios +========= + +1. Cluster nodes that are frequently rebuilt + + Default settings work well; machines do not float, and a per-client + password is not required. + +2. :ref:`NAT Howto nat_howto` + + * Build client records in advance with ``bcfg2-admin``, setting a uuid + for each new client. + + * Set the address attribute for each to the address of the NAT. + + * Optionally, set a per-client password for each, and set into secure + mode. + + .. note:: + + This will require the use of the uuid and password from each + client, and will require that they come through the NAT address. + +Building bcfg2.conf automatically +================================= + +This is a TCheetah template that automatically constructs per-client +`bcfg2.conf` from the per-client metadata:: + + [communication] + protocol = xmlrpc/ssl + #if $self.metadata.uuid != None + user = $self.metadata.uuid + #end if + #if $self.metadata.password != None + password = $self.metadata.password + #else + password = my-password-foobar + #end if + + [components] + bcfg2 = https://localhost:6789 + +In this setup, this will cause any clients that have uuids established +to be set to use them in `bcfg2.conf`. It will also cause any clients +with passwords set to use them instead of the global password. + +How Authentication Works +======================== + +#. First, the client is associated with a client record. If the client + specifies a uuid, it uses this instead of the results of a dns or + address lookup. + +#. Next, the ip address is verified against the client record. If the + address doesn't match, then the client must be set to + location=floating + +#. Finally, the password is verified. If the client is set to secure + mode, the only its per-client password is accepted. If it is not set + to secure mode, then either the global password or per-client password + will be accepted + +Failure during any of these stages results in authentication +failure. Note that clients set into secure mode that do not have +per-client passwords set will not be able to connect. + +SSL Cert-based client authentication +==================================== + +SSL-based client authentication is supported. This requires several +things: + +#. Certificate Authority (to sign all keys) + +#. Server key and cert signed by the CA + +#. Client key and cert signed by the CA + +A variety of CAs can be used, but these keys can be simply generated +using the following set of steps: + +#. Setup a CA + + http://www.flatmtn.com/article/setting-openssl-create-certificates + +#. Create keys for each client and server, signing them with the CA + signing cert + + http://www.flatmtn.com/article/setting-ssl-certificates-apache + + .. note:: + The client CN must be the FQDN of the client (as returned by a + reverse DNS lookup of the ip address. Otherwise, you will end up + with an error message on the client that looks like:: + + Server failure: Protocol Error: 401 Unauthorized + Failed to download probes from bcfg2 + Server Failure + + You will also see an error message on the server that looks + something like:: + + cmssrv01 bcfg2-server[9785]: Got request for cmssrv115 from incorrect address 131.225.206.122 + cmssrv01 bcfg2-server[9785]: Resolved to cmssrv115.fnal.gov + +#. Distribute the keys and certs to the appropriate locations + +#. Copy the ca cert to clients, so that the server can be authenticated + +Clients authenticating themselves with a certificate will be +authenticated that way first; clients can be setup to either +authenticate solely with certs, use certs with a fallback to password, +or password only. Also a bootstrap mode will be added shortly; this +will allow a client to authenticate with a password its first time, +requiring a certificate all subsequent times. This behavior can be +controlled through the use of the auth attribute in +`Metadata/clients.xml`:: + + <Clients> + <Client name='testclient' auth='cert'/> + </Clients> + +Allowed values are: + + +---------------+------------------------------------------+ + | **Auth Type** | **Meaning** | + +---------------+------------------------------------------+ + | cert | Certificates must be used | + +---------------+------------------------------------------+ + | cert+password | Certificate or password may be used | + +---------------+------------------------------------------+ + | bootstrap | Password can be used for one client run, | + | | after that certificate is required | + +---------------+------------------------------------------+ diff --git a/doc/appendix/guides/centos.txt b/doc/appendix/guides/centos.txt new file mode 100644 index 000000000..91c1f268f --- /dev/null +++ b/doc/appendix/guides/centos.txt @@ -0,0 +1,565 @@ +.. -*- mode: rst -*- + +.. _EPEL: http://fedoraproject.org/wiki/EPEL + +.. _guide-centos: + +===================== +Quickstart for CentOS +===================== + +This is a complete getting started guide for CentOS. With this document +you should be able to install a Bcfg2 server and a Bcfg2 client. + +Install Bcfg2 +============= + +The fastest way to get Bcfg2 onto your system is to use Yum or +your preferred package management tool. We'll be using the ones +that are distributed through EPEL_, but depending on your aversion +to risk you could download an RPM from other places as well. See +:ref:`getting_started-using_bcfg2-with-centos` for information about +building Bcfg2 from source and making your own packages. + +Using EPEL +---------- + +Make sure EPEL_ is a valid repository on your server. The `instructions +<http://fedoraproject.org/wiki/EPEL/FAQ#howtouse>`_ on how to do this +basically say:: + + [root@centos ~]# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm + +.. note:: + + You will have to adjust this command to match your architecture and + the current EPEL release. + +Install the bcfg2-server and bcfg2 RPMs:: + + [root@centos ~]# yum install bcfg2-server bcfg2 + +Your system should now have the necessary software to use Bcfg2. The +next step is to set up your Bcfg2 :term:`repository`. + +Initialize your repository +========================== + +Now that you're done with the install, you need to initialize your +repository and setup your ``/etc/bcfg2.conf``. ``bcfg2-admin init`` +is a tool which allows you to automate this:: + + [root@centos ~]# bcfg2-admin init + Store bcfg2 configuration in [/etc/bcfg2.conf]: + Location of bcfg2 repository [/var/lib/bcfg2]: + Input password used for communication verification (without echoing; leave blank for a random): + What is the server's hostname: [centos] + Input the server location [https://centos:6789]: + Input base Operating System for clients: + 1: Redhat/Fedora/RHEL/RHAS/Centos + 2: SUSE/SLES + 3: Mandrake + 4: Debian + 5: Ubuntu + 6: Gentoo + 7: FreeBSD + : 1 + Generating a 2048 bit RSA private key + .........................+++ + ..................+++ + writing new private key to '/etc/bcfg2.key' + ----- + Signature ok + subject=/C=US=ST=Illinois/L=Argonne/CN=centos + Getting Private key + Repository created successfuly in /var/lib/bcfg2 + +Change responses as necessary. + +Start the server +================ + +You are now ready to start your bcfg2 server for the first time:: + + [root@centos ~]# /sbin/service bcfg2-server start + +To verify that everything started ok, look for the running daemon and check the logs:: + + [root@centos ~]# /etc/init.d/service bcfg2-server status + [root@centos ~]# tail /var/log/messages + Mar 29 12:42:26 centos bcfg2-server[5093]: service available at https://centos:6789 + Mar 29 12:42:26 centos bcfg2-server[5093]: serving bcfg2-server at https://centos:6789 + Mar 29 12:42:26 centos bcfg2-server[5093]: serve_forever() [start] + Mar 29 12:42:41 centos bcfg2-server[5093]: Handled 16 events in 0.007s + +Run bcfg2 to be sure you are able to communicate with the server:: + + [root@centos ~]# bcfg2 -vqn + No ca is specified. Cannot authenticate the server with SSL. + No ca is specified. Cannot authenticate the server with SSL. + Loaded plugins: fastestmirror + Loading mirror speeds from cached hostfile + Excluding Packages in global exclude list + Finished + Loaded tool drivers: + Action Chkconfig POSIX YUMng + + Phase: initial + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 + + + Phase: final + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 + + No ca is specified. Cannot authenticate the server with SSL. + +The ca message is just a warning, meaning that the client does not +have sufficient information to verify that it is talking to the +correct server. This can be fixed by distributing the ca certificate +from the server to all clients. By default, this file is available in +``/etc/bcfg2.crt`` on the server. Copy this file to the client (with a +bundle) and add the ca option to ``bcfg2.conf`` pointing at the file, +and the client will be able to verify it is talking to the correct server +upon connection:: + + [root@centos ~]# cat /etc/bcfg2.conf + + + [communication] + protocol = xmlrpc/ssl + password = N41lMNeW + ca = /etc/bcfg2.crt + + [components] + bcfg2 = https://centos:6789 + +Now if you run the client, no more warning:: + + [root@centos ~]# bcfg2 -vqn + Loaded plugins: fastestmirror + Loading mirror speeds from cached hostfile + Excluding Packages in global exclude list + Finished + Loaded tool drivers: + Action Chkconfig POSIX YUMng + + Phase: initial + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 + + + Phase: final + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 208 + +Bring your first machine under Bcfg2 control +============================================ + +Now it is time to get your first machine's configuration into your +Bcfg2 :term:`repository`. Let's start with the server itself. + + +Setup the `Packages`_ plugin +---------------------------- + +.. _Packages: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Packages + +First, replace **Pkgmgr** with **Packages** in the plugins +line of ``bcfg2.conf``. Then create Packages layout (as per +:ref:`packages-exampleusage`) in ``/var/lib/bcfg2`` + +.. note:: I am using the RawURL syntax here since we are using `mrepo`_ + to manage our yum mirrors. + +.. _mrepo: http://dag.wieers.com/home-made/mrepo/ + +.. code-block:: xml + + <Sources> + <!-- CentOS (5.4) sources --> + <YUMSource> + <Group>centos5.4</Group> + <RawURL>http://mrepo/centos5-x86_64/RPMS.os</RawURL> + <Arch>x86_64</Arch> + </YUMSource> + <YUMSource> + <Group>centos5.4</Group> + <RawURL>http://mrepo/centos5-x86_64/RPMS.updates</RawURL> + <Arch>x86_64</Arch> + </YUMSource> + <YUMSource> + <Group>centos5.4</Group> + <RawURL>http://mrepo/centos5-x86_64/RPMS.extras</RawURL> + <Arch>x86_64</Arch> + </YUMSource> + </Sources> + +Due to the `Magic Groups`_, we need to modify our Metadata. Let's +add a **centos5.4** group which inherits a **centos** group +(this should replace the existing **redhat** group) present in +``/var/lib/bcfg2/Metadata/groups.xml``. The resulting file should look +something like this + +.. _Magic Groups: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Packages#MagicGroups + +.. code-block:: xml + + <Groups version='3.0'> + <Group profile='true' public='true' default='true' name='basic'> + <Group name='centos5.4'/> + </Group> + <Group name='centos5.4'> + <Group name='centos'/> + </Group> + <Group name='ubuntu'/> + <Group name='debian'/> + <Group name='freebsd'/> + <Group name='gentoo'/> + <Group name='centos'/> + <Group name='suse'/> + <Group name='mandrake'/> + <Group name='solaris'/> + </Groups> + +.. note:: + When editing your xml files by hand, it is useful to occasionally run + `bcfg2-repo-validate` to ensure that your xml validates properly. + +The final thing we need is for the client to have the proper +arch group membership. For this, we will make use of the +:ref:`unsorted-dynamic_groups` capabilities of the Probes plugin. Add +Probes to your plugins line in ``bcfg2.conf`` and create the Probe.:: + + [root@centos ~]# grep plugins /etc/bcfg2.conf + plugins = Base,Bundler,Cfg,Metadata,Packages,Probes,Rules,SSHbase + [root@centos ~]# mkdir /var/lib/bcfg2/Probes + [root@centos ~]# cat /var/lib/bcfg2/Probes/groups + #!/bin/sh + + echo "group:`uname -m`" + +Now we restart the bcfg2-server:: + + [root@centos ~]# /etc/init.d/bcfg2-server restart + +If you tail ``/var/log/syslog`` now, you will see the Packages plugin in +action, updating the cache. + +Start managing packages +----------------------- + +Add a base-packages bundle. Let's see what happens when we just populate +it with the *yum* package. + +.. code-block:: xml + + [root@centos ~]# cat /var/lib/bcfg2/Bundler/base-packages.xml + <Bundle name='base-packages'> + <Package name='yum'/> + </Bundle> + +You need to reference the bundle from your Metadata. The resulting +profile group might look something like this + +.. code-block:: xml + + <Group profile='true' public='true' default='true' name='basic'> + <Bundle name='base-packages'/> + <Group name='centos5.4'/> + </Group> + +Now if we run the client, we can see what this has done for us.:: + + [root@centos ~]# bcfg2 -vqn + Running probe groups + Probe groups has result: + x86_64 + Loaded plugins: fastestmirror + Loading mirror speeds from cached hostfile + Excluding Packages in global exclude list + Finished + Loaded tool drivers: + Action Chkconfig POSIX YUMng + Package pam failed verification. + + Phase: initial + Correct entries: 94 + Incorrect entries: 1 + Total managed entries: 95 + Unmanaged entries: 113 + + In dryrun mode: suppressing entry installation for: + Package:pam + + Phase: final + Correct entries: 94 + Incorrect entries: 1 + Package:pam + Total managed entries: 95 + Unmanaged entries: 113 + +Interesting, our **pam** package failed verification. What does this +mean? Let's have a look:: + + [root@centos ~]# rpm --verify pam + ....L... c /etc/pam.d/system-auth + +Sigh, it looks like the default RPM install for pam fails to verify +using its own verification process (trust me, it's not the only one). At +any rate, I was able to get rid of this particular issue by removing the +symlink and running ``yum reinstall pam``. + +As you can see, the Packages plugin has generated the dependencies +required for the yum package automatically. The ultimate goal should +be to move all the packages from the **Unmanaged** entries section to +the **Managed** entries section. So, what exactly *are* those Unmanaged +entries?:: + + [root@centos ~]# bcfg2 -veqn + Running probe groups + Probe groups has result: + x86_64 + Loaded plugins: fastestmirror + Loading mirror speeds from cached hostfile + Excluding Packages in global exclude list + Finished + Loaded tool drivers: + Action Chkconfig POSIX YUMng + Extra Package openssh-clients 4.3p2-36.el5_4.4.x86_64. + Extra Package libuser 0.54.7-2.1el5_4.1.x86_64. + ... + + Phase: initial + Correct entries: 95 + Incorrect entries: 0 + Total managed entries: 95 + Unmanaged entries: 113 + + + Phase: final + Correct entries: 95 + Incorrect entries: 0 + Total managed entries: 95 + Unmanaged entries: 113 + Package:at + Package:avahi + Package:avahi-compat-libdns_sd + ... + +Now you can go through these and continue adding the packages you want +to your Bundle. After a while, I ended up with a minimal bundle that +looks like this + +.. code-block:: xml + + <Bundle name='base-packages'> + <Package name='bcfg2-server'/> + <Package name='exim'/> + <Package name='grub'/> + <Package name='kernel'/> + <Package name='krb5-workstation'/> + <Package name='m2crypto'/> + <Package name='openssh-clients'/> + <Package name='openssh-server'/> + <Package name='prelink'/> + <Package name='redhat-lsb'/> + <Package name='rpm-build'/> + <Package name='rsync'/> + <Package name='sysklogd'/> + <Package name='vim-enhanced'/> + <Package name='yum'/> + </Bundle> + +Now when I run the client, you can see I have only one unmanaged +package:: + + [root@centos ~]# bcfg2 -veqn + Running probe groups + Probe groups has result: + x86_64 + Loaded plugins: fastestmirror + Loading mirror speeds from cached hostfile + Excluding Packages in global exclude list + Finished + Loaded tool drivers: + Action Chkconfig POSIX YUMng + Extra Package gpg-pubkey e8562897-459f07a4.None. + Extra Package gpg-pubkey 217521f6-45e8a532.None. + + Phase: initial + Correct entries: 187 + Incorrect entries: 0 + Total managed entries: 187 + Unmanaged entries: 16 + + + Phase: final + Correct entries: 187 + Incorrect entries: 0 + Total managed entries: 187 + Unmanaged entries: 16 + Package:gpg-pubkey + Service:atd + Service:avahi-daemon + Service:bcfg2-server + ... + +The gpg-pubkey packages are special in that they are not really +packages. Currently, the way to manage them is using :ref:`BoundEntries +<boundentries>`. So, after adding them, our Bundle now looks like this + +.. note:: This does not actually control the contents of the files, + you will need to do this part separately (see below). + +.. code-block:: xml + + <Bundle name='base-packages'> + <BoundPackage name="gpg-pubkey" type="rpm"> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> + </BoundPackage> + <Package name='bcfg2-server'/> + <Package name='exim'/> + <Package name='grub'/> + <Package name='kernel'/> + <Package name='krb5-workstation'/> + <Package name='m2crypto'/> + <Package name='openssh-clients'/> + <Package name='openssh-server'/> + <Package name='prelink'/> + <Package name='redhat-lsb'/> + <Package name='rpm-build'/> + <Package name='rsync'/> + <Package name='sysklogd'/> + <Package name='vim-enhanced'/> + <Package name='yum'/> + </Bundle> + +To actually push the gpg keys out via Bcfg2, you will need to manage the +files as well. This can be done by adding Path entries for each of the +gpg keys you want to manage + +.. code-block:: xml + + <Bundle name='base-packages'> + <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5'/> + <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL'/> + <BoundPackage name="gpg-pubkey" type="rpm"> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> + </BoundPackage> + <Package name='bcfg2-server'/> + <Package name='exim'/> + <Package name='grub'/> + <Package name='kernel'/> + <Package name='krb5-workstation'/> + <Package name='m2crypto'/> + <Package name='openssh-clients'/> + <Package name='openssh-server'/> + <Package name='prelink'/> + <Package name='redhat-lsb'/> + <Package name='rpm-build'/> + <Package name='rsync'/> + <Package name='sysklogd'/> + <Package name='vim-enhanced'/> + <Package name='yum'/> + </Bundle> + +Then add the files to Cfg:: + + mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 + cp /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 !$/RPM-GPG-KEY-CentOS-5 + mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL + cp /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL !$/RPM-GPG-KEY-EPEL + +Now, running the client shows only unmanaged Service entries. Woohoo! + +Manage services +--------------- + +Now let's clear up the unmanaged service entries by adding the following +entries to our bundle. + +.. code-block:: xml + + <!-- basic services --> + <Service name='atd'/> + <Service name='avahi-daemon'/> + <Service name='bcfg2-server'/> + <Service name='crond'/> + <Service name='cups'/> + <Service name='gpm'/> + <Service name='lvm2-monitor'/> + <Service name='mcstrans'/> + <Service name='messagebus'/> + <Service name='netfs'/> + <Service name='network'/> + <Service name='postfix'/> + <Service name='rawdevices'/> + <Service name='sshd'/> + <Service name='syslog'/> + +...and bind them in Rules + +.. code-block:: xml + + [root@centos ~]# cat /var/lib/bcfg2/Rules/services.xml + <Rules priority='1'> + <!-- basic services --> + <Service type='chkconfig' status='on' name='atd'/> + <Service type='chkconfig' status='on' name='avahi-daemon'/> + <Service type='chkconfig' status='on' name='bcfg2-server'/> + <Service type='chkconfig' status='on' name='crond'/> + <Service type='chkconfig' status='on' name='cups'/> + <Service type='chkconfig' status='on' name='gpm'/> + <Service type='chkconfig' status='on' name='lvm2-monitor'/> + <Service type='chkconfig' status='on' name='mcstrans'/> + <Service type='chkconfig' status='on' name='messagebus'/> + <Service type='chkconfig' status='on' name='netfs'/> + <Service type='chkconfig' status='on' name='network'/> + <Service type='chkconfig' status='on' name='postfix'/> + <Service type='chkconfig' status='on' name='rawdevices'/> + <Service type='chkconfig' status='on' name='sshd'/> + <Service type='chkconfig' status='on' name='syslog'/> + </Rules> + +Now we run the client and see there are no more unmanaged entries! :: + + [root@centos ~]# bcfg2 -veqn + Running probe groups + Probe groups has result: + x86_64 + Loaded plugins: fastestmirror + Loading mirror speeds from cached hostfile + Excluding Packages in global exclude list + Finished + Loaded tool drivers: + Action Chkconfig POSIX YUMng + + Phase: initial + Correct entries: 205 + Incorrect entries: 0 + Total managed entries: 205 + Unmanaged entries: 0 + + + Phase: final + Correct entries: 205 + Incorrect entries: 0 + Total managed entries: 205 + Unmanaged entries: 0 + +Dynamic (web) reports +===================== + +See installation instructions at :ref:`server-reports-install` diff --git a/doc/unsorted/converging_rhel5.txt b/doc/appendix/guides/converging_rhel5.txt index 9d508e5e4..9d508e5e4 100644 --- a/doc/unsorted/converging_rhel5.txt +++ b/doc/appendix/guides/converging_rhel5.txt diff --git a/doc/appendix/guides/fedora.txt b/doc/appendix/guides/fedora.txt new file mode 100644 index 000000000..f3a5a3929 --- /dev/null +++ b/doc/appendix/guides/fedora.txt @@ -0,0 +1,477 @@ +.. -*- mode: rst -*- + +.. This guide is based on the Centos guide. + +.. _guide-fedora: + +====== +Fedora +====== + +This guide is work in progess. + + +This is a complete getting started guide for Fedora. With this +document you should be able to install a Bcfg2 server, a Bcfg2 client, +and change the ``/etc/motd`` file on the client. + +Install Bcfg2 From RPM +====================== + +The fastest way to get Bcfg2 onto your system is to use ``yum`` +or PackageKit. `` +um`` will pull all dependencies of Bcfg2 +automatically in. :: + + $ su -c 'yum install bcfg2-server bcfg2' + +Your system should now have the necessary software to use Bcfg2. +The next step is to set up your Bcfg2 :term:`repository`. + +Initialize your repository +========================== + +Now that you're done with the install, you need to initialize your +repository and setup your ``/etc/bcfg2.conf``. ``bcfg2-admin init`` +is a tool which allows you to automate this: + +.. code-block:: sh + + # bcfg2-admin init + Store bcfg2 configuration in [/etc/bcfg2.conf]: + Location of bcfg2 repository [/var/lib/bcfg2]: + Directory /var/lib/bcfg2 exists. Overwrite? [y/N]:y + Input password used for communication verification (without echoing; leave blank for a random): + What is the server's hostname: [config01.local.net] + Input the server location [https://config01.local.net:6789]: + Input base Operating System for clients: + 1: Redhat/Fedora/RHEL/RHAS/Centos + 2: SUSE/SLES + 3: Mandrake + 4: Debian + 5: Ubuntu + 6: Gentoo + 7: FreeBSD + : 1 + Generating a 1024 bit RSA private key + .......................................................++++++ + .....++++++ + writing new private key to '/etc/bcfg2.key' + ----- + Signature ok + subject=/C=US/ST=Illinois/L=Argonne/CN=config01.local.net + Getting Private key + Repository created successfuly in /var/lib/bcfg2 + +Change responses as necessary. + +Start the server +================ + +You are now ready to start your bcfg2 server for the first time:: + + $ su -c '/etc/init.d/bcfg2-server start' + Starting Configuration Management Server: bcfg2-server [ OK ] + +To verify that everything started ok, look for the running daemon and +check the logs: + +.. code-block:: sh + + $ su -c 'tail /var/log/messages' + May 16 14:14:57 config01 bcfg2-server[2746]: service available at https://config01.local.net:6789 + May 16 14:14:57 config01 bcfg2-server[2746]: serving bcfg2-server at https://config01.local.net:6789 + May 16 14:14:57 config01 bcfg2-server[2746]: serve_forever() [start] + May 16 14:14:57 config01 bcfg2-server[2746]: Handled 16 events in 0.009s + + +Run ``bcfg2`` to be sure you are able to communicate with the server: + +.. code-block:: sh + + $ su -c 'bcfg2 -vqne' + + /usr/lib/python2.6/site-packages/Bcfg2/Client/Tools/rpmtools.py:23: DeprecationWarning: the md5 module is deprecated; use hashlib instead + import md5 + Loaded plugins: presto, refresh-packagekit + Loaded tool drivers: + Action Chkconfig POSIX YUMng + Extra Package imsettings-libs 0.108.0-2.fc13.i686. + Extra Package PackageKit-device-rebind 0.6.4-1.fc13.i686. + ... + Extra Package newt-python 0.52.11-2.fc13.i686. + Extra Package pulseaudio-gdm-hooks 0.9.21-6.fc13.i686. + + Phase: initial + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 1314 + + + Phase: final + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 1314 + Package:ConsoleKit Package:jasper-libs Package:pcsc-lite-libs + Package:ConsoleKit-libs Package:java-1.5.0-gcj Package:perf + ... + Package:iw Package:pcre Service:sshd + Package:jack-audio-connection-kit Package:pcsc-lite Service:udev-post + +The ``bcfg2.conf`` file contains only standard plugins so far. + +.. code-block:: sh + + $ su -c 'cat /etc/bcfg2.conf' + + [server] + repository = /var/lib/bcfg2 + plugins = Base,Bundler,Cfg,Metadata,Pkgmgr,Rules,SSHbase + + [statistics] + sendmailpath = /usr/lib/sendmail + database_engine = sqlite3 + # 'postgresql', 'mysql', 'mysql_old', 'sqlite3' or 'ado_mssql'. + database_name = + # Or path to database file if using sqlite3. + #<repository>/etc/brpt.sqlite is default path if left empty + database_user = + # Not used with sqlite3. + database_password = + # Not used with sqlite3. + database_host = + # Not used with sqlite3. + database_port = + # Set to empty string for default. Not used with sqlite3. + web_debug = True + + [communication] + protocol = xmlrpc/ssl + password = test1234 + certificate = /etc/bcfg2.crt + key = /etc/bcfg2.key + ca = /etc/bcfg2.crt + + [components] + bcfg2 = https://config01.local.net:6789 + + +Add the machines to Bcfg2 +------------------------- + +``bcfg2-admin`` can be used to add a machine to Bcfg2 easily. You +need to know the Fully Qualified Domain Name (FQDN) of ever system +you want to control through Bcfg2. :: + + bcfg2-admin client add <FQDN machine> + +Bring your first machine under Bcfg2 control +-------------------------------------------- + +Now it is time to get the first machine's configuration into the +Bcfg2 repository. The server will be the first machine. It's +already in the ``Metadata/client.xml``. + + +Setup the `Packages`_ plugin +++++++++++++++++++++++++++++ + +.. _Packages: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Packages + +First, replace **Pkgmgr** with **Packages** in the plugins +line of ``bcfg2.conf``. Then create `Packages/` directory in +``/var/lib/bcfg2`` :: + + $ su -c 'mkdir /var/lib/bcfg2/Packages' + +Create a ``config.xml`` file for the packages in +``/var/lib/bcfg2/Packages`` with the following content. Choose a +mirror near your location according the `Mirror list`_ . + +.. _Mirror list: http://mirrors.fedoraproject.org/publiclist/ + +.. code-block:: xml + + <Sources> + <YUMSource> + <Group>fedora-13</Group> + <URL>ftp://fedora.tu-chemnitz.de/pub/linux/fedora/linux/releases/</URL> + <Version>13</Version> + <Component>Fedora</Component> + <Arch>i386</Arch> + <Arch>x86_64</Arch> + </YUMSource> + </Sources> + +.. _Magic Groups: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Packages#MagicGroups + +Due to the `Magic Groups`_, we need to modify our Metadata. Let's +add a **fedora13** group which inherits a **fedora** group +(this should replace the existing **redhat** group) present in +``/var/lib/bcfg2/Metadata/groups.xml``. The resulting file should look +something like this + +.. code-block:: xml + + <Groups version='3.0'> + <Group profile='true' public='true' default='true' name='basic'> + <Group name='fedora13'/> + </Group> + <Group name='fedora13'/> + <Group name='fedora'/> + <Group name='ubuntu'/> + <Group name='debian'/> + <Group name='freebsd'/> + <Group name='gentoo'/> + <Group name='fedora'/> + <Group name='suse'/> + <Group name='mandrake'/> + <Group name='solaris'/> + </Groups> + +.. note:: + When editing your xml files by hand, it is useful to occasionally + run ``bcfg2-repo-validate`` to ensure that your xml validates + properly. + +Add a probe ++++++++++++ + +The next step for the client will be to have the proper +arch group membership. For this, we will make use of the +:ref:`server-plugins-grouping-dynamic_groups` capabilities of +the Probes plugin. Add **Probes** to your plugins line in ``bcfg2.conf`` +and create the Probe: + +.. code-block:: sh + + $ su -c 'mkdir /var/lib/bcfg2/Probes' + $ su -c 'cat /var/lib/bcfg2/Probes/groups' + #!/bin/sh + + echo "group:`uname -m`" + +Now a restart of ``bcfg2-server`` is needed:: + + $ su -c '/etc/init.d/bcfg2-server restart' + +To test the Probe just run ``bcfg2 -vqn``. + +.. code-block:: xml + + $ su -c 'bcfg2 -vqn' + Running probe group + Probe group has result: + group:i686 + ... + +Start managing packages ++++++++++++++++++++++++ + +Add a base-packages bundle. Let's see what happens when we just populate +it with the *yum* package. Create the ``base-packages.xml`` in your +``Bundler/`` directory with a entry for ``yum``. + +.. code-block:: xml + + $ cat /var/lib/bcfg2/Bundler/base-packages.xml + <Bundle name='base-packages'> + <Package name='yum'/> + </Bundle> + +You need to reference the bundle from your ``group.xml``. The resulting +profile group might look something like this + +.. code-block:: xml + + <Group profile='true' public='true' default='true' name='basic'> + <Bundle name='base-packages'/> + <Group name='fedora13'/> + </Group> + +Now if we run the client, we can see what this has done for us.:: + + output + +As you can see, the Packages plugin has generated the dependencies +required for the yum package automatically. The ultimate goal should +be to move all the packages from the **Unmanaged** entries section +to the **Managed** entries section. So, what exactly *are* those +Unmanaged entries?:: + + output + +Now you can go through these and continue adding the packages you +want to your Bundle. After a while, I ended up with a minimal bundle +that looks like this + +.. code-block:: xml + + <Bundle name='base-packages'> + + </Bundle> + +Now when I run the client, you can see I have only one unmanaged +package:: + + outout + +The gpg-pubkey packages are special in that they are not really +packages. Currently, the way to manage them is using +:ref:`BoundEntries <boundentries>`. So, after adding them, our +Bundle now looks like this + +.. note:: This does not actually control the contents of the files, + you will need to do this part separately (see below). + +.. code-block:: xml + + <Bundle name='base-packages'> + <BoundPackage name="gpg-pubkey" type="rpm"> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> + </BoundPackage> + <Package name='bcfg2-server'/> + <Package name='exim'/> + <Package name='grub'/> + <Package name='kernel'/> + <Package name='krb5-workstation'/> + <Package name='m2crypto'/> + <Package name='openssh-clients'/> + <Package name='openssh-server'/> + <Package name='prelink'/> + <Package name='redhat-lsb'/> + <Package name='rpm-build'/> + <Package name='rsync'/> + <Package name='sysklogd'/> + <Package name='vim-enhanced'/> + <Package name='yum'/> + </Bundle> + +To actually push the gpg keys out via Bcfg2, you will need to manage +the files as well. This can be done by adding Path entries for each +of the gpg keys you want to manage + +.. code-block:: xml + + <Bundle name='base-packages'> + <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5'/> + <Path name='/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL'/> + <BoundPackage name="gpg-pubkey" type="rpm"> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5" version="e8562897" release="459f07a4"/> + <Instance simplefile="/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL" version="217521f6" release="45e8a532"/> + </BoundPackage> + <Package name='bcfg2-server'/> + <Package name='exim'/> + <Package name='grub'/> + <Package name='kernel'/> + <Package name='krb5-workstation'/> + <Package name='m2crypto'/> + <Package name='openssh-clients'/> + <Package name='openssh-server'/> + <Package name='prelink'/> + <Package name='redhat-lsb'/> + <Package name='rpm-build'/> + <Package name='rsync'/> + <Package name='sysklogd'/> + <Package name='vim-enhanced'/> + <Package name='yum'/> + </Bundle> + +Then add the files to Cfg:: + + mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 + cp /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5 !$/RPM-GPG-KEY-CentOS-5 + mkdir -p Cfg/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL + cp /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL !$/RPM-GPG-KEY-EPEL + +Now, running the client shows only unmanaged Service entries. Woohoo! + +Manage services ++++++++++++++++ + +Now let's clear up the unmanaged service entries by adding the +following entries to our bundle... + +.. code-block:: xml + + <!-- basic services --> + <Service name='atd'/> + <Service name='avahi-daemon'/> + <Service name='bcfg2-server'/> + <Service name='crond'/> + <Service name='cups'/> + <Service name='gpm'/> + <Service name='lvm2-monitor'/> + <Service name='mcstrans'/> + <Service name='messagebus'/> + <Service name='netfs'/> + <Service name='network'/> + <Service name='postfix'/> + <Service name='rawdevices'/> + <Service name='sshd'/> + <Service name='syslog'/> + +...and bind them in Rules + +.. code-block:: xml + + [root@centos ~]# cat /var/lib/bcfg2/Rules/services.xml + <Rules priority='1'> + <!-- basic services --> + <Service type='chkconfig' status='on' name='atd'/> + <Service type='chkconfig' status='on' name='avahi-daemon'/> + <Service type='chkconfig' status='on' name='bcfg2-server'/> + <Service type='chkconfig' status='on' name='crond'/> + <Service type='chkconfig' status='on' name='cups'/> + <Service type='chkconfig' status='on' name='gpm'/> + <Service type='chkconfig' status='on' name='lvm2-monitor'/> + <Service type='chkconfig' status='on' name='mcstrans'/> + <Service type='chkconfig' status='on' name='messagebus'/> + <Service type='chkconfig' status='on' name='netfs'/> + <Service type='chkconfig' status='on' name='network'/> + <Service type='chkconfig' status='on' name='postfix'/> + <Service type='chkconfig' status='on' name='rawdevices'/> + <Service type='chkconfig' status='on' name='sshd'/> + <Service type='chkconfig' status='on' name='syslog'/> + </Rules> + +Now we run the client and see there are no more unmanaged entries! :: + + $ su -c 'bcfg2 -veqn' + + +Adding Plugins +++++++++++++++ + +Git +--- + +.. _Git tutorial: http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html + +Adding the :ref:`server-plugins-version-git` plugins can preserve +versioning information. The first step is to add *Git* to your +plugin line:: + + plugins = Base,Bundler,Cfg,...,Git + +For tracking the configuration files in the ``/var/lib/bcfg2`` +directory a git repository need to be established:: + + git init + +For more detail about the setup of git please refer to a `git tutorial`_. +The first commit can be the empty or the allready populated directory:: + + git add . && git commit -a + +While running ``bcfg2-info`` the following line will show up:: + + Initialized git plugin with git directory = /var/lib/bcfg2/.git + + + + + diff --git a/doc/unsorted/gentoo.txt b/doc/appendix/guides/gentoo.txt index 4a549210d..023e6ac24 100644 --- a/doc/unsorted/gentoo.txt +++ b/doc/appendix/guides/gentoo.txt @@ -1,6 +1,6 @@ .. -*- mode: rst -*- -.. _unsorted-gentoo: +.. _guide-gentoo: ====== Gentoo diff --git a/doc/unsorted/nat_howto.txt b/doc/appendix/guides/nat_howto.txt index 131c0c533..131c0c533 100644 --- a/doc/unsorted/nat_howto.txt +++ b/doc/appendix/guides/nat_howto.txt diff --git a/doc/appendix/guides/ubuntu.txt b/doc/appendix/guides/ubuntu.txt new file mode 100644 index 000000000..a4790ffed --- /dev/null +++ b/doc/appendix/guides/ubuntu.txt @@ -0,0 +1,479 @@ +.. -*- mode: rst -*- + +.. _guide-ubuntu: + +====== +Ubuntu +====== + +.. note:: + + This particular how to was done on lucid, but should apply to any + other `stable`__ version of Ubuntu. + +__ ubuntu-releases_ +.. _ubuntu-releases: https://wiki.ubuntu.com/Releases + +Install Bcfg2 +============= + +We first need to install the server. For this example, we will use the +bcfg2 server package from the bcfg2 `PPA`_ (note that there is also a +version available in the ubuntu archives, but it is not as up to date). + +.. _PPA: https://launchpad.net/~bcfg2/+archive/ppa + +Add the Ubuntu PPA listing to your APT sources +---------------------------------------------- + +See http://trac.mcs.anl.gov/projects/bcfg2/wiki/PrecompiledPackages#UbuntuLucid + +Install bcfg2-server +-------------------- +:: + + aptitude install bcfg2-server + +Remove the default configuration preseeded by the ubuntu package:: + + root@lucid:~# rm -rf /etc/bcfg2* /var/lib/bcfg2 + +Initialize your repository +========================== + +Now that you're done with the install, you need to intialize your +repository and setup your bcfg2.conf. bcfg2-admin init is a tool which +allows you to automate this process.:: + + root@lucid:~# bcfg2-admin init + Store bcfg2 configuration in [/etc/bcfg2.conf]: + Location of bcfg2 repository [/var/lib/bcfg2]: + Input password used for communication verification (without echoing; leave blank for a random): + What is the server's hostname: [lucid] + Input the server location [https://lucid:6789]: + Input base Operating System for clients: + 1: Redhat/Fedora/RHEL/RHAS/Centos + 2: SUSE/SLES + 3: Mandrake + 4: Debian + 5: Ubuntu + 6: Gentoo + 7: FreeBSD + : 5 + Generating a 1024 bit RSA private key + .......................................................................................+++ + ...++++++ + writing new private key to '/etc/bcfg2.key' + ----- + Signature ok + subject=/C=US/ST=Illinois/L=Argonne/CN=lucid + Getting Private key + Repository created successfuly in /var/lib/bcfg2 + + +Of course, change responses as necessary. + +Start the server +================ + +You are now ready to start your bcfg2 server for the first time.:: + + root@lucid:~# /etc/init.d/bcfg2-server start + root@lucid:~# tail /var/log/syslog + Dec 17 22:07:02 lucid bcfg2-server[17523]: serving bcfg2-server at https://lucid:6789 + Dec 17 22:07:02 lucid bcfg2-server[17523]: serve_forever() [start] + Dec 17 22:07:02 lucid bcfg2-server[17523]: Processed 16 fam events in 0.502 seconds. 0 coalesced + +Run bcfg2 to be sure you are able to communicate with the server:: + + root@lucid:~# bcfg2 -vqn + Loaded tool drivers: + APT Action DebInit POSIX + + Phase: initial + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 382 + + + Phase: final + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 382 + +Bring your first machine under Bcfg2 control +============================================ + +Now it is time to get your first machine's configuration into your Bcfg2 +repository. Let's start with the server itself. + +Setup the `Packages`_ plugin +---------------------------- + +.. _Packages: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Packages + +Replace Pkgmgr with Packages in the plugins line of ``bcfg2.conf``:: + + root@lucid:~# cat /etc/bcfg2.conf + [server] + repository = /var/lib/bcfg2 + plugins = Base,Bundler,Cfg,Metadata,Packages,Rules,SSHbase + + [statistics] + sendmailpath = /usr/lib/sendmail + database_engine = sqlite3 + # 'postgresql', 'mysql', 'mysql_old', 'sqlite3' or 'ado_mssql'. + database_name = + # Or path to database file if using sqlite3. + #<repository>/etc/brpt.sqlite is default path if left empty + database_user = + # Not used with sqlite3. + database_password = + # Not used with sqlite3. + database_host = + # Not used with sqlite3. + database_port = + # Set to empty string for default. Not used with sqlite3. + web_debug = True + + [communication] + protocol = xmlrpc/ssl + password = secret + certificate = /etc/bcfg2.crt + key = /etc/bcfg2.key + ca = /etc/bcfg2.crt + + [components] + bcfg2 = https://lucid:6789 + +Create Packages layout (as per :ref:`packages-exampleusage`) in +``/var/lib/bcfg2`` + +.. code-block:: xml + + root@lucid:~# mkdir /var/lib/bcfg2/Packages + root@lucid:~# cat /var/lib/bcfg2/Packages/config.xml + <Sources> + <APTSource> + <Group>ubuntu-lucid</Group> + <URL>http://us.archive.ubuntu.com/ubuntu</URL> + <Version>lucid</Version> + <Component>main</Component> + <Component>multiverse</Component> + <Component>restricted</Component> + <Component>universe</Component> + <Arch>amd64</Arch> + <Arch>i386</Arch> + </APTSource> + </Sources> + +Due to the `Magic Groups`_, we need to modify our Metadata. Let's add +an **ubuntu-lucid** group which inherits the **ubuntu** group already +present in ``/var/lib/bcfg2/Metadata/groups.xml``. The resulting file +should look something like this + +.. _Magic Groups: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Packages#MagicGroups + +.. code-block:: xml + + <Groups version='3.0'> + <Group profile='true' public='true' default='true' name='basic'> + <Group name='ubuntu-lucid'/> + </Group> + <Group name='ubuntu-lucid'> + <Group name='ubuntu'/> + </Group> + <Group name='ubuntu'/> + <Group name='debian'/> + <Group name='freebsd'/> + <Group name='gentoo'/> + <Group name='redhat'/> + <Group name='suse'/> + <Group name='mandrake'/> + <Group name='solaris'/> + </Groups> + +.. note:: + When editing your xml files by hand, it is useful to occasionally run + `bcfg2-repo-validate` to ensure that your xml validates properly. + +The last thing we need is for the client to have the proper +arch group membership. For this, we will make use of the +:ref:`server-plugins-grouping-dynamic_groups` capabilities of the Probes plugin. Add +Probes to your plugins line in ``bcfg2.conf`` and create the Probe. + +.. code-block:: sh + + root@lucid:~# grep plugins /etc/bcfg2.conf + plugins = Base,Bundler,Cfg,Metadata,Packages,Probes,Rules,SSHbase + root@lucid:~# mkdir /var/lib/bcfg2/Probes + root@lucid:~# cat /var/lib/bcfg2/Probes/groups + #!/bin/sh + + ARCH=`uname -m` + case "$ARCH" in + "x86_64") + echo "group:amd64" + ;; + "i686") + echo "group:i386" + ;; + esac + +Now we restart the bcfg2-server:: + + root@lucid:~# /etc/init.d/bcfg2-server restart + Stopping Configuration Management Server: * bcfg2-server + Starting Configuration Management Server: * bcfg2-server + root@lucid:~# tail /var/log/syslog + Dec 17 22:36:47 lucid bcfg2-server[17937]: Packages: File read failed; falling back to file download + Dec 17 22:36:47 lucid bcfg2-server[17937]: Packages: Updating http://us.archive.ubuntu.com/ubuntu//dists/lucid/main/binary-amd64/Packages.gz + Dec 17 22:36:54 lucid bcfg2-server[17937]: Packages: Updating http://us.archive.ubuntu.com/ubuntu//dists/lucid/multiverse/binary-amd64/Packages.gz + Dec 17 22:36:55 lucid bcfg2-server[17937]: Packages: Updating http://us.archive.ubuntu.com/ubuntu//dists/lucid/restricted/binary-amd64/Packages.gz + Dec 17 22:36:56 lucid bcfg2-server[17937]: Packages: Updating http://us.archive.ubuntu.com/ubuntu//dists/lucid/universe/binary-amd64/Packages.gz + Dec 17 22:37:27 lucid bcfg2-server[17937]: Failed to read file probed.xml + Dec 17 22:37:27 lucid bcfg2-server[17937]: Loading experimental plugin(s): Packages + Dec 17 22:37:27 lucid bcfg2-server[17937]: NOTE: Interfaces subject to change + Dec 17 22:37:27 lucid bcfg2-server[17937]: service available at https://lucid:6789 + Dec 17 22:37:27 lucid bcfg2-server[17937]: serving bcfg2-server at https://lucid:6789 + Dec 17 22:37:27 lucid bcfg2-server[17937]: serve_forever() [start] + Dec 17 22:37:28 lucid bcfg2-server[17937]: Processed 17 fam events in 0.502 seconds. 0 coalesced + +Start managing packages +----------------------- + +Add a base-packages bundle. Let's see what happens when we just populate +it with the ubuntu-standard package. + +.. code-block:: xml + + root@lucid:~# cat /var/lib/bcfg2/Bundler/base-packages.xml + <Bundle name='base-packages'> + <Package name='ubuntu-standard'/> + </Bundle> + +You need to reference the bundle from your Metadata. The resulting +profile group might look something like this + +.. code-block:: xml + + <Group profile='true' public='true' default='true' name='basic'> + <Bundle name='base-packages'/> + <Group name='ubuntu-lucid'/> + </Group> + +Now if we run the client in debug mode (-d), we can see what this has +done for us.:: + + root@lucid:~# bcfg2 -vqdn + Running probe groups + Probe groups has result: + amd64 + Loaded tool drivers: + APT Action DebInit POSIX + The following packages are specified in bcfg2: + ubuntu-standard + The following packages are prereqs added by Packages: + adduser debconf hdparm libdevmapper1.02.1 libk5crypto3 libparted1.8-12 libxml2 passwd upstart + apt debianutils info libdns53 libkeyutils1 libpci3 logrotate pciutils usbutils + aptitude dmidecode install-info libelf1 libkrb5-3 libpopt0 lsb-base perl-base wget + at dnsutils iptables libept0 libkrb5support0 libreadline5 lshw popularity-contest zlib1g + base-files dosfstools libacl1 libgcc1 liblwres50 libreadline6 lsof psmisc + base-passwd dpkg libattr1 libgdbm3 libmagic1 libselinux1 ltrace readline-common + bsdmainutils ed libbind9-50 libgeoip1 libmpfr1ldbl libsigc++-2.0-0c2a man-db rsync + bsdutils file libc-bin libgmp3c2 libncurses5 libssl0.9.8 memtest86+ sed + cpio findutils libc6 libgssapi-krb5-2 libncursesw5 libstdc++6 mime-support sensible-utils + cpp ftp libcap2 libisc50 libpam-modules libusb-0.1-4 ncurses-bin strace + cpp-4.4 gcc-4.4-base libcomerr2 libisccc50 libpam-runtime libuuid1 netbase time + cron groff-base libcwidget3 libisccfg50 libpam0g libxapian15 parted tzdata + + Phase: initial + Correct entries: 101 + Incorrect entries: 0 + Total managed entries: 101 + Unmanaged entries: 281 + + + Phase: final + Correct entries: 101 + Incorrect entries: 0 + Total managed entries: 101 + Unmanaged entries: 281 + +As you can see, the Packages plugin has generated the dependencies +required for the ubuntu-standard package for us automatically. The +ultimate goal should be to move all the packages from the **Unmanaged** +entries section to the **Managed** entries section. So, what exactly *are* +those Unmanaged entries?:: + + root@lucid:~# bcfg2 -vqen + Running probe groups + Probe groups has result: + amd64 + Loaded tool drivers: + APT Action DebInit POSIX + + Phase: initial + Correct entries: 101 + Incorrect entries: 0 + Total managed entries: 101 + Unmanaged entries: 281 + + + Phase: final + Correct entries: 101 + Incorrect entries: 0 + Total managed entries: 101 + Unmanaged entries: 281 + Package:apparmor + Package:apparmor-utils + Package:apport + ... + +Now you can go through these and continue adding the packages you want to +your Bundle. Note that ``aptitude why`` is useful when trying to figure +out the reason for a package being installed. Also, deborphan is helpful +for removing leftover dependencies which are no longer needed. After a +while, I ended up with a minimal bundle that looks like this + +.. code-block:: xml + + <Bundle name='base-packages'> + <Package name='bash-completion'/> + <Package name='bcfg2-server'/> + <Package name='debconf-i18n'/> + <Package name='deborphan'/> + <Package name='diffutils'/> + <Package name='e2fsprogs'/> + <Package name='fam'/> + <Package name='grep'/> + <Package name='grub-pc'/> + <Package name='gzip'/> + <Package name='hostname'/> + <Package name='krb5-config'/> + <Package name='krb5-user'/> + <Package name='language-pack-en-base'/> + <Package name='linux-generic'/> + <Package name='linux-headers-generic'/> + <Package name='login'/> + <Package name='manpages'/> + <Package name='mlocate'/> + <Package name='ncurses-base'/> + <Package name='openssh-server'/> + <Package name='python-fam'/> + <Package name='tar'/> + <Package name='ubuntu-minimal'/> + <Package name='ubuntu-standard'/> + <Package name='vim'/> + <Package name='vim-runtime'/> + + <!-- PreDepends --> + <Package name='dash'/> + <Package name='initscripts'/> + <Package name='libdbus-1-3'/> + <Package name='libnih-dbus1'/> + <Package name='lzma'/> + <Package name='mountall'/> + <Package name='sysvinit-utils'/> + <Package name='sysv-rc'/> + + <!-- vim dependencies --> + <Package name='libgpm2'/> + <Package name='libpython2.6'/> + </Bundle> + +As you can see below, I no longer have any unmanaged packages. :: + + root@lucid:~# bcfg2 -vqen + Running probe groups + Probe groups has result: + amd64 + Loaded tool drivers: + APT Action DebInit POSIX + + Phase: initial + Correct entries: 247 + Incorrect entries: 0 + Total managed entries: 247 + Unmanaged entries: 10 + + + Phase: final + Correct entries: 247 + Incorrect entries: 0 + Total managed entries: 247 + Unmanaged entries: 10 + Service:bcfg2 Service:fam Service:killprocs Service:rc.local Service:single + Service:bcfg2-server Service:grub-common Service:ondemand Service:rsync Service:ssh + +Manage services +--------------- + +Now let's clear up the unmanaged service entries by adding the following +entries to our bundle... + +.. code-block:: xml + + <!-- basic services --> + <Service name='bcfg2'/> + <Service name='bcfg2-server'/> + <Service name='fam'/> + <Service name='grub-common'/> + <Service name='killprocs'/> + <Service name='ondemand'/> + <Service name='rc.local'/> + <Service name='rsync'/> + <Service name='single'/> + <Service name='ssh'/> + + +...and bind them in Rules + +.. code-block:: xml + + root@lucid:~# cat /var/lib/bcfg2/Rules/services.xml + <Rules priority='1'> + <!-- basic services --> + <Service type='deb' status='on' name='bcfg2'/> + <Service type='deb' status='on' name='bcfg2-server'/> + <Service type='deb' status='on' name='fam'/> + <Service type='deb' status='on' name='grub-common'/> + <Service type='deb' status='on' name='killprocs'/> + <Service type='deb' status='on' name='ondemand'/> + <Service type='deb' status='on' name='rc.local'/> + <Service type='deb' status='on' name='rsync'/> + <Service type='deb' status='on' name='single'/> + <Service type='deb' status='on' name='ssh'/> + </Rules> + +Now we run the client and see there are no more unmanaged entries! :: + + root@lucid:~# bcfg2 -vqn + Running probe groups + Probe groups has result: + amd64 + Loaded tool drivers: + APT Action DebInit POSIX + + Phase: initial + Correct entries: 257 + Incorrect entries: 0 + Total managed entries: 257 + Unmanaged entries: 0 + + All entries correct. + + Phase: final + Correct entries: 257 + Incorrect entries: 0 + Total managed entries: 257 + Unmanaged entries: 0 + + All entries correct. + +Dynamic (web) reports +===================== + +See installation instructions at :ref:`server-reports-install` diff --git a/doc/getting_started/using-bcfg2-info.txt b/doc/appendix/guides/using-bcfg2-info.txt index 1fab0ea07..5f77a3c70 100644 --- a/doc/getting_started/using-bcfg2-info.txt +++ b/doc/appendix/guides/using-bcfg2-info.txt @@ -1,17 +1,17 @@ .. -*- mode: rst -*- -.. _getting_started-using_bcfg2_info: +.. _guide-using_bcfg2_info: ================ Using bcfg2-info ================ -*bcfg2-info* is a tool for introspecting server functions. It is +``bcfg2-info`` is a tool for introspecting server functions. It is useful for understanding how the server is interpreting your repository. It consists of the same logic executed by the server to process the repository and produce configuration specifications, just -without all of the network communication code. Think of *bcfg2-info* -as *bcfg2-server* on a stick. It is a useful location to do testing +without all of the network communication code. Think of ``bcfg2-info`` +as ``bcfg2-server`` on a stick. It is a useful location to do testing and staging of new configuration rules, prior to deployment. This is particularly useful when developing templates, or developing Bcfg2 plugins. @@ -33,9 +33,9 @@ First, fire up the bcfg2-info interpreter. > At this point, the server core has been loaded up, all plugins have -been loaded, and the *bcfg2-info* has both read the initial state of +been loaded, and the ``bcfg2-info`` has both read the initial state of the Bcfg2 repository, as well as begun monitoring it for changes. Like -*bcfg2-server*, *bcfg2-info* monitors the repository for changes, +*bcfg2-server*, ``bcfg2-info`` monitors the repository for changes, however, unlike *bcfg2-server*, it does not process change events automatically. File modification events can be processed by explicitly calling the **update** command. This will process the events, @@ -54,7 +54,7 @@ This explicit update process allows you to control the update process, as well as see the precise changes caused by repository modifications. -*bcfg2-info* has several builtin commands that display the state of +``bcfg2-info`` has several builtin commands that display the state of various internal server core state. These are most useful for examining the state of client metadata, either for a single client, or for clients overall. @@ -106,27 +106,27 @@ commands. Debugging and Developing Bcfg2 ============================== -*bcfg2-info* loads a full Bcfg2 server core, so it provides the ideal +``bcfg2-info`` loads a full Bcfg2 server core, so it provides the ideal environment for developing and debugging Bcfg2. Because it is hard to automate this sort of process, we have only implemented two commands -in *bcfg2-info* to aid in the process. +in ``bcfg2-info`` to aid in the process. **profile** The profile command produces python profiling information for - other *bcfg2-info* commands. This can be used to track + other ``bcfg2-info`` commands. This can be used to track performance problems in configuration generation. **debug** - The debug command exits the *bcfg2-info* interpreter loop and drops + The debug command exits the ``bcfg2-info`` interpreter loop and drops to a python interpreter prompt. The Bcfg2 server core is available in this namespace as "self". Full documentation for the server core is out of scope for this document. This capability is most useful to call into plugin methods, often with setup calls or the enabling of diagnostics. - It is possible to return to the *bcfg2-info* command loop by + It is possible to return to the ``bcfg2-info`` command loop by exiting the python interpreter with ^D. - There is built-in support for IPython in *bcfg2-info*. If IPython - is installed, dropping into debug mode in *bcfg2-info* will use + There is built-in support for IPython in ``bcfg2-info``. If IPython + is installed, dropping into debug mode in ``bcfg2-info`` will use the IPython interpreter by default. diff --git a/doc/getting_started/using-bcfg2-with-centos.txt b/doc/appendix/guides/using-bcfg2-with-centos.txt index 93875d4c5..7c452f422 100644 --- a/doc/getting_started/using-bcfg2-with-centos.txt +++ b/doc/appendix/guides/using-bcfg2-with-centos.txt @@ -43,9 +43,9 @@ Now you can install the rest of the prerequisites:: Build Packages from source ########################## -* After installing git, clone the master branch:: +* After installing subversion, check out a copy of trunk :: - [root@centos redhat]# git clone git://git.mcs.anl.gov/bcfg2.git + [root@centos redhat]# svn co https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2 * Install the ``fedora-packager`` package :: diff --git a/doc/appendix/guides/vcs.txt b/doc/appendix/guides/vcs.txt new file mode 100644 index 000000000..207337c30 --- /dev/null +++ b/doc/appendix/guides/vcs.txt @@ -0,0 +1,112 @@ +.. -*- mode: rst -*- + +.. _guide-vcs: + +======================= +Version control systems +======================= + +The sections in this guide do only cover the basics steps in the setup +of the different version control system for the usage with the Bcfg2 +plugin support. More more details about + +Git +=== + +.. _Git tutorial: http://www.kernel.org/pub/software/scm/git/docs/gittutorial.html + +Adding the :ref:`server-plugins-version-git` plugins can preserve +versioning information. The first step is to add **Git** to your +plugin line:: + + plugins = Base,Bundler,Cfg,...,Git + +For tracking the configuration files in the ``/var/lib/bcfg2`` +directory a git repository need to be established:: + + git init + +For more detail about the setup of git please refer to a `git tutorial`_. +The first commit can be the empty or the already populated directory:: + + git add . && git commit -a + +While running ``bcfg2-info`` the following line will show up:: + + Initialized git plugin with git directory = /var/lib/bcfg2/.git + +Mercurial +========= + +For the :ref:`server-plugins-version-hg` plugin are the same changes +needed as for git. :: + + plugins = Base,Bundler,Cfg,...,Mercurial + +The repository must be initialized:: + + hg init + +Mercurial will not commit the files to the repository until a user name +is defined in ``/var/lib/bcfg2/.hg/`` + +.. code-block:: sh + + cat <<END_ENTRY >> /var/lib/bcfg2/.hg/hgrc + [ui] + username = Yor name <you@example.com> + END_ENTRY + +Now you are able to make submissions to the repository:: + + hg commit + +While running ``bcfg2-info`` the following line will show up:: + + Initialized hg plugin with hg directory = /var/lib/bcfg2/.hg + +Darcs +===== + +If you wish to use the :ref:`server-plugins-version-darcs` plugin an +entry has to be made in the ``bcfg2.conf`` file.:: + + plugins = Base,Bundler,Cfg,...,Darcs + +The dracs repository must be initialized:: + + darcs initialize + +To commit to the darcs repository an author must be added to the +``_darcs/prefs/author`` file. If the ``author`` file is missing, +darcs will ask you to enter your e-mail address. + +.. code-block:: sh + + cat <<END_ENTRY >> /var/lib/bcfg2/_darcs/prefs/author + you@example.com + END_ENTRY + +All files in the ``/var/lib/bcfg2`` should be added to darcs now:: + + darcs add * + +After that you can submit them to the repository:: + + darcs record + +While running ``bcfg2-info`` the following line will show up:: + + Initialized Darcs plugin with darcs directory = /var/lib/bcfg2/_darcs + +Cvs +=== + +If you wish to use the :ref:`server-plugins-version-darcs` plugin an +entry has to be made in the ``bcfg2.conf`` file.:: + + plugins = Base,Bundler,Cfg,...,Cvs + +The CVS repository must be initialized:: + + cvs -d /var/lib/bcfg2 init diff --git a/doc/appendix/index.txt b/doc/appendix/index.txt new file mode 100644 index 000000000..1bac69a3d --- /dev/null +++ b/doc/appendix/index.txt @@ -0,0 +1,27 @@ +.. -*- mode: rst -*- + +.. _appendix-index: + +======== +Appendix +======== + +Bcfg2 is based on a client-server architecture. The client is +responsible for interpreting (but not processing) the configuration +served by the server. This configuration is literal, so no local +process is required. After completion of the configuration process, +the client uploads a set of statistics to the server. This section +will describe the goals and then the architecture motivated by it. + + +.. toctree:: + :maxdepth: 2 + + files + configuration + contributors + books + papers + articles + guides + tools diff --git a/doc/appendix/papers.txt b/doc/appendix/papers.txt new file mode 100644 index 000000000..54f9de5dd --- /dev/null +++ b/doc/appendix/papers.txt @@ -0,0 +1,44 @@ +.. -*- mode: rst -*- + +.. _appendix-papers: + +====== +Papers +====== + + +* Configuration Life-Cycle Management on the TeraGrid. + + * Ti Leggett, Cory Lueninghoener, and Narayan Desai + * In Proceedings of TeraGrid '07 Conference, June 2007 + +* `A Scalable Approach To Deploying And Managing Appliances <http://workspace.globus.org/papers/Scalable_Approach_To_Deploying_And_Managing_Appliances.pdf>`_ + + * Rick Bradshaw, Narayan Desai, Tim Freeman, and Kate Keahey + * In Proceedings of the TeraGrid '07 Conference, June 2007 + +* `Bcfg2 - Konfigurationsmanagement Für Heterogene Umgebungen <ftp://ftp.mcs.anl.gov/pub/bcfg/papers/20070301-FFG2007-bcfg2-paper.pdf>`_ + + * Marko Jung, Robert Gogolok + * In Proceedings of German Unix User Group's Frühjahrsfachgespräch 2007, March 2007. + +* `Directing Change Using Bcfg2 <ftp://ftp.mcs.anl.gov/pub/bcfg/papers/directing-change-with-bcfg2.pdf>`_ + + * Narayan Desai, Rick Bradshaw, Joey Hagedorn, and Cory Lueninghoener + * In Proceedings of the Twentieth Large Install System Administration Conference (LISA XX), December 2-9, 2006, Washington D.C., USA, 2006. + +* `A Case Study in Configuration Management Tool Deployment <ftp://ftp.mcs.anl.gov/pub/bcfg/papers/bcfg2-lisa05-deployment.pdf>`_ + + * Narayan Desai, Rick Bradshaw, Scott Matott, Sandra Bittner, Susan Coghlan, Remy Evard, Cory Leunighhoener, Ti Leggett, J.P. Navarro, Gene Rackow, Craig Stacey, and Tisha Stacey + * In Proceedings of the Nineteenth Large Install System Administration Conference (LISA XIX), December 4-9, 2005, San Diego, CA, USA, 2005. + +* `Bcfg2: A Pay As You Go Approach to Configuration Complexity <ftp://ftp.mcs.anl.gov/pub/bcfg/papers/pay-as-you-go.pdf>`_ + + * Narayan Desai + * In Proceedings of the 2005 Australian Unix Users Group (AUUG2005), October 16-21, 2005, Sydney, Australia, 2005. + +* `Bcfg: A Configuration Management Tool for Heterogenous Environments <ftp://ftp.mcs.anl.gov/pub/bcfg/papers/bcfg-cluster2003.pdf>`_ + + * Narayan Desai, Andrew Lusk, Rick Bradshaw, and Remy Evard + * In Proceedings of the 5th IEEE International Conference on Cluster Computing (CLUSTER03), pages 500-503. IEEE Computer Society, 2003. + diff --git a/doc/appendix/tools.txt b/doc/appendix/tools.txt new file mode 100644 index 000000000..040823504 --- /dev/null +++ b/doc/appendix/tools.txt @@ -0,0 +1,14 @@ +.. -*- mode: rst -*- + +.. _appendix-tools: + +===== +Tools +===== + +In the ``tools/`` directory are several tools collected. Those tools +can help you to maintain your Bcfg2 configuration, to make the initial +setup easier, or to do some other tasks. + + +http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/bcfg2/tools diff --git a/doc/architecture/client.txt b/doc/architecture/client.txt new file mode 100644 index 000000000..77da25621 --- /dev/null +++ b/doc/architecture/client.txt @@ -0,0 +1,112 @@ +.. -*- mode: rst -*- + +.. _architecture-client: + +The Bcfg2 Client +================ + +The Bcfg2 client performs all client configuration or reconfiguration +operations. It renders a declarative configuration specification, provided +by the Bcfg2 server, into a set of configuration operations which will, +if executed, attempt to change the client's state into that described by +the configuration specification. Conceptually, the Bcfg2 client serves to +isolate the Bcfg2 server and specification from the imperative operations +required to implement configuration changes. + +This isolation allows declarative specifications to be manipulated +symbolically on the server, without needing to understand the properties +of the underlying system tools. In this way, the Bcfg2 client acts +as a sort of expert system that *knows* how to implement declarative +configuration changes. + +The operation of the Bcfg2 client is intended to be as simple as +possible. The normal configuration process consists of four main steps: + +* **Probe Execution** + + During the probe execution stage, the client connects to the server + and downloads a series of probes to execute. These probes reveal + local facts to the Bcfg2 server. For example, a probe could discover + the type of video card in a system. The Bcfg2 client returns this + data to the server, where it can influence the client configuration + generation process. + +* **Configuration Download and Inventory** + + The Bcfg2 client now downloads a configuration specification from the + Bcfg2 server. The configuration describes the complete target state + of the machine. That is, all aspects of client configuration should + be represented in this specification. For example, all software + packages and services should be represented in the configuration + specification. The client now performs a local system inventory. + This process consists of verifying each entry present in the + configuration specification. After this check is completed, heuristic + checks are executed for configuration not included in the configuration + specification. We refer to this inventory process as 2-way validation, + as first we verify that the client contains all configuration that + is included in the specification, then we check if the client has + any extra configuration that isn't present. This provides a fairly + rigorous notion of client configuration congruence. Once the 2-way + verification process has been performed, the client has built a list of + all configuration entries that are out of spec. This list has two parts: + specified configuration that is incorrect (or missing) and unspecified + configuration that should be removed. + +* **Configuration Update** + + The client now attempts to update its configuration to match the + specification. Depending on options, changes may not (or only partially) + be performed. First, if extra configuration correction is enabled, + extra configuration can be removed. Then the remaining changes + are processed. The Bcfg2 client loops while progress is made in the + correction of these incorrect configuration entries. This loop results + in the client being able to accomplish all it will be able to during + one execution. Once all entries are fixed, or no progress is being + made, the loop terminates. Once all configuration changes that can be + performed have been, bundle dependencies are handled. Bundle groupings + result in two different behaviors. Contained entries are assumed + to be inter-dependent. To address this, the client re-verifies each + entry in any bundle containing an updates configuration entry. Also, + services contained in modified bundles are restarted. + +* **Statistics Upload** + + Once the reconfiguration process has concluded, the client reports + information back to the server about the actions it performed during the + reconfiguration process. Statistics function as a detailed return code + from the client. The server stores statistics information. Information + included in this statistics update includes (but is not limited to): + + * Overall client status (clean/dirty) + * List of modified configuration entries + * List of uncorrectable configuration entries + * List of unmanaged configuration entries + +Architecture Abstraction +------------------------ + +The Bcfg2 client internally supports the administrative tools available +on different architectures. For example, ``rpm`` and ``apt-get`` are +both supported, allowing operation of Debian, Redhat, SUSE, and Mandriva +systems. The client toolset is determined based on the availability of +client tools. The client includes a series of libraries which describe +how to interact with the system tools on a particular platform. + +Three of the libraries exist. There is a base set of functions, which +contain definitions describing how to perform POSIX operations. Support +for configuration files, directories, symlinks, hardlinks, etc., are +included here. Two other libraries subclass this one, providing support +for Debian and rpm-based systems. + +The Debian toolset includes support for apt-get and update-rc.d. These +tools provide the ability to install and remove packages, and to install +and remove services. + +The Redhat toolset includes support for rpm and chkconfig. Any other +platform that uses these tools can also use this toolset. Hence, all +of the other familiar rpm-based distributions can use this toolset +without issue. + +Other platforms can easily use the POSIX toolset, ignoring support for +packages or services. Alternatively, adding support for new toolsets +isn't difficult. Each toolset consists of about 125 lines of python code. diff --git a/doc/architecture/config-spec.txt b/doc/architecture/config-spec.txt new file mode 100644 index 000000000..57b131318 --- /dev/null +++ b/doc/architecture/config-spec.txt @@ -0,0 +1,54 @@ +.. -*- mode: rst -*- + +.. _architecture-config-spec: + +The Literal Configuration Specification +======================================= + +Literal configuration specifications are served to clients by the +Bcfg2 server. This is a differentiating factor for Bcfg2; all other +major configuration management systems use a non-literal configuration +specification. That is, the clients receive a symbolic configuration that +they process to implement target states. We took the literal approach +for a few reasons: + +* A small list of configuration element types can be defined, each of + which can have a set of defined semantics. This allows the server to + have a well-formed model of client-side operations. Without a static + lexicon with defined semantics, this isn't possible. This allows the + server, for example, to record the update of a package as a coherent + event. +* Literal configurations do not require client-side processing. Removing + client-side processing reduces the critical footprint of the tool. + That is, the Bcfg2 client (and the tools it calls) need to be + functional, but the rest of the system can be in any state. Yet, + the client will receive a correct configuration. +* Having static, defined element semantics also requires that all + operations be defined and implemented in advance. The implementation + can maximize reliability and robustness. In more ad-hoc setups, these + operations aren't necessarily safely implemented. + +The Structure of Specifications +------------------------------- + +Configuration specifications contain some number of clauses. Two types +of clauses exist. Bundles are groups of inter-dependent configuration +entities. The purpose of bundles is to encode installation-time +dependencies such that all new configuration is properly activated +during reconfiguration operations. That is, if a daemon configuration +file is changed, its daemon should be restarted. Another example of +bundle usage is the reconfiguration of a software package. If a package +contains a default configuration file, but it gets overwritten by an +environment-specific one, then that updated configuration file should +survive package upgrade. The purpose of bundles is to describe services, +or reconfigured software packages. Independent clauses contain groups +of configuration entities that aren't related in any way. This provides a +convenient mechanism that can be used for bulk installations of software. + +Each of these clauses contains some number of configuration entities. A +number of configuration entities exist including Path, Package, Service, +etc. Each of these correspond to the obvious system item. Configuration +specifications can get quite large; many systems have specifications +that top one megabyte in size. An example of one is included in an +appendix. These configurations can be written by hand, or generated by +the server. diff --git a/doc/architecture/design.txt b/doc/architecture/design.txt new file mode 100644 index 000000000..7008c601f --- /dev/null +++ b/doc/architecture/design.txt @@ -0,0 +1,77 @@ +.. -*- mode: rst -*- + +.. _architecture-design: + +Design Considerations +===================== + +This section will discuss several aspects of the design of Bcfg2, and the +particular use cases that motivated them. Initially, this will consist +of a discussion of the system metadata, and the intended usage model +for package indices as well. + +System Metadata +--------------- + +Bcfg2 system metadata describes the underlying patterns in system +configurations. It describes commonalities and differences between these +specifications in a rigorous way. The groups used by Bcfg2's metadata are +responsible for differentiating clients from one another, and building +collections of allocatable configuration. + +The Bcfg2 metadata system has been designed with several high-level +goals in mind. Flexibility and precision are paramount concerns; no +configuration should be undescribable using the constructs present in +the Bcfg2 repository. We have found (generally the hard way) that any +assumptions about the inherent simplicity of configuration patterns tend +to be wrong, so obscenely complex configurations must be representable, +even if these requirements seem illogical during the implementation. + +In particular, we wanted to streamline several operations that commonly +occurred in our environment. + +* Copying one node's profile to another node. + + In many environments, many nodes are instances of a common configuration + specification. They all have similar roles and software. In our + environment, desktop machines were the best example of this. Other than + strictly per-host configuration like SSH keys, all desktop machines + use a common configuration specification. This trivializes the process + of creating a new desktop machine. + +* Creating a specialized version of an existing profile. + + In environments with highly varied configurations, departmental + infrastructure being a good example, "another machine like X but with + extra software" is a common requirement. For this reason, it must be + trivially possible to inherit most of a configuration specification + from some more generic source, while being able to describe overriding + aspects in a convenient fashion. + +* Compose several pre-existing configuration aspects to create a new profile. + + The ability to compose configuration aspects allows the easy creation + of new profiles based on a series of predefined set of configuration + specification fragments. The end result is more agility in environments + where change is the norm. + + In order for a classing system to be comprehensive, it must be usable in + complex ways. The Bcfg2 metadata system has constructs that map cleanly + to first-order logic. This implies that any complex configuration + pattern can be represented (at all) by the metadata, as first-order + logic is provably comprehensive. (There is a discussion later in the + document describing the metadata system in detail, and showing how it + corresponds to first-order logic) + +These use cases motivate several of the design decisions that we +made. There must be a many to one correspondence between clients and +groups. Membership in a given profile group must imbue a client with +all of its configuration properties. + +Package Management +------------------ + +The interface provided in the Bcfg2 repository for package specification +was designed with automation in mind. The goal was to support an +append only interface to the repository, so that users do not need to +continuously re-write already existing bits of specification. diff --git a/doc/architecture/goals.txt b/doc/architecture/goals.txt new file mode 100644 index 000000000..395b574a0 --- /dev/null +++ b/doc/architecture/goals.txt @@ -0,0 +1,47 @@ +.. -*- mode: rst -*- + +.. _architecture-goals: + +Goals +===== + +* **Model configurations using declarative semantics.** + + Declarative semantics maximize the utility of configuration management + tools; they provide the most flexibility for the tool to determine + the right course of action in any given situation. This means that + users can focus on the task of describing the desired configuration, + while leaving the task of transitioning clients states to the tool. + +* **Configuration descriptions should be comprehensive.** + + This means that configurations served to the client should be sufficient + to reproduce all desired functionality. This assumption allows the + use of heuristics to detect extra configuration, aiding in reliable, + comprehensive configuration definitions. + +* **Provide a flexible approach to user interactions.** + + Most configuration management systems take a rigid approach to user + interactions; that is, either the client system is always correct, + or the central system is. This means that users are forced into an + overly proscribed model where the system asserts where correct data + is. Configuration data modification is frequently undertaken on both + the configuration server and clients. Hence, the existence of a single + canonical data location can easily pose a problem during normal tool + use. Bcfg2 takes a different approach. + +The default assumption is that data on the server is correct, however, +the client has the option to run in another mode where local changes are +catalogued for server-side integration. If the Bcfg2 client is run in dry +run mode, it can help to reconcile differences between current client +state and the configuration described on the server. The Bcfg2 client +also searches for extra configuration; that is, configuration that is +not specified by the configuration description. When extra configuration +is found, either configuration has been removed from the configuration +description on the server, or manual configuration has occurred on the +client. Options related to two-way verification and removal are useful +for configuration reconciliation when interactive access is used. + +* Plugins and administrative applications. +* Incremental operations. diff --git a/doc/architecture/index.txt b/doc/architecture/index.txt new file mode 100644 index 000000000..d5e034d34 --- /dev/null +++ b/doc/architecture/index.txt @@ -0,0 +1,24 @@ +.. -*- mode: rst -*- + +.. _architecture-index: + +====================== +Architecture in Detail +====================== + +Bcfg2 is based on a client-server architecture. The client is +responsible for interpreting (but not processing) the configuration +served by the server. This configuration is literal, so no local +process is required. After completion of the configuration process, +the client uploads a set of statistics to the server. This section +will describe the goals and then the architecture motivated by it. + + +.. toctree:: + :maxdepth: 1 + + goals + client + server + config-spec + design diff --git a/doc/architecture/server.txt b/doc/architecture/server.txt new file mode 100644 index 000000000..1341f9e0a --- /dev/null +++ b/doc/architecture/server.txt @@ -0,0 +1,65 @@ +.. -*- mode: rst -*- + +.. _architecture-server: + +The Bcfg2 Server +================ + +The Bcfg2 server is responsible for taking a network description and +turning it into a series of configuration specifications for particular +clients. It also manages probed data and tracks statistics for clients. + +The Bcfg2 server takes information from two sources when generating +client configuration specifications. The first is a pool of metadata that +describes clients as members of an aspect-based classing system. That is, +clients are defined in terms of aspects of their behavior. The other is +a file system repository that contains mappings from metadata to literal +configuration. These are combined to form the literal configuration +specifications for clients. + +The Configuration Specification Construction Process +---------------------------------------------------- + +As we described in the previous section, the client connects to the server +to request a configuration specification. The server uses the client's +metadata and the file system repository to build a specification that +is tailored for the client. This process consists of the following steps: + +* **Metadata Lookup** + + The server uses the client's IP address to initiate the metadata + lookup. This initial metadata consists of a (profile, image) tuple. If + the client already has metadata registered, then it is used. If not, + then default values are used and stored for future use. This metadata + tuple is expanded using some profile and class definitions also included + in the metadata. The end result of this process is metadata consisting + of hostname, profile, image, a list of classes, a list of attributes + and a list of bundles. + +* **Abstract Configuration Construction** + + Once the server has the client metadata, it is used to create + an abstract configuration. An abstract configuration contains + all of the configuration elements that will exist in the final + specification **without** any specifics. All entries will be typed + (i.e. the tagname will be one of Package, Path, Action, etc) and will + include a name. These configuration entries are grouped into bundles, + which document installation time interdependencies. + +* **Configuration Binding** + + The abstract configuration determines the structure of the client + configuration, however, it doesn't yet contain literal configuration + information. After the abstract configuration is created, each + configuration entry must be bound to a client-specific value. The Bcfg2 + server uses plugins to provide these client-specific bindings. The Bcfg2 + server core contains a dispatch table that describes which plugins can + handle requests of a particular type. The responsible plugin is located + for each entry. It is called, passing in the configuration entry and + the client's metadata. The behavior of plugins is explicitly undefined, + so as to allow maximum flexibility. The behaviours of the stock plugins + are documented elsewhere in this manual. Once this binding process + is completed, the server has a literal, client-specific configuration + specification. This specification is complete and comprehensive; the + client doesn't need to process it at all in order to use it. It also + represents the totality of the configuration specified for the client. diff --git a/doc/authentication.txt b/doc/authentication.txt index 9b1bdc187..ff3b1d901 100644 --- a/doc/authentication.txt +++ b/doc/authentication.txt @@ -14,9 +14,9 @@ Scenarios Default settings work well; machines do not float, and a per-client password is not required. -2. `NAT_HOWTO <http://trac.mcs.anl.gov/projects/bcfg2/wiki/NAT_HOWTO>`_ +2. :ref:`NAT Howto nat_howto` - * Build client records in advance with bcfg2-admin, setting a uuid + * Build client records in advance with ``bcfg2-admin``, setting a uuid for each new client. * Set the address attribute for each to the address of the NAT. @@ -33,7 +33,7 @@ Building bcfg2.conf automatically ================================= This is a TCheetah template that automatically constructs per-client -bcfg2.conf from the per-client metadata:: +`bcfg2.conf` from the per-client metadata:: [communication] protocol = xmlrpc/ssl @@ -43,14 +43,14 @@ bcfg2.conf from the per-client metadata:: #if $self.metadata.password != None password = $self.metadata.password #else - password = my-password-foobat + password = my-password-foobar #end if [components] bcfg2 = https://localhost:6789 In this setup, this will cause any clients that have uuids established -to be set to use them in bcfg2.conf. It will also cause any clients +to be set to use them in `bcfg2.conf`. It will also cause any clients with passwords set to use them instead of the global password. How Authentication Works @@ -123,7 +123,7 @@ or password only. Also a bootstrap mode will be added shortly; this will allow a client to authenticate with a password its first time, requiring a certificate all subsequent times. This behavior can be controlled through the use of the auth attribute in -Metadata/clients.xml:: +`Metadata/clients.xml`:: <Clients> <Client name='testclient' auth='cert'/> diff --git a/doc/client/tools/blast.txt b/doc/client/tools/blast.txt new file mode 100644 index 000000000..14eb53124 --- /dev/null +++ b/doc/client/tools/blast.txt @@ -0,0 +1,10 @@ +.. -*- mode: rst -*- + +.. _client-tools-blast: + +===== +Blast +===== + +Blastwave Packages. This tool driver is for blastwave packages on +solaris. diff --git a/doc/client/tools/chkconfig.txt b/doc/client/tools/chkconfig.txt new file mode 100644 index 000000000..35cc0969e --- /dev/null +++ b/doc/client/tools/chkconfig.txt @@ -0,0 +1,15 @@ +.. -*- mode: rst -*- + +.. _client-tools-chkconfig: + +========= +Chkconfig +========= + +Tool to manage services (primarily on Red Hat based distros). + +.. note:: Start and stop are standard arguments, but the one for reload + isn't consistent across services. You can specify which argument + to use with the `restart` property in Service tags. Example: + ``<Service name="ftp" restart="condrestart" status="on" + type="chkconfig">`` diff --git a/doc/client/tools/debinit.txt b/doc/client/tools/debinit.txt new file mode 100644 index 000000000..33d20f89c --- /dev/null +++ b/doc/client/tools/debinit.txt @@ -0,0 +1,9 @@ +.. -*- mode: rst -*- + +.. _client-tools-debinit: + +======= +DebInit +======= + +Debian Service Support; exec's update-rc.d to configure services. diff --git a/doc/client/tools/encap.txt b/doc/client/tools/encap.txt new file mode 100644 index 000000000..40508ac7a --- /dev/null +++ b/doc/client/tools/encap.txt @@ -0,0 +1,9 @@ +.. -*- mode: rst -*- + +.. _client-tools-encap: + +===== +Encap +===== + +`Encap <http://www.encap.org>`_ Packages. diff --git a/doc/client/tools/freebsdinit.txt b/doc/client/tools/freebsdinit.txt new file mode 100644 index 000000000..7df2518bc --- /dev/null +++ b/doc/client/tools/freebsdinit.txt @@ -0,0 +1,9 @@ +.. -*- mode: rst -*- + +.. _client-tools-freebsdinit: + +=========== +FreeBSDInit +=========== + +FreeBSD Service Support. Only bundle updates will work. diff --git a/doc/client/tools/freebsdpackage.txt b/doc/client/tools/freebsdpackage.txt new file mode 100644 index 000000000..a460fb41c --- /dev/null +++ b/doc/client/tools/freebsdpackage.txt @@ -0,0 +1,10 @@ +.. -*- mode: rst -*- + +.. _client-tools-freebsdpackage: + +============== +FreeBSDPackage +============== + +FreeBSD Packages. Verifies packages and their version numbers but can't +install packages. diff --git a/doc/client/tools/index.txt b/doc/client/tools/index.txt index 878a221ee..7f6e6b667 100644 --- a/doc/client/tools/index.txt +++ b/doc/client/tools/index.txt @@ -2,154 +2,31 @@ .. _client-tools-index: -Client Tool Drivers -=================== - -Client tool drivers allow Bcfg2 to execute configuration operations by -interfacing with platform and distribution specific tools. - -Tool drivers handle any reconfiguration or verification operation. So -far we have tools that primarily deal with packaging systems and service -management. The POSIX tool also handles file system and permissions/groups -operations. - -To write your own tool driver, to handle a new packaging format, or new -service architecture see :ref:`development-index-writingtooldrivers` - -When the Bcfg2 client is run, it attempts to instantiate each of these -drivers. The succeeding list of drivers are printed as a debug message -after this process has completed. Drivers can supercede one another, -for example, the Yum driver conflicts (and unloads) the RPM driver. This -behavior can be overridden by running the Bcfg2 client with the -D -flag. This flag takes a colon delimited list of drivers to use on -the system. +Available client tools +====================== + +Client tool drivers allow Bcfg2 to execute configuration operations +by interfacing with platform and distribution specific tools. + +Tool drivers handle any reconfiguration or verification operation. +So far we have tools that primarily deal with packaging systems +and service management. The POSIX tool also handles file system +and permissions/groups operations. To write your own tool driver, +to handle a new packaging format, or new service architecture see +:ref:`development-client-driver`. + +When the Bcfg2 client is run, it attempts to instantiate each of +these drivers. The succeeding list of drivers are printed as a +debug message after this process has completed. Drivers can +supercede one another, for example, the Yum driver conflicts (and +unloads) the RPM driver. This behavior can be overridden by running +the Bcfg2 client with the ``-D`` flag. This flag takes a colon +delimited list of drivers to use on the system. Currently these are the tool drivers that are distributed with Bcfg2: -Action ------- - -Pre and post-install tests and actions. This driver executes commands -and supplies status information to the Bcfg2 server via the statistics -mechanism. It can also be used to prevent bundle installation when -pre-conditions are not met. See the UsingActions page for more details. - -APT ---- - -Debian Packages. This tool driver is used to handle packages on dpkg -based systems and employs the "apt" executable. Extra information can be -found at :ref:`client-tools-apt`. - -Blast ------ - -Blastwave Packages. This tool driver is for blastwave packages on solaris - -Chkconfig ---------- - -Tool to manage services (primarily on Redhat based distros). - -.. note:: Start and stop are standard arguments, but the one for reload - isn't consistent across services. You can specify which argument - to use with the `restart` property in Service tags. Example: - ``<Service name="ftp" restart="condrestart" status="on" - type="chkconfig">`` - -DebInit -------- - -Debian Service Support; exec's update-rc.d to configure services. - -Encap ------ - -`Encap <http://www.encap.org>`_ Packages. - -FreeBSDInit ------------ - -FreeBSD Service Support. Only bundle updates will work. - -FreeBSDPackage --------------- - -FreeBSD Packages. Verifies packages and their version numbers but can't -install packages. - -launchd -------- - -Mac OS X Services. To use this tool, you must maintain a standard launch -daemon .plist file in ``/Library/LaunchDaemons/`` (example ssh.plist) -and setup a ``<Service name="com.openssh.sshd" type="launchd" status="on" -/>`` entry in your config to load or unload the service. Note the name -is the ''Label'' specified inside of the .plist file - -Portage -------- - -Support for Gentoo Packages. - -POSIX ------ - -Files and Permissions are handled by the POSIX driver. Usage well -documented other places. - -RcUpdate --------- - -Uses the rc-update executable to manage services on distributions such -as Gentoo. - -RPM ---- - -.. warning:: Deprecated in favor of :ref:`RPMng <client-tools-yumng>` - -Executes rpm to manage packages most often on redhat based systems. - -RPMng ------ - -Next-generation RPM tool, will be default in upcoming release. Handles -RPM sublties like epoch and prelinking and 64-bit platforms better than -RPM client tool. :ref:`client-tools-yumng` - -SMF ---- - -Solaris Service Support. - -Example legacy run service (lrc): - -.. code-block:: xml - - <BoundService name='/etc/rc2_d/S47pppd' FMRI='lrc:/etc/rc2_d/S47pppd' status='off' type='smf'/> - -SYSV ----- - -Handles System V Packaging format that is available on Solaris. - -Upstart -------- - -Upstart service support. Uses `Upstart`_ to configure services. - -.. _Upstart: http://upstart.ubuntu.com/ - -Yum ---- - -.. warning:: Deprecated in favor of :ref:`YUMng <client-tools-yumng>` - -Handles RPMs using the YUM package manager. - -YUMng ------ +.. toctree:: + :maxdepth: 1 + :glob: -Handles RPMs using the YUM package manager. Handles sublties better than -the Yum client tool. :ref:`client-tools-yumng` + * diff --git a/doc/client/tools/launchd.txt b/doc/client/tools/launchd.txt new file mode 100644 index 000000000..59a872297 --- /dev/null +++ b/doc/client/tools/launchd.txt @@ -0,0 +1,13 @@ +.. -*- mode: rst -*- + +.. _client-tools-launchd: + +======= +launchd +======= + +Mac OS X Services. To use this tool, you must maintain a standard launch +daemon .plist file in ``/Library/LaunchDaemons/`` (example ssh.plist) +and setup a ``<Service name="com.openssh.sshd" type="launchd" status="on" +/>`` entry in your config to load or unload the service. Note the name +is the ''Label'' specified inside of the .plist file diff --git a/doc/client/tools/portage.txt b/doc/client/tools/portage.txt new file mode 100644 index 000000000..7a91408b1 --- /dev/null +++ b/doc/client/tools/portage.txt @@ -0,0 +1,9 @@ +.. -*- mode: rst -*- + +.. _client-tools-portage: + +======= +Portage +======= + +Support for Gentoo Packages. diff --git a/doc/client/tools/posix.txt b/doc/client/tools/posix.txt new file mode 100644 index 000000000..c4da38332 --- /dev/null +++ b/doc/client/tools/posix.txt @@ -0,0 +1,10 @@ +.. -*- mode: rst -*- + +.. _client-tools-posix: + +===== +POSIX +===== + +Files and Permissions are handled by the POSIX driver. Usage well +documented other places. diff --git a/doc/client/tools/rcupdate.txt b/doc/client/tools/rcupdate.txt new file mode 100644 index 000000000..4547d3b39 --- /dev/null +++ b/doc/client/tools/rcupdate.txt @@ -0,0 +1,10 @@ +.. -*- mode: rst -*- + +.. _client-tools-rcupdate: + +======== +RcUpdate +======== + +Uses the rc-update executable to manage services on distributions such +as Gentoo. diff --git a/doc/client/tools/rpm.txt b/doc/client/tools/rpm.txt new file mode 100644 index 000000000..17d367b55 --- /dev/null +++ b/doc/client/tools/rpm.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _client-tools-rpm: + +=== +RPM +=== + +.. warning:: Deprecated in favor of :ref:`RPMng <client-tools-yumng>` + +Executes rpm to manage packages most often on redhat based systems. diff --git a/doc/client/tools/rpmng.txt b/doc/client/tools/rpmng.txt new file mode 100644 index 000000000..c6d6cddf1 --- /dev/null +++ b/doc/client/tools/rpmng.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _client-tools-rpmng: + +===== +RPMng +===== + +Next-generation RPM tool, will be default in upcoming release. Handles +RPM sublties like epoch and prelinking and 64-bit platforms better than +RPM client tool. diff --git a/doc/client/tools/smf.txt b/doc/client/tools/smf.txt new file mode 100644 index 000000000..4829a14ed --- /dev/null +++ b/doc/client/tools/smf.txt @@ -0,0 +1,15 @@ +.. -*- mode: rst -*- + +.. _client-tools-smf: + +=== +SMF +=== + +Solaris Service Support. + +Example legacy run service (lrc): + +.. code-block:: xml + + <BoundService name='/etc/rc2_d/S47pppd' FMRI='lrc:/etc/rc2_d/S47pppd' status='off' type='smf'/> diff --git a/doc/client/tools/sysv.txt b/doc/client/tools/sysv.txt new file mode 100644 index 000000000..c08614c52 --- /dev/null +++ b/doc/client/tools/sysv.txt @@ -0,0 +1,9 @@ +.. -*- mode: rst -*- + +.. _client-tools-sysv: + +==== +SYSV +==== + +Handles System V Packaging format that is available on Solaris. diff --git a/doc/client/tools/upstart.txt b/doc/client/tools/upstart.txt new file mode 100644 index 000000000..60d0cf4ef --- /dev/null +++ b/doc/client/tools/upstart.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _client-tools-upstart: + +======= +Upstart +======= + +Upstart service support. Uses `Upstart`_ to configure services. + +.. _Upstart: http://upstart.ubuntu.com/ diff --git a/doc/client/tools/yum.txt b/doc/client/tools/yum.txt new file mode 100644 index 000000000..fcb5244df --- /dev/null +++ b/doc/client/tools/yum.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _client-tools-yum: + +=== +Yum +=== + +.. warning:: Deprecated in favor of :ref:`YUMng <client-tools-yumng>` + +Handles RPMs using the YUM package manager. diff --git a/doc/development/client-driver.txt b/doc/development/client-driver.txt new file mode 100644 index 000000000..19da9cb67 --- /dev/null +++ b/doc/development/client-driver.txt @@ -0,0 +1,75 @@ +.. -*- mode: rst -*- + +.. _development-client-driver: + +Writing A Client Tool Driver +============================ + +This page describes the step-by-step process of writing a client tool +driver for a configuration element type. The included example describes +an existing driver, and the process that was used to create it. + +#. Pick a name for the driver. In this case, we picked the name RPM. +#. Add "RPM" to the ``__all__ list`` in ``src/lib/Client/Tools/__init__.py`` +#. Create a file in ``src/lib/Client/Tools`` with the same name (RPM.py) +#. Create a class in this file with the same name (``class RPM``) + + * If it handles ``Package`` entries, subclass ``Bcfg2.Client.Tools.PkgTool`` + (from here referenced as branch [P]) + * If it handles ``Service`` entries, subclass ``Bcfg2.Client.Tools.SvcTool`` + (from here referenced as branch [S]) + * Otherwise, ``subclass Bcfg2.Client.Tools.Tool`` (from here referenced + as branch [T]) + +#. Set ``__name__`` to "RPM" +#. Add any required executable programs to ``__execs__`` +#. Set ``__handles__`` to a list of (``entry.tag``, ``entry.get('type')``) + tuples. This determines which entries the Tool module can be used + on. In this case, we set ``__handles__ = [('Package', 'rpm')]``. +#. Add verification. This method should return True/False depending + on current entry installation status. + + * [T] Add a Verify<entry.tag> method. + * [P] Add a VerifyPackage method. + * [S] Add a VerifyService method. + * In the failure path, the current state of failing entry + attributes should be set in the entry, to aid in auditing. + (For example, if a file should be mode 644, and is currently + mode 600, then set attribute current_perms='600' in the input + entry) + +#. Add installation support. This method should return True/False + depending on the results of the installation process. + + * [T,S] Add an Install<entry.tag> method. + * [P] The PkgTool baseclass has a generic mechanism for performing + all-at-once installations, followed, in the case of failures, + by single installations. To enable this support, set the pkgtype + attribute to the package type handled by this driver. Set the + ``pkgtool`` to a tuple ("command string %s", ("per-package string + format", [list of package entry fields])). For RPM, we have + ``setup pkgtool = ("rpm --oldpackage --replacepkgs --quiet -U %s", ("%s", ["url"]))`` + +#. Implement entry removal + + * [T,S] Implement a ``Remove`` method that removes all specified + entries (``prototype Remove(self, entries)``) + * [P] Implement a ``RemovePackages`` that removes all specified + entries (same prototype as Remove) + +#. Add a ``FindExtra`` method that locates entries not included in the + configuration. This may or may not be required, certain drivers + do not have the capability to find extra entries. +#. [P] Package drivers require a ``RefreshPackages`` method that updates + the internal representation of the package database. + +Writing Tool Driver Methods +--------------------------- + +#. Programs can be run using ``self.cmd.run``. This function returns a + (return code, stdout list) tuple. +#. The configuration is available as ``self.config`` +#. Runtime options are available in a dictionary as ``self.setup`` +#. Informational, error, and debug messages can be produced by + running ``self.logger.info/error/debug``. + diff --git a/doc/development/documentation.txt b/doc/development/documentation.txt new file mode 100644 index 000000000..8223670b8 --- /dev/null +++ b/doc/development/documentation.txt @@ -0,0 +1,75 @@ +.. -*- mode: rst -*- + +.. _development-documentation: + +Documentation +============= + +There are two parts of documentation in the Bcfg2 project: + +* The wiki +* The manual + + +The wiki +-------- +.. _Wiki: http://trac.mcs.anl.gov/projects/bcfg2/wiki +.. _Trac: http://trac.edgewall.org/ +.. _OpenID: https://openid.org/ +.. _MCS: http://www.mcs.anl.gov/ +.. _Argonne National Laboratory: http://www.anl.gov/ + +A python-based Trac_ instance is used for the Bcfg2 website. The +Wiki_ part of the website can be edited after you have successful +logged in. For the login is a vaild OpenID provider needed and an +interaction from an administrator. Please request your access to +the Wiki_ on the :ref:`mailinglist` or in the :ref:`ircchannel`. + + +The manual +---------- +.. _rst: http://en.wikipedia.org/wiki/ReStructuredText +.. _Sphinx: http://sphinx.pocoo.org +.. _Docutils: + +The source for the manual is located in the `doc/` directory in the +SVN repository or in the source tarball. All files are written in +rst_ (ReStructuredText). For the build process we are using Sphinx_. + +Building the Manual +^^^^^^^^^^^^^^^^^^^ + +* Install the prerequisites. Docutils_ and Sphinx_ are needed to build. + + * For Debian (Lenny) the tools are available in the `backports <http://www.backports.org/dokuwiki/doku.php?id=instructionst>`_ repository; installation can be done with the following:: + + apt-get -t lenny-backports install python-sphinx + + * The needed tools for Fedora based systems are in the `Fedora Package Collection <https://admin.fedoraproject.org/pkgdb>`_; installation can be done easily with Yum:: + + yum -y install python-sphinx python-docutils + + * Additionally, to build the PDF version: + + * LaTeX + * pdftex + +* Download the source. Please refer to :ref:`source` for more details. + +* Building the HTML version, run the following command in the `doc/` directory. The output will appear in `../build/sphinx/html`:: + + python setup.py build_sphinx + +* Building the PDF version :: + + python setup.py build_sphinx --builder=latex + cd build/sphinx/latex + make + +The latest version of the manual +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The latest version of the manual can always be found +`on the Four Kitchens server <http://doc.bcfg2.fourkitchens.com/>`_. + +This is an auto-updated from the `Launchpad mirror <https://code.launchpad.net/~vcs-imports/bcfg2/trunk>`_. diff --git a/doc/unsorted/emacs_snippet.txt b/doc/development/emacs_snippet.txt index 471a3b244..2905a2b45 100644 --- a/doc/unsorted/emacs_snippet.txt +++ b/doc/development/emacs_snippet.txt @@ -1,12 +1,14 @@ .. -*- mode: rst -*- -.. _unsorted-emacs_snippet: +.. _development-emacs_snippet: -======================================= -Using Bcfg2 with Emacs + YASnippet mode -======================================= +====================== +Emacs + YASnippet mode +====================== -This page describes using emacs with YASnippet mode with a set of snippets that allow quick composition of bundles and base files. More snippets are under development. +This page describes using emacs with YASnippet mode with a set of +snippets that allow quick composition of bundles and base files. +More snippets are under development. #. Download YASnippet from http://code.google.com/p/yasnippet/ #. Install it into your emacs load path (typically ~/.emacs.d/site-lisp) @@ -48,6 +50,11 @@ This page describes using emacs with YASnippet mode with a set of snippets that #. One quick M-x eval-current-buffer, and this code is enabled -Each of these snippets activates on the opening element, ie <Bundle. After this string is entered, but before entering a space, press <TAB>, and the snippet will be expanded. The template will be inserted into the text with a set of input prompts, which default to overwrite mode and can be tabbed through. +Each of these snippets activates on the opening element, ie <Bundle. +After this string is entered, but before entering a space, press <TAB>, +and the snippet will be expanded. The template will be inserted into +the text with a set of input prompts, which default to overwrite mode +and can be tabbed through. -The code above only works for bundles and base, but will be expanded to support other xml files as well. +The code above only works for bundles and base, but will be expanded +to support other xml files as well. diff --git a/doc/development/index.txt b/doc/development/index.txt index 74621815f..222978c98 100644 --- a/doc/development/index.txt +++ b/doc/development/index.txt @@ -6,315 +6,49 @@ Bcfg2 Development ================= -There are many ways to get involved in Bcfg2 development. Here we will -outline some things that can help you get familiar with the various -areas of the Bcfg2 code. +There are several ways users can contribute to the Bcfg2 project. -Tips for Bcfg2 Development --------------------------- +* Developing code +* Testing prereleases +* Reporting bugs +* Adding to the common repository +* Improving the wiki and writing documentation -#. Focus on either the client or server code. This focuses the development process down to the precise pieces of code that matter for the task at hand. - * If you are developing a client driver, then write up a small configuration specification that includes the needed characteristics. - * If you are working on the server, run ``bcfg2-info`` and use to assess the code. +This section will outline some things that can help you get familiar +with the various areas of the Bcfg2 code. -#. Use the python interpreter. One of python's most appealing features is interactive use of the interpreter. +Send patches to the :ref:`mailinglist` or create a trac +`ticket <https://trac.mcs.anl.gov/projects/bcfg2/newticket>`_ +with the patch included. In order to submit a ticket via the +trac system, you will need to create a session by clicking on the +`Preferences <https://trac.mcs.anl.gov/projects/bcfg2/prefs>`_ link and +filling out/saving changes to the form. In order to be considered for +mainline inclusion, patches need to be BSD licensed. The most convenient +way to prepare patches is by using ``git diff`` inside of a source tree +checked out of git. - * If you are developing for the client-side, run ``python -i /usr/sbin/bcfg2`` with the appropriate bcfg2 options. This will cause the python interpreter to continue running, leaving all variables intact. This can be used to examine data state in a convenient fashion. - * If you are developing for the server side, use ``bcfg2-info`` and the "debug" option. This will leave you at a python interpreter prompt, with the server core loaded in the variable "bcore". +The source tree can be checked out by running:: -#. Use ``pylint`` obsessively. It raises a lot of style-related warnings which can be ignored, but most all of the errors are legitimate. -#. If you are doing anything with Regular Expressions, `Kodos`_ and `re-try`_ are your friends. - - -.. _Kodos: http://kodos.sourceforge.net -.. _re-try: http://re-try.appspot.com - - -Environment setup for development -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -* Check out a copy of the code:: - - git clone git://git.mcs.anl.gov/bcfg2.git - -* Create link to src/lib:: - - cd bcfg2 - ln -s src/lib Bcfg2 - -* Add ``bcfg2/src/sbin`` to your PATH environment variable -* Add ``bcfg2`` to your PYTHONPATH environment variable - -.. _development-index-writingtooldrivers: - -Writing A Client Tool Driver -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This page describes the step-by-step process of writing a client tool -driver for a configuration element type. The included example describes -an existing driver, and the process that was used to create it. - -#. Pick a name for the driver. In this case, we picked the name RPM. -#. Add "RPM" to the __all__ list in ``src/lib/Client/Tools/__init__.py`` -#. Create a file in ``src/lib/Client/Tools`` with the same name (RPM.py) -#. Create a class in this file with the same name (class RPM) - - * If it handles Package entries, subclass Bcfg2.Client.Tools.PkgTool (from here referenced as branch [P]) - * If it handles Service entries, subclass Bcfg2.Client.Tools.SvcTool (from here referenced as branch [S]) - * Otherwise, subclass Bcfg2.Client.Tools.Tool (from here referenced as branch [T]) - -#. Set __name__ to "RPM" -#. Add any required executable programs to __execs__ -#. Set __handles__ to a list of (entry.tag, entry.get('type')) tuples. This determines which entries the Tool module can be used on. In this case, we set __handles__ = [('Package', 'rpm')]. -#. Add verification. This method should return True/False depending on current entry installation status. - - * [T] Add a Verify<entry.tag> method. - * [P] Add a VerifyPackage method. - * [S] Add a VerifyService method. - * In the failure path, the current state of failing entry attributes should be set in the entry, to aid in auditing. [[BR]] (For example, if a file should be mode 644, and is currently mode 600, then set attribute current_perms='600' in the input entry) - -#. Add installation support. This method should return True/False depending on the results of the installation process. - - * [T,S] Add an Install<entry.tag> method. - * [P] The PkgTool baseclass has a generic mechanism for performing all-at-once installations, followed, in the case of failures, by single installations. To enable this support, set the pkgtype attribute to the package type handled by this driver. Set the pkgtool to a tuple ("command string %s", ("per-package string format", [list of package entry fields])). For RPM, we have setup pkgtool = ("rpm --oldpackage --replacepkgs --quiet -U %s", ("%s", ["url"])) - -#. Implement entry removal - - * [T,S] Implement a Remove method that removes all specified entries (prototype Remove(self, entries)) - * [P] Implement a !RemovePackages that removes all specified entries (same prototype as Remove) - -#. Add a FindExtra method that locates entries not included in the configuration. This may or may not be required, certain drivers do not have the capability to find extra entries. -#. [P] Package drivers require a !RefreshPackages method that updates the internal representation of the package database. - -Writing Tool Driver Methods -""""""""""""""""""""""""""" - -#. Programs can be run using self.cmd.run. This function returns a (return code, stdout list) tuple. -#. The configuration is available as self.config -#. Runtime options are available in a dictionary as self.setup -#. Informational, error, and debug messages can be produced by running self.logger.info/error/debug. - -Bcfg2 Plugin development ------------------------- - -While the Bcfg2 server provides a good interface for representing -general system configurations, its plugin interface offers the ability -to implement configuration interfaces and representation tailored to -problems encountered by a particular site. This chapter describes what -plugins are good for, what they can do, and how to implement them. - -Bcfg2 Plugins -^^^^^^^^^^^^^ - -Bcfg2 plugins are loadable python modules that the Bcfg2 server loads at -initialization time. These plugins can contribute to the functions already -offered by the Bcfg2 server or can extend its functionality. In general, -plugins will provide some portion of the configuration for clients, with a -data representation that is tuned for a set of common tasks. Much of the -core functionality of Bcfg2 is implemented by several plugins, however, -they are not special in any way; new plugins could easily supplant one -or all of them. - -The following table describes the various functions of bcfg2 plugins. - -+--------------------+---------------------------------------------+ -| Name | Description | -+====================+=============================================+ -| Probes | Plugins can issue commands to collect | -| | client-side state (like hardware inventory) | -| | to include in client configurations | -+--------------------+---------------------------------------------+ -| ConfigurationEntry | Plugins can construct a list of per-client | -| List | configuration entry lists to include in | -| | client configurations. | -+--------------------+---------------------------------------------+ -| ConfigurationEntry | Literal values for configuration entries | -| contents | | -+--------------------+---------------------------------------------+ -| XML-RPC functions | Plugins can export function calls that | -| | expose internal functions. | -+--------------------+---------------------------------------------+ - -Writing Bcfg2 Plugins -^^^^^^^^^^^^^^^^^^^^^ - -Bcfg2 plugins are python classes that subclass from -Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must be set -in the new plugin. These values dictate how the new plugin will behave -with respect to the above four functions. The following table describes -all important member fields. - -+-----------------+-----------------------------------+--------------------------+ -| Name | Description | Format | -+=================+===================================+==========================+ -| __name__ | The name of the plugin | string | -+-----------------+-----------------------------------+--------------------------+ -| __version__ | The plugin version (generally | string | -| | tied to revctl keyword expansion) | | -+-----------------+-----------------------------------+--------------------------+ -| __author__ | The plugin author. | string | -+-----------------+-----------------------------------+--------------------------+ -| __rmi__ | Set of functions to be exposed as | List of function names | -| | XML-RPC functions | (strings) | -+-----------------+-----------------------------------+--------------------------+ -| Entries | Multidimentional dictionary of | Dictionary of | -| | keys that point to the function | ConfigurationEntityType, | -| | used to bind literal contents for | Name keys, and function | -| | a given configuration entity. | reference values | -+-----------------+-----------------------------------+--------------------------+ -| BuildStructures | Function that returns a list of | Member function | -| | the structures for a given client | | -+-----------------+-----------------------------------+--------------------------+ -| GetProbes | Function that returns a list of | Member function | -| | probes that a given client should | | -| | execute | | -+-----------------+-----------------------------------+--------------------------+ -| ReceiveData | Function that accepts the probe | Member function | -| | results for a given client. | | -+-----------------+-----------------------------------+--------------------------+ - -Example Plugin -^^^^^^^^^^^^^^ - -.. code-block:: python - - import Bcfg2.Server.Plugin - class MyPlugin(Bcfg2.Server.Plugin.Plugin): - """An example plugin.""" - # All plugins need to subclass Bcfg2.Server.Plugin.Plugin - __name__ = 'MyPlugin' - __version__ = '1' - __author__ = 'me@me.com' - __rmi__ = ['myfunction'] - # myfunction is not available remotely as MyPlugin.myfunction - - def __init__(self, core, datastore): - Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) - self.Entries = {'Path':{'/etc/foo.conf': self.buildFoo}} - - def myfunction(self): - """Function for xmlrpc rmi call.""" - # Do something - return True - - def buildFoo(self, entry, metadata): - """Bind per-client information into entry based on metadata.""" - entry.attrib.update({'type':'file', 'owner':'root', 'group':'root', 'perms':'644'}) - entry.text = '''contents of foo.conf''' - -Example Connector -^^^^^^^^^^^^^^^^^ - -.. code-block:: python - - import Bcfg2.Server.Plugin - - class Foo(Bcfg2.Server.Plugin.Plugin, - Bcfg2.Server.Plugin.Connector): - """The Foo plugin is here to illustrate a barebones connector.""" - name = 'Foo' - version = '$Revision: $' - experimental = True - - def __init__(self, core, datastore): - Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) - Bcfg2.Server.Plugin.Connector.__init__(self) - self.store = XMLFileBacked(self.data, core.fam) - - def get_additional_data(self, metadata): - - mydata = {} - for data in self.store.entries['foo.xml'].data.get("foo", []): - - mydata[data] = "bar" - - return dict([('mydata', mydata)]) - - def get_additional_groups(self, meta): - return self.cgroups.get(meta.hostname, list()) - -Server Plugin Types -------------------- - -Generator -^^^^^^^^^ - -Generator plugins contribute to literal client configurations - -Structure -^^^^^^^^^ - -Structure Plugins contribute to abstract client configurations - -Metadata -^^^^^^^^ - -Signal metadata capabilities - -Connector -^^^^^^^^^ - -Connector Plugins augment client metadata instances - -Probing -^^^^^^^ - -Signal probe capability - -Statistics -^^^^^^^^^^ - -Signal statistics handling capability - -Decision -^^^^^^^^ - -Signal decision handling capability - -Version -^^^^^^^ - -Interact with various version control systems - -Writing Server Plugins ----------------------- - -Metadata -^^^^^^^^ - -If you would like to define your own Metadata plugin (to extend/change -functionality of the existing Metadata plugin), here are the steps to -do so. We will call our new plugin `MyMetadata`. - -#. Add MyMetadata.py - - .. code-block:: python - - __revision__ = '$Revision$' - - import Bcfg2.Server.Plugins.Metadata - - class MyMetadata(Bcfg2.Server.Plugins.Metadata.Metadata): - '''This class contains data for bcfg2 server metadata''' - __version__ = '$Id$' - __author__ = 'bcfg-dev@mcs.anl.gov' - - def __init__(self, core, datastore, watch_clients=True): - Bcfg2.Server.Plugins.Metadata.Metadata.__init__(self, core, datastore, watch_clients) - -#. Add MyMetadata to ``src/lib/Server/Plugins/__init__.py`` -#. Replace Metadata with MyMetadata in the plugins line of bcfg2.conf - - -Documentation -------------- - -One of the areas where everyone can help is with the documentation. *Insert verbiage on how people can help.* + git clone git://git.mcs.anl.gov/bcfg2.git +Users wishing to contribute on a regular basis can apply for direct +git access. Mail the :ref:`mailinglist` for details. .. toctree:: :maxdepth: 1 - + + tips + setup + client-driver + plugins + writing_plugins + plugin-types + writing_specification + server + testing + documentation docstyleguide + emacs_snippet + vim_snippet diff --git a/doc/development/plugin-types.txt b/doc/development/plugin-types.txt new file mode 100644 index 000000000..5f0a4771a --- /dev/null +++ b/doc/development/plugin-types.txt @@ -0,0 +1,46 @@ +.. -*- mode: rst -*- + +.. _development-plugin-types: + +Server Plugin Types +------------------- + +Generator +^^^^^^^^^ + +Generator plugins contribute to literal client configurations + +Structure +^^^^^^^^^ + +Structure Plugins contribute to abstract client configurations + +Metadata +^^^^^^^^ + +Signal metadata capabilities + +Connector +^^^^^^^^^ + +Connector Plugins augment client metadata instances + +Probing +^^^^^^^ + +Signal probe capability + +Statistics +^^^^^^^^^^ + +Signal statistics handling capability + +Decision +^^^^^^^^ + +Signal decision handling capability + +Version +^^^^^^^ + +Interact with various version control systems diff --git a/doc/development/plugins.txt b/doc/development/plugins.txt new file mode 100644 index 000000000..b6debba24 --- /dev/null +++ b/doc/development/plugins.txt @@ -0,0 +1,45 @@ +.. -*- mode: rst -*- + +.. _development-plugins: + +Bcfg2 Plugin development +------------------------ + +While the Bcfg2 server provides a good interface for representing +general system configurations, its plugin interface offers the ability +to implement configuration interfaces and representation tailored to +problems encountered by a particular site. This chapter describes what +plugins are good for, what they can do, and how to implement them. + +Bcfg2 Plugins +^^^^^^^^^^^^^ + +Bcfg2 plugins are loadable python modules that the Bcfg2 server loads at +initialization time. These plugins can contribute to the functions already +offered by the Bcfg2 server or can extend its functionality. In general, +plugins will provide some portion of the configuration for clients, with a +data representation that is tuned for a set of common tasks. Much of the +core functionality of Bcfg2 is implemented by several plugins, however, +they are not special in any way; new plugins could easily supplant one +or all of them. + +The following table describes the various functions of bcfg2 plugins. + ++--------------------+---------------------------------------------+ +| Name | Description | ++====================+=============================================+ +| Probes | Plugins can issue commands to collect | +| | client-side state (like hardware inventory) | +| | to include in client configurations | ++--------------------+---------------------------------------------+ +| ConfigurationEntry | Plugins can construct a list of per-client | +| List | configuration entry lists to include in | +| | client configurations. | ++--------------------+---------------------------------------------+ +| ConfigurationEntry | Literal values for configuration entries | +| contents | | ++--------------------+---------------------------------------------+ +| XML-RPC functions | Plugins can export function calls that | +| | expose internal functions. | ++--------------------+---------------------------------------------+ + diff --git a/doc/development/server.txt b/doc/development/server.txt new file mode 100644 index 000000000..0f594422e --- /dev/null +++ b/doc/development/server.txt @@ -0,0 +1,83 @@ +.. -*- mode: rst -*- + +.. _development-server-plugins: + +Writing Server Plugins +---------------------- + +Metadata +^^^^^^^^ + +If you would like to define your own Metadata plugin (to extend/change +functionality of the existing Metadata plugin), here are the steps to +do so. We will call our new plugin `MyMetadata`. + +#. Add MyMetadata.py + + .. code-block:: python + + __revision__ = '$Revision$' + + import Bcfg2.Server.Plugins.Metadata + + class MyMetadata(Bcfg2.Server.Plugins.Metadata.Metadata): + '''This class contains data for bcfg2 server metadata''' + __version__ = '$Id$' + __author__ = 'bcfg-dev@mcs.anl.gov' + + def __init__(self, core, datastore, watch_clients=True): + Bcfg2.Server.Plugins.Metadata.Metadata.__init__(self, core, datastore, watch_clients) + +#. Add MyMetadata to ``src/lib/Server/Plugins/__init__.py`` +#. Replace Metadata with MyMetadata in the plugins line of bcfg2.conf + +.. _development-server-packages: + +Packages +-------- + +In order to support a given client package tool driver, that driver +must support use of the auto value for the version attribute in Package +entries. In this case, the tool driver views the current state of +available packages, and uses the underlying package manager's choice of +correct package version in lieu of an explicit, centrally-specified, +version. This support enables Packages to provide a list of Package +entries with version='auto'. Currently, the APT and YUMng drivers support +this feature. Note that package management systems without any network +support cannot operate in this fashion, so RPMng and SYSV will never be +able to use Packages. Emerge, Zypper, IPS, and Blastwave all have the +needed features to be supported by Packages, but support has not yet +been written. + +Packages fills two major functions in configuration generation. The first +is to provide entry level binding support for Package entries included +in client configurations. This function is quite easy to implement; +Packages determines (based on client group membership) if the package +is available for the client system, and which type it has. Because +version='auto' is used, no version determination needs to be done. + +The second major function is more complex. Packages ensures that client +configurations include all package-level prerequisites for package entries +explicitly included in the configuration. In order to support this, +Packages needs to directly process network data for package management +systems (the network sources for apt or yum, for examples), process +these files, and build data structures describing prerequisites and the +providers of those functions/paths. To simplify implementations of this, +there is a generic base class (Bcfg2.Server.Plugins.Packages.Source) +that provides a framework for fetching network data via HTTP, processing +those sources (with subclass defined methods for processing the specific +format provided by the tool), a generic dependency resolution method, +and a caching mechanism that greatly speeds up server/bcfg2-info startup. + +Each source type must define: + +* a get_urls attribute (and associated urls property) that describes + the URLS where to get data from. +* a read_files method that reads and processes the downloaded files + +Sources may define a get_provides method, if provides are complex. For +example, provides in rpm can be either rpm names or file paths, so +multiple data sources need to be multiplexed. + +The APT source in ``src/lib/Server/Plugins/Packages.py`` provides a +relatively simple implementation of a source. diff --git a/doc/development/setup.txt b/doc/development/setup.txt new file mode 100644 index 000000000..08a85dac1 --- /dev/null +++ b/doc/development/setup.txt @@ -0,0 +1,19 @@ +.. -*- mode: rst -*- + +.. _development-setup: + + +Environment setup for development +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* Check out a copy of the code:: + + svn co https://svn.mcs.anl.gov/repos/bcfg/trunk/bcfg2 + +* Create link to ``src/lib``:: + + cd bcfg2 + ln -s src/lib Bcfg2 + +* Add ``bcfg2/src/sbin`` to your PATH environment variable +* Add ``bcfg2`` to your PYTHONPATH environment variable diff --git a/doc/development/specification_overview.png b/doc/development/specification_overview.png Binary files differnew file mode 100644 index 000000000..66774f359 --- /dev/null +++ b/doc/development/specification_overview.png diff --git a/doc/development/testing.txt b/doc/development/testing.txt new file mode 100644 index 000000000..45b1cbefc --- /dev/null +++ b/doc/development/testing.txt @@ -0,0 +1,74 @@ +.. -*- mode: rst -*- + +.. _development-testing: + +Testing +======= + +Testing Prereleases +------------------- + +Before each release, several prereleases will be tagged. It is +helpful to have users test these releases (when feasible) because +it is hard to replicate the full range of potential reconfiguration +situations; between different operating systems, system management +tools, and configuration specification variation, there can be +large differences between sites. + + +For more details please visit `Tracking Development Releases of Bcfg2 <http://trac.mcs.anl.gov/projects/bcfg2/wiki/TrackingDevelopmentTrunk>`_ . + + +Upgrade Testing +--------------- + +This section describes upgrade procedures to completely test the +client and server. These procedures can be used for either pre-release +testing, or for confidence building in a new release. + + +Server Testing +^^^^^^^^^^^^^^ + +1. Ensure that the server produces the same configurations for clients + + * Before the upgrade, generate all client configurations using the buildall subcommand of bcfg2-info. This subcommand takes a directory argument; it will generate one client configuration in each file, naming each according to the client name. + + .. code-block:: sh + + mgt1:~/bcfg# bcfg2-info + Filesystem check 1 of 25 + ... + > buildall /path/to/cf-old + Generated config for fs2.bgl.mcs.anl.gov in 1.97310400009 seconds + Generated config for fs13.bgl.mcs.anl.gov in 1.47958016396 seconds + ... + + Take notice of any messages produced during configuration generation. These generally reflect minor issues in the configuration specification. Ideally, they should be fixed. + + * Upgrade the server software + * Generate all client configurations in a second location using the new software. Any tracebacks reflect bugs, and should be filed in the ticketing system. Any new messages should be carefully examined. + * Compare each file in the old directory to those in the new directory using ``bcfg2-admin compare -r /old/directory /new/directory`` + + .. code-block:: sh + + mgt1:~/bcfg# bcfg2-admin compare -r cf-old/ cf-new/ + Entry: fs2.bgl.mcs.anl.gov.xml + Entry: fs2.bgl.mcs.anl.gov.xml good + Entry: fs13.bgl.mcs.anl.gov.xml + Entry: fs13.bgl.mcs.anl.gov.xml good + Entry: login1.bgl.mcs.anl.gov.xml + ConfigFile /bin/whatami contents differ + ConfigFile /bin/whatami differs (in bundle softenv) + Entry: login1.bgl.mcs.anl.gov.xml bad + + This can be used to compare configurations for single clients, or different clients. + +2. Compare old and new group diagrams (using ``bcfg2-admin viz``) + +Client Testing +^^^^^^^^^^^^^^ + +Run the client in dry-run and non-dry-run mode; ensure that multiple +runs produce consistent results. + diff --git a/doc/development/tips.txt b/doc/development/tips.txt new file mode 100644 index 000000000..03ffb4871 --- /dev/null +++ b/doc/development/tips.txt @@ -0,0 +1,23 @@ +.. -*- mode: rst -*- + +.. _development-tips: + +Tips for Bcfg2 Development +-------------------------- + +#. Focus on either the client or server code. This focuses the development process down to the precise pieces of code that matter for the task at hand. + + * If you are developing a client driver, then write up a small configuration specification that includes the needed characteristics. + * If you are working on the server, run ``bcfg2-info`` and use to assess the code. + +#. Use the python interpreter. One of python's most appealing features is interactive use of the interpreter. + + * If you are developing for the client-side, run ``python -i /usr/sbin/bcfg2`` with the appropriate bcfg2 options. This will cause the python interpreter to continue running, leaving all variables intact. This can be used to examine data state in a convenient fashion. + * If you are developing for the server side, use ``bcfg2-info`` and the "debug" option. This will leave you at a python interpreter prompt, with the server core loaded in the variable "bcore". + +#. Use ``pylint`` obsessively. It raises a lot of style-related warnings which can be ignored, but most all of the errors are legitimate. +#. If you are doing anything with Regular Expressions, `Kodos`_ and `re-try`_ are your friends. + + +.. _Kodos: http://kodos.sourceforge.net +.. _re-try: http://re-try.appspot.com diff --git a/doc/development/vim_snippet.txt b/doc/development/vim_snippet.txt new file mode 100644 index 000000000..a01c1cfda --- /dev/null +++ b/doc/development/vim_snippet.txt @@ -0,0 +1,65 @@ +.. -*- mode: rst -*- + +.. _development-vim_snippet: + +=================== +Vim Snippet Support +=================== + +This page describes using vim with snipMate and a set of snippets +that allow quick composition of bundles and base files. + +#. Download snipMate from http://www.vim.org/scripts/script.php?script_id=2540 +#. Install it using the install instructions (unzip snipMate.zip -d ~/.vim or equivalent, e.g. $HOME\vimfiles on Windows) +#. Add the following to ``~/.vim/snippets/xml.snippets`` + + .. code-block:: cl + + # Bundle + snippet <Bundle + <Bundle name='${1:bundlename}'> + ${2} + </Bundle> + # Base + snippet <Base + <Base> + ${1} + </Base> + # Group + snippet <Group + <Group name='${1:groupname}'> + ${2} + </Group> + # ConfigFile + snippet <Config + <ConfigFile name='${1:filename}'/> + # Service + snippet <Service + <Service name='${1:svcname}'/> + # Package + snippet <Package + <Package name='${1:packagename}'/> + # Action + snippet <Action + <Action name='${1:name}'/> + # Directory + snippet <Directory + <Directory name='${1:name}'/> + # SymLink + snippet <SymLink + <SymLink name='${1:name}'/> + # Permissions + snippet <Permissions + <Permissions name='${1:name}'/> + + +#. Save and start editing away! + +Each of these snippets activates on the opening element, ie <Bundle>. +After this string is entered, but before entering a space, press <TAB>, +and the snippet will be expanded. The template will be inserted into +the text with a set of input prompts, which default to overwrite mode +and can be tabbed through. + +The code above only works for bundles and base, but will be expanded +to support other xml files as well. diff --git a/doc/development/writing_plugins.txt b/doc/development/writing_plugins.txt new file mode 100644 index 000000000..40e077e43 --- /dev/null +++ b/doc/development/writing_plugins.txt @@ -0,0 +1,104 @@ +.. -*- mode: rst -*- + +.. _development-write-plugins: + +Writing Bcfg2 Plugins +===================== + +Bcfg2 plugins are python classes that subclass from +Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must be set +in the new plugin. These values dictate how the new plugin will behave +with respect to the above four functions. The following table describes +all important member fields. + ++-----------------+-----------------------------------+--------------------------+ +| Name | Description | Format | ++=================+===================================+==========================+ +| __name__ | The name of the plugin | string | ++-----------------+-----------------------------------+--------------------------+ +| __version__ | The plugin version (generally | string | +| | tied to revctl keyword expansion) | | ++-----------------+-----------------------------------+--------------------------+ +| __author__ | The plugin author. | string | ++-----------------+-----------------------------------+--------------------------+ +| __author__ | The plugin author. | string | ++-----------------+-----------------------------------+--------------------------+ +| __rmi__ | Set of functions to be exposed as | List of function names | +| | XML-RPC functions | (strings) | ++-----------------+-----------------------------------+--------------------------+ +| Entries | Multidimentional dictionary of | Dictionary of | +| | keys that point to the function | ConfigurationEntityType, | +| | used to bind literal contents for | Name keys, and function | +| | a given configuration entity. | reference values | ++-----------------+-----------------------------------+--------------------------+ +| BuildStructures | Function that returns a list of | Member function | +| | the structures for a given client | | ++-----------------+-----------------------------------+--------------------------+ +| GetProbes | Function that returns a list of | Member function | +| | probes that a given client should | | +| | execute | | ++-----------------+-----------------------------------+--------------------------+ +| ReceiveData | Function that accepts the probe | Member function | +| | results for a given client. | | ++-----------------+-----------------------------------+--------------------------+ + +Example Plugin +-------------- + +.. code-block:: python + + import Bcfg2.Server.Plugin + class MyPlugin(Bcfg2.Server.Plugin.Plugin): + '''An example plugin''' + # All plugins need to subclass Bcfg2.Server.Plugin.Plugin + __name__ = 'MyPlugin' + __version__ = '1' + __author__ = 'me@me.com' + __rmi__ = ['myfunction'] + # myfunction is not available remotely as MyPlugin.myfunction + + def __init__(self, core, datastore): + Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) + self.Entries = {'Path':{'/etc/foo.conf': self.buildFoo}} + + def myfunction(self): + '''function for xmlrpc rmi call''' + #do something + return True + + def buildFoo(self, entry, metadata): + '''Bind per-client information into entry based on metadata''' + entry.attrib.update({'type':'file', 'owner':'root', 'group':'root', 'perms':'644'}) + entry.text = '''contents of foo.conf''' + +Example Connector +----------------- + +.. code-block:: python + + import Bcfg2.Server.Plugin + + class Foo(Bcfg2.Server.Plugin.Plugin, + Bcfg2.Server.Plugin.Connector): + '''The Foo plugin is here to illustrate a barebones connector''' + name = 'Foo' + version = '$Revision: $' + experimental = True + + def __init__(self, core, datastore): + Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) + Bcfg2.Server.Plugin.Connector.__init__(self) + self.store = XMLFileBacked(self.data, core.fam) + + def get_additional_data(self, metadata): + + mydata = {} + for data in self.store.entries['foo.xml'].data.get("foo", []): + + mydata[data] = "bar" + + return dict([('mydata', mydata)]) + + def get_additional_groups(self, meta): + return self.cgroups.get(meta.hostname, list()) + diff --git a/doc/development/writing_specification.txt b/doc/development/writing_specification.txt new file mode 100644 index 000000000..4e1c03483 --- /dev/null +++ b/doc/development/writing_specification.txt @@ -0,0 +1,54 @@ +.. -*- mode: rst -*- + +.. _development-writing_specification: + +=========================== +Writing Bcfg2 Specification +=========================== + +Bcfg2 specifications are logically divided in to three areas: + +* Metadata +* Abstract +* Literal + +The metadata portion of the configuration assigns a client to its profile +group and to its non-profile groups. The profile group is assigned +in ``Metadata/clients.xml`` and the non profile group assignments are in +``Metadata/groups.xml``. + +The group memberships contained in the metadata are then used to constuct +an abstract configuration for the client. An abstract configuration for +a client identifies the configuration entities (packages, configuration +files, service, etc) that a client requires, but it does not identify +them explicitly. For instance an abstract configuration may identify +that a client needs the Bcfg2 package with + +.. code-block:: xml + + <Package name=bcfg2/> + +but this does not explicitly identify that an RPM package version +0.9.2 should be loaded from http://rpm.repo.server/bcfg2-1.0.1-0.1.rpm. +The abstract configuration is defined in the xml configuration files +for the Base and Bundles plugins. + +A combination of a clients metadata (group memberships) and abstract +configuration is then used to generate the clients literal configuration. +For instance the above abstract configuration entry may generate a +literal configuration of + + +.. code-block:: xml + + <Package name='bcfg2' version='1.0.1-0.1' type='yum'/> + +A clients literal configuration is generated by a number of plugins that +handle the different configuration entities. + + +.. figure:: specification_overview.png + :width: 60% + :alt: Specification overview + :align: center + diff --git a/doc/faq/general.txt b/doc/faq/general.txt deleted file mode 100644 index 65db5883c..000000000 --- a/doc/faq/general.txt +++ /dev/null @@ -1,26 +0,0 @@ -.. _faq-general: - -FAQ: General -============ - -**What does Bcfg2 stand for?** - -Initially, Bcfg stood for the bundle configuration system. Bcfg2 is the -second major revision. At this point, the acronym is meaningless, but -the name has stuck. Luckily, Bcfg2 googles better than Bcfg does. No, -seriously. Try it. All I know is that I have no interest in a billion -cubic feet of gas. - -**What architectures does Bcfg2 support?** - -Bcfg2 should run on any POSIX compatible operating system, however direct -support for an operating system's package and service formats are limited -by the currently available :ref:`client-tools-index` (although new client -tools are pretty easy to add). The following is an incomplete but more -exact list of platforms on which Bcfg2 works. - -* GNU/Linux deb based distros -* GNU/Linux rpm based distros -* Solaris pkg based -* Gentoo portage based -* OSX (POSIX/launchd support) diff --git a/doc/faq/index.txt b/doc/faq/index.txt deleted file mode 100644 index 7f2024619..000000000 --- a/doc/faq/index.txt +++ /dev/null @@ -1,11 +0,0 @@ -.. _faq-index: - -========= -Bcfg2 FAQ -========= - -.. toctree:: - :maxdepth: 2 - - general - client diff --git a/doc/getting_help/error-messages.txt b/doc/getting_help/error-messages.txt new file mode 100644 index 000000000..72d1f1f40 --- /dev/null +++ b/doc/getting_help/error-messages.txt @@ -0,0 +1,45 @@ +.. -*- mode: rst -*- + +.. _error-messages: + +============== +Error Messages +============== + +This page describes error messages produced by Bcfg2 and steps that +can be taken to remedy them. + ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Error | Location | Meaning | Repair | ++===========================================================================+==========+=====================================================================================================+=========+ +| Incomplete information for entry <EntryTag>:<EntryName>; cannot verify | Client | The described entry is not fully specified by the server, so no verification can be performed. | [#f1]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Incomplete information for entry <EntryTag>:<EntryName>; cannot install | Client | The described entry is not fully specified by the server, so no verification can be performed. | [#f2]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| The following entries are not handled by any tool: <EntryTag>:<EntryName> | Client | The client cannot figure out how to handle this entry. | [#f3]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| no server x509 fingerprint; no server verification performed! | Client | The client is unable to verify the server | [#f4]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Failed to bind entry: <EntryTag> <EntryName> | Server | The server was unable to find a suitable version of entry for client. | [#f5]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Failed to bind to socket | Server | The server was unable to bind to the tcp server socket. | [#f6]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Failed to load ssl key <path> | Server | The server was unable to read and process the ssl key. | [#f7]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Failed to read file <path> | Server | The server failed to read the specified file | [#f8]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Failed to parse file <path> | Server | The server failed to parse the specified XML file | [#f9]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ +| Client metadata resolution error for <IP> | Server | The server cannot resolve the client hostname or the client is associated with a non-profile group. | [#f10]_ | ++---------------------------------------------------------------------------+----------+-----------------------------------------------------------------------------------------------------+---------+ + +.. [#f1] This entry is not being bound. Ensure that a version of this entry applies to this client. +.. [#f2] This entry is not being bound. Ensure that a version of this entry applies to this client. +.. [#f3] Add a type to the generator definition for this entry +.. [#f4] Run bcfg2-admin fingerprint on the server and add it to the client bcfg2.conf as mentioned here +.. [#f5] This entry is not being bound. Ensure that a version of this entry applies to this client. +.. [#f6] Ensure that another instance of the daemon (or any other process) is not listening on the same port. +.. [#f7] Ensure that the key is readable by the user running the daemon and that it is well-formed. +.. [#f8] Ensure that this file still exists; a frequent cause is the deletion of a temp file. +.. [#f9] Ensure that the file is properly formed XML. +.. [#f10] Fix hostname resolution for the client or ensure that the profile group is properly setup. diff --git a/doc/faq/client.txt b/doc/getting_help/faq/client.txt index e196568bb..a230a84bb 100644 --- a/doc/faq/client.txt +++ b/doc/getting_help/faq/client.txt @@ -1,3 +1,5 @@ +.. -*- mode: rst -*- + .. _faq-client: FAQ: Client @@ -8,7 +10,7 @@ FAQ: Client This means that the client does not have a copy of the CA for the server, so it can't verify that it is talking to the right server. To fix this issue, copy ``/etc/bcfg2.crt`` from the server to ``/etc/bcfg2.ca`` -on the client. Then add the following on the client.:: +on the client. Then add the following on the client. :: [communication] ca = /etc/bcfg2.ca diff --git a/doc/getting_help/faq/general.txt b/doc/getting_help/faq/general.txt new file mode 100644 index 000000000..f08bfb7b2 --- /dev/null +++ b/doc/getting_help/faq/general.txt @@ -0,0 +1,72 @@ +.. -*- mode: rst -*- + +.. _faq-general: + +FAQ: General +============ + +**What does Bcfg2 stand for?** + +Initially, Bcfg stood for the bundle configuration system. Bcfg2 is the +second major revision. At this point, the acronym is meaningless, but +the name has stuck. Luckily, Bcfg2 googles better than Bcfg does. No, +seriously. Try it. All I know is that I have no interest in a billion +cubic feet of gas. + +**What architectures does Bcfg2 support?** + +Bcfg2 should run on any POSIX compatible operating system, however direct +support for an operating system's package and service formats are limited +by the currently available :ref:`client-tools-index` (although new client +tools are pretty easy to add). The following is an incomplete but more +exact list of platforms on which Bcfg2 works. + +* GNU/Linux deb based distros +* GNU/Linux rpm based distros +* Solaris pkg based +* Gentoo portage based +* OSX (POSIX/launchd support) + +**What pre-requisites are needed to run Bcfg2?** + +Please visit the :ref:`prerequisites` section in the manual. + +**Why won't bcfg2-server start?** + +If your server doesn't seem to be starting and you see no error +messages in your server logs, try running it in the foreground to +see why. + +**Why am I getting a traceback?** + +If you get a traceback, please let us know. You can file a +`ticket <https://trac.mcs.anl.gov/projects/bcfg2/newticket>`_, send +the traceback to the :ref:`mailinglist`, or hop on +:ref:`ircchannel` and let us know. + +**What is the most common cause of "The following entries are not handled by any tool"?** + +Often it corresponds to entries that aren't bound by the server (for which you'll get error messages on the server). You should try inspecting the logs on the server to see what may be the cause. + +**How can I figure out if error is client or server side?** + +* Cache a copy of the client using ``bcfg2 -c /tmp/config.xml`` +* Search for the entry of interest +* If it looks correct, then there is a client issue + +This file contains all aspects of client configuration. It is structured as a series of bundles and base entries. + +.. note:: + + Most often the entry is not correct and the issue lies in the specification. + +**Where are the server log messages?** + +The bcfg2-server process logs to syslog facility LOG_DAEMON. The server produces a series of messages upon a variety of events and errors. + +**Is there a way to check if all repository XML files conform to schemas?** + +Bcfg2 comes with XML schemas describing all of the XML formats used in +the server repository. A validation command ``bcfg2-repo-validate`` is +included with the source distribution and all packages. Run it with +the ``-v`` flag to see each file and the results if its validation. diff --git a/doc/getting_help/faq/index.txt b/doc/getting_help/faq/index.txt new file mode 100644 index 000000000..27aa5303f --- /dev/null +++ b/doc/getting_help/faq/index.txt @@ -0,0 +1,17 @@ +.. -*- mode: rst -*- + +.. _faq-index: + +=== +FAQ +=== + +The Frequently Asked Questions (FAQ) answers the most common questions +about Bcfg2. At the moment the FAQ is splitted into a general +and a client specfic section. + +.. toctree:: + :maxdepth: 2 + + general + client diff --git a/doc/getting_help/index.txt b/doc/getting_help/index.txt index bc713bcbe..308f31df1 100644 --- a/doc/getting_help/index.txt +++ b/doc/getting_help/index.txt @@ -1,33 +1,40 @@ .. -*- mode: rst -*- -.. _help-index: +.. _gettinghelp-index: -======================= -Getting Help with Bcfg2 -======================= +.. _IRC logs: http://colabti.org/irclogger/irclogger_logs/bcfg2 + + +Getting Help +============ Having trouble? We'd like to help! -* Try the :ref:`FAQ <faq-index>` -- it's got answers to many common questions. -* Looking for specific information? Try the :ref:`genindex`, :ref:`modindex` or the :ref:`detailed table of contents <contents>`. -* Search for information in the :ref:`help-mailinglist`. -* Visit our :ref:`help-irc` channel. +There are several ways to get in touch with the community around Bcfg2. -.. _report-a-bug: +* Try the :ref:`FAQ <faq-index>` -- it's got answers to many common + questions. -Report A Bug -============ +* Looking for specific information? Try the :ref:`genindex`, + :ref:`modindex`, or the :ref:`detailed table of contents <contents>`. + +* Search for information in the :ref:`mailinglist`. -Report bugs with Bcfg2 on the `Trac ticket tracker`_. +* Ask a question in the :ref:`ircchannel`, or search the `IRC logs`_ + to see if its been asked before. -.. _Bcfg2 mailing list archives: http://trac.mcs.anl.gov/projects/bcfg2/wiki/MailingList -.. _Trac ticket tracker: http://trac.mcs.anl.gov/projects/bcfg2/wiki +Note that the IRC channel tends to be much busier than the mailing +list; use whichever seems most appropriate for your query, but don't +let the lack of mailing list activity make you think the project isn't +active. -Other ways to get in touch -========================== .. toctree:: - :maxdepth: 2 + :maxdepth: 1 - irc + reporting mailinglist + ircchannel + faq/index + error-messages + manpages diff --git a/doc/getting_help/irc.txt b/doc/getting_help/irc.txt deleted file mode 100644 index 37d2099d3..000000000 --- a/doc/getting_help/irc.txt +++ /dev/null @@ -1,45 +0,0 @@ -.. -*- mode: rst -*- - -.. _help-irc: - -=== -IRC -=== - -The Bcfg2 IRC channel is `#bcfg2 on chat.freenode.net`_. It is home -to both support and development discussions. If you have a question, -suggestion, or just want to know about Bcfg2, please drop in and say hi. - -.. _#bcfg2 on chat.freenode.net: irc://chat.freenode.net/bcfg2 - -Archives are available at: http://colabti.org/irclogger/irclogger_logs/bcfg2. - -.. raw:: html - - <form method="get" action="http://colabti.org/irclogger/irclogger_log_search/bcfg2"> - <input type=text name=search value='' size=48> - <input type=submit value="Search #bcfg2 irc logs"> - <input type=hidden name=action value=search> <br> - in the timespan - <input type=text name=timespan value='' size=17> - (yyyymmdd-yyyymmdd) - <p>Options:<br> - <label><input type=checkbox name=nick value="checked">Search in the nick field</label><br> - <label><input type=checkbox name=text checked value="checked">Search in the text field</label><br> - </form> - -Administrative Note -=================== - -If the IRC logging stops working for a while, coordinate on #bcfg2 and -then bug **feb** on #irclogger (freenode), and stick around on that -channel until you get an answer (**feb** is great, but busy and in a -non-US time zone). Actually as long as **ilogger2** is logged in you -should be okay (**feb** looks at the logs). - -If you have private logs for the period of time **ilogger2** was off the -channel, you can convert them to the `format shown here`_, and **feb** -will incorporate them into the online logs. Be sure to strip out any -private messages in your logs first :-) - -.. _format shown here: http://colabti.org/irclogger/irclogger_log/bcfg2?date=2008-03-21,Fri;raw=on diff --git a/doc/getting_help/ircchannel.txt b/doc/getting_help/ircchannel.txt new file mode 100644 index 000000000..b97e5f6a6 --- /dev/null +++ b/doc/getting_help/ircchannel.txt @@ -0,0 +1,28 @@ +.. -*- mode: rst -*- + +.. _Freenode: http://chat.freenode.net +.. _#bcfg2: irc://chat.freenode.net/bcfg2 + +.. _ircchannel: + +=========== +IRC Channel +=========== + +The Bcfg2 IRC channel is `#bcfg2`_ on `Freenode`_. It is home to both support and development discussions. If you have a question, suggestion, or just want to know about Bcfg2, please drop in and say hi. + +Archives are available at: http://colabti.org/irclogger/irclogger_logs/bcfg2 + +.. raw:: html + + <form method="get" action="http://colabti.org/irclogger/irclogger_log_search/bcfg2"> + <input type=text name=search value='' size=48> + <input type=submit value="Search #bcfg2 irc logs"> + <input type=hidden name=action value=search> <br> + in the timespan + <input type=text name=timespan value='' size=17> + (yyyymmdd-yyyymmdd) + <p>Options:<br> + <label><input type=checkbox name=nick value="checked">Search in the nick field</label><br> + <label><input type=checkbox name=text checked value="checked">Search in the text field</label><br> + </form> diff --git a/doc/getting_help/manpages.txt b/doc/getting_help/manpages.txt new file mode 100644 index 000000000..14c22a92e --- /dev/null +++ b/doc/getting_help/manpages.txt @@ -0,0 +1,16 @@ +.. -*- mode: rst -*- + +.. _manpages: + +============ +Manual pages +============ + +These are copies of the bcfg2 manual pages created with man2html. +The most recent versions of these man pages are always in the `man/` +directory in the source. + +.. toctree:: + :glob: + + manpages/* diff --git a/doc/getting_help/manpages/bcfg2-admin.txt b/doc/getting_help/manpages/bcfg2-admin.txt new file mode 100644 index 000000000..4440c7134 --- /dev/null +++ b/doc/getting_help/manpages/bcfg2-admin.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _manpages-bcfg2-admin: + +=========== +bcfg2-admin +=========== + +The man page of ``bcfg2-admin``. :: + + man bcfg2-admin diff --git a/doc/getting_help/manpages/bcfg2-build-reports.txt b/doc/getting_help/manpages/bcfg2-build-reports.txt new file mode 100644 index 000000000..217c25627 --- /dev/null +++ b/doc/getting_help/manpages/bcfg2-build-reports.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _manpages-bcfg2-build-reports: + +=================== +bcfg2-build-reports +=================== + +The man page of ``bcfg2-build-reports``. :: + + man bcfg2-build-reports diff --git a/doc/getting_help/manpages/bcfg2-conf.txt b/doc/getting_help/manpages/bcfg2-conf.txt new file mode 100644 index 000000000..623521a67 --- /dev/null +++ b/doc/getting_help/manpages/bcfg2-conf.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _manpages-bcfg2.conf: + +========== +bcfg2.conf +========== + +The man page of ``bcfg2.conf``. :: + + man bcfg2.conf diff --git a/doc/getting_help/manpages/bcfg2-info.txt b/doc/getting_help/manpages/bcfg2-info.txt new file mode 100644 index 000000000..751905e0b --- /dev/null +++ b/doc/getting_help/manpages/bcfg2-info.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _manpages-bcfg2-info: + +========== +bcfg2-info +========== + +The man page of ``bcfg2-info``. :: + + man bcfg2-info diff --git a/doc/getting_help/manpages/bcfg2-repo-validate.txt b/doc/getting_help/manpages/bcfg2-repo-validate.txt new file mode 100644 index 000000000..b13bd928b --- /dev/null +++ b/doc/getting_help/manpages/bcfg2-repo-validate.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _manpages-bcfg2-repo-validate: + +=================== +bcfg2-repo-validate +=================== + +The man page of ``bcfg2-repo-validate``. :: + + man bcfg2-repo-validate diff --git a/doc/getting_help/manpages/bcfg2-server.txt b/doc/getting_help/manpages/bcfg2-server.txt new file mode 100644 index 000000000..27002789e --- /dev/null +++ b/doc/getting_help/manpages/bcfg2-server.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _manpages-bcfg2-server: + +============ +bcfg2-server +============ + +The man page of ``bcfg2-server``. :: + + man bcfg2-server diff --git a/doc/getting_help/manpages/bcfg2.txt b/doc/getting_help/manpages/bcfg2.txt new file mode 100644 index 000000000..4e93ab449 --- /dev/null +++ b/doc/getting_help/manpages/bcfg2.txt @@ -0,0 +1,11 @@ +.. -*- mode: rst -*- + +.. _manpages-bcfg2: + +===== +bcfg2 +===== + +The man page of bcfg2. :: + + man bcfg2 diff --git a/doc/getting_help/reporting.txt b/doc/getting_help/reporting.txt new file mode 100644 index 000000000..cd53b1206 --- /dev/null +++ b/doc/getting_help/reporting.txt @@ -0,0 +1,12 @@ +.. -*- mode: rst -*- + +.. _reporting: + +.. _tracker: http://trac.mcs.anl.gov/projects/bcfg2/wiki + +============== +Reporting bugs +============== + +Report bugs with Bcfg2 in our `tracker`_. + diff --git a/doc/getting_started/index.txt b/doc/getting_started/index.txt index 2c05c5238..786c70290 100644 --- a/doc/getting_started/index.txt +++ b/doc/getting_started/index.txt @@ -2,57 +2,245 @@ .. _getting_started-index: -=========== -Using Bcfg2 -=========== +=============== +Getting started +=============== -These are many of the resources available to help new users get started. +The steps below should get you from just thinking about a +configuration management system to an operational installation of +Bcfg2. If you get stuck, be sure to check the :ref:`mailinglist` +or to drop in on our :ref:`ircchannel`. -* For the impatient there is the :ref:`quickstart-index` page. -* :ref:`help-index` has information when you are troubleshooting or need to ask a question. -* If you find anything wrong or missing please :ref:`report-a-bug` to let us know. +Get and Install Bcfg2 Server +============================ -.. toctree:: - :maxdepth: 2 +We recommend running the server on a Linux machine for ease of +deployment due to the availability of packages for the dependencies. - using-bcfg2-info - using-bcfg2-with-centos +First, you need to download and install Bcfg2. The section +:ref:`installation-index` in this manual describes the steps to take. +To start, you will need to install the server on one machine and the +client on one or more machines. Yes, your server can also be a client +(and should be by the time your environment is fully managed). -Must Reads -========== +.. _Bcfg2 download page: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Download +.. _Install page: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Install + +Set up Repository +================= + +The next step after installing the Bcfg2 packages is to configure the +server. You can easily set up a personalized default configuration by +running, on the server, :: + + bcfg2-admin init -* `Documentation` -* `WritingSpecification` -* `Bcfg2 Architecture <Doc/Architecture>` -* `ServerSideOverview` -* `UpgradeTesting` -* `Troubleshooting` -* `Bcfg2 Server Plugins <Plugins>` +You will be presented with a series of questions that will build a +Bcfg2 configuration file in ``/etc/bcfg2.conf``, set up a skeleton +repository (in ``/var/lib/bcfg2`` by default), help you create ssl +certificates, and do any other similar tasks needed to get you +started. + +Once this process is done, you can start the Bcfg2 server:: + + /etc/init.d/bcfg2-server start + +You can try it out by running the Bcfg2 client on the same machine, +acting like it is your first client. + +.. note:: + + The following command will tell the client to run in no-op mode, + meaning it will only check the client against the repository and + report any changes it sees. It won't make any changes (partially + because you haven't populated the repository with any + yet). However, nobody is perfect - you can make a typo, our + software can have bugs, monkeys can break in and hit enter before + you are done. Don't run this command on a production system if you + don't know what it does and aren't prepared for the + consequences. We don't know of anybody having problems with it + before, but it is better to be safe than sorry. + +And now for the command:: + + bcfg2 -q -v -n -Misc/Others -=========== +That can be translated as "bcfg2 quick verbose no-op". The output +should be something similar to:: -* `Probes <Plugins/Probes>` -* `AgentMode` -* `DynamicGroups` -* `Client Tool Drivers <ClientTools>` -* `Developing for Bcfg2 <DevelopingForBcfg2>` -* `Howtos` + Loaded tool drivers: + Chkconfig POSIX PostInstall RPM -Reporting -========= + Phase: initial + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 242 + + + Phase: final + Correct entries: 0 + Incorrect entries: 0 + Total managed entries: 0 + Unmanaged entries: 242 + +Perfect! We have started out with an empty configuration, and none of +our configuration elements are correct. It doesn't get much cleaner +than that. But what about those unmanaged entries? Those are the extra +configuration elements (probably all packages at the moment) that +still aren't managed. Your goal now is to migrate each of those plus +any it can't see up to the "Correct entries" line. + +Populate Repository +=================== + +Finally, you need to populate your repository. Unfortunately, from +here on out we can't write up a simple recipe for you to follow to get +this done. It is very dependent on your local configuration, your +configuration management goals, the politics surrounding your +particular machines, and many other similar details. We can, however, +give you guidance. + +After the above steps, you should have a toplevel repository structure +that looks like:: + + bcfg-server:~ # ls /var/lib/bcfg2 + Bundler/ Cfg/ Metadata/ Pkgmgr/ Rules/ SSHbase/ etc/ + +The place to start is the Metadata directory, which contains two +files: ``clients.xml`` and ``groups.xml``. Your current +``clients.xml`` will look pretty close to: + +.. code-block:: xml + + <Clients version="3.0"> + <Client profile="basic" pingable="Y" pingtime="0" name="bcfg-server.example.com"/> + </Clients> + +The ``clients.xml`` file is just a series of ``<Client />`` tags, each +of which describe one host you manage. Right now we only manage one +host, the server machine we just created. This machine is bound to the +``basic`` profile, is pingable, has a pingtime of ``0``, and has the +name ``bcfg-server.example.com``. The two "ping" parameters don't +matter to us at the moment, but the other two do. The name parameter +is the fully qualified domain name of your host, and the profile +parameter maps that host into the ``groups.xml`` file. + +Our simple ``groups.xml`` file looks like: + +.. code-block:: xml + + <Groups version='3.0'> + <Group profile='true' public='false' name='basic'> + <Group name='suse'/> + </Group> + <Group name='ubuntu' toolset='debian'/> + <Group name='debian' toolset='debian'/> + <Group name='redhat' toolset='rh'/> + <Group name='suse' toolset='rh'/> + <Group name='mandrake' toolset='rh'/> + <Group name='solaris' toolset='solaris'/> + </Groups> + +There are two types of groups in Bcfg: profile groups +(``profile='true'``) and non-profile groups +(``profile='false'``). Profile groups can act as top-level groups to +which clients can bind, while non-profile groups only exist as members +of other groups. In our simple starter case, we have a profile group +named ``basic``, and that is the group that our first client bound +to. Our first client is a SuSE machine, so it contains the ``suse`` +group. Of course, ``bcfg2-admin`` isn't smart enough to fill out the +rest of your config, so the ``suse`` group further down is empty. + +Let's say the first thing we want to set up on our machine is the +message of the day. To do this, we simply need to create a Bundle and +add that Bundle to an appropriate group. In this simple example, we +start out by adding + +.. code-block:: xml + + <Bundle name='motd'/> + +to the ``basic`` group. + +Next, we create a motd.xml file in the Bundler directory: + +.. code-block:: xml + + <Bundle name='motd' version='2.0'> + <Path name='/etc/motd' /> + </Bundle> + +Now when we run the client, we get slightly different output:: + + Loaded tool drivers: + Chkconfig POSIX PostInstall RPM + Incomplete information for entry Path:/etc/motd; cannot verify + + Phase: initial + Correct entries: 0 + Incorrect entries: 1 + Total managed entries: 1 + Unmanaged entries: 242 + + In dryrun mode: suppressing entry installation for: + Path:/etc/motd + + Phase: final + Correct entries: 0 + Incorrect entries: 1 + Total managed entries: 1 + Unmanaged entries: 242 + +We now have an extra unmanaged entry, bringing our total number of +managed entries up to one. To manage it we need to copy ``/etc/motd`` +to ``/var/lib/bcfg2/Cfg/etc/motd/``. Note the layout of that path: all +plain-text config files live in the Cfg directory. The directory +structure under that directory directly mimics your real filesystem +layout, making it easy to find and add new files. The last directory +is the name of the file itself, so in this case the full path to the +motd file would be ``/var/lib/bcfg2/Cfg/etc/motd/motd``. Copy your +real ``/etc/motd`` file to that location, run the client again, and +you will find that we now have a correct entry:: + + Loaded tool drivers: + Chkconfig POSIX PostInstall RPM + + Phase: initial + Correct entries: 1 + Incorrect entries: 0 + Total managed entries: 1 + Unmanaged entries: 242 + + + Phase: final + Correct entries: 1 + Incorrect entries: 0 + Total managed entries: 1 + Unmanaged entries: 242 + +Done! Now we just have 242 (or more) entries to take care of! + +:ref:`server-plugins-structures-bundler-index` is a relatively easy +directory to populate. You can find many samples of Bundles in the +`Bundle Repository`_, many of which can be used without editing. + +.. _Bundle Repository: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Plugins/Bundler/examples + +Next Steps +========== -* :ref:`server-reports-index` +Several other utilities can help from this point on: -Examples -======== +``bcfg2-info`` is a utility that instantiates a copy of the Bcfg2 server +core (minus the networking code) for examination. From this, you can +directly query: -* `Demo Repository <GroupNames>` -* `Plugins/Bundler/examples` -* `Plugins/Probes/examples` -* `Plugins/TGenshi/examples` +* Client Metadata +* Which entries are provided by particular plugins +* Client Configurations -Man Pages -========= +Run ``bcfg2-info``, and type help and the prompt when it comes up. -* `Manual Pages <ManualPages>` +``bcfg2-admin`` can perform a variety of repository maintenance +tasks. Run ``bcfg2-admin`` help for details. diff --git a/doc/installation/distributions.txt b/doc/installation/distributions.txt new file mode 100644 index 000000000..5a86e81d5 --- /dev/null +++ b/doc/installation/distributions.txt @@ -0,0 +1,17 @@ +.. -*- mode: rst -*- + +.. _distributions: + +=========================== +Distribution-specific notes +=========================== + +The installation of Bcfg2 on a specific distribution depends on +the used package management tool and the disposability in the +distribution's package :term:`repository` +. + +.. toctree:: + :glob: + + distro/* diff --git a/doc/installation/distro/archlinux.txt b/doc/installation/distro/archlinux.txt new file mode 100644 index 000000000..75ff59c49 --- /dev/null +++ b/doc/installation/distro/archlinux.txt @@ -0,0 +1,17 @@ +.. -*- mode: rst -*- + +.. _installation-archlinux: + +.. _Arch Linux: http://www.archlinux.org/ +.. _AUR: http://aur.archlinux.org/packages.php?ID=20979 + + +ArchLinux +========= + +Packages for `Arch Linux`_ are available in the Arch User Repository (AUR_). +Just use `pacman` to perform the installation :: + + pacman -S bcfg2 bcfg2-server + + diff --git a/doc/installation/distro/debian.txt b/doc/installation/distro/debian.txt new file mode 100644 index 000000000..67a4f1d9d --- /dev/null +++ b/doc/installation/distro/debian.txt @@ -0,0 +1,24 @@ +.. -*- mode: rst -*- + +.. _debian: + +.. _Wiki: http://trac.mcs.anl.gov/projects/bcfg2/wiki/PrecompiledPackages#UnofficialDebianRepository + +Debian +====== + +Packages of Bcfg2 are available for Debian Lenny, Debian Squeeze, and +Debian Sid. The fastest way to get Bcfg2 onto your Debian system +is to use `apt-get` or `aptitude`. :: + + sudo aptitude install bcfg2 bcfg2-server + +If you want to use unofficial packages from Bcfg2. Add the following line +to your `/etc/apt/sources.list` file :: + + deb ftp://ftp.mcs.anl.gov/pub/bcfg/debian sarge/ + +Now just run `aptitute` in the way it is mentioned above. + +For more details about running prerelease version of Bcfg2 on Debian +systems, please refer to the Wiki_. diff --git a/doc/installation/distro/fedora.txt b/doc/installation/distro/fedora.txt new file mode 100644 index 000000000..92771b974 --- /dev/null +++ b/doc/installation/distro/fedora.txt @@ -0,0 +1,23 @@ +.. -*- mode: rst -*- + +.. _Packages: https://admin.fedoraproject.org/pkgdb/acls/name/bcfg2 +.. _Fedora: https://www.fedoraproject.org + +.. _fedora-installation: + +Fedora +====== + +The fastest way to get Bcfg2 Packages_ onto your Fedora_ system is to +use `yum` or PackageKit. Yum will pull in all dependencies of Bcfg2 +automatically. :: + + $ su -c 'yum install bcfg2-server bcfg2' + +Be aware that the latest release of Bcfg2 may only be available for the +Development release of Fedora (Rawhide). With the activation of the +Rawhide repository of Fedora you will be able to install it. :: + + $ su -c 'yum install --enablerepo=rawhide bcfg2-server bcfg2' + +This way is not recommanded on productive systems. Only for testing. diff --git a/doc/installation/distro/gentoo.txt b/doc/installation/distro/gentoo.txt new file mode 100644 index 000000000..5497a01b2 --- /dev/null +++ b/doc/installation/distro/gentoo.txt @@ -0,0 +1,27 @@ +.. -*- mode: rst -*- + +.. _gentoo-installation: + +.. _Freenode: http://chat.freenode.net +.. _#bcfg2: irc://chat.freenode.net/bcfg2 + + +Gentoo +====== + +Early in July 2008, Bcfg2 was added to the Gentoo portage tree. So far +it's only keyworded for ~x86, but we hope to see it soon in the amd64 and +x64-solaris ports. If you're using Gentoo on some other architecture, it +should still work provided that you have a reasonably up to date Python; +try adding `app-admin/bcfg2 ~*` to your `/etc/portage/package.keywords` +file. + +If you don’t use portage to install Bcfg2, you’ll want to make sure you +have all the prerequisites installed first. For a server, you’ll need: + +* ``app-admin/gamin`` or ``app-admin/fam`` +* ``dev-python/lxml`` + +Clients will need at least: + +* ``app-portage/gentoolkit`` diff --git a/doc/installation/distro/osx.txt b/doc/installation/distro/osx.txt new file mode 100644 index 000000000..8b7a5a95d --- /dev/null +++ b/doc/installation/distro/osx.txt @@ -0,0 +1,29 @@ +.. -*- mode: rst -*- + +.. _osx-installation: + +.. _Freenode: http://chat.freenode.net +.. _#bcfg2: irc://chat.freenode.net/bcfg2 + + +OS X +==== + +Once macports is installed:: + + port install bcfg2 + +Using native OS X python +------------------------ + +First, make sure you have Xcode installed as you need `packagemaker` which +comes bundled in the Developer tools. + +Clone the git source:: + + git clone git://git.mcs.anl.gov/bcfg2.git + +Change to the osx directory and type make. Your new package should be located +at bcfg2-'''$VERSION'''.pkg (where '''$VERSION''' is that which is specified +in setup.py). + diff --git a/doc/installation/distro/rhel.txt b/doc/installation/distro/rhel.txt new file mode 100644 index 000000000..49e43179f --- /dev/null +++ b/doc/installation/distro/rhel.txt @@ -0,0 +1,31 @@ +.. -*- mode: rst -*- + +.. _CentOS: http://www.centos.org/ +.. _Red Hat/RHEL: http://www.redhat.com/rhel/ +.. _Scientific Linux: http://www.scientificlinux.org/ +.. _EPEL: http://fedoraproject.org/wiki/EPEL +.. _RPMForge: https://rpmrepo.org/RPMforge + +.. _rhel-installation: + +RHEL / Centos / Scientific Linux +================================ + +While you can go about building all these things from source, this +section will try and meet the dependencies using packages from EPEL_ +[#f1]_. The *el5* and the soon available *el6* package should be compatible +with `CentOS`_ 5.x/6.x and `Scientific Linux`_. + +EPEL_:: + + [root@centos ~]# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm + +Install the bcfg2-server and bcfg2 RPMs:: + + [root@centos ~]# yum install bcfg2-server bcfg2 + +.. note:: + + The latest package for *el5* is only available in the testing repository. + +.. [#f1] For more details check the EPEL_ `instructions <http://fedoraproject.org/wiki/EPEL/FAQ#howtouse>`_ diff --git a/doc/installation/distro/ubuntu.txt b/doc/installation/distro/ubuntu.txt new file mode 100644 index 000000000..694cce865 --- /dev/null +++ b/doc/installation/distro/ubuntu.txt @@ -0,0 +1,36 @@ +.. -*- mode: rst -*- + +.. _ubuntu-installation: + +.. _Ubuntu: http://www.ubuntu.com +.. _Wiki: http://trac.mcs.anl.gov/projects/bcfg2/wiki/PrecompiledPackages#UbuntuLaunchpadBcfg2PPA + +Ubuntu +====== + +The steps to bring Bcfg2 onto your Ubuntu_ system depends on your +release. + +* Dapper + Add the following lines to `/etc/apt/sources.list` :: + + deb ftp://ftp.mcs.anl.gov/pub/bcfg/ubuntu dapper/ + deb http://archive.ubuntu.com/ubuntu dapper universe + deb-src http://archive.ubuntu.com/ubuntu dapper universe + +* Edgy + Add the following lines to `/etc/apt/sources.list` :: + + deb ftp://ftp.mcs.anl.gov/pub/bcfg/ubuntu edgy/ + deb http://archive.ubuntu.com/ubuntu edgy universe + deb-src http://archive.ubuntu.com/ubuntu edgy universe + +* Feisty + Those packages are available from the Ubuntu_ repositories. + +To install the packages, just lauch the following command :: + + sudo aptitude install bcfg2 bcfg2-server + +For more details about running prerelease version of Bcfg2 on Ubuntu_ +systems, please refer to the Wiki_. diff --git a/doc/installation/index.txt b/doc/installation/index.txt new file mode 100644 index 000000000..9f04d4b52 --- /dev/null +++ b/doc/installation/index.txt @@ -0,0 +1,23 @@ +.. -*- mode: rst -*- + +.. _installation-index: + +============ +Installation +============ + +Before installing, you will need to choose a machine to be the Bcfg2 +server. We recommend a Linux-based machine for this purpose, but the +server will work on any supported operating system. Note that you may +eventually want to run a web server on this machine for reporting and +serving up package repositories. The server package only needs to be +installed on your designated Bcfg2 server machine. The clients package +needs to be installed on any machine you plan to manage by Bcfg2. + +.. toctree:: + :maxdepth: 2 + + prerequisites + source + packages + distributions diff --git a/doc/installation/packages.txt b/doc/installation/packages.txt new file mode 100644 index 000000000..a15e3e98e --- /dev/null +++ b/doc/installation/packages.txt @@ -0,0 +1,81 @@ +.. -*- mode: rst -*- + +.. _packages: + +.. _CentOS: http://www.centos.org/ +.. _Red Hat/RHEL: http://www.redhat.com/rhel/ +.. _Scientific Linux: http://www.scientificlinux.org/ +.. _EPEL: http://fedoraproject.org/wiki/EPEL +.. _RPMForge: https://rpmrepo.org/RPMforge + + +Building packages from source +============================= + +The Bcfg2 distribution contains two different spec files. + +Building from Tarball +--------------------- + +* Copy the tarball to `/usr/src/packages/SOURCES/` +* Extract another copy of it somewhere else (eg: `/tmp`) and retrieve + the `misc/bcfg2.spec` file +* Run :: + + rpmbuild -ba bcfg2.spec + +* The resulting RPMs will be in `/usr/src/packages/RPMS/` and SRPMs + in `/usr/src/packages/SRPMS` + +Building from an GIT Checkout +----------------------------- + +* Change to the `redhat/` directory in the working copy +* Run :: + + make + +* The resulting RPMs will be in `/usr/src/redhat/RPMS/ `and SRPMs + in `/usr/src/redhat/SRPMS` and will have the SVN revision appended + +Building RPM packages with ``rpmbuild`` +--------------------------------------- + +While you can go about building all these things from source, this +how to will try and meet the dependencies using packages from EPEL_. +The *el5* package should be compatible with CentOS 5.x. + +* Installation of the EPEL_ repository package :: + + [root@centos ~]# rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm + +* Now you can install the rest of the prerequisites :: + + [root@centos ~]# yum install python-genshi python-cheetah python-lxml + +* After installing git, check out the master branch :: + + [root@centos redhat]# git clone git://git.mcs.anl.gov/bcfg2.git + +* Install the ``fedora-packager`` package :: + + [root@centos ~]# yum install fedora-packager + +* A directory structure for the RPM build process has to be established. :: + + [you@centos ~]$ rpmdev-setuptree + +* Change to the *redhat* directory of the checked out Bcfg2 source:: + + [you@centos ~]$ cd bcfg2/redhat/ + +* In the particular directory is a ``Makefile`` which will do the job of + building the RPM packages. You can do this as root, but it's not + recommanded :: + + [you@centos redhat]$ make + +* Now the new RPM package can be installed. Please adjust the path to + your RPM package :: + + [root@centos ~]# rpm -ihv /home/YOU/rpmbuild/RPMS/noarch/bcfg2-server-1.0.0-0.2r5835.noarch.rpm diff --git a/doc/installation/prerequisites.txt b/doc/installation/prerequisites.txt new file mode 100644 index 000000000..e4ea715b0 --- /dev/null +++ b/doc/installation/prerequisites.txt @@ -0,0 +1,47 @@ +.. -*- mode: rst -*- + +.. _prerequisites: + +Prerequisites +============= + +Bcfg2 has several server side prerequisites and a minimal set of +client side requirements. This page describes the prerequisite +software situation on all supported platforms. The table describes +what software is needed on the client and server side. + + +Bcfg2 Client +------------ + +===================================== =================== ============================== +Software Version Requires +===================================== =================== ============================== +libxml2 (if lxml is used) Any +libxslt (if lxml is used) Any libxml2 +python 2.3-2.4, 2.5 [#f1]_ +lxml or elementtree [#f2]_ Any lxml: libxml2, libxslt, python +python-apt [#f3]_ Any python +debsums (if APT tool driver is used) Any +===================================== =================== ============================== + + +.. [#f1] python 2.5 works with elementtree. +.. [#f2] elementtree is included in python 2.5 and later. +.. [#f3] python-apt is only required on platforms that use apt, such as Debian and Ubuntu. + +Bcfg2 Server +------------ + +===================================== ============= ============================== +Software Version Requires +===================================== ============= ============================== +libxml2 2.6.24+ +libxslt Any libxml2 +python 2.2-2.5 +lxml 0.9+ lxml: libxml2, libxslt, python +gamin or fam Any +python-gamin or python-fam Any gamin or fam, python +M2crypto Any python, openssl +===================================== ============= ============================== + diff --git a/doc/installation/source.txt b/doc/installation/source.txt new file mode 100644 index 000000000..e21f7150f --- /dev/null +++ b/doc/installation/source.txt @@ -0,0 +1,64 @@ +.. -*- mode: rst -*- + +.. _Download: http://trac.mcs.anl.gov/projects/bcfg2/wiki/Download +.. _GPG: http://pgpkeys.mit.edu:11371/pks/lookup?op=get&search=0x75BF2C177F7D197E +.. _GPGKey: ftp://ftp.mcs.anl.gov/pub/bcfg/bcfg2-1.1.0.tar.gz.gpg +.. _Tarball: ftp://ftp.mcs.anl.gov/pub/bcfg/bcfg2-1.1.0.tar.gz + +.. |md5sum| replace:: 13593938daf7e8b9a81cb4b677dc7f99 +.. |version| replace:: 1.1.0 +.. |rel_date| replace:: 9/27/10 + +.. _source: + +Download +======== + +Tarball +------- + +The Bcfg2 source tarball can be grabbed from the Download_ page. + +========= ======== ======= ============== ===================== +Version URL GPG key Release Date md5sum +========= ======== ======= ============== ===================== +|version| Tarball_ GPGKey_ |rel_date| |md5sum| +========= ======== ======= ============== ===================== + +The full command to use with ``wget`` are listed below. Please +replace ``<version>`` with |version|. :: + + wget ftp://ftp.mcs.anl.gov/pub/bcfg/bcfg2-<version>.tar.gz + wget ftp://ftp.mcs.anl.gov/pub/bcfg/bcfg2-<version>.tar.gz.gpg + +All tarballs are signed with GPG key `7F7D197E <GPGKey>`_. You can verify +your download by importing the key and running :: + + $ gpg --recv-keys 0x75bf2c177f7d197e + $ gpg --verify bcfg2-<version>.tar.gz.gpg bcfg2-<version>.tar.gz + +For older or prepreleases please visit the Download_ wiki page. + + +Git checkout +------------ + +You can also get the latest (possibly broken) code via git :: + + git clone git://git.mcs.anl.gov/bcfg2.git + +Installation from source +======================== + +If you are working with the release tarball of Bcfg2 you need to +untar it before you can go on with the installation :: + + tar -xzf bcfg2-<version>.tar.gz + +Now you can build Bcfg2 with. If you are working with a SVN checkout +no <version> need to be specified. :: + + cd bcfg2-<version> + python setup.py install --prefix=/install/prefix + +This will install both the client and server on that machine. diff --git a/doc/server/plugins/generators/tcheetah.txt b/doc/server/plugins/generators/tcheetah.txt index 52a0f3264..e1ad600a2 100644 --- a/doc/server/plugins/generators/tcheetah.txt +++ b/doc/server/plugins/generators/tcheetah.txt @@ -27,7 +27,7 @@ located in in a ``TCheetah`` subdirectory of your repository, usually files, ``template`` and ``info``. The template is a standard Cheetah template with two additions: -* `self.metadata` is the client's :ref:`metadata <server-plugins-grouping-metadata-clientmetadata>` +* `self.metadata` is the client's metadata * `self.properties` is an xml document of unstructured data The ``info`` file is formatted like ``:info`` files from Cfg. @@ -44,7 +44,19 @@ Permissions entry and a Path entry to handle the same file. self.metadata variables ======================= -self.metadata is an instance of the class ClientMetadata and documented :ref:`here <server-plugins-grouping-metadata-clientmetadata>`. +The following variables are available for self.metadata: + +* hostname +* bundles +* groups +* categories +* probes +* uuid +* password + +self.metadata is an instance of the class +ClientMetadata of file `Bcfg2/Server/Plugins/Metadata.py +<http://trac.mcs.anl.gov/projects/bcfg2/browser/trunk/bcfg2/src/lib/Server/Plugins/Metadata.py>`_. self.properties =============== diff --git a/doc/server/plugins/generators/tgenshi/index.txt b/doc/server/plugins/generators/tgenshi/index.txt index 425b3a289..cd9bcf152 100644 --- a/doc/server/plugins/generators/tgenshi/index.txt +++ b/doc/server/plugins/generators/tgenshi/index.txt @@ -48,10 +48,8 @@ supported. Inside of templates =================== -* **metadata** is the client's :ref:`metadata <server-plugins-grouping-metadata-clientmetadata>` -* **properties.properties** is an xml document of unstructured data -* **name** is the path name specified in bcfg -* **path** is the path to the TGenshi template +* metadata is the client's metadata +* properties.properties is an xml document of unstructured data See the genshi `documentation <http://genshi.edgewall.org/wiki/Documentation>`_ for examples of diff --git a/doc/server/plugins/grouping/metadata.txt b/doc/server/plugins/grouping/metadata.txt index 22c967262..96fc70ff5 100644 --- a/doc/server/plugins/grouping/metadata.txt +++ b/doc/server/plugins/grouping/metadata.txt @@ -225,70 +225,3 @@ Probes The metadata plugin includes client-side probing functionality. This is fully documented :ref:`here <server-plugins-probes-index>`. - -.. _server-plugins-grouping-metadata-clientmetadata: - -ClientMetadata -============== - -A special client metadata class is available to the TGenshi and TCheetah -plugins. - -+----------------------+------------------------------------------------+-------------------+ -| Attribute | Description | Value | -+======================+================================================+===================+ -| hostname | Client hostname | String | -+----------------------+------------------------------------------------+-------------------+ -| profile | Client profile | String | -+----------------------+------------------------------------------------+-------------------+ -| aliases | Client aliases | List | -+----------------------+------------------------------------------------+-------------------+ -| addresses | Adresses this client is known by | List | -+----------------------+------------------------------------------------+-------------------+ -| groups | Groups this client is a member of | List | -+----------------------+------------------------------------------------+-------------------+ -| categories | Categories of this clients groups | List | -+----------------------+------------------------------------------------+-------------------+ -| uuid | uuid identifier for this client | String | -+----------------------+------------------------------------------------+-------------------+ -| password | bcfg password for this client | String | -+----------------------+------------------------------------------------+-------------------+ -| connectors | connector plugins known to this client | List | -+----------------------+------------------------------------------------+-------------------+ -| query | MetadataQuery object | MetadataQuery | -+----------------------+------------------------------------------------+-------------------+ - -| - -+------------------------------+------------------------------------------------+-------------------+ -| Method | Description | Value | -+==============================+================================================+===================+ -| inGroup(group) | True if this client is a memnber of 'group' | Boolean | -+------------------------------+------------------------------------------------+-------------------+ -| group_in_category(category) | Returns the group in 'category' if the client | String | -| | is a member of 'category', otherwise '' | | -+------------------------------+------------------------------------------------+-------------------+ - -MetadataQuery -------------- - -This class provides query routines for the servers Metadata. - -+------------------------------+------------------------------------------------+-------------------+ -| Method | Description | Value | -+==============================+================================================+===================+ -| by_name(client) | Get ClientMetadata object for 'client' | ClientMetadata | -+------------------------------+------------------------------------------------+-------------------+ -| names_by_groups(group) | | | -+------------------------------+------------------------------------------------+-------------------+ -| names_by_profiles(profile) | All clients names in 'profile' | List | -+------------------------------+------------------------------------------------+-------------------+ -| all_clients() | All known client hostnames | List | -+------------------------------+------------------------------------------------+-------------------+ -| all_groups() | All known group names | List | -+------------------------------+------------------------------------------------+-------------------+ -| all_groups_in_category(cat) | All groups in category 'cat' | List | -+------------------------------+------------------------------------------------+-------------------+ -| all() | Get ClientMetadata for all clients | List | -+------------------------------+------------------------------------------------+-------------------+ - diff --git a/doc/server/plugins/index.txt b/doc/server/plugins/index.txt index 61f91da86..126331325 100644 --- a/doc/server/plugins/index.txt +++ b/doc/server/plugins/index.txt @@ -68,15 +68,6 @@ Literal Configuration (Generators) Each of these plugins has a corresponding subdirectory with the same name in the Bcfg2 repository. -Connector Plugins ------------------ - -.. toctree:: - :maxdepth: 2 - :glob: - - connectors/* - Statistics Plugins ------------------ @@ -112,4 +103,5 @@ More details can be found in :ref:`server-plugins-plugin-roles` plugin-roles probes/index + properties trigger diff --git a/doc/server/plugins/properties.txt b/doc/server/plugins/properties.txt new file mode 100644 index 000000000..fa8bfd884 --- /dev/null +++ b/doc/server/plugins/properties.txt @@ -0,0 +1,46 @@ +.. -*- mode: rst -*- + +.. _server-plugins-properties: + +========== +Properties +========== + +The Properties plugin is a connector plugin that adds information from +properties files into client metadata instances. + +Enabling Properties +=================== + +First, ``mkdir /var/lib/bcfg2/Properties``. Each property XML file goes +in this directory. Each will automatically be cached by the server, +and reread/reparsed upon changes. Add **Properties** to your ``plugins`` +line in ``/etc/bcfg2.conf``. + +Data Structures +=============== + +Properties adds a new dictionary to client metadata instances that maps +property file names to PropertyFile instances. PropertyFile instances +contain parsed XML data as the "data" attribute. + +Usage +===== + +Specific property files can be referred to in +templates as metadata.Properties[<filename>]. The +data attribute is an LXML element object. (Documented +`here <http://codespeak.net/lxml/tutorial.html#the-element-class>`_) + +Currently, no access methods are defined for this data, but as we +formulate common use cases, we will add them to the !PropertyFile class +as methods. This will simplify templates. + +Accessing Properties contest from TGenshi +========================================= + +Access contents of ``Properties/auth.xml`` + +:: + + ${metadata.Properties['auth.xml'].data.find('file').find('bcfg2.key').text} diff --git a/doc/server/plugins/structures/bundler/index.txt b/doc/server/plugins/structures/bundler/index.txt index da49b299e..7b4cd7b6f 100644 --- a/doc/server/plugins/structures/bundler/index.txt +++ b/doc/server/plugins/structures/bundler/index.txt @@ -29,7 +29,7 @@ The following is an annotated copy of a bundle: .. code-block:: xml - <Bundle name='ssh' version='2.0'> + <Bundle name='ssh' version='2.0'> <Path name='/etc/ssh/ssh_host_dsa_key'/> <Path name='/etc/ssh/ssh_host_rsa_key'/> <Path name='/etc/ssh/ssh_host_dsa_key.pub'/> diff --git a/doc/unsorted/contribute.txt b/doc/unsorted/contribute.txt deleted file mode 100644 index fab9d09a1..000000000 --- a/doc/unsorted/contribute.txt +++ /dev/null @@ -1,81 +0,0 @@ -.. -*- mode: rst -*- - -.. _unsorted-contribute: - -===================== -Contributing to Bcfg2 -===================== - -There are several ways users can contribute to the Bcfg2 project. - -* Developing code -* Testing prereleases -* Adding to the common repository -* Improving the wiki - -Development -=========== - -Send patches to the [wiki:MailingList bcfg mailing list] or create -a trac [https://trac.mcs.anl.gov/projects/bcfg2/newticket ticket] -with the patch included. In order to submit a ticket via the -trac system, you will need to create a session by clicking on the -[https://trac.mcs.anl.gov/projects/bcfg2/prefs Preferences] link and -filling out/saving changes to the form. In order to be considered for -mainline inclusion, patches need to be BSD licensed. The most convenient -way to prepare patches is by using git diff inside of a source tree -checked out of subversion. The source tree can be checked out by running:: - - git clone git://git.mcs.anl.gov/bcfg2.git - -Users wishing to contribute on a regular basis can apply for direct -subversion access. Mail the mailing list for details. - -Several resources for developers exist in the wiki: - -* [wiki:DevelopmentTips] -* [wiki:Development/WritingPlugins Writing Bcfg2 Server Plugins] -* [wiki:Architecture] -* [wiki:WritingClientToolDrivers] -* [wiki:Bcfg2SubversionHowto] -* [wiki:LearningPython] -* [wiki:UsingRcache] - -Bcfg2 is the result of a lot of work by many different people. They are -listed on the [wiki:Contributors contributors page.] - -Feel free to drop in during a [wiki:CodeSprintIdeas code sprint] if you -would like to help out with some easier problems. - -Testing Prereleases -=================== - -Before each release, several prereleases will be tagged. It is helpful -to have users test these releases (when feasible) because it is hard to -replicate the full range of potential reconfiguration situations; between -different operating systems, system management tools, and configuration -specification variation, there can be large differences between sites. - -See the [wiki:TrackingDevelopmentTrunk] page for a better view of changes -in the prereleases. - -Adding to the Common Repository -=============================== - -The Bcfg2 common repository is a set of portable examples that new -repositories can be based on. This repo has several parts. The first -is a series of group definitions that describe a wide range of client -systems. The second is a set of portable bundles that have been ported -to use these groups. This makes these bundles transparently portable -across new client architectures. - -This approach has several benefits to users - -* Configuration specification can be shared across sites where appropriate -* This common configuration specification can be reused, allowing sites to migrate to new systems that other sites have already ported the common repository to -* Setup of new systems becomes a lot easier. - -Improving the wiki -================== - -Mail the [wiki:MailingList mailing list] for an account on the wiki. diff --git a/doc/unsorted/developing_for_bcfg2.txt b/doc/unsorted/developing_for_bcfg2.txt deleted file mode 100644 index 9d3e77bd7..000000000 --- a/doc/unsorted/developing_for_bcfg2.txt +++ /dev/null @@ -1,111 +0,0 @@ -.. -*- mode: rst -*- - -.. _unsorted-developing_for_bcfg2: - -==================== -Developing For Bcfg2 -==================== - -While the Bcfg2 server provides a good interface for representing -general system configurations, its plugin interface offers the -ability to implement configuration interfaces and representation -tailored to problems encountered by a particular site. This -chapter describes what plugins are good for, what they can do, and -how to implement them. - -Bcfg2 Plugins -============= - -Bcfg2 plugins are loadable python modules that the Bcfg2 server -loads at initialization time. These plugins can contribute to -the functions already offered by the Bcfg2 server or can extend -its functionality. In general, plugins will provide some portion -of the configuration for clients, with a data representation -that is tuned for a set of common tasks. Much of the core -functionality of Bcfg2 is implemented by several plugins, -however, they are not special in any way; new plugins could -easily supplant one or all of them. - -The following table describes the various functions of bcfg2 plugins. - -|| '' '''Name''' '' || '' '''Description''' '' || -|| Probes || Plugins can issue commands to collect client-side state (like hardware inventory) to include in client configurations || -|| !ConfigurationEntry List || Plugins can construct a list of per-client configuration entry lists to include in client configurations. || -|| !ConfigurationEntry contents || Literal values for configuration entries. || -|| XML-RPC functions || Plugins can export function calls that expose internal functions. || - -Writing Bcfg2 Plugins -===================== - -Bcfg2 plugins are python classes that subclass from -Bcfg2.Server.Plugin.Plugin. Several plugin-specific values must -be set in the new plugin. These values dictate how the new -plugin will behave with respect to the above four functions. -The following table describes all important member fields. - -|| '' '''Name''' '' || '' '''Description''' '' || '' '''Format''' '' || -|| __name__ || The name of the plugin || string || -|| __version__ || The plugin version (generally tied to revctl keyword expansion). || string || -|| __author__ || The plugin author. || string || -|| __rmi__ || Set of functions to be exposed as XML-RPC functions || List of function names (strings) || -|| Entries || Multidimentional dictionary of keys that point to the function [[BR]] used to bind literal contents for a given configuration entity. || Dictionary of !ConfigurationEntityType, Name keys and function reference values || -|| !BuildStructures || Function that returns a list of the structures for a given client || Member function || -|| !GetProbes || Function that returns a list of probes that a given client should execute || Member function || -|| !ReceiveData || Function that accepts the probe results for a given client. || Member function || - -Example Plugin -============== - -.. code-block:: python - - import Bcfg2.Server.Plugin - class MyPlugin(Bcfg2.Server.Plugin.Plugin): - '''An example plugin''' - # All plugins need to subclass Bcfg2.Server.Plugin.Plugin - __name__ = 'MyPlugin' - __version__ = '1' - __author__ = 'me@me.com' - __rmi__ = ['myfunction'] - # myfunction is not available remotely as MyPlugin.myfunction - - def __init__(self, core, datastore): - Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) - self.Entries = {'Path':{'/etc/foo.conf': self.buildFoo}} - - def myfunction(self): - '''function for xmlrpc rmi call''' - #do something - return True - - def buildFoo(self, entry, metadata): - '''Bind per-client information into entry based on metadata''' - entry.attrib.update({'type':'file', 'owner':'root', 'group':'root', 'perms':'644'}) - entry.text = '''contents of foo.conf''' - -Example Connector -================= - -.. code-block:: python - - import Bcfg2.Server.Plugin - - class Foo(Bcfg2.Server.Plugin.Plugin, - Bcfg2.Server.Plugin.Connector): - '''The Foo plugin is here to illustrate a barebones connector''' - name = 'Foo' - version = '$Revision: $' - experimental = True - - def __init__(self, core, datastore): - Bcfg2.Server.Plugin.Plugin.__init__(self, core, datastore) - Bcfg2.Server.Plugin.Connector.__init__(self) - self.store = XMLFileBacked(self.data, core.fam) - - def get_additional_data(self, metadata): - mydata = {} - for data in self.store.entries['foo.xml'].data.get("foo", []): - mydata[data] = "bar" - return dict([('mydata', mydata)]) - - def get_additional_groups(self, meta): - return self.cgroups.get(meta.hostname, list()) diff --git a/doc/unsorted/development_tips.txt b/doc/unsorted/development_tips.txt deleted file mode 100644 index 29924e645..000000000 --- a/doc/unsorted/development_tips.txt +++ /dev/null @@ -1,20 +0,0 @@ -.. -*- mode: rst -*- - -.. _unsorted-development_tips: - -========================== -Tips for Bcfg2 Development -========================== - -#. Focus on either the client or server code. This focuses the development process down the the precise pieces of code that matter for the task at hand. - - * If you are developing a client driver, then write up a small configuration specification that includes the needed characteristics. - * If you are working on the server, run bcfg2-info and use to assess the code - -#. Use the python interpreter. One of python's most appealing features is interactive use of the interpreter. - - * If you are developing for the client-side, run "python -i /usr/sbin/bcfg2" with the appropriate bcfg2 options. This will cause the python interpreter to continue running, leaving all variables intact. This can be used to examine data state in a convenient fashion. - * If you are developing for the server side, use bcfg2-info and the "debug" option. This will leave you at a python interpreter prompt, with the server core loaded in the variable "bcore". - -#. Use pylint obsessively. It raises a lot of style-related warnings which can be ignored, but most all of the errors are legitimate. -#. If you are doing anything with Regular Expressions, [http://kodos.sourceforge.net/ Kodos - The Python Regular Expression Debugger] and [http://re-try.appspot.com/ re-try] are your friends. diff --git a/doc/unsorted/development_writing_plugins.txt b/doc/unsorted/development_writing_plugins.txt deleted file mode 100644 index cc0bd7c00..000000000 --- a/doc/unsorted/development_writing_plugins.txt +++ /dev/null @@ -1,77 +0,0 @@ -.. -*- mode: rst -*- - -.. _unsorted-development_writing_plugins: - -=============== -Writing Plugins -=============== - -Server Plugin Types -=================== - -Generator ---------- - -Generator plugins contribute to literal client configurations - -Structure ---------- - -Structure Plugins contribute to abstract client configurations - -Metadata --------- - -Signal metadata capabilities - -Connector ---------- - -Connector Plugins augment client metadata instances - -Probing -------- - -Signal probe capability - -Statistics ----------- - -Signal statistics handling capability - -Decision --------- - -Signal decision handling capability - -Version -------- - -Interact with various version control systems - -Writing Server Plugins -====================== - -Metadata --------- - -If you would like to define your own Metadata plugin (to extend/change functionality of the existing Metadata plugin), here are the steps to do so. We will call our new plugin `MyMetadata`. - -#. Add MyMetadata.py - - .. code-block:: python - - __revision__ = '$Revision$' - - import Bcfg2.Server.Plugins.Metadata - - class MyMetadata(Bcfg2.Server.Plugins.Metadata.Metadata): - '''This class contains data for Bcfg2 server metadata''' - __version__ = '$Id$' - __author__ = 'bcfg-dev@mcs.anl.gov' - - def __init__(self, core, datastore, watch_clients=True): - Bcfg2.Server.Plugins.Metadata.Metadata.__init__(self, core, datastore, watch_clients) - -#. Add MyMetadata to `src/lib/Server/Plugins/__init__.py` -#. Replace Metadata with MyMetadata in the plugins line of bcfg2.conf diff --git a/doc/unsorted/dynamic_groups.txt b/doc/unsorted/dynamic_groups.txt deleted file mode 100644 index 11535dc8b..000000000 --- a/doc/unsorted/dynamic_groups.txt +++ /dev/null @@ -1,27 +0,0 @@ -.. -*- mode: rst -*- - -.. _unsorted-dynamic_groups: - -============== -Dynamic Groups -============== - -Bcfg2 supports the use of dynamic groups. These groups are not included -in a client's profile group, but instead are derived from the results -of probes executed on the client. These dynamic groups need not already -exist in ``Metadata/groups.xml``. If a dynamic group is defined in -``Metadata/groups.xml``, clients that include this group will also get -all included groups and bundles. - -Setting up dynamic groups -========================= - -In order to define a dynamic group, setup a probe that outputs the text -based on system properties:: - - group:groupname - -This output is processed by the Bcfg2 server, and results in dynamic -group membership in groupname for the client. See the :ref:`Probes -<server-plugins-probes-index>` page for a more thorough description -of probes. diff --git a/doc/unsorted/install.txt b/doc/unsorted/install.txt deleted file mode 100644 index 6589261ab..000000000 --- a/doc/unsorted/install.txt +++ /dev/null @@ -1,47 +0,0 @@ -.. -*- mode: rst -*- - -.. _unsorted-install: - -============ -Installation -============ - -Prerequisites -============= - -First, install the prerequisite libraries. See the [wiki:Prereqs] page for more information. - -Bcfg2 -===== - -Before installing, you will need to choose a machine to be the Bcfg2 server. We recommend a Linux-based machine for this purpose, but the server will work on any supported operating system. Note that you may eventually want to run a web server on this machine for reporting and serving up package repositories. - -* '''From packages:''' The easiest way to install Bcfg2 is from packages for your operating system. You can grab packages (and source packages) for various OSes from the [wiki:Download] page. Install them as you would any other packages. The server package only needs to be installed on your designated Bcfg2 server machine; the clients package needs to be installed on any machine you plan to manage by Bcfg2. -* '''From source:''' The Bcfg2 source tarball can also be grabbed from the [wiki:Download] page. After untarring the file, you can build Bcfg2 with {{{python setup.py install --prefix=/install/prefix}}} This will install both the client and server on that machine. - -Configuration -============= - -Once Bcfg2 is installed, head over to the [wiki:QuickStart] to get it configured and up-and-running. - - -OS X -==== - -Using native OS X python ------------------------- - -First, make sure you have Xcode installed as you need `packagemaker` which comes bundled in the Developer tools. - -Clone the git source:: - - git clone git://git.mcs.anl.gov/bcfg2.git - -Change to the osx directory and type make. Your new package should be located at bcfg2-'''$VERSION'''.pkg (where '''$VERSION''' is that which is specified in setup.py). - -Using macports --------------- - -Once macports is installed:: - - port install bcfg2 diff --git a/doc/unsorted/learningpython.txt b/doc/unsorted/learningpython.txt deleted file mode 100644 index 3810d3899..000000000 --- a/doc/unsorted/learningpython.txt +++ /dev/null @@ -1,23 +0,0 @@ -.. -*- mode: rst -*- - -.. _unsorted-learningpython: - -=============== -Learning Python -=============== - -Learning -======== - -* MIT OpenCourseWare [http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-189January--IAP--2008/CourseHome/index.htm A Gentle Introduction to Programming Using Python] which primarily uses [http://www.greenteapress.com/thinkpython/thinkpython.html Think Python: How to Think Like a Computer Scientist]. -* [http://www.diveintopython.org/ Dive into Python], available in (free) online and dead trees formats. Good introduction, but a bit dated at this point (2004). -* djbclark recommends [http://hetland.org/writing/beginning-python/ Beginning Python: From Novice to Professional] -* djbclark recommends [http://www.oreilly.com/catalog/pythoncook2/ Python Cookbook, 2nd Edition] - -References -========== - -* desai recommends "Python: Essential Reference, 3rd Edition", which is available [http://safari.samspublishing.com/0672328623 online], in [http://www.samspublishing.com/bookstore/product.asp?isbn=0768664985&rl=1 pdf], and as a [http://www.samspublishing.com/bookstore/product.asp?isbn=0672328623&rl=1 book]. -* djbclark recommends [http://www.oreilly.com/catalog/pythonpr3/ Python Pocket Reference, 3rd Edition] -* [http://www.oreilly.com/catalog/pythonian2/ Python in a Nutshell, 2nd Edition] -* lueningh recommends the official [http://docs.python.org/lib/lib.html library reference] for, say, when you just need to know what all of a string object's methods are. diff --git a/doc/unsorted/writing_specification.txt b/doc/unsorted/writing_specification.txt index 5a75165bf..eced54841 100644 --- a/doc/unsorted/writing_specification.txt +++ b/doc/unsorted/writing_specification.txt @@ -166,7 +166,8 @@ The following is an annotated copy of a bundle: .. code-block:: xml - <Bundle name='ssh' version='2.0'> + <Bundle revision='$Revision: 2668 $' name='ssh' version='2.0' + origin='https://svn.mcs.anl.gov/repos/bcfg/trunk/repository/Bundler/ssh.xml'> <Path name='/etc/ssh/ssh_host_dsa_key'/> <Path name='/etc/ssh/ssh_host_rsa_key'/> <Path name='/etc/ssh/ssh_host_dsa_key.pub'/> |
