Hello World
This is a sample paragraph followed by a bulleted list:
Figure 13-1 shows what the HTML looks like in a browser.
.. figure :: 9349f1301.png
:target: _images/9349f1301.png
:scale: 70
:alt: Figure 13-1. The generated HTML displayed in a browser
Figure 13-1. The generated HTML displayed in a browser
.. index ::
single: overview of; reStructuredText language
Although this might look plain, you are free to apply your own style sheet to the output produced, and because the HTML is well constructed, you can do a lot with the generated output.
.. tip ::
You might be interested to know that this book is written entirely in reStructuredText and that many of the articles on the Pylons wiki are written in reStructuredText as well.
You can also generate HTML output from reStructuredText source programmatically. The following demonstrates this:
::
from docutils import core
from docutils.writers.html4css1 import Writer
def rstify(string):
w = Writer()
result = core.publish_parts(string, writer=w)['html_body']
# Strip the first and last lines
result = '\n'.join(result.split('\n')[1:-2])
return result
.. index ::
single: docutils.core.publish_parts() function
To generate HTML from a string containing reStructuredText, you use the ``docutils.core.publish_parts()`` function. This returns a dictionary containing different parts of the HTML document. In most cases where you are generating HTML programmatically, it is likely you’ll want only the ``html_body`` part because you will make up the rest of the HTML yourself. The previous ``rstify()`` function does just that. The ``docutils`` package is actually very modular, and with a little effort you can create some very powerful customizations.
Rather than trying to explain the reStructuredText syntax in the book, I’ll refer you to some excellent resources online that will teach you everything you need to know:
.. index ::
single: reStructuredText documentation; web sites
*reStructuredText primer*
This is an excellent introduction to reStructuredText. It forms part of the Sphinx documentation; you can find it at http://sphinx.pocoo.org/rest.html.
.. index ::
single: overview of; reStructuredText language
*reStructuredText home page*
You can find the authoritative reStructruredText documentation at http://docutils.sourceforge.net/rst.html. Of particular value is the Quick reStructuredText guide at http://docutils.sourceforge.net/docs/user/rst/quickref.html.
Introducing Sphinx
==================
.. index ::
single: description of; Sphinx tool
Although using reStructedText as a stand-alone documentation tool is very useful, reStructuredText can also be used within docstrings to document individual functions, classes, and methods.
This functionality brings with it certain possibilities. Wouldn’t it be handy, for example, if there were a tool for extracting module documentation but that also understood reStructuredText for formatting that documentation? Well, there is just such a tool; I’ve mentioned it a number of times already, and it is called *Sphinx*.
Sphinx is a tool that translates a set of reStructuredText source files into various output formats including HTML and LaTeX (which can then be used to produce a PDF), automatically producing cross-references and indexes. The focus of Sphinx is handwritten documentation, but as you’ll see shortly, Sphinx can also be used to automatically generate documentation from source code.
You can install Sphinx with Easy Install like this (it requires Python 2.4 or newer to run):
::
$ easy_install "Sphinx==0.4.2"
.. index ::
single: description of; Sphinx tool
Sphinx uses Jinja for its templating support, so Easy Install will install Jinja too if you didn’t install it in Chapter 5. Jinja currently expects a compiler to be present if you install it from source, so Mac OS X users will need to have Xcode installed (it includes GCC) or use a binary version.
Using Sphinx
------------
.. index ::
single: documenting SimpleSite project using; Sphinx tool
Let’s use Sphinx to document the SimpleSite project. You’ll notice that the source directory already contains a ``docs`` directory. This is a great place to set up Sphinx. It already contains an ``index.txt`` file, but you’ll replace this with one generated by Sphinx, so delete it because it currently contains instructions about how to generate documentation with a tool called Pudge, which Sphinx supercedes.
Create a new Sphinx build like this, and answer the questions:
::
$ cd SimpleSite/docs
$ rm index.txt
$ sphinx-quickstart
Enter ``SimpleSite`` as the project name, and choose 0.1.0 as the version number. Use ``.txt`` as the file extension. Accept the defaults for everything else except ``autodoc``. You *do* want to use Sphinx’s ``autodoc`` extensions, which will pick up documentation from docstrings in your Pylons project’s source code, so enter ``y`` to that question.
.. index ::
single: docs directory
The ``docs`` directory will contain these files once the utility has finished:
``Makefile``
Generated on non-Windows platforms if you answered ``y`` to the makefile question at the end of the wizard. I won’t describe it here because all the functionality it contains can be reached via the command line directly.
``conf.py``
Contains all the Sphinx configuration for your project. There are quite a few options, but they are well commented in the ``conf.py`` file and documented on the Sphinx web site at http://sphinx.pocoo.org.
.. index ::
single: documenting SimpleSite project using; Sphinx tool
``index.txt``
This file represents your entire documentation for the project.
.. index ::
single: index.txt file
Here’s ``index.txt``:
::
.. SimpleSite Documentation documentation master file, created by sphinx-quickstart on Thu Oct 2 14:16:47 2008.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to SimpleSite Documentation's documentation!
====================================================
Contents:
.. toctree::
:maxdepth: 2
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
.. index ::
single: sphinx-build command
Now that Sphinx is configured, let’s build the documentation. The ``sphinx-build`` command takes the source directory and output directory commands. In this case, the source directory is the current directory (``docs``), and the output directory is a new subdirectory called ``.build``. The ``-b html`` option tells Sphinx you want HTML output (the default). Run the command like this:
::
$ sphinx-build -b html . ./.build
Sphinx v0.4.2, building html
trying to load pickled env... not found
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading... index
pickling the env... done
checking consistency...
writing output... index
finishing...
writing additional files... genindex modindex search
copying static files...
dumping search index...
build succeeded.
.. index ::
single: documenting SimpleSite project using; Sphinx tool
After the build has succeeded, the subdirectories ``.build``, ``.static``, and ``.templates`` will be present in the ``docs`` directory.
.. note ::
Unix and Mac OS X users should note that folders beginning with ``.`` are often hidden in the file browser software, so you may not see these directories unless you use the command line.
.. index ::
single: .build directory
The ``.build`` directory contains the HTML output. Open the ``index.html`` file in a web browser; Figure 13-2 shows what you’ll see.
.. figure :: 9349f1302.png
:target: _images/9349f1302.png
:scale: 70
:alt: Figure 13-2. The initial output generated by Sphinx
Figure 13-2. The initial output generated by Sphinx
.. index ::
single: documenting SimpleSite project using; Sphinx tool
You can easily customize the look and feel of the generated documents using templates. See the Sphinx documentation for the details.
Now let’s add two files to the ``docs`` directory. These will form the basis for our user and developer documentation, respectively. Add ``user_guide.txt`` with this content:
::
User Guide
==========
This will contain instructions for end users of the application.
Add ``developer_guide.txt`` with this content:
::
Developer Guide
===============
This software is documented in detail in the SimpleSite tutorial
chapters of the book *The Definitive Guide to Pylons* available
under an open source license at http://pylonsbook.com. You should
read those chapters to discover how SimpleSite is developed.
.. index ::
single: toctree directive (Sphinx)
The reStructuredText format doesn’t have any option for linking documents, so Sphinx provides its own called ``toctree``. The ``toctree`` directive should have a list of all the documents you want included in the documentation. Update the ``toctree`` directive in the ``index.txt`` file to look like this (notice that the file names shouldn’t contain the ``.txt`` extensions and that there is a blank line before the document list):
::
.. toctree::
:maxdepth: 2
user_guide
developer_guide
When choosing names for your documents, you should avoid using ``genindex``, ``modindex``, and ``search`` and instead choose names that don’t start with an ``_`` character.
Now rebuild the documentation by running the ``sphinx-build`` command again:
::
$ sphinx-build -b html . ./.build
If you add a file to the directory structure but forget to include it in the ``toctree``, Sphinx will show you a warning like this when you try to build the documentation:
::
checking consistency...
WARNING: /Users/james/SimpleSite/docs/developer_guide.txt:: document isn't included in any toctree
.. index ::
single: documenting SimpleSite project using; Sphinx tool
Once the build completes successfully, you will see that the new ``index.html`` file has two links in the ``Contents`` section. Sphinx automatically uses the titles from the documents themselves rather than using their file names. If you click the links, you will see the guides have been correctly generated. What is more, you’ll also find the search box to the left side also works. Sphinx compiles a search index in ``.build/searchindex.json`` as part of the build process, and it uses this to search your documentation.
Documenting Python Source Code
------------------------------
.. index ::
single: documenting Python source code using; Sphinx tool
Although being able to write paragraphs of text is very useful a lot of the time, you will want to be able to document Python source code directly. To facilitate this, Sphinx adds a number of markup constructs to the standard ones supported by reStructuredText.
Let’s add a new file called ``api.txt`` to contain some API documentation. You’ll also need to add it to the ``toctree`` directive in ``index.txt``.
Let’s start by adding a title and a brief summary to the file:
::
API Documentation
=================
This page contains some basic documentation for the SimpleSite project. To
understand the project completely please refer to the documentation on the
Pylons Book website at http://pylonsbook.com or read the source code directly.
Now let’s add some information about the ``simplesite`` and ``simplesite.controllers`` modules. To do this, you might write the following:
::
The :mod:`simplesite` Module
----------------------------
.. module:: simplesite
Contains all the controllers, model and templates as sub-modules.
The :mod:`controllers` Module
-----------------------------
.. module:: simplesite.controllers
Contains all the controllers. The most important of which is
:class:`PageController`.
Let’s also document the page controller since so far it contains the majority of the application’s logic. You might add this:
::
.. class:: PageController
The :class:`PageController` is responsible for displaying pages as well as
allowing users to add, edit, delete and list pages.
.. method:: PageController.view(self[, id=None])
When a user visits a URL such as ``/view/page/1`` the :class:`PageController`
class's :meth:`view` action is called to render the page.
.. index ::
single: documenting Python source code using; Sphinx tool
If you rebuild the documentation this time, the API documentation page will look like Figure 13-3.
.. figure :: 9349f1303.png
:target: _images/9349f1303.png
:scale: 70
:alt: Figure 13-3. The API Documentation page
Figure 13-3. The API Documentation page
.. index ::
single: Index page; SimpleSite application
More interestingly, Sphinx has understood that the page documents the ``simplesite`` and ``simplesite.controllers`` modules, and it has recognized that ``PageController`` is in the ``simplesite.controllers`` module. With this knowledge, it has been able to produce both a Global Module Index and a general Index page. Figure 13-4 shows the Index page.
.. figure :: 9349f1304.png
:target: _images/9349f1304.png
:scale: 70
:alt: Figure 13-4. The Index page
Figure 13-4. The Index page
.. tip ::
.. index ::
single: documenting Python source code using; Sphinx tool
All the current Python documentation is written in reStructuredText and generated in this manner by Sphinx. You can see it online at http://docs.python.org.
Automatically Generating Documentation
--------------------------------------
.. index ::
single: automatically generating documentation using; Sphinx tool
single: autodoc extensions (Sphinx)
If you have a lot of modules, classes, methods, and functions to document, it can become very tedious to document them all manually, particularly if you also duplicate much of the documentation in the docstrings themselves. Sphinx provides the ``sphinx.ext.autodoc`` extensions for this purpose. The ``autodoc`` extension automatically extracts docstrings from the objects you tell it about and includes them as part of your Sphinx documentation.
Let’s use it to extract the docstring from the SimpleSite ``lib/helpers.py`` module. Add the following to the end of ``api.txt``:
::
The :mod:`helpers` Module
-------------------------
.. automodule:: simplesite.lib.helpers
For ``autodoc`` to work, it must be able to import the ``simplesite.lib.helpers`` module. This means the SimpleSite application must be installed in the same virtual Python environment you are running Sphinx with. Run this command to install it in develop mode so that any changes you make are immediately available to Sphinx:
::
$ python setup.py develop
Now you can build the documentation again:
::
$ sphinx-build -b html . ./.build
.. index ::
single: automatically generating documentation using; Sphinx tool
You’ll see that ``autodoc`` has found the correct docstring and used it to generate the necessary documentation. In addition to the ``.. automodule::`` construct, there are others to handle classes, functions, and methods. There is also a range of options for each of the constructs. One of the more useful options for ``..automodule::`` is ``:members:`` which can be used to automatically document each of the objects which are directly contained in a module but not imported classes or functions.
::
.. automodule::
:members:
You can also specify just the members you want to document.
::
.. automodule::
:members: foo, Bar
Syntax Highlighting
-------------------
.. index ::
single: syntax highlighting; Sphinx tool
single: Pygments package
Sometimes it is useful to show code examples. In reStructuredText, you would normally just use an empty ``::`` directive to tell reStructuredText to display the following text verbatim, but Sphinx allows you to be slightly more sophisticated. If you install a package called Pygments, Sphinx will be able to automatically highlight source code.
It is usually installed along with Sphinx, but you can also install Pygments directly like this:
::
$ easy_install "Pygments==0.11.1"
You then mark blocks of code in your source by using a ``.. code-block::`` directive. The Pygments short name for the type of code the block represents should be added immediately after the ``::`` characters and the code itself in an indented block after that. Pygments can highlight many different types of code including HTML, Mako templates, a Python interactive console, and more. A full list of the different lexers for source code highlighting, as well as their corresponding short names, is available at http://pygments.org/docs/lexers/.
Let’s add some documentation about the FormEnocde schema being used in the page controller as an example. Add this to ``api.txt`` just before the heading for the ``helpers`` module. The Pygments short name for Python code blocks is ``python``, so this is what you add after the ``.. code-block::`` directive:
::
The page controller makes use of a FormEncode schema to validate the page
data it receives. Here is the schema it uses:
.. code-block:: python
class NewPageForm(formencode.Schema):
allow_extra_fields = True
filter_extra_fields = True
content = formencode.validators.String(
not_empty=True,
messages={
'empty': 'Please enter some content for the page.'
}
)
heading = formencode.validators.String()
title = formencode.validators.String(not_empty=True)
As you can see the schema includes validators for the title, heading
and content.
.. index ::
single: syntax highlighting; Sphinx tool
If you save this and rebuild the documentation, you will see the example syntax nicely highlighted (see Figure 13-5).
.. figure :: 9349f1305.png
:target: _images/9349f1305.png
:scale: 70
:alt: Figure 13-5. Highlighted source code
Figure 13-5. Highlighted source code
Sometimes the code you are demonstrating will contain a mixture of two different types of source code. For example, it might be Mako syntax that also contains HTML or Mako that also contains CSS. In these cases, Pygments provides lexers that you can use via their short names, which are ``html+mako`` and ``html+css``, respectively.
.. index ::
single: Sphinx documentation; web sites
single: syntax highlighting; Sphinx tool
As you can see, Sphinx is a powerful and useful tool. It is well worth reading the documentation at http://sphinx.pocco.org to find out exactly what it can do. Two areas that are beyond the scope of this chapter but are nonetheless worth investigating for yourself are Sphinx’s extensive cross-referencing and indexing tools and its ability to generate LaTeX output that can be used to produce high-quality book-style PDF documents.
Summary
=======
In this chapter, you saw every aspect of documenting a Pylons project from using docstrings to learning reStructuredText and building documentation with Sphinx. You’ve even learned how tests can be integrated into documentation.
Documenting a project properly can really help other users of your code. This chapter has given you the knowledge and tools you need to create really good documentation for your Pylons project.
.. index ::
single: Chap 13-Documentation
In the next chapter, you’ll return to the SimpleSite tutorial to add a range of new features and learn more about Pylons development as you do.