Gather All Your Development Documentation In One Place

Want to have always the latest documentation and database schema available? Then build it on your CI!

This post is for people who reject writing development documentation using MsWord, and use text-based tools (e.g. Asciidoc, reStructuredText etc.).

It is also a continuation of my adventures as a team lead of a new project (read the first part here).

Initial Frustration

As we all know "Every good work of software starts by scratching a developer's personal itch". My itch was, that I lost track of development documentation that we write.

Ok, so we have dozens of projects, many of them with some developers documentation (which contains architecture diagrams, some notes on design decisions, testing, releasing etc.). The documentation is written using reStructuredText, which means it is purely textual, and you need to execute some magic commands (make in our case) to generate HTML, PDF or whatever you want.

BTW, have you ever used. reStructuredText with PlantUML? It works pretty cool!

So one day I got frustrated with browsing SVN in search of documentation of some project, and started wondering how nice that would be, to have the latest version of all documentation we have gathered in one place. Preferably in HTML format, so we can easily access it with browsers.

Are you interested in
unit testing?
If so,see my book
"Practical Unit Testing
with TestNG and Mockito"

Script & CI

The first thing was to set up a common layout for all projects. I decided that documentation should be kept in docs directory (what a creative directory name, don't you agree?) in root of each project. This way it is easy to find it and to automate the process. This required moving few directories around. No problems.

Now to the main points which are a build script and CI server configuration.

The script does two things:

  • fetches sources of documentation from SVN,
  • for each fetched documentation, it executes command to generate HTML output,
  • creates an "index" page with links to all documents so it is easy to browse them

As for our CI (Jenkins), it did not require a lot of effort. Jenkins can run any type of build scripts (shell scripts included), so this was just a matter of configuring the job.

The tricky part was to set failure criterias for this build. I managed to do it using Jenkins Text-finder Plugin. Basically, it allows to specify what patterns (regular expressions) to look for in console output of the build, and fail the occur (or pass if they occur - depending on configuration). So I did some bad things in the script (like messing up SVN paths) to figure out what expressions should I look for. Unfortunately, this plugin does not offer enough flexibility to meet all my needs, but somehow I managed to set it.

The result

The result is, that every night, all documentation sources are fetched from SVN and build. A nice index page is generated, which contains not only links to each documentation, but also information about last modification date (of each project documentation) and a nick of developer who modified it. See below (and click to enlarge):

Database Structure Information

Then the build script was enhanced with another capability. Now it also connects to some database (on testing environment) and fetches its structure. We use SchemaSpy for this. It generates a really nice HTML page with all information on your database schema. Later on we added it to our index page, so it is also easily accessible.

If we find it useful, we will also generate database schema (probably using Liquibase), but as for now, this is not required.

Conclusion

Concluding, this is not something that made our productivity boost, but this is a really useful thing to have. And it does not cost you much to set up a similar solution.

On the other hand, it does not solve the real issue with documentation ("is it up-to-date?"), but at least it makes the documentation being used more, and hence updated more frequently. Which is a step in the right direction.

Pros:

  • it looks pretty cool :),
  • everyone knows where to find documentation,
  • everyone uses the latest documentation,
  • some basic quality checks of documentation generation process,
  • easy to present your documentation and prove (to your PM or client) that you have really done something.

Cons:

  • when build fails Jenkins points to the last person who modified the script, which is not always the real culprit

P.S. And if you like pure text-based things, you should definitely learn about Slidy + AsciiDoc combo. Awesome!

SchemaSpy

Tomek, do you know if SchemaSpy analyzes database artifacts like triggers and stored procedures and their relationship with tables? I can extract this metadata from the database using custom SQL queries, but it would be nice, if SchemaSpy could include/process this information somehow.

BTW. Please fix the link to SchemaSpy. Currently it points to liquidbase.

AFAIK SchemaSpy it does not support triggers etc

Hello Anonymous,

it seems that SchemaSpy does not support triggers/stored procedures - http://sourceforge.net/tracker/index.php?func=detail&aid=3414009&group_i...

Link fixed, thanks for letting me know.

--
Tomek

You're welcome. Thanks for

You're welcome. Thanks for pointing this tool out. I glanced at the webpage some time ago, but was not convinced it works well. I tried it now on out DB and it worked. The deletion order feature is noteworthy.

--
B

Click to enlarge

after clicking i receive 404 error :-(

sorry, fixed!

and thanks for letting me know

If you are using the Jira, why not use wiki + jira as document

Wiki to collect and organize all documents so that you do not need the jenkins at all for this purpose?

Please comment using