8. Add-on packages for AutoGen

This chapter includes several programs that either work closely with AutoGen (extracting definitions or providing special formatting functions), or leverage off of AutoGen technology. There is also a formatting library that helps make AutoGen possible.

AutoOpts ought to appear in this list as well, but since it is the primary reason why many people would even look into AutoGen at all, I decided to leave it in the list of chapters.

8.1 Automated Finite State Machine
8.2 Combined RPC Marshalling
8.3 Automated Event Management
8.4 Invoking columns
8.5 Invoking getdefs
8.6 Invoking xml2ag
8.7 Replacement for Stdio Formatting Library		The extensible format printing library.

8.1 Automated Finite State Machine

The templates to generate a finite state machine in C or C++ is included with AutoGen. The documentation is not. The documentation is in HTML format for viewing, or you can download FSM.

8.2 Combined RPC Marshalling

The templates and NFSv4 definitions are not included with AutoGen in any way. The folks that designed NFSv4 noticed that much time and bandwidth was wasted sending queries and responses when many of them could be bundled. The protocol bundles the data, but there is no support for it in rpcgen. That means you have to write your own code to do that. Until now. Download this and you will have a large, complex example of how to use AutoXDR for generating the marshalling and unmarshalling of combined RPC calls. There is a brief example on the web, but you should download AutoXDR.

8.3 Automated Event Management

Large software development projects invariably have a need to manage the distribution and display of state information and state changes. In other words, they need to manage their software events. Generally, each such project invents its own way of accomplishing this and then struggles to get all of its components to play the same way. It is a difficult process and not always completely successful. This project helps with that.

AutoEvents completely separates the tasks of supplying the data needed for a particular event from the methods used to manage the distribution and display of that event. Consequently, the programmer writing the code no longer has to worry about that part of the problem. Likewise the persons responsible for designing the event management and distribution no longer have to worry about getting programmers to write conforming code.

This is a work in progress. See my web page on the subject, if you are interested. I have some useful things put together, but it is not ready to call a product.

8.4 Invoking columns

This program was designed for the purpose of generating compact, columnized tables. It will read a list of text items from standard in or a specified input file and produce a columnized listing of all the non-blank lines. Leading white space on each line is preserved, but trailing white space is stripped. Methods of applying per-entry and per-line embellishments are provided. See the formatting and separation arguments below.

This program is used by AutoGen to help clean up and organize its output.

See `autogen/agen5/fsm.tpl' and the generated output `pseudo-fsm.h'.

This function was not implemented as an expression function because either it would have to be many expression functions, or a provision would have to be added to provide options to expression functions. Maybe not a bad idea, but it is not being implemented at the moment.

A side benefit is that you can use it outside of AutoGen to columnize input, a la the ls command.

This section was generated by AutoGen, the aginfo template and the option descriptions for the columns program. It documents the columns usage text and option meanings.

This software is released under the GNU General Public License.

8.4.1 columns usage help (-?)
8.4.13 by-columns option
8.4.4 col-width option (-w)
8.4.3 columns option (-c)
8.4.7 first-indent option
8.4.10 format option (-f)
8.4.6 indent option (-I)
8.4.14 input option (-i)
8.4.12 line-separation option
8.4.11 separation option (-S)
8.4.9 sort option (-s)
8.4.5 spread option
8.4.8 tab-width option
8.4.2 width option (-W)

8.4.1 columns usage help (-?)

This is the automatically generated usage text for columns:

@exampleindent 0

columns - Columnize Input Text - Ver. 1.1 USAGE: columns [ -<flag> [<val>] | --<name>[{=| }<val>] ]... Flg Arg Option-Name Description -W Num width Maximum Line Width -c Num columns Desired number of columns -w Num col-width Set width of each column Num spread maximum spread added to column width -I Str indent Line prefix or indentation Str first-indent First line prefix - requires these options: indent Num tab-width tab width -s opt sort Sort input text -f Str format Formatting string for each input -S Str separation Separation string - follows all but last Str line-separation string at end of all lines but last no by-columns Print entries in column order -i Str input Input file (if not stdin) -v opt version Output version information and exit -? no help Display usage information and exit -! no more-help Extended usage information passed thru pager Options are specified by doubled hyphens and their name or by a single hyphen and the flag character. This program was designed for the purpose of generating compact, columnized tables. It will read a list of text items from standard in or a specified input file and produce a columnized listing of all the non-blank lines. Leading white space on each line is preserved, but trailing white space is stripped. Methods of applying per-entry and per-line embellishments are provided. See the formatting and separation arguments below. This program is used by AutoGen to help clean up and organize its output. please send bug reports to: autogen-bugs@lists.sf.net

@exampleindent 4

8.4.2 width option (-W)

This is the "maximum line width" option. This option specifies the full width of the output line, including any start-of-line indentation. The output will fill each line as completely as possible, unless the column width has been explicitly specified. If the maximum width is less than the length of the widest input, you will get a single column of output.

8.4.3 columns option (-c)

This is the "desired number of columns" option. Use this option to specify exactly how many columns to produce. If that many columns will not fit within line_width, then the count will be reduced to the number that fit.

8.4.4 col-width option (-w)

This is the "set width of each column" option. Use this option to specify exactly how many characters are to be allocated for each column. If it is narrower than the widest entry, it will be over-ridden with the required width.

8.4.5 spread option

This is the "maximum spread added to column width" option. Use this option to specify exactly how many characters may be added to each column. It allows you to prevent columns from becoming too far apart.

8.4.6 indent option (-I)

This is the "line prefix or indentation" option. If a number, then this many spaces will be inserted at the start of every line. Otherwise, it is a line prefix that will be inserted at the start of every line.

8.4.7 first-indent option

This is the "first line prefix" option.

This option has some usage constraints. It:

• must appear in combination with the following options: indent.

If a number, then this many spaces will be inserted at the start of the first line. Otherwise, it is a line prefix that will be inserted at the start of that line.

8.4.8 tab-width option

This is the "tab width" option. If an indentation string contains tabs, then this value is used to compute the ending column of the prefix string.

8.4.9 sort option (-s)

This is the "sort input text" option. Causes the input text to be sorted. If an argument is supplied, it is presumed to be a pattern and the sort is based upon the matched text. If the pattern starts with or consists of an asterisk (*), then the sort is case insensitive.

8.4.10 format option (-f)

This is the "formatting string for each input" option. If you need to reformat each input text, the argument to this option is interpreted as an sprintf(3) format that is used to produce each output entry.

8.4.11 separation option (-S)

This is the "separation string - follows all but last" option. Use this option if, for example, you wish a comma to appear after each entry except the last.

8.4.12 line-separation option

This is the "string at end of all lines but last" option. Use this option if, for example, you wish a backslash to appear at the end of every line, except the last.

8.4.13 by-columns option

This is the "print entries in column order" option. Normally, the entries are printed out in order by rows and then columns. This option will cause the entries to be ordered within columns. The final column, instead of the final row, may be shorter than the others.

8.4.14 input option (-i)

This is the "input file (if not stdin)" option. This program normally runs as a filter, reading from standard input, columnizing and writing to standard out. This option redirects input to a file.

8.5 Invoking getdefs

#line nnn "source-file-name" entry_type = { name = entry_name; ... };

The ellipsis "..." is filled in by text found between the two delimiters, using the following rules:

1. Each entry is located by the pattern "\n[^*\n]*\\*[ \t]*([a-z][a-z0-9_]*):". Fundamentally, it finds a line that, after the first asterisk on the line, contains whitespace then a name and is immediately followed by a colon. The name becomes the name of the attribute and what follows, up to the next attribute, is its value.
2. If the first character of the value is either a single or double quote, then you are responsible for quoting the text as it gets inserted into the output definitions.
3. All the leading text on a line is stripped from the value. The leading text is everything before the first asterisk, the asterisk and all the whitespace characters that immediately follow it. If you want whitespace at the beginnings of the lines of text, you must do something like this:

   * mumble: * " this is some\n" * " indented text."

4. If the `<entry-name>' is followed by a comma, the word `ifdef' (or 'ifndef') and a name `if_name', then the above entry will appear as:

   #ifdef if_name #line nnn "source-file-name" entry_type = { name = entry_name; ... }; #endif

5. If you use of the subblock option, you can specify a nested value, See section 8.5.6 subblock option. That is, this text:

   * arg: int, this, what-it-is

   with the `--subblock=arg=type,name,doc' option would yield:

   arg = { type = int; name = this; doc = what-it-is; };

This section was generated by AutoGen, the aginfo template and the option descriptions for the getdefs program. It documents the getdefs usage text and option meanings.

This software is released under the GNU General Public License.

8.5.1 getdefs usage help
8.5.17 agarg option
8.5.9 assign option
8.5.15 autogen option
8.5.18 base-name option
8.5.10 common-assign option
8.5.11 copy option
8.5.2 defs-to-get option
8.5.8 filelist option
8.5.4 first-index option
8.5.5 input option
8.5.13 linenum option
8.5.7 listattr option
8.5.3 ordering option
8.5.14 output option
8.5.12 srcfile option
8.5.6 subblock option
8.5.16 template option

8.5.1 getdefs usage help

This is the automatically generated usage text for getdefs:

@exampleindent 0

getdefs - AutoGen Definition Extraction Tool - Ver. 1.4 USAGE: getdefs [ <option-name>[{=| }<val>] ]... Arg Option-Name Description Str defs-to-get Regexp to look for after the "/*=" opt ordering Alphabetize or use named file - disabled as --no-ordering - enabled by default Num first-index The first index to apply to groups Str input Input file to search for defs - may appear multiple times - default option for unnamed options Str subblock subblock definition names - may appear multiple times Str listattr attribute with list of values - may appear multiple times opt filelist Insert source file names into defs Definition insertion options Arg Option-Name Description Str assign Global assignments - may appear multiple times Str common-assign Assignments common to all blocks - may appear multiple times Str copy File(s) to copy into definitions - may appear multiple times opt srcfile Insert source file name into each def opt linenum Insert source line number into each def Definition output disposition options: Arg Option-Name Description Str output Output file to open - an alternate for autogen opt autogen Invoke AutoGen with defs - disabled as --no-autogen - enabled by default Str template Template Name Str agarg AutoGen Argument - prohibits these options: output - may appear multiple times Str base-name Base name for output file(s) - prohibits these options: output Auto-supported Options: Arg Option-Name Description opt version Output version information and exit no help Display usage information and exit no more-help Extended usage information passed thru pager opt save-opts Save the option state to an rc file Str load-opts Load options from an rc file - disabled as --no-load-opts - may appear multiple times All arguments are named options. If no ``input'' argument is provided or is set to simply "-", and if ``stdin'' is not a ``tty'', then the list of input files will be read from ``stdin''. The following option preset mechanisms are supported: - reading file /dev/null This program extracts AutoGen definitions from a list of source files. Definitions are delimited by `/*=<entry-type> <entry-name>\n' and `=*/\n'. From that, this program creates a definition of the following form: #line nnn "source-file-name" entry_type = { name = entry_name; ... }; The ellipsis '...' is filled in by text found between the two delimiters, with everything up through the first sequence of asterisks deleted on every line. There are two special ``entry types'': * The entry_type enclosure and the name entry will be omitted and the ellipsis will become top-level definitions. -- The contents of the comment must be a single getdefs option. The option name must follow the double hyphen and its argument will be everything following the name. This is intended for use with the ``subblock'' and ``listattr'' options. please send bug reports to: autogen-bugs@lists.sf.net

@exampleindent 4

8.5.2 defs-to-get option

This is the "regexp to look for after the "/*="" option. If you want definitions only from a particular category, or even with names matching particular patterns, then specify this regular expression for the text that must follow the /*=.

8.5.3 ordering option

This is the "alphabetize or use named file" option.

This option has some usage constraints. It:

• is enabled by default.

By default, ordering is alphabetical by the entry name. Use, no-ordering if order is unimportant. Use ordering with no argument to order without case sensitivity. Use ordering=<file-name> if chronological order is important. getdefs will maintain the text content of file-name. file-name need not exist.

8.5.4 first-index option

This is the "the first index to apply to groups" option. By default, the first occurrance of a named definition will have an index of zero. Sometimes, that needs to be a reserved value. Provide this option to specify a different starting point.

8.5.5 input option

This is the "input file to search for defs" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

All files that are to be searched for definitions must be named on the command line or read from stdin. If there is only one input option and it is the string, "-", then the input file list is read from stdin. If a command line argument is not an option name and does not contain an assignment operator (=), then it defaults to being an input file name. At least one input file must be specified.

8.5.6 subblock option

This is the "subblock definition names" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

This option is used to create shorthand entries for nested definitions. For example, with:

using subblock thus

--subblock=arg=argname,type,null

and defining an arg thus

arg: this, char *

will then expand to:

arg = { argname = this; type = "char *"; };

8.5.7 listattr option

This is the "attribute with list of values" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

This option is used to create shorthand entries for definitions that generally appear several times. That is, they tend to be a list of values. For example, with:
listattr=foo defined, the text:
foo: this, is, a, multi-list will then expand to:
foo = 'this', 'is', 'a', 'multi-list';
The texts are separated by the commas, with the white space removed. You may use characters other than commas by starting the value string with a punctuation character other than a single or double quote character.

8.5.8 filelist option

This is the "insert source file names into defs" option. Inserts the name of each input file into the output definitions. If no argument is supplied, the format will be:

infile = '%s';

If an argument is supplied, that string will be used for the entry name instead of infile.

8.5.9 assign option

This is the "global assignments" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The argument to each copy of this option will be inserted into the output definitions, with only a semicolon attached.

8.5.10 common-assign option

This is the "assignments common to all blocks" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The argument to each copy of this option will be inserted into each output definition, with only a semicolon attached.

8.5.11 copy option

This is the "file(s) to copy into definitions" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

The content of each file named by these options will be inserted into the output definitions.

8.5.12 srcfile option

This is the "insert source file name into each def" option. Inserts the name of the input file where a definition was found into the output definition. If no argument is supplied, the format will be:

srcfile = '%s';

If an argument is supplied, that string will be used for the entry name instead of srcfile.

8.5.13 linenum option

This is the "insert source line number into each def" option. Inserts the line number in the input file where a definition was found into the output definition. If no argument is supplied, the format will be:

linenum = '%s';

If an argument is supplied, that string will be used for the entry name instead of linenum.

8.5.14 output option

This is the "output file to open" option.

This option has some usage constraints. It:

• is a member of the autogen class of options.

If you are not sending the output to an AutoGen process, you may name an output file instead.

8.5.15 autogen option

This is the "invoke autogen with defs" option.

This option has some usage constraints. It:

• is enabled by default.
• is a member of the autogen class of options.

This is the default output mode. Specifying no-autogen is equivalent to output=-. If you supply an argument to this option, that program will be started as if it were AutoGen and its standard in will be set to the output definitions of this program.

8.5.16 template option

This is the "template name" option. Specifies the template name to be used for generating the final output.

8.5.17 agarg option

This is the "autogen argument" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.
• must not appear in combination with any of the following options: output.

This is a pass-through argument. It allows you to specify any arbitrary argument to be passed to AutoGen.

8.5.18 base-name option

This is the "base name for output file(s)" option.

This option has some usage constraints. It:

• must not appear in combination with any of the following options: output.

When output is going to AutoGen, a base name must either be supplied or derived. If this option is not supplied, then it is taken from the template option. If that is not provided either, then it is set to the base name of the current directory.

8.6 Invoking xml2ag

• The --override-tpl command line option
• A top level XML attribute named, "template"

The base-name for the output will similarly be either:

• The --base-name command line option.
• The base name of the `.xml' file.

The definitions derived from XML generally have an extra layer of definition. Specifically, this XML input:

<mumble attr="foo"> mumble-1 <grumble> grumble, grumble, grumble. </grumble>mumble, mumble </mumble>

Will get converted into this:

mumble = { grumble = { text = 'grumble, grumble, grumble'; }; text = 'mumble-1'; text = 'mumble, mumble'; };

Please notice that some information is lost. AutoGen cannot tell that "grumble" used to lie between the mumble texts. Also please note that you cannot assign:

grumble = 'grumble, grumble, grumble.';

because if another "grumble" has an attribute or multiple texts, it becomes impossible to have the definitions be the same type (compound or text values).

This section was generated by AutoGen, the aginfo template and the option descriptions for the xml2ag program. It documents the xml2ag usage text and option meanings.

This software is released under the GNU General Public License.

8.6.1 xml2ag usage help (-?)
8.6.6 base-name option (-b)
8.6.21 define option (-D)
8.6.7 definitions option
8.6.13 equate option
8.6.5 lib-template option (-l)
8.6.9 load-functions option (-F)
8.6.8 load-scheme option (-S)
8.6.15 loop-limit option
8.6.2 output option (-O)
8.6.4 override-tpl option (-T)
8.6.11 select-suffix option (-o)
8.6.19 show-defs option
8.6.20 show-shell option
8.6.10 skip-suffix option (-s)
8.6.12 source-time option
8.6.3 templ-dirs option (-L)
8.6.16 timeout option (-t)
8.6.18 trace-out option
8.6.17 trace option
8.6.22 undefine option (-U)
8.6.14 writable option

8.6.1 xml2ag usage help (-?)

This is the automatically generated usage text for xml2ag:

@exampleindent 0

xml2ag - XML to AutoGen Definiton Converter - Ver. 5.5.4 USAGE: xml2ag [ -<flag> [<val>] | --<name>[{=| }<val>] ]... [ <def-file> ] Flg Arg Option-Name Description -O Str output Output file in lieu of AutoGen processing -L Str templ-dirs Template search directory list - may appear multiple times -T Str override-tpl Override template file -l Str lib-template Library template file - may appear multiple times -b Str base-name Base name for output file(s) Str definitions Definitions input file -S Str load-scheme Scheme code file to load -F Str load-functions Load scheme callout library -s Str skip-suffix Omit the file with this suffix - may appear multiple times -o opt select-suffix specify this output suffix - may appear multiple times no source-time set mod times to latest source Str equate characters considered equivalent no writable Allow output files to be writable - disabled as --not-writable Num loop-limit Limit on increment loops it must lie in one of the ranges: -1 exactly, or 1 to 16777216 -t Num timeout Time limit for servers it must lie in the range: 0 to 3600 KWd trace tracing level of detail Str trace-out tracing output file or filter no show-defs Show the definition tree no show-shell Show shell commands -D Str define name to add to definition list - may appear multiple times -U Str undefine definition list removal pattern - an alternate for define -v opt version Output version information and exit -? no help Display usage information and exit -! no more-help Extended usage information passed thru pager Options are specified by doubled hyphens and their name or by a single hyphen and the flag character. This program will convert any arbitrary XML file into equivalent AutoGen definitions, and invoke AutoGen. The valid trace option keywords are: nothing templates block-macros expressions everything The template will be derived from either: * the ``--override-tpl'' command line option * a top level XML attribute named, "template" The ``base-name'' for the output will similarly be either: * the ``--base-name'' command line option * the base name of the .xml file please send bug reports to: autogen-bugs@lists.sf.net

@exampleindent 4

8.6.2 output option (-O)

This is the "output file in lieu of autogen processing" option. By default, the output is handed to an AutoGen for processing. However, you may save the definitions to a file instead.

8.6.3 templ-dirs option (-L)

This is the "template search directory list" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.4 override-tpl option (-T)

This is the "override template file" option. Pass-through AutoGen argument

8.6.5 lib-template option (-l)

This is the "library template file" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.6 base-name option (-b)

This is the "base name for output file(s)" option. Pass-through AutoGen argument

8.6.7 definitions option

This is the "definitions input file" option. Pass-through AutoGen argument

8.6.8 load-scheme option (-S)

This is the "scheme code file to load" option. Pass-through AutoGen argument

8.6.9 load-functions option (-F)

This is the "load scheme callout library" option. Pass-through AutoGen argument

8.6.10 skip-suffix option (-s)

This is the "omit the file with this suffix" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.11 select-suffix option (-o)

This is the "specify this output suffix" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.

Pass-through AutoGen argument

8.6.12 source-time option

This is the "set mod times to latest source" option. Pass-through AutoGen argument

8.6.13 equate option

This is the "characters considered equivalent" option. Pass-through AutoGen argument

8.6.14 writable option

This is the "allow output files to be writable" option. Pass-through AutoGen argument

8.6.15 loop-limit option

This is the "limit on increment loops" option. Pass-through AutoGen argument

8.6.16 timeout option (-t)

This is the "time limit for servers" option. Pass-through AutoGen argument

8.6.17 trace option

This is the "tracing level of detail" option. Pass-through AutoGen argument

8.6.18 trace-out option

This is the "tracing output file or filter" option. Pass-through AutoGen argument

8.6.19 show-defs option

This is the "show the definition tree" option. Pass-through AutoGen argument

8.6.20 show-shell option

This is the "show shell commands" option. Pass-through AutoGen argument

8.6.21 define option (-D)

This is the "name to add to definition list" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.
• is a member of the define class of options.

Pass-through AutoGen argument

8.6.22 undefine option (-U)

This is the "definition list removal pattern" option.

This option has some usage constraints. It:

• may appear an unlimited number of times.
• is a member of the define class of options.

Pass-through AutoGen argument

8.7 Replacement for Stdio Formatting Library

Using the `printf' formatting routines in a portable fashion has always been a pain, and this package has been way more pain than anyone ever imagined. Hopefully, with this release of snprintfv, the pain is now over for all time.

The issues with portable usage are these:

1. Argument number specifiers are often either not implemented or are buggy. Even GNU libc, version 1 got it wrong.

2. ANSI/ISO "forgot" to provide a mechanism for computing argument lists for vararg procedures.

3. The argument array version of printf (`printfv()') is not generally available, does not work with the native printf, and does not have a working argument number specifier in the format specification. (Last I knew, anyway.)

4. You cannot fake varargs by calling `vprintf()' with an array of arguments, because ANSI does not require such an implementation and some vendors play funny tricks because they are allowed to.

These four issues made it impossible for AutoGen to ship without its own implementation of the `printf' formatting routines. Since we were forced to do this, we decided to make the formatting routines both better and more complete :-). We addressed these issues and added the following features to the common printf API:

1. The formatted output can be written to
   • a string allocated by the formatting function (`asprintf()').
   • a file descriptor instead of a file stream (`dprintf()').
   • a user specified stream (`stream_printf()').

2. The formatting functions can be augmented with your own functions. These functions are allowed to consume more than one character from the format, but must commence with a unique character. For example,

   "%{struct stat}\n"

   might be used with '{' registered to a procedure that would look up "struct stat" in a symbol table and do appropriate things, consuming the format string through the '}' character.

Gary V. Vaughan was generous enough to supply this implementation. Many thanks!!

For further details, the reader is referred to the snprintfv documentation. These functions are also available in the template processing as `sprintf' (see section 3.5.25 `sprintf' - format a string), `printf' (see section 3.5.20 `printf' - format to stdout), `fprintf' (see section 3.5.6 `fprintf' - format to a file), and `shellf' (see section 3.5.24 `shellf' - format a string, run shell).