API reference

API reference for TTP module.

class ttp.ttp(data='', template='', log_level='WARNING', log_file=None, base_path='', vars=None, dynamic_path_key_to_int=False)

Template Text Parser main class to load data, templates, lookups, variables and dispatch data to parser object to parse in single or multiple processes, construct final results and run outputs.

Parameters

  • data – (obj) file object or OS path to text file or directory with text files with data to parse

  • template – (obj) one of: file object, OS path to template text file, OS path to directory with templates text files, template text string, TTP Templates path (ttp://x/y/z), OS path to template text file within TTP_TEMPLATES_DIR directory, where TTP_TEMPLATES_DIR is an environment variable.

  • base_path – (str) base OS path prefix to load data from for template’s inputs

  • log_level – (str) level of logging “DEBUG”, “INFO”, “WARNING”, “ERROR”, “CRITICAL”

  • log_file – (str) path where to save log file

  • vars – (dict) dictionary of variables to make available to ttp parser

  • dynamic_path_key_to_int – (bool) if True converts dynamic path keys to integers, False by default

Example:

from ttp import ttp
parser = ttp(data="/os/path/to/data/dir/", template="/os/path/to/template.txt")
parser.parse()
result = parser.result(format="json")
print(result[0])

Or if TTP_TEMPLATES_DIR=/absolute/os/path/to/templates/dir/:

from ttp import ttp
parser = ttp(data="/os/path/to/data/dir/", template="template.txt")
parser.parse()
result = parser.result(format="json")
print(result[0])
add_function(fun, scope, name=None, add_ttp=False)

Method to add custom function in _ttp_ dictionary. Function can be referenced in template depending on scope.

Parameters

  • fun - function reference

  • scope - scope to add function to

  • name - optional, name to use within templates, by default equal to function __name__

  • add_ttp - boolean, on True will add _ttp_ dictionary in function’s global scope

scope options

  • match - used for match variables

  • group - used for groups

  • input - used for inputs

  • output - used for outputs

  • returners - used for output returners

  • formatters - used for output returners

  • variable - used as template variable getter

  • macro - used as macro function

Warning

add_function should be called before template loaded in parser

Custom functions should use first argument to hold data to process, additional args and kwargs will be supplied to function if provided in template.

TTP passes output tag attributes to returner and formatter functions, attributes need to be unpacked using, for instance, **kwargs.

For template variable getters functions first argument supplied is an input text data, second argument is datum name, equal to filename if loaded from file

Function return content differ depending on scope:

  • match - must return tuple of two items

  • group - must return tuple of two items

  • input - must return tuple of two items

  • output - must return single element containing processing results

  • returners - not returns expected

  • formatters - must return single element containing processing results

  • variable - must return single element to assign to variable

  • macro - can return processing results, True or False

For match, group and input functions TTP expects in return tuple of two elements where first element should contain processing results, second element can be True, False, None or dictionary. Second item can influence processing logic following these rules:

  • if second item is False - results invalidated and discarded

  • if second item is True or None - first item replaces originally supplied data, processing continues

  • if second item is dictionary - supported by match scope only, dictionary merged with results

Example:

def group_cust_fun(data, *args, **kwargs):
    if kwargs.get("upper") == True:
        data["description"] = data["description"].upper()
    return data, None

template = '''
<input load="text">
interface Lo1
 description this interface has description
 ip address 1.1.1.1 32
</input>

<group myFun="upper=True">
interface {{ interface }}
 description {{ description | ORPHRASE }}
 ip address {{ ip }} {{ mask }}
</group>
'''

parser = ttp()
parser.add_function(group_cust_fun, scope="group", name="myFun")
parser.add_template(template)
parser.parse()
add_input(data, input_name='Default_Input', template_name='_root_template_', groups=None)

Method to load data to be parsed. Data associated with certain input of input_name and template of template_name.

Warning

add_input should be called only after templates added

Parameters

  • data text data or OS path to text file or directory with text files with data to parse. Also can be structured data - list or dictionary - will be passed to input as is, so that it can be pre-processed using input macro function(s)

  • input_name (str) name of the input to put data in, default is Default_Input

  • groups (list) list of group names to use to parse this input data

  • template_name (str) name of the template to add input for

add_lookup(name, text_data='', include=None, load='python', key=None)

Method to add lookup table data to all templates loaded so far. Lookup is a text representation of structure that can be loaded into python dictionary using one of the available loaders - python, csv, ini, yaml, json.

Parameters

  • name (str) name to assign this lookup table for referencing

  • text_data (str) text to load lookup table/dictionary from

  • include (str) absolute or relative /os/path/to/lookup/table/file.txt

  • load (str) name of TTP loader to use to load table data

  • key (str) specify key column for csv loader to construct dictionary

include can accept relative OS path - relative to the directory where TTP will be invoked either using CLI tool or as a module

add_template(template, template_name='_root_template_', filters=None)

Method to load TTP templates into the parser.

Parameters

  • template file object or OS path to text file with template

  • template_name (str) name of the template

  • filters (list) list of templates’ names to load,

filters attribute allow to filter the list of template names that should be loaded. Checks done against child templates as well. For templates specified in filter list, groups/macro/inputs/etc. will not be loaded and no results produced.

add_vars(vars)

Method to add variables to ttp and its templates to reference during parsing

Parameters

  • vars dictionary of variables to make available to ttp parser

clear_input(template_name='_all_')

Method to delete all input data for all or some templates, can be used prior to adding new set of data to parse with same templates, instead of re-initializing ttp object.

Parameters

  • template_name (str) name of the template to clear input for, clears for all templates by default

clear_result(templates=None)

Method to clear parsing results for templates.

Parameters

  • templates (list or str) - name of template(s) to clear results for, if not provided will clear results for all templates.

get_input_load()

Method to retrieve input tag text load. Using input load attribute, text data can be loaded into python structure using one of the supported loaders, for instance if text data structured using YAML, YAML loader can be used to produce python native structure, that structure will be returned by this method.

Primary use case is to specify parameters within TTP input that can be used by other applications/scrips.

Returns

Dictionary of {“template_name”: {“input_name”: “input load data”}} across all templates, where input_name set to input name attribute value, by default it is “Default_Input”, and template_name set to name of the template, by default it is “_root_template_”

Warning

inputs load can override one another if combination of template_name and input_name is not unique.

get_template()

Method to return a string of fully constructed TTP templates. This is mainly to assist with troubleshooting extend tags processing.

parse(one=False, multi=False)

Method to parse data with templates.

Parameters

  • one (boolean) if set to True will run parsing in single process

  • multi (boolean) if set to True will run parsing in multiprocess

By default one and multi set to False and TTP will run parsing following below rules:

  1. if one or multi set to True run in one or multi process

  2. if overall data size is less then 5Mbyte, use single process

  3. if overall data size is more then 5Mbytes, use multiprocess

In addition to 3 TTP will check if number of input data items more then 1, if so multiple processes will be used and one process otherwise.

result(templates=None, structure='list', **kwargs)

Method to get parsing results, supports basic filtering based on templates’ names, results can be formatted and returned to specified returner.

Parameters

  • templates (list or str) names of the templates to return results for

  • structure (str) structure type, valid values - list, dictionary or flat_list

kwargs - can contain any attributes supported by output tags, for instance:

  • format (str) output formatter name - yaml, json, raw, pprint, csv, table, tabulate

  • functions (str) reference output functions to run results through

Example:

from ttp import ttp
parser = ttp(data="/os/path/to/data/dir/", template="/os/path/to/template.txt")
parser.parse()
json_result = parser.result(format="json")[0]
yaml_result = parser.result(format="yaml")[0]
print(json_result)
print(yaml_result)

Returns

By default template results set to per_input and structure set to list, returns list such as:

[
   [ template_1_input_1_results,
     template_1_input_2_results,
     ...
     template_1_input_N_results ],
   [ template_2_input_1_results,
     template_2_input_2_results,
     ...
]

If template results set to per_template and structure set to list, returns list such as:

[
   [ template_1_input_1_2...N_joined_results ],
   [ template_2_input_1_2...N_joined_results ]
]

If template results set to per_input and structure set to dictionary, returns dictionary such as:

{
   template_1_name: [
     input_1_results,
     input_2_results,
     ...
     input_N_results
    ],
   template_2_name: [
     input_1_results,
     input_2_results
    ],
     ...
}

If template results set to per_template and structure set to dictionary, returns dictionary such as:

{
   template_1_name: input_1_2...N_joined_results,
   template_2_name: input_1_2...N_joined_results
}

If structure set to flat_list, results will be combined across all templates in a list of dictionaries. For instance, with structure set to list result might look like this:

[[[{'interface': 'Lo0', 'ip': '192.168.0.1', 'mask': '32'},
   {'interface': 'Lo1', 'ip': '1.1.1.1', 'mask': '32'}],
  [{'interface': 'Lo2', 'ip': '2.2.2.2', 'mask': '32'},
   {'interface': 'Lo3', 'ip': '3.3.3.3', 'mask': '32'}]]]

But with structure set to flat_list it will be flattened to this:

[{'interface': 'Lo0', 'ip': '192.168.0.1', 'mask': '32'},
 {'interface': 'Lo1', 'ip': '1.1.1.1', 'mask': '32'},
 {'interface': 'Lo2', 'ip': '2.2.2.2', 'mask': '32'},
 {'interface': 'Lo3', 'ip': '3.3.3.3', 'mask': '32'}]
set_input(data, input_name='Default_Input', template_name='_root_template_', groups=None)

Method to replace existing templates inputs data with new set of data. This method is alias to clear_input and add_input methods.

Warning

set_input should be called only after templates added

Parameters

  • data text data or OS path to text file or directory with text files with data to parse Also can be structured data - list or dictionary - will be passed to input as is, so that it can be pre-processed using input macro function(s)

  • input_name (str) name of the input to put data in, default is Default_Input

  • groups (list) list of group names to use to parse this input data

  • template_name (str) name of the template to set input for