How to parse hierarchical configuration data

TTP can use simple templates that does not contain much hierarchy (same as the data that parsed by them), but what to do if we want to extract information from below text:

router bgp 12.34
 address-family ipv4 unicast
  router-id 1.1.1.1
 !
 vrf CT2S2
  rd 102:103
  !
  neighbor 10.1.102.102
   remote-as 102.103
   address-family ipv4 unicast
    send-community-ebgp
    route-policy vCE102-link1.102 in
    route-policy vCE102-link1.102 out
   !
  !
  neighbor 10.2.102.102
   remote-as 102.103
   address-family ipv4 unicast
    route-policy vCE102-link2.102 in
    route-policy vCE102-link2.102 out
   !
  !
 vrf AS65000
  rd 102:104
  !
  neighbor 10.1.37.7
   remote-as 65000
   address-family ipv4 labeled-unicast
    route-policy PASS-ALL in
    route-policy PASS-ALL out

In such a case we have to use ttp groups to define nested, hierarchical structure, sample template might look like this:

<group name="bgp_cfg">
router bgp {{ ASN }}
 <group name="ipv4_afi">
 address-family ipv4 unicast {{ _start_ }}
  router-id {{ bgp_rid }}
 </group>

 <group name="vrfs">
 vrf {{ vrf }}
  rd {{ rd }}

  <group name="neighbors">
  neighbor {{ neighbor }}
   remote-as {{ neighbor_asn }}
   <group name="ipv4_afi">
   address-family ipv4 unicast {{ _start_ }}
    send-community-ebgp {{ send_community_ebgp | set("Enabled") }}
    route-policy {{ RPL_IN }} in
    route-policy {{ RPL_OUT }} out
   </group>
  </group>
 </group>
</group>

Above data and template can be saved in two files and run using ttp CLI tool with command:

ttp -d "/path/to/data/file.txt" -t "/path/to/template.txt" --outputter yaml

These results will be printed to screen:

- bgp_cfg:
    ASN: '12.34'
    ipv4_afi:
      bgp_rid: 1.1.1.1
    vrfs:
    - neighbors:
      - ipv4_afi:
          RPL_IN: vCE102-link1.102
          RPL_OUT: vCE102-link1.102
          send_community_ebgp: Enabled
        neighbor: 10.1.102.102
        neighbor_asn: '102.103'
      - ipv4_afi:
          RPL_IN: vCE102-link2.102
          RPL_OUT: vCE102-link2.102
        neighbor: 10.2.102.102
        neighbor_asn: '102.103'
      rd: 102:103
      vrf: CT2S2
    - neighbors:
      - ipv4_afi:
          RPL_IN: PASS-ALL
          RPL_OUT: PASS-ALL
      - neighbor: 10.1.37.7
        neighbor_asn: '65000'
      rd: 102:104
      vrf: AS65000

Not too bad, but let’s say we want VRFs to be represented as a dictionary with VRF names as keys, same goes for neighbors - we want them to be a dictionary with neighbor IPs as a key, we can use TTP dynamic path feature together with path formatters to accomplish exactly that, here is the template:

<group name="bgp_cfg">
router bgp {{ ASN }}
 <group name="ipv4_afi">
 address-family ipv4 unicast {{ _start_ }}
  router-id {{ bgp_rid }}
 </group>
 !
 <group name="vrfs.{{ vrf }}">
 vrf {{ vrf }}
  rd {{ rd }}
  !
  <group name="peers.{{ neighbor }}**">
  neighbor {{ neighbor }}
   remote-as {{ neighbor_asn }}
   <group name="ipv4_afi">
   address-family ipv4 unicast {{ _start_ }}
    send-community-ebgp {{ send_community_ebgp | set("Enabled") }}
    route-policy {{ RPL_IN }} in
    route-policy {{ RPL_OUT }} out
   </group>
  </group>
 </group>
</group>

After parsing TTP will print these structure:

- bgp_cfg:
    ASN: '12.34'
    ipv4_afi:
      bgp_rid: 1.1.1.1
    vrfs:
      AS65000:
        peers:
          10.1.37.7:
            ipv4_afi:
              RPL_IN: PASS-ALL
              RPL_OUT: PASS-ALL
            neighbor_asn: '65000'
        rd: 102:104
      CT2S2:
        peers:
          10.1.102.102:
            ipv4_afi:
              RPL_IN: vCE102-link1.102
              RPL_OUT: vCE102-link1.102
              send_community_ebgp: Enabled
            neighbor_asn: '102.103'
          10.2.102.102:
            ipv4_afi:
              RPL_IN: vCE102-link2.102
              RPL_OUT: vCE102-link2.102
            neighbor_asn: '102.103'
        rd: 102:103

That’s better, but what actually changed to have such a different results, well, not to much by the look of it, but quite a lot in fact.

TTP group’s name attribute actually used as a path where to save group parsing results within results tree, to denote different levels dot symbol can be used, that is how we get new vrf and peers keys in the output.

In addition we used TTP dynamic path feature by introducing {{ vrf }} and {{ neighbor }} in the name of the group, that will be dynamically substituted with matching results.

Moreover, we also have to use double star ** path formatter to tell TTP that {{ neighbor }} child content should be kept as a dictionary and not transformed into list (default behavior) whenever we add new data to that portion of results tree.