Skip to content

Network Automation

My journey with Network & Cloud Automation

Menu
  • Beginner
  • DevOps-NetDevOps
  • Network Automation
    • Docker
    • Python Libraries
      • NAPALM
      • Netmiko
      • Jinja2
      • Scrapli
      • Yang
  • Cloud Automation
    • Terraform
  • Python šŸ Tips and Tricks
Menu
textfsm for network engineers

How to Parse data with TextFSM

Posted on August 23, 2021August 21, 2021 by Gurpreet Kochar

TextFsm is available as a plugin to various device interaction libraries like Netmiko but what if we need to use TEXTFSM as a standalone python package, it is also available. Conditions where you don’t have access to the devices but have outputs from the device in a text file. We have already seen how to use TextFSM with Netmiko here. Let’s see how to parse data with TextFSM as a standalone package. Assume we have below directory structure

╰─ tree                                                                      ─╯
.
ā”œā”€ā”€ 01_textfsm.py
ā”œā”€ā”€ Input
│   ā”œā”€ā”€ abc-aust-001-as01.txt
│   ā”œā”€ā”€ abc-aust-001-cs01.txt
│   ā”œā”€ā”€ abc-aust-006-as01.txt
│   ā”œā”€ā”€ abc-aust-006-as02.txt
│   ā”œā”€ā”€ abc-aust-006-as03.txt
│   ā”œā”€ā”€ abc-aust-006-as04.txt
│   ā”œā”€ā”€ abc-newz-001-as01.txt
│   ā”œā”€ā”€ abc-newz-001-as02.txt
│   ā”œā”€ā”€ abc-newz-001-as03.txt
│   ā”œā”€ā”€ abc-newz-001-as04.txt
│   ā”œā”€ā”€ abc-newz-001-as05.txt
│   ā”œā”€ā”€ abc-newz-001-as07.txt
│   ā”œā”€ā”€ abc-newz-001-as08.txt
│   ā”œā”€ā”€ abc-newz-001-as09.txt
│   ā”œā”€ā”€ abc-newz-001-as10.txt
│   ā”œā”€ā”€ abc-newz-001-as11.txt
│   ā”œā”€ā”€ abc-newz-001-cs01.txt
│   └── abc-newz-003-cs01.txt
ā”œā”€ā”€ Output

Script blueprint:-

  1. Read all files in the input directory
  2. For each command we want to parse, load the correct textfsm template from the templates folder
  3. After textfsm has parsed the data, convert data into tabular format.
  4. Write to excel using openpyxl or pandas
import textfsm
import os
import pandas as pd
from datetime import datetime
import ntc_templates


if __name__ == "__main__":
  # Find the folder where ntc_templates are installed.
	templates = os.path.dirname(os.path.dirname(ntc_templates.__file__)) + '/ntc_templates/templates'
	#print(templates)
  current_time = datetime.now()
	#command list for each command that is in the input files you want to parse
	commands = ['show_version', 'show_interfaces']
	# Create a seperate df for each command thereby seperate sheet for each command for all devices.
	df_list = []
	for command in commands:
		print(f"Parsing the output of {command}\n!")
		template = _load_template(command, templates) # _load_template function will load correct template
		#print(dir(template))
		df_list.append([_parse_each_file(), command]) # _parse_each_file will prepare a dataframe containing parsed content of each file.
	_write_to_excel(df_list) # this function will write the dataframes for each command into an excel file.

## Let's now see each function as it occurs in the above blueprint code.
  1. Load template
def _load_template(command, templates):
	with open(f"{templates}/cisco_ios_{command}.textfsm") as f:
		return textfsm.TextFSM(f)

2. Parse each file using TextFsm template

def _parse_each_file():
	df = pd.DataFrame()
	for file in sorted(os.listdir('Input')):
		template.Reset() # otherwise entires from next loop item adds to the previous loop item,
		with open('Input/' + file) as f:
			text = f.read()

		df_parsed = pd.DataFrame(template.ParseTextToDicts(text))
    """Calls ParseText and turns the result into list of dicts.
    List items are dicts of rows, dict key is column header and value is column value. """

		df_parsed.insert(0, 'DEVICE_NAME', file[:-4])
		df = df.append(df_parsed)
	return df

3. After all parsing is complete, we need to write the dataframes to output sheet.

def _write_to_excel(df_list):
	writer = pd.ExcelWriter('Output/parsed_commands_' + current_time.strftime("%Y-%m-%d-%H-%M") + '.xlsx', engine='xlsxwriter')
	for df, sheetname in df_list:
	    df.to_excel(writer, sheet_name=sheetname, index=False)
	    workbook = writer.book
	    worksheet = writer.sheets[sheetname]
	    cell_format = workbook.add_format()
	    cell_format.set_text_wrap()
	    cell_format.set_align('center')
	    cell_format.set_align('vcenter')
	    # worksheet.set_column('A:E', 20, cell_format)
	writer.save()

Putting all the pieces together and executing the script it finally gets this output. Notice separate tabs for each command.

Within a fraction of seconds, you will be able to parse data of well-known command outputs into separate tabs for later reference without requiring any understanding of writing any parsers for parsing the text into fields you want.

I hope this was informative. If you have any suggestions/comments, please feel free to contact me below.

Know someone who may benefit? Share this:

  • Tweet
  • Click to share on Telegram (Opens in new window) Telegram
  • Click to share on WhatsApp (Opens in new window) WhatsApp
  • Click to email a link to a friend (Opens in new window) Email
  • More
  • Click to print (Opens in new window) Print
  • Click to share on Reddit (Opens in new window) Reddit
  • Share on Tumblr
  • Pocket

Like this:

Like Loading...

Related

Leave a ReplyCancel reply

All Blog Posts
My Resume

Upcoming Posts

Sorry - nothing planned yet!

Recent Posts

  • How to backup configuration to TFTP Server using Ansible – Part II
  • How to backup network devices using Ansible – Part I
  • Netmiko SSH Proxy/JumpServer
  • A short note on SASE
  • Understanding Ansible

Recent Comments

  1. Jack on Multithreading with Python for Network Engineers
  2. LifeCanvas on [Theory] Multithreading vs Multiprocessing vs AsyncIO
  3. Jasper Horng on Netmiko SSH Proxy/JumpServer
  4. asdfasdf on Python API Using FASTAPI – UPDATE – PUT – PATCH – Part V
  5. Gurpreet Kochar on Python Scrapli AsyncIO Usage

Archives

  • September 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
Topic Request / Suggestion
Loading
© 2025 Network Automation | Powered by Minimalist Blog WordPress Theme
 

Loading Comments...
 

    %d