Cisco does have a bug toolkit to programmatically fetch the vulnerability information but it doesn’t have any such API for SDWAN vulnerability information yet. So I had the option to create a web scraper to fetch that information into an excel sheet and update it automatically at a click of a button. Let’s take a look at a very straightforward code that you can modify to suit your requirements.
Cisco posts its SDWAN vulnerability information at this URL.
Intent:- Click on each URL on this web page automatically and gather the required information from each URL and dump it into an excel sheet.
STEP1:-
setup imports and write a method to fetch all URLs from cisco’s SDWAN vulnerability URL
from bs4 import BeautifulSoup
import requests
import os
URL_SDWAN = "https://www.cisco.com/c/en/us/support/routers/sd-wan/products-security-advisories-list.html"
class SDWAN_BS4:
def __init__(self):
self.OUTPUT = os.path.join(os.getcwd(), 'OUTPUT')
def get_all_vuln_urls(self):
all_urls = []
r = requests.get(URL_SDWAN)
if r.ok:
soup = BeautifulSoup(r.text, 'lxml')
hrefs = soup.find('div', class_="no-groups").find_all('a')
all_urls = [f"https://www.cisco.com/{href.get('href')}" for href in hrefs]
return all_urlsExecuting the above piece of code gives us a list of all SDWAN Vulnerabilities that have been published
[
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sd-wan-Fhqh8pKX.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sd-wan-jOsuRJCc.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sd-wan-credentials-ydYfskzZ.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-maapi-cmdinjec-znUYTuC.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdw-mpls-infodisclos-MSSRFkZq.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-vmanage-infdis-LggOP9sE.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sd-wan-fuErCWwF.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdw-auth-bypass-65aYqcS2.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sd-wan-vmanage-4TbynnhZ.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-dos-Ckn5cVqW.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-privesc-QVszVUPy.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-vmanageinfdis-LKrFpbv.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sd-wan-vmanage-9VZO4gfU.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-arbfile-7Qhd9mCn.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-enumeration-64eNnDKy.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-xss-eN75jxtW.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-vmaninfdis3-OvdR6uu8.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-auth-bypass-Z3Zze5XC.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-cmdinj-nRHKgfHX.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-xml-ext-entity-q6Z7uVUg.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-cql-inject-c7z9QqyB.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-info-disclos-gGvm9Mfu.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-YuTVWqy.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-authorization-b-GUEpSLK.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwanvman-infodis1-YuQScHB.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdw-sqlinj-HDJUeEAX.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-vdaemon-bo-RuzzEA2.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-privesc-vman-kth3c82B.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-sigverbypass-gPYXd6Mk.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-dir-trav-Bpwc5gtm.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-pathtrav-Z5mCVsjf.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-vinfdis-MC8L58dj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-infodis-2-UPO232DG.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-cql-inject-72EhnUc.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-sqlinjm-xV8dsjq5.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-cmdinjm-9QMSmgcn.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-dosmulti-48jJuEUP.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-abyp-TnGFHrS.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-bufovulns-B5NrSHbj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-file-Y2JSRNRb.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanpt2-FqLuefsS.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmxss2-NL4KSSVR.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanxsshi-9KHEqRpM.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanxss1-XhJCymBt.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-cedge-filt-bypass-Y6wZMqm4.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-privilege-zPmMf73k.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanuafw-ZHkdGGEy.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanxss2-ugJyqxWF.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanx3-vrZbOqqD.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanx2-KpFVSUc.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vepeshlg-tJghOQcA.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vepestd-8C3J9Vc.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vsoln-arbfile-gtsEYxns.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vman-traversal-hQh24tmk.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-escalation-Jhqs5Skf.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vepescm-BjgQm4vJ.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vepegr-4xynYLUj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwan-xss-xXeLFpC3.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdbufof-h5f5VSeL.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-uabvman-SYGzt8Bv.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-clibypvman-sKcLf2L.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanrce-4jtWT28P.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdw-dos-KWOdyHnB.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdscred-HfWWfqBj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sivm-M8wugR9O.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmdirtrav-eFdAxsJg.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanxss-z7bhvHpy.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanxml-Aj4GFEKd.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmandowndir-CVGvdKM3.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmpresc-SyzcS4kC.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanage-v78FubGV.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-emvman-3y6LuTcZ.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-vmanwebid-5QWMcCvt.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20200318-vmanage-cypher-inject.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwpresc-ySJGvE9.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwclici-cvrQpH9v.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sdwanbo-QKcABnS2.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20200318-vmanage-xss.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20200122-sdwan-cmd-inject.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20200122-sdwan-sqlinj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20200122-sdwan-priv-esc.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20200122-sdwan-sql-inject.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20191120-vman-csrf.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190807-sd-wan-bypass.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190619-sdwan-cmdinj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190619-sdwan-privesca.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190619-sdwan-privilescal.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190123-sdwan-escal.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190123-sdwan-unaccess.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190123-sdwan-sol-escal.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190123-sdwan-bo.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20190123-sdwan-file-write.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20181003-sd-wan-bypass.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180905-sd-wan-escalation.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180905-sd-wan-injection.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180905-sd-wan-validation.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sd-wan-bo.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sd-wan-code-ex.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sd-wan-cmd-inject.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sdwan-coinj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sdwan-cmdinj.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sdwan-fo.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sdwan-cmdnjct.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sdwan-ci.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sdwan-dos.html',
'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-20180718-sdwan-cx.html'
]STEP2:- Write a method to access information inside each of these URLs and gather the required information. In this case, I am going to fetch information with the below fields in a dictionary. I assume this is all we need in most of the cases when doing vulnerability assessment.
d = {
"Advisory ID" : advisory_id,
"URL" : url,
"CVE" : cve,
"First Published" : first_published,
"Last Updated" : last_updated,
"BUG IDs" : '\n'.join(ids).strip(),
"BUG URL" : '\n'.join(urls).strip(),
"CVE" : '\n'.join(cve).strip(),
"Summary" : summary,
"Vulnerable Products" : vuln_products,
"Non Vulnerable" : non_vuln_products,
"Workaround" : workaround,
"Fixed Software" : fixed_software
}This method is the heart of this script and can get a little tricky. The intent here is to mimic how HTML on that webpage has stored information and extract information from HTML. If you inspect that webpage, you will find the same elements as mentioned in the below code where the information we need exists. Take some time to review especially if you are not at all familiar with web scrapping.
def get_vuln_detail(self, url):
r = requests.get(url)
if r.ok:
soup = BeautifulSoup(r.text, 'lxml')
advisory_id = soup.find('div', id="divpubidvalue").text
URL = url
first_published = soup.find('div', id='ud-published').find('div', class_='divLabelContent').text.replace(u'\xa0', u' ')
try:
last_updated = soup.find('div', id='ud-last-updated').find('div', class_='divLabelContent').text.replace(u'\xa0', u' ')
except Exception:
last_updated = 'Not published'
try:
bugs = soup.find('span', id='fullddtscontent_content').find_all('a')
except Exception:
bugs = soup.find('div', class_='ddtsList').find_all('a')
urls = [url.get('href') for url in bugs if '#' not in url.get('href')]
ids = [url.text for url in bugs]
try:
cve = soup.find('span', id='fullcvecontent_content').text
cve = cve.spit(',')
except Exception:
cve = soup.find('div', class_='CVEList').find_all('div', class_='inlineblock divPaddingTen')
cve = [x.text for x in cve]
summary = soup.find('div', id="summaryfield").text.strip()
vuln_products = soup.find('div', id="vulnerableproducts").text.strip()
non_vuln_products = soup.find('div', id="productsconfirmednotvulnerable").text.strip()
workaround = soup.find('div', id='workaroundsfield').text.strip()
fixed_software = soup.find('div', id='fixedsoftfield').text.strip()
d = {
"Advisory ID" : advisory_id,
"URL" : url,
"CVE" : cve,
"First Published" : first_published,
"Last Updated" : last_updated,
"BUG IDs" : '\n'.join(ids).strip(),
"BUG URL" : '\n'.join(urls).strip(),
"CVE" : '\n'.join(cve).strip(),
"Summary" : summary,
"Vulnerable Products" : vuln_products,
"Non Vulnerable" : non_vuln_products,
"Workaround" : workaround,
"Fixed Software" : fixed_software
}
return dThe above method gives us output in the below format
{
'Advisory ID': 'cisco-sa-sd-wan-Fhqh8pKX',
'URL': 'https://www.cisco.com//c/en/us/support/docs/csa/cisco-sa-sd-wan-Fhqh8pKX.html',
'CVE': 'CVE-2021-1546',
'First Published': '2021 September 22 16:00 GMT',
'Last Updated': 'Not published',
'BUG IDs': 'CSCvx79335',
'BUG URL': 'https://bst.cloudapps.cisco.com/bugsearch/bug/CSCvx79335',
'Summary': 'A vulnerability in the CLI of Cisco\xa0SD-WAN Software could allow an authenticated, local attacker to access sensitive information.\nThis vulnerability is due to improper protections on file
access through the CLI. An attacker could exploit this vulnerability by running a CLI command that targets an arbitrary file on the local system. A successful exploit could allow the attacker to return portions
of an arbitrary file, possibly resulting in the disclosure of sensitive information.\nCisco\xa0has released software updates that address this vulnerability. There are no workarounds that address this
vulnerability.\nThis advisory is available at the following link:https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-sd-wan-Fhqh8pKX',
'Vulnerable Products': 'At the time of publication, this vulnerability affected the following Cisco\xa0products if they were running a vulnerable release of Cisco\xa0SD-WAN Software:\n\nSD-WAN vBond
Orchestrator Software\nSD-WAN vEdge Cloud Routers\nSD-WAN vEdge Routers\nSD-WAN vManage Software\nSD-WAN vSmart Controller Software\n\nFor information about which Cisco\xa0software releases were vulnerable, see
the Fixed Software section of this advisory. See the Details section in the bug ID(s) at the top of this advisory for the most complete and current information.',
'Non Vulnerable': 'Only products listed in the Vulnerable Products section of this advisory are known to be affected by this vulnerability.\nCisco\xa0has confirmed that this vulnerability does not affect
Cisco\xa0IOS XE SD-WAN Software.',
'Workaround': 'There are no workarounds that address this vulnerability.',
'Fixed Software': 'When considering software upgrades, customers are advised to regularly consult the advisories for Cisco\xa0products, which are available from the Cisco\xa0Security Advisories page, to
determine exposure and a complete upgrade solution.\nIn all cases, customers should ensure that the devices to be upgraded contain sufficient memory and confirm that current hardware and software configurations
will continue to be supported properly by the new release. If the information is not clear, customers are advised to contact the Cisco\xa0Technical Assistance Center (TAC) or their contracted maintenance
providers.\nFixed Releases\nAt the time of publication, the release information in the following table(s) was accurate.\xa0See the Details section in the bug ID(s) at the top of this advisory for the most
complete and current information.\nThe left column lists Cisco\xa0software releases, and the right column indicates whether a release was affected by the vulnerability described in this advisory and which
release included the fix for this vulnerability.\n\n\n\nCisco\xa0SD-WAN Software Release\nFirst Fixed Release\n\n\n\n\n18.4\nMigrate to a fixed release.\n\n\n19.2\nMigrate to a fixed
release.\n\n\n20.3\xa0\nMigrate to a fixed release.\n\n\n20.4\n20.4.2\n\n\n20.5\n20.5.2\n\n\n20.6\n20.6.1'
}STEP3:- Create a method to write to an excel sheet. It takes in a list of data frames, output folder, and filename as an argument and writes each dataframe to a separate sheet of workbook like in this case.
def write_to_excel(self, df_list, folder, file_name):
save_file_name = f"{folder}/{file_name}_{datetime.now().strftime('%d-%b-%Y')}.xlsx"
writer = pd.ExcelWriter(save_file_name, engine='xlsxwriter')
for df, sheetname, index in df_list:
df.to_excel(writer, sheet_name=sheetname, index=index)
writer.save()STEP4:- Iterate over each URL and prepare the data frames that we need to create excel sheets.
def create_sdwan_vuln_report(self):
all_urls = self.get_all_vuln_urls()
self.data = [self.get_vuln_detail(url) for url in all_urls]
df = pd.DataFrame(self.data)
df_exploded = df.copy()
df_exploded['BUG IDs'] = df_exploded['BUG IDs'].str.split('\n')
df_exploded = df_exploded.explode('BUG IDs')
df_list = [(df, 'SDWAN_VULNS', False), (df_exploded, 'RAW', False)]
self.write_to_excel(df_list, self.OUTPUT, 'sdwan_vulnerabilities_bs4')STEP5:- Bringing it all together
from bs4 import BeautifulSoup
import requests
import pandas as pd
import os
from datetime import datetime
URL_SDWAN = "https://www.cisco.com/c/en/us/support/routers/sd-wan/products-security-advisories-list.html"
class SDWAN_BS4:
def __init__(self):
self.OUTPUT = os.path.join(os.getcwd(), 'OUTPUT')
if not os.path.exists(self.OUTPUT):
os.mkdir(self.OUTPUT)
def get_all_vuln_urls(self):
all_urls = []
r = requests.get(URL_SDWAN)
if r.ok:
soup = BeautifulSoup(r.text, 'lxml')
hrefs = soup.find('div', class_="no-groups").find_all('a')
all_urls = [f"https://www.cisco.com/{href.get('href')}" for href in hrefs]
return all_urls
def get_vuln_detail(self, url):
r = requests.get(url)
if r.ok:
soup = BeautifulSoup(r.text, 'lxml')
advisory_id = soup.find('div', id="divpubidvalue").text
URL = url
first_published = soup.find('div', id='ud-published').find('div', class_='divLabelContent').text.replace(u'\xa0', u' ')
try:
last_updated = soup.find('div', id='ud-last-updated').find('div', class_='divLabelContent').text.replace(u'\xa0', u' ')
except Exception:
last_updated = 'Not published'
try:
bugs = soup.find('span', id='fullddtscontent_content').find_all('a')
except Exception:
bugs = soup.find('div', class_='ddtsList').find_all('a')
urls = [url.get('href') for url in bugs if '#' not in url.get('href')]
ids = [url.text for url in bugs]
try:
cve = soup.find('span', id='fullcvecontent_content').text
cve = cve.spit(',')
except Exception:
cve = soup.find('div', class_='CVEList').find_all('div', class_='inlineblock divPaddingTen')
cve = [x.text for x in cve]
summary = soup.find('div', id="summaryfield").text.strip()
vuln_products = soup.find('div', id="vulnerableproducts").text.strip()
non_vuln_products = soup.find('div', id="productsconfirmednotvulnerable").text.strip()
workaround = soup.find('div', id='workaroundsfield').text.strip()
fixed_software = soup.find('div', id='fixedsoftfield').text.strip()
d = {
"Advisory ID" : advisory_id,
"URL" : url,
"CVE" : cve,
"First Published" : first_published,
"Last Updated" : last_updated,
"BUG IDs" : '\n'.join(ids).strip(),
"BUG URL" : '\n'.join(urls).strip(),
"CVE" : '\n'.join(cve).strip(),
"Summary" : summary,
"Vulnerable Products" : vuln_products,
"Non Vulnerable" : non_vuln_products,
"Workaround" : workaround,
"Fixed Software" : fixed_software
}
return d
def write_to_excel(self, df_list, folder, file_name):
save_file_name = f"{folder}/{file_name}_{datetime.now().strftime('%d-%b-%Y')}.xlsx"
writer = pd.ExcelWriter(save_file_name, engine='xlsxwriter')
for df, sheetname, index in df_list:
df.to_excel(writer, sheet_name=sheetname, index=index)
writer.save()
def create_sdwan_vuln_report(self):
all_urls = self.get_all_vuln_urls()
self.data = [self.get_vuln_detail(url) for url in all_urls]
df = pd.DataFrame(self.data)
df_exploded = df.copy()
df_exploded['BUG IDs'] = df_exploded['BUG IDs'].str.split('\n')
df_exploded = df_exploded.explode('BUG IDs')
df_list = [(df, 'SDWAN_VULNS', False), (df_exploded, 'RAW', False)]
self.write_to_excel(df_list, self.OUTPUT, 'sdwan_vulnerabilities_bs4')
if __name__ == '__main__':
sdwan = SDWAN_BS4()
sdwan.create_sdwan_vuln_report()At this point, you should have an excel sheet that looks like this ( unformatted but it does contain everything you need). Also, you could add print statements, logs, progress bars to make it more verbose at the moment, you will see nothing when it’s working. Another enhancement could be to use threads to make the script faster since there are tens of URLs we need to fetch information from. Search for multithreading and multiprocessing in the search bar on the top right for the blog about those.
You could add a couple of more methods to format the excel sheet to your desire as well. Here is my implementation of openpyxl formatting.
def _color_formatting(self, workbook):
# Add a format. Light red fill with dark red text.
red = workbook.add_format({'bg_color': '#FFC7CE',
'font_color': '#9C0006'})
# Add a format. Yellow fill with dark yellow text.
yellow = workbook.add_format({'bg_color': '#FFEB9C',
'font_color': '#9C6500'})
# Add a format. Green fill with dark green text.
green = workbook.add_format({'bg_color': '#C6EFCE',
'font_color': '#006100'})
cell_format = workbook.add_format()
cell_format.set_text_wrap()
cell_format.set_align('center')
cell_format.set_align('vcenter')
header_format = workbook.add_format({
'bold': True,
'text_wrap': True,
'valign': 'center',
'align': 'vcenter',
'fg_color': '#CC00CC',
'font_size': 12,
'border': 1})
return red, yellow, green, cell_format, header_format
def write_to_excel(self, df_list, folder, file_name):
save_file_name = f"{folder}/{file_name}_{datetime.now().strftime('%d-%b-%Y')}.xlsx"
writer = pd.ExcelWriter(save_file_name, engine='xlsxwriter')
for df, sheetname, index in df_list:
df.to_excel(writer, sheet_name=sheetname, startrow=1, header=False, index=index) # strip default header
workbook = writer.book
red, yellow, green, cell_format, header_format = self._color_formatting(workbook)
worksheet = writer.sheets[sheetname]
worksheet.set_column('A:AC', 20, cell_format)
new_cols = df.columns.values #
if index:
new_cols = list(df.index.names) + list(df.columns.values)
for col_num, value in enumerate(new_cols): # Adding custom header
worksheet.write(0, col_num, value, header_format) # for all sheets.
worksheet.set_row(0, 25) # Set header row width
writer.save()At this point, you should have a sheet that looks like this
Implementing multi-threads should be relatively easy. Take a look at below search results.
https://networkautomationlane.in/?s=multithreading
