Brandon Bennett

Network Automation Engineer

Info

Location
Evergreen, CO 80439
Phone
720.284.2903
Email
brandon@brbe.me
Github
github.com/nemith

Skills

Languages
Go, Python, Rust, SLAX/Junos scripting, C/C++, Ruby
Protocols
BGP, OSPF, IS-IS, MPLS (LDP, RSVP, RSVP-TE), QoS, DNS, DHCP, ZTP, HSRP, NETCONF, SNMP, YANG/OpenConfig, TACACS
Systems
Linux, BSD, Junos, IOS-XR, NX-OS, Cumulus, FBOSS
Frameworks/Technologies
Django, Ansible, Chef, Netbox, MySQL/Postgres, Apache Zookeeper, git, mercurial (hg), svn

Certifications

  • CCIE R&S #19406 (Expired)
  • JNCIE-ER #46 (Expired)
  • JNCIE-M #658 (Expired)

Profile

Network engineer with a passion for automating the mundane to improve operation efficiency and cut down on user errors. 20 years of experience in networking, half of which has been focused on building tools and automation. Deep experience with multiple network operating systems, Linux, and large scale networks.

Work History

Network Production Engineer, Meta

Feb 2021 - Present
  • Worked as a Production Engineer embedded inside the DC SDN Controller team to help optimize for reliability and fast/accurate deployments. Improved dashboard and alarms for easier on-call shifts.
  • Implemented a plan to help harden critical automation tooling after a major Facebook outage. Organized work for 3 critical systems and help hand-off of the systems to engineers unfamiliar with them. Implemented critical bug and guardrails to help prevent similar issues going forward.

Network Automation Manager, Subspace

Sep 2020 - Feb 2021
  • Started the foundation for a network automation team inside Subspace to solve deployments and config changes for devices located in pops around the globe.
  • Worked closely with recruiting to define the network automation role and what engineers to target.
  • Started work on a network modeling system so we could define out device and circuit modeling to eventually be used to generate configuration. Implemented a sync script to keep new/old systems in sync.
  • Worked to help define a "bootstrap" console server that could be used to ZTP provision the rest of the network to cut down on a multiple days turn up into hours. Worked with external vendors and internal teams to select correct hardware.

Network Production Engineering Manager, Facebook (now Meta)

Sep 2018 - Sep 2020
  • Supported a team of six supporting various systems for device life-cycle, ZTP, network auditing, and controlled network drains.
  • Worked to restructure the team to help aid the merging of overlapping tools and to work closer with Software Engineering counterparts in the organization.
  • Created and ran the first network tooling meetup for all teams building or working with network tooling from across multiple offices across the globe to give updates/presentations and collaborate on different systems together.
  • Help build a community team comprised of 5-7 engineers from multiple organizations to support the Go programming language.

Network Production Engineering, Facebook (now Meta)

Dec 2013 - Sep 2018
  • Re-implemented a full BGP mesh script to intelligently create a full iBGP mesh config for all datacenter/edge routers and apply the correct policy to them.
  • Re-implemented a MPLS mesh script to create RSVP tunnels between all remote sides based on traffic demand, capacity, site type, and static configuration. Increasing code extendability, and vastly sped up build times from tens of minutes down to seconds.
  • Wrote a system to store vendor firmware/OS images an serve them over HTTP, TFTP, FTP, or SFTP from multiple datacenters utilizing internal blob storage
  • Worked on a small team to design and implement a new device life-cycle system (provisioning/ztp, reconfiguring/rebuild, and decomissioning, etc) for multiple different device roles and network operating systems
    • Fully distributed system with a Zookeeper based queue system and very resilient to system outages. If workers died other workers would pick up the work
    • Processes were spit up into small independent binaries that did the work and could be reused across multiple life-cycle workflows
    • Used to perform over 4 million jobs over the life of the system to date.
  • Worked with the initial turn up of Facebook's Express Backbone, a network with a custom MPLS SR stack running on commodity switches (Arista 7500/7800) series. Design the initial network configuration, addressing, and port policies. Created a program to automate generation of full configs for this new system.
  • Helped supported Go for all of Facebook and integrating into build systems, test runners, internal library support, for over 300 developers. Helped build tools and provide support for integrating Oslo merger into Facebook's megarepo.

Sr. Systems Engineer, Juniper Networks

Jul 2011 - Dec 2013
  • Presales engineer in the commercial space covering CO, NM, UT, and WY including the entire Juniper product lines with emphasis on advanced routing and switching on the EX-series and MX-series products.
  • Assisted and led customer designs including large security, campus, and data center deployments utilizing Juniper equipment.
  • Presented in front of Juniper customers and partners utilizing both self-generated and corporate created content on latest industry trends and various Juniper solutions.
  • Wrote commit and event scripts in SLAX replicating Ciscos IOS RPM feature to secure a multi-million dollar deal with O3B Networks

Sr. Voice Engineer, tw telecom (now Lumen Technologies)

Feb 2010 - Jul 2011
  • Solely designed and managed a project to deploy a multi cluster Cisco Communications Manger to over 140 sites with about 2500 phones using the latest recommended practices including SIP gateway configuration, SRST, call globalization/localization, and unified messaging.
  • Worked closely with engineering teams to internally trial a new SIP trunk product offering and providing valuable feedback to assist with the product development.
  • Led the technical discussions on improving the call center to better facilitate our customer utilizing the latest technology available from our vendors.
  • Collaborated with the internal networking lead architect to design and implement new Cisco Nexus equipment as new core routers for the main IT datacenter during an emergency upgrade in under a week.

Solutions Engineer, Accuvant (now Optiv)

Mar 2009 - Feb 2010
  • Lead the company for both pre and post routing and switching resources often being pulled in for deals outside of primary region in the routing and switching domain.
  • Presented in joint lunch and learns with Juniper in future directions of networking and equipment all across the country.
  • Sold, designed, and implemented a flexible and multi-purpose proof-of-concept/development lab for Agilent Technologies to test real world protocol encapsulations using Juniper M-series, J-series, and EX-series products with various protocols stacks including 8+ layers of QinQ, 4+ layers of MPLS labels, L2TPv3, GRE, and IPIP
  • Consulted with Agilent Technologies product development and software/hardware development teams as a subject matter expert in network protocols, design, and overall functions to help with their next-gen protocol analyzer product
  • Wrote custom software in Python to convert proprietary backup data from Cisco ACS to Juniper Steel-Belted Radius

Network Architect, tw telecom (now Lumen Technologies)

Feb 2006 - Mar 2008
  • Redesigned an antiquated, data center to a highly resilient, high-speed network utilizing Catalyst 6500 VSS and including a new cabling and rack infrastructure to utilize gigabit and ten-gigabit connections. Phased move from old equipment to new equipment with full interoperability for low/no impact migration
  • Planned and executed a multi-phased network merger of two overlapping networks to facilitate the merger of Xspedius Communications in late 2006 only using NAT when absolutely required and a 1+ year plan to remove any workarounds required during the merger with the goal to have one network architecture.
  • Designed the network infrastructure for a new NOC from the ground up using Layer 3 access layer with OSPF and SSM. Designed everything from cabling and rack placement to the fiber uplinks and full network design and configuration templates.

Network Engineer, M.D.C. Holdings

May 2004 - Feb 2006
  • Designed migrated redundant internet connection from a dual-NAT setup to a dual-homed BGP load sharing architecture with two Tier 1 OSP network providers.
  • Designed migrated network from a legacy point-to-point T1 design to 30+ sites to a carrier provided MPLS L3VPN network using BGP for communication between PE and CE and EIGRP for any local routers at a site. Legacy T1 network was redesigned to be redundant backup to the MPLS provider.
  • Designed and implemented a full Cisco IP Telephony system including Cisco Callmanager, Unity and IPCC Express for over 500 users in 3 months and steadily increased it to 1500 users in 20 sites in less than one year.

Accomplishments

  • Presented a talk on Facebook's adaptable network provisioning system at NANOG71 and NANOG73.
  • Worked with the NANOG Program Committee to create the very first NANOG Hackathon.
    • Helped host the hackathons for NANOG 67, NANOG 70 and NANOG 71.
    • Gave a beginners tutorial on the Go and GoBGP for the hackathon in NANOG 71.
  • Proposed and added support for test enumeration (go test list) in the Go programming language
  • Wrote the Go NETCONF library for Juniper Networks
  • Invited to work in a small workshop of 12 engineers in rewriting the JNCIE-ER to become the JNCIE-ENT.
    • Solely authored questions in the IGP and QOS sections.
  • Invited by Learning@Cisco as a panel of 12 members to help develop the new CCNP SP
  • Contributor to the Cyanogenmod open source Android project.
    • Device maintainer for the Nook Color and Google Nexus 7 GSM
    • various other OS/build system contributions.