Firewall Upgrade

Mainstream Firewall Upgrade

Upgrading the Mainstream firewalls from ASA5520 hardware to ASA5550 hardware.

Steps

• Coordinate potential outage with *certain* customers. The immediate impact will be as follows:
  • All established TCP connections will be reset and must re-establish.
  • Customers that connect directly into our network over the Internet (like Technicolor) may briefly lose that ability.
  • For NOC hosts and devices that route via the firewall, an ARP timeout of up to two minutes may occur.
  • For NOC hosts and devices that route via a switch or router, a clear arp may be required on that switch or router.

---

• When ready for switch, here's the prep work:
  • This is a list of devices to console into for the next steps, in the order of priority. This list likely needs to expand. Use a terminal server for as many as possible:
    • distprod1
    • slcigw1
    • slcigw2
    • slcxnetsw1
    • vsat1tx1
    • vsat2tx1
    • slccorpl3sw1 (in the wiring closet)
    • slcukgw1
  • On the previous list of devices, make a console connection and do the following:
    • log in.
    • run the "clear arp" command.
    • if using an actual console cable, disconnect, but do not log out.
    • For connections via a terminal server, just leave the window open.
  • Set up a PC with a number of continuous pings so we can see what loses connectivity.
    • A list of things to ping and what probably needs 'clear arp' if that ping completely stops responding:
      • distprod1 (distprod1)
      • 10.5.3.32 (distprod1)
      • vsat1tx1 (vsat1tx1)
      • 192.168.150.7 (vsat1tx1)
      • vsat1pbr1 (vsat1tx1)
      • vsat2tx1 (vsat2tx1)
      • 192.168.160.7 (vsat2tx1)
      • vsat2pbr1 (vsat2tx1)
      • slcigw1 (slcigw1)
      • 209.210.71.209 (slcigw1)
      • slcigw2 (slcigw2)
      • 24.104.64.225 (slcigw2)
      • slcxnetsw1 (slcxnetsw1)
      • dvbmr1 (slcxnetsw1)
      • slcukgw1 (slcukgw1)
      • albus (slcukgw1)
      • slccorpl3sw1 (slccorpl3sw1) [can likely do this one via telnet]
      • mcp (slccorpl3sw1)

---

• Actual steps required to switch:
  • Do prep work as previously listed.
  • Failover existing pair to the secondary firewall (rack 3).
  • Remove network cables from primary firewall (rack 2).
  • Partially load network cables into new firewall, so they do not make contact but are ready to push in.
  • Connect console cable to new primary firewall, log in.
  • Power down new primary firewall.
  • Push Gi0/3 cable into new firewall.
  • Power up new firewall, check to see if it will establish a failover pair.
  • If failover is established (not expecting this to work):
    • Push remaining primary cables in, verify that connectivity is good.
    • Initiate failover.
    • Verify ping connectivity and customer data flow.
    • If we're still going to engage in customer contact, begin that now.
    • Move cables from old secondary firewall to new secondary firewall.
    • Verify that failover communication is good.
    • Maintenance complete. Zero downtime.
  • If failover is not established between the different hardware:
    • Unplug Gi0/3 cable from primary but leave in jack.
    • Unplug all cables from old secondary firewall, leave in jacks.
    • Push all cables into new primary firewall.
    • Check continuous pings to determine which devices need a clear arp.
    • For each device that needs work, console in and execute the last command, which should be 'clear arp'.
    • Verify ping connectivity and customer data flow.
    • If we're still going to engage in customer contact, begin that now.
    • Move secondary firewall cables from old unit to new unit.
    • Verify failover communication.
    • Maintenance complete.