United Airlines has moved to synchronise its patch cycles across Windows and Linux servers and cut administrative effort out of applying updates to applications running on top of its infrastructure.
The airline operates three data centres in Chicago, Denver and San Francisco, which collectively house nine Oracle converged Oracle SuperClusters, as well as HP blade servers running Oracle Linux.
It also has a large number of Windows systems, many of which it inherited via its 2010 merger with Continental Airlines.
Unix engineering senior manager Marshall Weymouth told the recent Oracle Openworld 2017 conference that the airline's security team wanted the Windows and Linux teams to synchronise their patch cycles.
This was primarily to reduce pain for sysadmins, security, change management, application teams and airline customers.
"Patching and updating meant rebooting servers that would affect applications co-ordinating 5000 daily departures for our 700-plane fleet," Weymouth said.
“When you say 'reboot' to change management people, they panic. You need 10 days to discuss it. Then you have to bubble it up to application management teams and that was [all] pretty painful."
On the Linux environment alone, patching took up 54 hours of sysadmin time over a 28-day period of rolling upgrades. That typically resulted in about seven hours of downtime, which affected end users and travellers.
Updating applications hosted on the infrastructure was also painful; an update to the main United.com website usually required updates to application files and code additions to the underlying database.
United tested Ksplice, a technology acquired by Oracle, to support its patching process. Ksplice is primarily used to patch runtime security vulnerabilities and stability bugs.
Weymouth said the tool allowed him to meet security compliance without having to "go through a lengthy change management process that adds a ton of operating expenses" to the process.
He said sysadmin time demands had dropped from 54 hours to 7 hours per cycle, and that patched or updated servers could continue to operate without requiring a reboot.
Weymouth said Ksplice could also be used to roll back updates from any point in "microseconds" using a simple command.
He said that the increased incidence of nation-state attacks on large organisations meant there was now more pressure to improve security and protection processes.
“We were being pushed really hard by the security team,” Weymouth said. “We knew we had to close the security gaps."
Sholto Macpherson travelled to Oracle OpenWorld 2017 as a guest of Oracle.
