Woodside Energy is in the process of replacing its continuous integration and infrastructure-as-code tools, shifting instead to cloud-native AWS services.
Senior cognitive and cloud solutions architect Luke Evans told a recent event in Perth that the gas giant had a “bit of a love-hate relationship” with parts of its DevOps toolchain, singling out CI/CD server TeamCity and infrastructure-as-code tool Terraform.
“TeamCity builds our packages [and] Terraform does our deploys," Evans said.
“The spoiler alert is that I actually still really like TeamCity and Terraform. I don't think there's any problem with those technologies or those tools.”
However, with both tools forming part of the process for getting code into the cloud, Evans saw room for improvement.
“It's all about the little things,” he said.
“There were some little things that we hadn't noticed before ... that started to surface.
“Like any team, we want to improve and get better, and we want to mature the relationship that we have with our technology and our tools.
“So we wanted to do more, get more value out of our tools and we wanted to add more automation so we didn't have to do menial tasks.”
Evans said that automating processes felt harder than it needed to be.
“The TeamCity API wasn't quite the best, and the Terraform Enterprise API wasn't much better,” he said.
“It felt like what we were left with was a ‘sticky tape and glue’ style architecture. It just didn't feel right.”
Evans said he also wanted to get away from having to be hands-on with server used to host TeamCity.
“I don't enjoy managing servers at all; kudos to those people who do,” he said.
Evans also wanted a solution to the “dreaded TeamCity build queue”, a reference to the propensity for TeamCity users to have problems with delays or otherwise optimising the queue to get new code into production.
“If you have TeamCity you would know [about] that,” he said.
“With TeamCity, you're limited by your licensing power. Because we were sharing this TeamCity instance with other teams, it was either our build was holding them up or they're build was holding us up.
“We all just want to code. We don't want to have to manage/stroke the TeamCity server.”
Evans also believed that the existing tools were “a little bit expensive” and, with multiple teams using them, it was hard to apportion costs.
“Because we were sharing it with other teams, it was hard to know who should pay for what and who was paying for what,” he said.
The company is now taking “the big plunge”, replacing TeamCity with AWS CodeBuild and Terraform Enterprise with AWS CodePipeline.
The move is gradual; “we didn't change all our components at once,” Evans said.
“Even now, we still have a couple of components using TeamCity.
“So it's a journey but we’ve made the leap to do that.”
So far, Evans said that the shift had been valuable in lowering costs (“we only pay while our build is running”) and reducing blockages.
It had also been easier to automate more parts of the process.
“We don't need voodoo magic to be able to stitch pipelines together to create that ‘glue and sticky tape’ pipeline,” he said.
“We've swapped that for CodePipeline, and it means we can run integration tests, performance tests and any other tests or anything we want to do sequentially or parallel in that one pipeline, which makes it easier to automate.”
Evans noted the transition was still a work-in-progress, in part as Woodside waited on new functionality or capabilities to be introduced into the AWS tools.
“No relationship is successful without compromise, and although we are well on our way to being happily ever after I believe and we'll get there, there are a couple of gaps in CodeBuild and CodePipeline,” he said.
“For example, versioning is a little hard in CodeBuild, but we just find ways to work around it. Also, the process of rolling back in CodePipeline is not entirely ideal. It’s a little bit manual - but we think that “The pros certainly outweigh the cons, in our case.
“We're betting on AWS improving CodeBuild and CodePipeline. We think it's just a matter of time [before we get to where we want to be].”
