The Commonwealth Bank of Australia is naming its machine learning models after superheroes and has them "competing with each other every day" to provide the best customer service outcomes.
The bank is in the process of accelerating its use of machine learning through a partnership with H2O.ai that it hopes will produce both superhero-like models and model builders.
Senior product owner Avinav Goel told a H2O.ai webinar yesterday that the “people aspect” of the program is “easily [his] favourite” part.
He said the bank has “literally gamified the art of modelling among our analysts and decision scientists”.
“We told them, 'Hey, you are not building propensity models, you are building 'superheroes' who are going to compete with each other, every day for that better customer response," Goel said.
“We had a 'Captain America' for the benefits finder model, leading the charts in week one, while 'Thor', the business model, beat him hands down in week two, and 'Ironman' starting with smashing results in weeks one and two, but then dipping in week three, telling us that, 'Hey, this calls for re-scoring the model'."
Gamifying the program had resulted in more insights being created and shared by teams, and had also resulted in a "community of learners in the bank, across geographies.”
Goel said the CBA teams would ask questions and gain answers from one another “in real time, through group chats or discussion boards.”
“These mates, if I can call them [that], and their 'superheroes' [models] exist in a proud universe in CBA. We call this the ‘Superverse’," he said.
Scaling up CEE
One of the bank's earliest AI/ML endeavours was the customer engagement engine or CEE, which suggests next-best conversations to have with each banking customer.
While CEE is equipped “with adequate science, business rules and adaptive models to personalise” conversations today, Goel said the bank wanted to also make it future proof, but was acutely aware of the amount of resource required to do so.
This led into its adoption of the H2O.ai platform and the considerable upskilling effort now underway across the bank.
“Done the right way, we knew this will also bring analytics closer to the rest of the organisation," Goel said.
Goel said the first H2O.ai integrations into CEE started small and experimentally, with the aim to increase the proportion of customers that engaged with a conversation, offer, or other actionable item being put in front of them.
Prior to the changes the CEE had produced “all in all, a good customer outcome”.
However after deploying the new models, “two out of three customers in the H2O segment were encouraged to take action, highlighting the success of using H2O.ai as a path to improve on the CEE.
The bank said it had seen similar engagement improvements with other existing tools, such as its 'benefits finder' government rebate search tool, a 'saving habit' tool, and its 'bill sense' payment prediction product, by augmenting them with models built in H2O.ai.
