Bloxy AI Chatbot
The rapid emergence of AI prompted Infoblox to embark on a proof of concept that we could build our own large language model service for our product. We wanted a use case that would not evaluated the technology, but also one where we could evaluate customers’ perceptions of the usability, accuracy, and value. We chose a chatbot use case as it was a means of rapidly evaluating our solution.
We found that users found the solution to be useful and trustworthy, although the content is limited to specific scenarios. After design iterations, 90% of users tested could successfully interact with the chatbot.
Design Objectives
Users consider content useful and valuable
Responses are deemed to be valid
Users can successfully interact with the chatbot
Chatbot Scenarios
The chatbot - named Bloxy - leveraged the large volume of help content we had to add configuration and troubleshooting of of networks. Some common tasks Bloxy supports are:
Configuring DNS fallback mechanism
Configuring Data Connector
Getting DHCP Scope / IP utilization data in a high level view
Debugging / Troubleshooting issues in network (OPH, DNS & DHCP)
Log analysis (Lack of clear & relevant information for debugging)
Understanding status / health / performance of the network
Network provisioning i.e. Deploying VMs
Understanding last known good configuration after something breaks
The intention is for Bloxy to answer questions that our customer support personnel often answer.
“Most of the questions that we get from customers are from a troubleshooting perspective, right? They have something that is broken or not working, and then they contact us to get it fixed. So that’s like the majority of the cases that I would say up almost around like 70 to 80%.
”
Initial Concepts
Final Concepts for Beta
Early designs revealed some flaws:
too text-heavy
hard to scan
no clear call to action (CTA)
did not repeat the prompt
Our pilot designs reflected improvements
tabular information rather than straight text
clearer CTAs
clear repetition of user’s prompt
Results and Learnings
Users found content useful and trustworthy
People prefer using the chatbot to looking up content themselves
90% of users successfully interacted with the chatbot
In addition to standard configuration support, users want more advice based on their environment
People want content to be personalized
“Where can I create a network of x size? Or where do I have continuous address space of x amount?”
“The things that that network person needs versus someone in security, they are quite different. I’m sure there’s awareness of who’s using it.”