Proactive User Experiences with Dan Poltar, Jimmy Vo, Anthony Lopez, and Kyle Baptiste

Jimmy Vo (top, left), Anthony Lopez (bottom, left), Maple So (center), Dan Poltar (top, right), and Kyle Baptiste (bottom, right). Graphic: Rebecca Wilkerson

Interviewer: Maple So

Welcome to a new episode of Machine Learning, Artificial Intelligence, and Data Science in the Knowledge-Driven Enterprise. Our host, Maple So, discusses business drivers, future vision, technologies, and the framework supporting proactive user experiences in information technology.

Proactive user experiences in Information Technology applies Artificial Intelligence to reduce end-user friction. In this episode of the MITRE Knowledge Driven Enterprise Podcast, Dan Poltar, Jimmy Vo, Anthony Lopez, and Kyle Baptiste share their insights on efforts to enhance the MITRE Service Desk.

Resources mentioned in this episode:

SupportWorld Live by HDI
White, Steve. 29 June 2020. “How to Scale Remote-Worker Services and Support Through ITSM.” Gartner. https://www.gartner.com/en/documents/3986923
Abbas, Amna, et al. “Chatbot Accessibility Playbook.” MITRE Chatbot Accessibility Playbook, The MITRE Corporation, 2021, https://www.mitre.org/sites/default/files/2022-06/pr-22-1494-chatbot-accessibility-playbook.pdf.

Click below to listen to podcast:

Podcast transcript

Maple (00:17):

Hello, everyone. Welcome to MITRE’s Knowledge-Driven Enterprise podcast. I am your host, Maple So. For today’s topic on Machine Learning, Artificial Intelligence, and Data Science, today I’ll be discussing proactive user experiences with Dan Poltar, Anthony Lopez, Kyle Baptiste, and Jimmy Vo.

Dan, Anthony, Kyle, and Jimmy, would you like to introduce yourself and briefly share with the listeners about your role and experience at MITRE?

Dan (00:48):

Sure. My name is Dan Poltar. I’m a Portfolio Manager within our corporate IT organization, managing our operational intelligence portfolio. My portfolio is focused on driving the adoption of innovative solutions and using operational data sources to sense our environment so that we can provide operational intelligence to system problems, which increase our speed to mission impact for not just our user community, but also our operational teams.

Jimmy (01:15):

Hi, I’m Jimmy Vo. I’m the Department Manager for the Service Desk. My department’s made up of our Service Desk, our Digital Adoption team, as well as our Problem Management team. I’ve been in the industry for about 22 years. I pretty much started right after college. I’ve lived the entire life cycle of a Service Desk agent, so I know the ins and outs. I also went over to our brethren in cybersecurity at InfoSec for a short stint, and then I came back to run the department. And now I’m looking for innovative ways to help our users even further.

Anthony (01:56):

Hi, everybody. My name’s Anthony Lopez. I’m a Technologist for MITRE. I work with our corporate IT division, and I work for Jimmy at the Service Desk. So, making sure that our team has the right tools and processes in place to meet our customer’s needs.

Kyle (02:10):

Hello everybody. My name is Kyle Baptiste. I am a summer intern here at MITRE. I am studying up in Rochester, New York at Rochester Institute of Technology. I’m getting my Bachelor’s in Innovative Technology Development and my time here, I’ve been working with Anthony, Jimmy, and Dan on assisting them in providing the innovative and proactive IT engagement to help our end-users.

Maple (02:33):

Thank you for everyone’s introductions. All right, Dan, can you share the business drivers associated with proactive user experiences?

Dan (02:43):

In general, our concept can be applied to several domains, such as, Retail, Finance to name a few. Today I’m going to focus–as you’ve heard on the introductions–I’m going to focus on IT service and support. So, to answer your question, our primary goal or North Star is ready to evolve and continuously mature our IT services and support that we provide so that IT is not necessarily front of mind with our user community day to day. We want them focused on the work at hand and not necessarily IT all the time. We want to identify and reduce any friction points across IT before they occur or impact someone’s experience. Really being able to engage our users in the moment while enabling them to be more productive and efficient. But let me also provide a little bit more clarity with regards to part of your question around the proactive nature. We’re in fact focused on many aspects that cross the spectrum of proactive, predictive, and prescriptive.

Dan (03:43):

I’ll talk about proactive engagement first. That’s where we’re developing and explore different technologies that enabled us to act before something happens or before someone thinks they need help. And a very basic example of proactive engagement, so that everybody’s familiar with it, would be when someone’s hard drive is filling up. Well, if that situation is not addressed, that could introduce a number of issues that impact that user’s experience. There are other examples where providing guidance based on someone installing a new application in the moment when they need it is another aspect where proactive engagement can benefit. The last example I’ll give in this proactive nature is when we have someone join the organization or move into a new part of the organization, we can sense that change and we can invoke proactive measures to help them onboard, understand their new group they’re working with, or the people that are working around them.

Dan (04:43):

When you think about predictive elements, we’re looking to integrate experiences that forecast potential issues to which we can resolve through automation or by providing self-service guidance. An example that I’ll use today is most enterprises leverage some form of a VPN to connect their business. In some cases, those sessions are set to expire and require re-authentication. This situation can be exasperated or problematic for some users when that session is set to expire during a critical business meeting or similar task. By leveraging data about our users, in this case calendar free busy information, coupled with information about our user’s connectivity status, we can predict when a conflict may occur and that provides us the opportunity to influence that moment and really take an action that can reduce a potential negative user experience. In this case, initiating contact with that individual to reset their session before that upcoming event makes it a more promising user experience.

Dan (05:48):

The other element and last element that I’ll talk about in my response is prescriptive, which leads us to develop models or solutions that enable us to surface deeper insights and understanding about attributes that contribute to some of these events. The most basic example would include things like what other users are in the same situation at that moment. Are they in the same geographic location? Are they using similar devices? Do they have the same configuration or software? Those attributes assist us, not just identifying the impacted community, but also narrowing our engineers and service owners to the specifics of a situation which reduce the time in which our users may be impacted. These three components all contribute to us developing a framework, which you’ll hear more about later, which elevates our user experiences, reduces the demand on our human agents, which allows them to focus on our more complex work, while also leveraging the ever-increasing amount of data that’s around any enterprise these days.

Maple (06:52):

Thank you, Dan. How is your effort related to IT service support and Artificial Intelligence?

Dan (07:00):

So, there are several technologies and techniques we are using, but also experimenting with to continue evolving our user journeys. First and foremost, our virtual agent solution, which is a conversational AI platform that understands natural language but can also manage dynamic conversations so traditional user conversations can go in many branches in any moment based on the need. So, having that dynamic ability to understand that natural language is very important. We have previous experiences with virtual agents, which helped us realize that traditional solutions based on dialogue trees, for example, are difficult to develop, costly to manage, and not susceptible to scale. They also lead to conversations that come across as scripted and lead to dead ends or less than desired experiences. I think we all can probably relate to an experience we’ve had with a virtual agent where it did not go as well as planned. In addition, we’ve all witnessed in the last couple years how the pandemic has changed, or you could say accelerated, how we communicate and collaborate with our peers.

Dan (08:07):

Recognizing this, we’ve embedded our virtual agent into that ecosystem. So, our users are not just organized around the tool they use, but they’re also accustomed to that tool and familiar with it so it’s easy to use. We’re also scaling automation into our solution, eliminating the repetitive work that intelligent systems are great at or becoming better at every day. But typically, those tasks are not those that humans prefer to do, and they actually want to avoid them, in some cases. Another significant aspect–which most organizations are in the process or have realized–is that data: they’re sitting on a ton of data. It’s constantly being generated and it’s changing. So, we’re realizing that leveraging our data assets, treating them as strategic assets, in my opinion, is going to serve as the accelerator and key driver for a number of organizational transformations, regardless of industry. There was a quote I came across recently, I think it’s maybe a couple years old now, but it came out of the Executive Office of the President, I think it was the National Science and Technology Committee.

Dan (09:11):

It’s along the lines of, “An AI-enabled world demands data in a data literate citizenry.” That really stuck with me and further led me to think about that problem. I then took some courses at MIT (Massachusetts Institute of Technology) and I attended a course labeled “Data Monetization”. I walked away from that really thinking that I needed to contribute to our organization’s data maturity, but also demonstrate what is possible with data. So, developing that deeper understanding and appreciation for data and serving as a champion across our enterprise about the opportunities data could provide. As we talk about data, keeping data top of mind, we’re actively working to mature our data understanding, including things like definitions, catalogs, making sure data’s accessible to the organization, but most importantly (in my opinion) make sure our data maintains some sort of governance structure. These solutions and models we develop are only going to be as good as the data that we’re feeding them, so our data strategy must remain a focus.

Dan (10:15):

So, Maple, in summary for what I’ve talked about this morning, we are integrating and experimenting with a variety of techniques and technologies with a couple things in mind. We want to provide exceptional user experiences that not only solve problems for our end-users before they happen or while they’re happening, but also make sure that they’re personalized, in time, and in context. We want to enable actionable operational insights and solutions which increase our efficiency as an IT organization. We want to enable capabilities that are always available regardless of time or day. We don’t always have someone available to help.

Dan (10:54):

As we’ve learned, people are working different ways, different places, different time zones, and so we want our solutions to be available no matter time or day. We want to experiment, we want to innovate, and we want to explore these new technologies, so that we can demonstrate the art of the possible. While doing that, we provide our team members additional opportunities to grow. They get to experiment with these new technologies, get to learn from what works and what doesn’t, really developing stronger, better solutions. And lastly, everything we do within Corporate IT, whatever we develop or whatever we learn, we want to be able to share that across MITRE with our sponsors and partners so they can leverage what we’ve learned or what we’ve developed.

Maple (11:37):

Jimmy, can you introduce us to MITRE’s Service Desk and share your vision for service and support?

Jimmy (11:44):

Yeah, absolutely. So, as I said earlier, I’ve been with the Service Desk since 2000. I’ve seen it actually change. We started off as a help desk handling just Tier 1 incidents and requests that come in. Very structured and pretty much by textbook of what a help desk does. People have problems, they call us, and we try to resolve. We’ve evolved a lot. We’ve put in a lot of best practices. We’ve grown our team and what our capabilities are and expanded in a lot of our services. So, we serve as the front-end face for our entire IT organization called ECIS, which stands for our Enterprise Computing Information and Security.

Dan (12:27):

Our ECIS organization provides our employees all of their systems, mobile devices, applications and services as well as our infrastructure in order for them to do their jobs. So, our Service Desk is here to support our employees when they’re challenged with any IT issue, whether it’s knowledge and guidance to use a product or service or fix an issue when an application or service breaks. The vision is to make IT as frictionless as possible for our employees so that they know how to use our products and services, but also when they have issues, they’re addressed before they’re even impacted. Truly, we want to make the experience for our users as frictionless as possible. Things should just work. That’s our vision. That’s what we’re aiming to do.

Maple (13:12):

Thank you, Jimmy. So, what is the overall vision for ECIS efforts in Information Technology support and reducing end-user friction?

Jimmy (13:24):

Sure. So, you know, speaking from my level, I would say our vision from an ECIS perspective is to provide a great IT experience at MITRE where employees can utilize all of our products, systems, and services effortlessly to help make the world safer. That’s MITRE’s mantra, that’s what we know our mission is. So, we work with our employees to do so. We get about 130,000 contacts a year where employees are reaching out to us because they have IT issues or have service requests. I see those as end-user friction points. Every time they have to call us, that means an end-user is having an issue with either our IT or our services. So, we track these service metrics–such as customer satisfaction, average speed, answer-abandonment rate, and talk times–as well as tracking operational metrics–such as first contact resolution, first level resolution, and ticket-to-contact ratio.

Jimmy (14:21):

We track and measure these Key Performance Indicators (what we call KPIs) to measure our service and how we’re performing. But we’re now transitioning to tracking, capturing, and measuring the user experience. Because when it comes down to it, they’re the most important aspect of our service. As I said, each call is an end-user friction with our IT, so something that they couldn’t fix themselves or maybe not know where to find the information to fix it, or an issue with the product or service that we operate. So, it’s great if each interaction with our Service Desk is fantastic, right? I expect my team to provide great customer service, to be professional and to resolve all the issues that come in. But if that one employee had to contact us more than 10 times a month, that’s a bad IT experience for that person. So, we should be able to identify and work to proactively fix their issues before they even have to experience it or be impacted by it.

Jimmy (15:21):

So, our team’s looking at proactively identifying potential friction points, defining what they are and how they affect our employees, and addressing them before it impacts our employees either by back-end changes or updates where we fix an issue before it affects our employee, or we notify them before they’re unable to work so that they can fix it themselves before they even have to call us. By leveraging technology like AI and Machine Learning to capture and collect data that may indicate potential issues, we can be innovative in how we address incidents and identify systemic problems, but then work with our service and product managers to address them proactively and take action so that we can resolve these issues before they impact our employees.

Maple (16:09):

Thank you, Jimmy. Anthony, can you discuss the technologies and framework supporting this proactive effort?

Anthony (16:17):

Yeah, absolutely. So, a lot of the core technologies here center around operationalizing all this data that we’re collecting. So, first and foremost, that digital experience monitoring. There’s a lot of tools in this space, also called was DEX, D-E-X, that just collect timing information and grab logs from endpoints. But we also want to be able to capture the data from the network sensors, network logs, as well as our identity platforms. So, digital experience monitoring is a tool set or class of tools and then you also have the ability to aggregate those logs. So, think of any sim tool, would be a good example of that log collection capability. To act on that data, we need tools within the Robotic Process Automation space (so, RPA), and then to engage with our users, our communication channel of choices, our virtual agent. So, those are the core technologies that we need to actually operationalize this data.

Anthony (17:18):

On top of that, we’ve put together a framework that allows us to more quickly compose campaigns, is what we’re calling them, which are automated engagements. So, what makes up the campaign is first and foremost, ‘What is the impact that the issue we’re going after? What is that impact of the issue to the end-user?’ So, is it maybe endpoint performance? Application performance? Is it endpoint stability? Is their computer crashing? And then next to that, the next column over, if you look at it like a matrix, the next column over would be the data source.

Anthony (17:55):

So, where are we exactly getting the trigger to identify that user impact is coming? And then from left to right, the following columns would be all the different ways that we can remediate or prevent that issue from happening. So as Jimmy mentioned, a lot of times we’re looking at a campaign, we identify a specific error that results in a downstream impact–so some end-user impact, like system stability—and, if we can, we would very much prefer to just prevent the issue from happening in the first place. So, it’s not only the tools and technology, but also that process and working with the engineers to see if we can prevent the issue from happening in the first place by some baseline configuration, for example. So, those are some of the tools and technologies, and this framework that we’ve put together to help us compose these campaigns.

Maple (18:46):

Thank you, Anthony, for sharing the technology and framework supporting this effort. So, can you share some detail on the methods used to engage your user community?

Anthony (18:58):

Absolutely. So, for these campaigns, our chosen method to engage within users is through our virtual agent. So, we have a chatbot. So, the framework kind of drives the campaign. So, as we detect an issue on an endpoint or at an identity system that issue–the detection method–is mapped to a remediation action. So, like, a knowledge-based article–and the end-user is sent a direct message through a chatbot with what they can do to fix that issue, the impact of that issue, and just a short snippet that will help them with identifying that this is something that you should take action on. So, our chosen communication method is our virtual agent within MITRE.

Maple (19:42):

As you deploy, Anthony, what are some of the benefits and/or challenges you’re experiencing or anticipating in the future?

Anthony (19:51):

Yeah, that’s a good question. We’re constantly running into new challenges. So, one of the most interesting benefits of proactive engagement is we can short step, or shortcut, short circuit the triage process. So, by using the data to drive the engagement, we’re kind of shifting the paradigm. The customer is no longer contacting us with what they interpret the problem to be. We’re initiating the engagement by using that data to drive what is the issue and then exactly how can we remediate it? So, we don’t have to do a lot of the triage anymore, assuming that we have good data triggers that are mapped to good solutions.

Anthony (20:35):

So, it’s really shifting the paradigm of how we provide support by again, mapping those known bad situations, those known bad conditions, and mapping them directly to a way that the customer can remediate issue. So, one of the challenges though with this, is that we do need to be very aware of how noisy we are being with these campaigns. We don’t want to cause too much messages to our end-users because then they’re just going to mute the ‘bot and not receive any of the notifications. So, we want to be very aware of exactly how many messages are actually being delivered to end-customers and make sure that we do set good “cool down” timeframes for these campaigns. So, that’s one of the cool benefits of this kind of engagement model and one of the challenges.

Maple (21:18):

Thank you, Anthony. What are some resources or references that may help our listeners learn more about proactive user experiences?

Anthony (21:29):

Sure. Yeah. So, there’s a conference that we try to go to. They just recently launched it again, “Support World Life” by HDI. So, a bunch of vendors got together, a lot of people from the industry do talks on what they’re doing new in their environment. So, there are good presentations at that conference and I’m looking forward to the next one. The last one just happened in May. So, probably around the same time next year. I would take a look at that. As we shifted to remote work, there was a really good Gartner article on scaling remote worker services and support through ITSM, which covers a lot of the core fundamentals that you need to do this kind of engagement. So, like, Service Desk portal design; how to do good self-service; and how to do the basics: the core basics and fundamentals of knowledge management.

Anthony (22:20):

And all of this is centered on managing the knowledge that you have [and] mapping it to those known indicators of a compromised user experience. I would also look at some digital employee experience tools. Those are critical to what we’re doing here at MITRE, as well as Robotic Process Automation that you can leverage to operationalize that data. Lastly, there’s also a MITRE chatbot accessibility playbook that you can get from the MITRE website. I’ll provide a link … which provides some good guidance on how you can build a good user experience that’s also accessible right within your organization. So, those are some of the resources I recommend that anyone interested in this take a look at.

Maple (23:03):

Thank you, Anthony. We’re joined by one of MITRE’s 2022 summer interns, Kyle, who had the opportunity to explore and develop solutions in support of this initiative. Kyle, can you describe your experiences and discuss some of the work you pursued this summer?

Kyle (23:21):

Yeah, no problem. So, one of the things that I really appreciated with working at MITRE is just the environment that they have available for everyone working here. I have one experience previously working at a different company as a summer intern. And while that one was very fun, it was more set up in a way that you’re kind of working in your department and those are the only people you’re interacting with on a day-to-day basis. And you have a lot of freedoms, but you just have lot of freedoms in your one area. Coming over to MITRE, MITRE has a really big initiative to inspire people to collaborate with everyone because that’s the easiest way to get innovation happening all around. And to that end, people are able to work kind of with anyone when it comes to working on different projects here and there, depending on what the project is and the security and other things like that.

Kyle (24:18):

But if it’s available, you’re able to just go around and have brainstorming sessions with people at kind of anytime you need to. So, I appreciate [that] because I was able to get, in my words, like, a larger sandbox in the company to interact and collaborate with different people in different departments that I didn’t even know existed previously. And being able to interact with those people was a really refreshing experience overall, because everyone is able to give immediate feedback on what you’re working on. And in that little on-demand brainstorming session, you’re able to come up with brand new ideas that you might not even thought of on your own. So, you can be working on a project to go to lunch, be talking to a coworker about something in your project, and then once you leave lunch, have a brand-new idea or a point of view to attack that problem at or just a new way of developing your project.

Kyle (25:18):

So, that has assisted me in my process throughout the entire summer by allowing me to rapidly prototype different ideas and different things almost every day because I was able to come into the office with an idea on how I want to continue working on it. And then just throughout the day by talking to different people, I’m able to change it, adapt it, evolve it in different ways, and be just extremely agile. And what’s allowed me to also utilize a lot of the tools that MITRE had available. MITRE has a vast amount of tools and services that they have access to, and there are a lot of knowledgeable people who know a lot about said tools or services. So, as you’re going around and adapting and evolving your project on a semi-day-to-day basis, for me–because I was just talking to almost everybody–I was able to meet a person, get feedback on my project, their input on what directions they think I might need to look into, and also might even learn about some brand new tools that are available to me that I might not even know existed.

Kyle (26:25):

Which was intriguing to me, and it helped me a lot throughout my entire process. And lastly, for some of the projects I was able to work on using this whole process of talking to people, rapidly prototyping, evolving it. I was able to assist Anthony with [the] framework, developing it, and utilizing the prototype framework that we have and develop it into a proper framework that we can use for different campaigns in the future (and even the current ones), so we can continue to use it more and more. Because we know it’s a tried-and-true method and it’s a nice template to begin with because as we find new data points and new areas that we know a problem’s going to happen here, we can then take it and put it into the template, map everything up, and then have a proactive use case available for people to get messages predictably and proactively. But also making sure we’re not making it too noisy. So, adding in different features here and there and kind of evolving it as we work on it was a very, very fun experience.

Maple (27:29):

Thank you, Kyle. Thank you for sharing the initiatives, experiences, and resources on proactive user experiences. It was a great pleasure getting to learn more about this topic. Thank you, Dan, Jimmy, Anthony, and Kyle. I’d also like to thank MITRE and the Knowledge Driven Enterprise for making our podcast possible. And again, thank you Dan, Jimmy, Anthony, and Kyle for sharing on the podcast today.

Maple So is a Data Analytics Engineer. She is curious about Artificial Intelligence and Machine Learning applications in enhancing user experience in the Information Technology domain.

MITRE’s mission-driven team is dedicated to solving problems for a safer world. Learn more about MITRE.

Thinking About Thinking While You’re Thinking

My Path as a Techie to Learning About Equity

Find My Colleagues

Building Resilient Teams and Cross-Collaboration Between Different Organizations, with Renee Rookwood

Great Power Competition – New Capabilities for New Challenges, with David D. Perkins

MITRE Platforms to Accelerate Mission Impact

Project Demodocus: Bringing Accessibility to the Masses

Encouraging the Next Generation of Innovative Thinkers

New book teaches how to apply agility principles to organizations

Sustainable IT Modernization: Unlocking the Secret to Success

Applying Design Thinking to Boost Federal Agency Problem Solving