Interview with Dan Ward, Rachel Gregorio, and Jessica Yu on MITRE’s Innovation Toolkit
Rachel Gregorio (left front), Jessica Yu (left back), Aileen Laughlin (seated), Dan Ward (Center), Stephanie Medicke (right), Kaylee White (right back), and Niall White (left front). Photo by: Jae Robinson
Interviewer: Cameron Boozarjomehri
Welcome to the fifth installment of the Knowledge-Driven Podcast. In this series, Software Systems Engineer Cameron Boozarjomehri interviews technical leaders at MITRE who have made knowledge sharing and collaboration an integral part of their practice.
Anyone with experience facing an important challenge or project understands that the job is easier when you have the right tools. The Innovation Toolkit (ITK) is a collection of methods and techniques curated by MITRE experts to help teams be more innovative. These tools stimulate insightful conversations by asking important questions and providing frameworks for capturing the results. Join Cameron as he interviews ITK team members Dan Ward, Rachel Gregorio, and Jessica Yu about collaboration and innovation at MITRE.
Click below to listen to podcast:
| Cameron: | 00:14 | Hello everyone, and welcome to MITRE’s Knowledge-Driven Podcast. I’m your host, Cameron Boozarjomehri, and today I will be discussing the Innovation Toolkit with MITRE’s Toolkit team. Would everyone like to go ahead and introduce themselves? |
| Jessica: | 00:26 | Hi. This is Jessica Yu. I’m a human factors engineer here at MITRE. |
| Dan: | 00:30 | Hi, everybody. I’m Dan Ward. I’m a systems engineer here at MITRE, although my business card says Innovation Catalyst. |
| Rachel: | 00:35 | My name’s Rachel Gregorio. I’m also a human factors engineer here. |
| Cameron: | 00:39 | It’s awesome to finally get to talk to you all. I’ve been really excited for this interview, mostly because I’ve heard nothing but amazing things about the Innovation Toolkit. I think I mentioned before we started recording that Tammy Freeman has sung your praises during our little interview, and I’ve been quite excited to get this one on the record. Let’s just jump right in with a little bit about your team and kind of how you all came together. |
| Dan: | 01:01 | Yeah, sure. We’re very much a multidisciplinary team, so we come from a lot of different backgrounds and represent three different departments, or four, maybe, now. We initially got to get together, frankly, to solve our own problems. We were working on a project together and realized that a lot of the approaches and techniques and methods that we use for making decisions and solving problems and delivering new capabilities were … we had some things in common with these approaches. |
| Dan: | 01:28 | We thought, “Gosh, wouldn’t it be nice if we were able to write this down and share them with each other a little more smoothly?” Then we realized, hey, if this is stuff we’re using and we’re sharing with each other, what if we shared them with other people in MITRE and then other people outside of MITRE? That’s sort of the origins of the Innovation Toolkit. |
| Rachel: | 01:46 | We had a team member who actually came across the concept of innovation toolkits, which are not new. They’ve been around. Industry, academia, are using them already, and what we found, checking out the existing toolkits out there, was that none of them were really tailored to meet MITRE’s needs, whether it be MITRE employees or MITRE sponsors and our customers. We wanted to make something that would really speak to them and feel comfortable and accessible for anyone. |
| Jessica: | 02:15 | We ended up actually looking out across industry and academia at different innovation toolkits and action toolkits, and did a survey. Brought back about, you know, 300 different tools, and then we began to curate that list and evaluate the different types of tools that we found to choose the ones that we thought would be the best fit for MITRE’s culture, and to use to work with our sponsors. Today’s Innovation Toolkit is our first iteration and our first try at coming up with a collection of useful tools. |
| Dan: | 02:50 | There’s about 24 tools in that kit, if I recall, and some of them are drawn from industry and kind of best practices. Others are tools that we developed ourselves, either from our own practice or our own research or just collaborative. We’d say, “We need a tool that helps us do this. It doesn’t exist, so we’ll make it and test it and play with it.” |
| Cameron: | 03:09 | What was the … I don’t want to say the organic moment, but what was kind of the question? What were the kind of needs that you saw MITRE not being able to fill, that you thought the Innovation Toolkit would be beneficial to bring to MITRE? |
| Rachel: | 03:22 | We were working with a company through MassChallenge, which is an incubator program that MITRE partners with, startups in the Seaport, trying to find applications for government, identifying ways to apply human-centered engineering practices to the products that they were developing, whether it was ergonomically or developing something that would work well for the people that were going to be using it. We all agreed that this was really a perfect opportunity for an innovation toolkit, had we had one, and I think that was our … that was kind of our aha, light bulb moment, where we said, “You know what, we should really make our own that works for our culture,” like Jessica mentioned. |
| Jessica: | 04:05 | Yeah. I think, as Dan mentioned earlier, we were using a lot of these techniques and strategies on our own project work, and they were really successful, but there are a limited number of us and a limited number of folks that we found knew how to do this, and we can’t be everywhere all the time. We came across folks that wanted help with creative problem-solving and trying to figure out like what the next step could be in the most innovative way, and the idea of having a toolkit and resources that we could give someone to use to get started, like in between the time when they told us about this problem and we could actually start working with them, we thought would be helpful. It was really trying to increase the capabilities of the general MITRE population, so that more folks could do this. |
| Dan: | 05:02 | Yeah. Kind of our two taglines we use a lot, we’re trying to democratize innovation, so we’re trying to help equip people with the skills and the tools and the understanding of what innovation is, what are the barriers to innovation, how do you overcome these barriers, what are the principles and practices that go into innovation. Really, innovation is not just a small group of people who own innovation. It’s really everybody’s job, so we want to democratize innovation. The other tagline we use a lot is to go beyond brainstorming. |
| Jessica: | 05:31 | I think a lot of our tools also are meant to bring people together. You can use them individually, but we also saw that these canvases and templates and tools could bring the team together for conversation, discussion, ideation, but also to make sure that after that, everyone was literally on the same page. |
| Rachel: | 05:53 | Consensus building. |
| Cameron: | 05:56 | I think this whole thing that a lot of people still are confused by is what is the toolkit. I have the toolkit up right now. The MITRE Innovation Toolkit is, “To help our colleagues, sponsors and collaborators jump start the innovation process.” I honestly would like us to expand on kind of what that really means in terms of the tangible things I can do when I come to the toolkit. |
| Rachel: | 06:17 | I think the best place to start in the toolkit is the Categories page, where we’ve made an initial grouping of the tools based on, like you said, Cameron, what you can do throughout the innovation process. One piece of it is the Generate Ideas, the Come Up With a Million Thoughts, Quantity Over Quality, Go Wild and Crazy, but then we also have categories for Understanding Your User, making sure that whatever product you design is designed for the right person and is actually going to solve their problems and challenges. |
| Rachel: | 06:47 | There’s a Frame Problem section where we actually help you walk through the process of identifying the problem that you’re looking to solve and asking that “why” question, why are we doing this, which oftentimes … you’d be surprised … people skip. There is a section about Reducing Complexity, so after you go wide and think about all the crazy ideas, you narrow it down to, okay, what’s feasible, what can we do within our constraints, what will actually work. |
| Rachel: | 07:17 | That’s exactly what we did with the toolkit, right? We’re eating our own dog food here, where we went out, we looked at 300-plus tools, and then we narrowed to a digestible set that was easy for people to understand. |
| Dan: | 07:30 | Yeah. That last category of tools is so important because it’s so underused, this idea of creation by subtraction, the idea of improving things by simplifying them, by taking things out. These tools aren’t that much harder to use than any of the other tools, but because they’re used so seldom, boy, there’s a lot of low-hanging fruit out there where we could make serious improvements really, really quickly by using a couple of these very simple simplifying tools. |
| Jessica: | 07:53 | I think these tools are different than the typical MITRE tool in that they are not software tools. They are low-tech thinking tools. You know, you print out a template and use it at your desk, or you can print out a bunch of them and use it with a room of 20 people, which we did earlier today. It’s meant to guide people through a thought process and to help people analyze situations and ecosystems and stakeholders and problems, so that they can figure out what the next step should be or what the right problem is. |
| Jessica: | 08:27 | We also … you know, once you use one, one tool, there’s an output for that tool, and we have created that concept of tool chains where you can use the output of one tool as the input to another, and then string these tools together to kind of get you through wherever you might be in the phase of a project. |
| Cameron: | 08:49 | I just want to take a minute to touch on your website, because I really appreciate how kind of the first thing you see is just this Let’s Get Started section, where you immediately … like I’m coming to you asking, kind of, “Where am I supposed to start,” and you’ve already done us the courtesy of taking those … I believe you said 300-some different toolkits, narrowing them down to, “I need an idea, I have an early idea, I have a mature idea, I need to identify problems and solutions.” You’ve kind of already broken it down into at what stage of the ideal life cycle am I at, so that I immediately know here are three very common, very popular tools I can use to start looking into this problem. |
| Rachel: | 09:26 | We developed those tool chains that you see on that first page initially for the MITRE Innovation Program, where employees were wanting to get involved and the departments were encouraging people to get involved, but we recognized that need of people not necessarily knowing where to start. I think a lot of people can relate to that problem of just, “I need an idea, I need to think of something, but I don’t know what I need to think about,” so the tools help you think through that process from start to finish. |
| Cameron: | 09:53 | Are these tools … what kind of experience do I need to use any of these tools? Are they purely software? Are they purely printable? |
| Dan: | 09:59 | I love the way Jessica often describes it, as delightfully low-tech. I just love that phrase. They are delightfully low-tech. Most of them are just something you can print out on a piece of paper and use to have a facilitated discussion. Many of these tools are about getting the right people in the right room at the right time and asking the right questions, and having some conversations that aren’t often terribly hard to have unless you don’t have them. Sort of seeding those questions, and helping teams and people work together more effectively. |
| Dan: | 10:27 | Like any tool, mastery takes time, so the more you use these tools, the better you get at them. Even for some simple tools, they can be a little tricky to use the first time. I mean, imagine a hammer, or think of a hammer. The first time you swing a hammer, you’re going to bend some nails. You’re going to hit yourself on the thumb. If you ever watch a master carpenter hammer something, boy, one hit and that nail goes straight in. Again, with any of these tools, the more you play with them, the more you experiment with them, you better you get. |
| Jessica: | 10:54 | Yeah, and I think there are some that are more approachable. If you’ve never had experience with … like the pre-mortem tool, right? It’s basically answer all the questions that are on the sheet of paper, and that will lead you through, right? You don’t need a lot of training there. Something that may be a little more sophisticated is a journey map or a service blueprint, where you’re really trying to detail and document a whole workflow, and what’s on the front stage and what’s on the back stage and what are the users’ experience, and so that might take a little bit more training. |
| Jessica: | 11:25 | Our team is very inclusive, and we’re doing a lot of outreach to folks to say, “Hey, if you have an interest in this and you want to facilitate, please join us, and we’ll teach you how. You can come sit in on our sessions and see how we do it, and then put your own style on it.” We don’t necessarily need to be the keepers of this, right? We wanted to create this so that people can take it and make it their own and tailor it. |
| Jessica: | 11:51 | What you see on our website and that you can download are general templates with instructions about when to use them and how to use them, but what we really love is when people take these tools and then change the wording a little bit so they fit better with the culture and the audience that you’re working with, because that’s when they’re really the most effective. |
| Dan: | 12:10 | Yeah. In fact, the content on the website, we published it with a Creative Commons license, to make it just as easy as we possibly could to share it, to download it, to remix it and to modify it, again depending on your particular team or project’s needs and interests. |
| Cameron: | 12:25 | Going back to the toolkit, I was hoping that we might be able to walk through maybe a scenario, just kind of a breakdown of what a normal person coming to the ITK might experience, saying what might be a way the ITK could help them. |
| Rachel: | 12:38 | Yeah. I think that’s a very common scenario that happens, where a sponsor comes to us asking for us to build something. We always recommend starting with framing the problem, as we mentioned as one of the categories. We encourage people to take a step back and take a second to say why do they want that. Before we jump into the development of the solution and the execution of the project, let’s define and make sure that we’re designing the right thing and we’re solving the right problem for the right person. I would look at the Frame the Problem tools, the Develop Plan tools and the Understand User tools before jumping into, “Let’s come up with a bunch of ideas on how to attack.” |
| Dan Ward: | 13:19 | Yeah. In other situations, you might come in with a sponsor, or a customer or user comes in and says, “Hey, I have a really complicated thing, can you help me to make sense of this?” In that case, again still maybe a problem-framing tool. What is the problem they’re trying to solve? Build a consensus around that problem. From there, you might fairly quickly jump into one of these simplifying tools and say, “All right, here’s are some tools and techniques to reduce the complexity of your process, your technology, your organization, your system, whatever it is you’re working on,” but it really depends on the unique situations of the ask. |
| Dan Ward: | 13:50 | That’s one of the roles that Team Toolkit can provide as well. We oftentimes help teams pick the right tool or apply the right tool. We can kind of walk them through that first application, but ultimately we want people to be able to take these things and run with it and kind of do it even without us, although we’re certainly here to help as needed. |
| Rachel: | 14:08 | A lot of times we end up looking at an existing system, and maybe they’re replacing an existing system or they’re looking to make upgrades, and the tool Rose, Bud, Thorn is in the Evaluate Options category, and that’s a really easy one to use where you just identify the roses, which are the positives, the thorns are the negatives, and the buds being the opportunities. It’s a really quick way to just get a bunch of ideas up on the board and then prioritize and sort through those as action items. If that’s the request that comes through, as, “Hey, can we take a look at improving this system,” that could be another place to start. |
| Cameron: | 14:46 | I think another thing about the Innovation Toolkit I find fascinating … and this goes back to my conversation with Tammy Freeman … was you kind of redefine innovation, or I guess you have a very novel definition of what innovation should be. I was hoping that you might be able to talk to that. |
| Dan Ward: | 15:02 | Yeah, sure. That’s one of the things we often start with, is we throw around this word “innovation,” but what do we really mean by it? We do often want to spend a little bit of time defining that term, because it’s one of those terms that gets overused and underdefined. I did a little bit of research into this area, and I found a couple of academic papers that … I think one identified 41 different definitions of innovation in use in academia and industry and business, and then one that ended up on 60 different definitions, so somewhere between 41 and 60. |
| Rachel: | 15:32 | That’s crazy. |
| Dan Ward: | 15:32 | Yeah. I couldn’t believe it. My favorite definition, and the one thing that 95% of these definitions all have in common, are really two elements, and those are novelty and impact. My favorite definition of innovation is novelty with impact, or doing something different that makes a difference. |
| Dan Ward: | 15:50 | What I love about that is it’s three words, so short enough to be memorable … novelty with impact … it’s clear, but it’s general enough to be applicable in a really broad range of applications, because “novelty” could refer to new technologies, new processes, new systems, new communication modes, new organizational structures. “Impact” can be saving time, saving money, making the thing more effective, making it safer, making it happier, making something more delicious. You know, whatever your measures of merit are. |
| Dan Ward: | 16:18 | The real benefit of that definition is it points to two really important questions. What kind of novelty am I trying to introduce? What kind of impact am I trying to have? Until we can answer those two questions, we really don’t know what we’re talking about in terms of the innovation we’re trying to deliver. |
| Dan Ward: | 16:32 | Sort of parenthetically, one thing we bump into a lot is this idea of, “Hey, we shouldn’t do innovation just for innovation’s sake.” I think that concept, I generally agree with what they’re trying to say when people say that, but if we define innovation as something different that makes a difference, then that’s as if we’re just going to say, “Don’t make a difference just for the sake of making a difference.” You can’t make a difference just for the sake of making a difference. There’s always that external value to it. |
| Dan Ward: | 17:01 | Really I think when people are saying don’t do innovation for innovation’s sake, that it’s possible to be different just for the sake of being different. That’s possible. Can we make a difference just for the sake of making a difference? Yeah. I think that’s always good, to make a positive difference in the world around us. Anyway, novelty with impact. That’s our general definition for innovation. |
| Cameron: | 17:20 | What I like about that is that a lot of people might think MITRE is technology, MITRE is coming up with a new invention, a new solution. They typically are thinking of like a physical deliverable or like a specific system, but in your definition of innovation, you’re saying we’re not just limited to technology. It can be a process improvement. It can be all sorts of the smaller details that go along with improving a system, that might not mean necessarily adding blockchain or adding machine learning or adding just web development in general. |
| Rachel: | 17:51 | Absolutely, and that’s one of the things that we encounter quite often when we facilitate, especially with MITRE employees who are brilliant engineers. They go to that technology answer right away when we’re looking for a solution, but we as facilitators, Innovation Toolkit practitioners, often bring up those questions to get them to think outside of that box. |
| Rachel: | 18:14 | We do say, “What if we communicated differently with these people? What if we broke down silos? What if we changed the process rather than the tool itself?” Kind of ask those hard questions, because I think everyone here knows how to think through the improvement of a tool itself, but not necessarily the process and people part. |
| Jessica: | 18:36 | I think of a human-centered engineer. We also advocate really heavily for people. |
| Rachel: | 18:42 | Yes. |
| Jessica: | 18:43 | Like keep the human in the loop, the users and the workers. |
| Rachel: | 18:47 | Where their pain is. |
| Jessica: | 18:49 | Yeah. We want to remind people that, like, yes, there is technology here and there’s a system, but there are people involved and people using it. The changes that we make do affect them, right, whether those are customers or sponsor employees or whoever that might be. Don’t forget the people. Don’t get wrapped up in the technology and just make those decisions in a vacuum. |
| Jessica: | 19:13 | That, I think, is where we spend a lot of our reminder time, as well as getting to why. We want to make sure that there is a value proposition that we’ve identified, and that’s another one of our tools. We push that pretty heavily, because second to what problem were you trying to solve, it’s like, well, what value are you adding? |
| Dan: | 19:34 | Yeah. I think so often, technology is novel but not impactful, whereas the people piece, that really puts that impact, the making a difference, front and center. That’s really, I think, what a lot of this is all about, because of the two, novelty and impact, if you’re only going to deliver one of those, deliver impact. Make a difference. Novelty is great, but it’s making a difference that is really what it’s all about. |
| Cameron: | 19:58 | I think we’ve gotten a lot of interesting discussion about the ITK, and I just wanted to go back and reconfirm. What is the goal for the Innovation Toolkit? |
| Rachel: | 20:07 | I would say that MITRE has been pushing … or not pushing, but they’ve been communicating the fact that MITRE has an innovative culture and we want to be innovative, and we should always be pushing the boundaries of innovation for our sponsors. It begs the question of how, and I think the Innovation Toolkit provides some process and some delineation around, okay, this is the goal, this is what we’re striving for as a company, but this is how you make it happen. |
| Rachel: | 20:37 | This is how you make projects more innovative. You think more about the user, you think more about the why, think more about the problem and collaborate with each other, so that you can be the most innovative you can be. |
| Dan: | 20:50 | Yeah. We want to help people understand what innovation is, how to do it, why to do it, when to do it, and not just in an academic sense but in a practical sense. With the teams that we’re part of on the projects that we’re working on, to really, like we said before, democratize innovation, to equip practitioners with the tools and the capability that is they need to do this “novelty with impact” thing. |
| Jessica: | 21:12 | Yeah. I think we’re enabling MITRE employees, but we’ve also put it into the world, so it’s publicly released and available, you know, at itk.mitre.org for anyone, so that our reach can be a little bit broader. Folks that do want to … or do need just a little bit of structure, and also these constraints, right, so that you can be creative. |
| Rachel: | 21:38 | Constraints breed creativity. |
| Jessica: | 21:40 | Yeah. Yeah, so that’s something that we provide and I think strive for. |
| Dan: | 21:45 | One of our goals with Team Toolkit too is to have fun. We really … we like each other. We enjoy working on these projects. This is a passion project for all of us. This isn’t any of our day job. We just sort of had a shared interest in doing this, and yeah, so we want to make sure that we are enjoying our time on this. |
| Cameron: | 22:02 | In closing, I was hoping you might be able to give us some final thoughts on anyone internal or maybe even external to MITRE who would want to get more involved with using the ITK to improve their business processes or their work. |
| Dan: | 22:15 | Yeah. I guess step one would be to go to the website, itk.mitre.org. We do have a blog there under I think the Community tab, where you can get … every week we have a new post, different facilitation tips or a description of a tool, and that’s a really easy way to get your foot in the door and get initially oriented on what the Innovation Toolkit is and how you might use it. Then just click around on the website and see what else you might find. |
| Rachel: | 22:40 | You’re always welcome to reach out to us. Our team is accessible at itk@mitre.org, so pretty similar addresses there. If you have any questions, if you say, “Hey, this tool looks really interesting, but I have no idea how to use it, can you help me,” yes, we can help. |
| Dan: | 22:57 | That’s right. |
| Rachel: | 22:59 | Absolutely take advantage of us as a resource. |
| Jessica: | 23:02 | I think just try it. Like pick a tool and simply attempt to use it with, you know, a group that you feel comfortable with, and you don’t even have to tell them that you are using a tool from the Innovation Toolkit. |
| Dan: | 23:14 | That’s right. |
| Jessica: | 23:14 | You can call it whatever you want, and just like integrate it into your meeting. Use different terms, so that the folks that you’re talking to know what you mean and that it’s clear, but from there, you will see what value you have and what applicability that tool might have. There are many choices, so I think just get started and try to use it, and then get back to us and let us know how it went and what feedback you might have. Because this is our first iteration, and we plan to expand and evolve the tool kit moving forward. |
| Cameron: | 23:49 | All right. Well, thank you all very much. This has been my discussion with Dan, Jessica and Rachel Gregorio, all members of the Innovation Toolkit team. Thank you to MITRE and the Knowledge-Driven Enterprise for making this show possible, and thank you to your entire team for making your platform possible. |
| Dan: | 24:05 | Thanks so much, Cameron. We really appreciate the opportunity to help tell the story. |
| Jessica: | 24:08 | Great talking to you. Thanks. |
Cameron Boozarjomehri is a Software Engineer and a member of MITRE’s Privacy Capability. His passion is exploring the applications and implications of emerging technologies and finding new ways to make those technologies accessible to the public.
© 2019 The MITRE Corporation. All rights reserved. Approved for public release. Distribution unlimited. (Case number 19-1618)
MITRE’s mission-driven team is dedicated to solving problems for a safer world. Learn more about MITRE.
See also:
Interview with Tammy Freeman on Redefining Innovation
Interview with Jesse Buonanno on Blockchain
Interview with Dr. Michael Balazs on Generation AI Nexus
Interview with Dr. Sanith Wijesinghe on Agile Connected Government
Is This a Wolf? Understanding Bias in Machine Learning
A Spin Around the Blockchain—Exploring Future Government Applications