Class 8, Part 2: DARPA as the Connected Model in the Innovation System & Government-Private Sector Interaction and the Example of Computing

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT Open CourseWare continue to offer high-quality educational resources for free. To make a donation or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu

WILLIAM BONVILLIAN: Rasheed, why don't you give us that point that you were making to me about DARPA.

RASHEED: Yes, so DARPA relies very heavily on this program manager structure. So they give these program managers a lot of power and a lot of leeway to do whatever they want. But they have-- they have to have a very distinct vision from the outset, like really when they start to be able to do things.

And so I was concerned that these program managers were going to basically be coming from the same places over and over again, and so have very similar visions. And you're going to be able to miss out on a lot of different talent that exists outside, or a lot of different new projects, or new fields, maybe, just like you're talking about-- these new collaboration between biology and kind of humans and computer science, things like that. But that was really my main concern, was program managers weren't being chosen as-- equitably isn't the word, but that the selection process of these program managers was actually-- might be too narrow.

WILLIAM BONVILLIAN: But they do get visionaries.

STUDENT: Sometimes. I mean, I think we mentioned that they only got white males [INAUDIBLE].

WILLIAM BONVILLIAN: OK, so I'm going to move us to Tammy Carleton. And Tammy's big perception about DARPA is this vision perception, that that's a critical factor that makes DARPA different. And there is a process that DARPA has developed for this radical technological visioning.

And essentially, she argues that there's six stages that they use to kind of get there. First, recruitment of great talent-- and the talent, they hear about talent from this DARPA network, which is now quite huge. Because all of the people that get DARPA contracts and awards, they stay in this kind of DARPA family.

And typically, researchers love to work for DARPA. Because working on DARPA-hard problems is really intriguing. So that's a community that tends to know each other, and be in communication, as well as the community of former DARPA program managers and officials.

So a lot of the talent comes through that network, which kind of knows what's going to work. So the recruitment and the vision get united in a way, in that recruitment process. Then comes the stage of the vision formulation.

The program manager has got to figure out what they're going to do with their four or five years. And they've got to formulate a radical vision, a right/left vision, what's going to come out of that pipeline, and what's the fundamental research that going to get me there. And it's driving for that breakthrough that's the organizing proposition.

Then, they've got to go through defining a program and its launch. Often, they will have a workshop, an invitation-only workshop, where they'll assemble 25, 30, 40 of the kind of best thinkers on a project area, and meet for a day or two days. I've watched one of those, which got held up at MIT as part of the beginnings of creating this Biological Technologies office.

And they brought in just an amazing collection, from MIT and from elsewhere, of thinkers about what some of the early projects might be to achieve some of these capabilities. So the Deputy Director of DARPA came up, and kind of helped run this workshop and think this through. The office wasn't set up for a while until the next DARPA director, but it really helped contribute to that portfolio. And it was a really serious effort.

So the program manager develops a portfolio approach of standing up a series of groups that are going to be working on this, and inter-relating them. And the program manager is also going to be responsible for making this technology transfer. So in other words, it's not enough just to do something cool.

You've got to move it into the implementation stage. And how is that going to happen? So the program manager has got to undertake all of these various steps.

So the criteria for being DARPA hard is that the project has to be technologically very challenging. It's got to extend beyond the current limits of technology and knowledge. Otherwise, they don't really want to do it.

It's got to be actionable, which means it can be made. It can be built. It can be produced. So if it can't be, why do it? You've got to be able to demonstrate that it's actionable.

Typically multidisciplinary-- that it's going to draw on a variety of areas of expertise. It's got to be far reaching. It's got to change things. It's got to be very ambitious at a big scale. "Don't do little things" is kind of a DARPA sub-rule.

And various techniques to flesh out the vision include those expert-only workshops, and a proof of concept effort. Sometimes, if it's some idea that looks promising, they'll do what DARPA calls a "seedling," just put a little money on it, and let somebody play with it, a real thinker play with it, and see what it might materialize into. So they've got that seedling capability.

There is no training system for program managers. They don't go through any kind of training. They just arrive.

And they kind of learn from other program managers what they're supposed to do. So there's a very strong kind of DARPA culture about how it operates, that gets communicated by word of mouth and kind of learning by doing. And it's informal exchange, but there is a very strong kind of operating rules set.

DARPA program managers really have to be quite entrepreneurial. And Martine, you were raising this earlier. They've got to stand up a whole new territory. And to do that, they've got to be highly entrepreneurial about how to bring that about.

And the decision to stand up a vision program-- this is not set up through peer review. There isn't consensus decision-making at DARPA. That program manager has got to have a really strong vision, and it's got to get really organized. And then it's got to withstand the office director and then the DARPA director really tearing through it, and really analyzing it.

So once you get that, though, the approvals can be very fast, once the decision is made to stand something up. And that the program manager really is-- to go back to that term that you liked, Steph-- a vision champion. They've got to champion this vision and make it happen. That's what the program manager has got to do.

So DARPA starts with division. That is not how industry operates. That's not how other government agencies operate.

Industry works largely on what's called a "stage gate system." So industry is going to want to do typically more incremental kind of advances, because the advances can snap onto markets that it's already got, and economic opportunities that it already sees, and technologies that it's already starting to explore. So it's typically going to want to do more incremental advances.

And it's only going to do those incremental advances if they can optimize them economically. So an industry R&D project has got to go through a series of stage gates, which essentially is a way of tearing down a large menu of stuff and getting to the most economically optimal. Industry stage gate process is not a technology visioning system.

It's not deciding on a vision and doing whatever is necessary to get there, which is the way DARPA operates. It's a much more limited set of procedural stage gates that a technology idea has got to go through. So DARPA just working on a very different kind of organizational model from the way in which things are done, either at other agencies or within industry.

And starting with this vision up front is a really interesting organizational idea, that you all should kind of keep in mind when you're operating out there, and setting up your own companies, and doing your own startups. So Max, you've got Tammy Carleton's piece.

MAX: Right. The key step within Carleton's five steps that I thought was most interesting was the fifth one, technology transfer, which is supposed to bridge the R&D-- the valley of death, as it were, that enables a different scientific idea to be commercialized, and to actually affect people all over the world, eventually. While reading, I kind of realized that DARPA seems like they have a pretty decent organization. They get things done relatively efficiently. They have a lot of money.

Why is it they have-- and I actually just decided to start googling, to try to figure out if DARPA had done anything related to fusion. And I couldn't find anything. So I'm curious, why not?

WILLIAM BONVILLIAN: Well, actually, DARPA deferred on energy technologies to ARPA-E, when ARPA-E got stood up, because DARPA views ARPA-E as a compatriot. And ARPA-E interestingly, has done some extremely interesting work on fusion.

STUDENT: I know Lockheed [INAUDIBLE] fusion--

WILLIAM BONVILLIAN: Right.

STUDENT: [INAUDIBLE]

MAX: Yeah, but Lockheed Martin's--

WILLIAM BONVILLIAN: Well, they have the Skunk Works.

MAX: They do.

WILLIAM BONVILLIAN: And DARPA's done an enormous amount of work with the Skunk Works over time, including stealth.

MAX: Granted, yes-- they definitely have a facility, and they definitely have the project managers. I just feel that it's a bit of an area outside of their expertise. They're going to accidentally make it fly and stuff.

WILLIAM BONVILLIAN: Right. And look-- another issue for DARPA is-- remember, these technologies have got to get stood up in the lifetime of the project.

MAX: Yeah, five years.

WILLIAM BONVILLIAN: Like a five year project. So if you can't do it within five years, it's very hard to put it on the DARPA agenda.

MAX: Well, you mentioned that some of the projects would get handed off to other project managers.

WILLIAM BONVILLIAN: Yes.

MAX: So that's how long a manager lasts, right?

WILLIAM BONVILLIAN: Yes.

MAX: But it's not always how long a project lasts.

WILLIAM BONVILLIAN: Yes. Sometimes, there can be intergenerational transfers. And then Licklider developed a different kind of system. He created a community that kept continuing with the same kind of ethos, working on that same set of agenda items, working them down one after another, and making progress year after year on each of them.

So that's a different kind of way that DARPA is going to be organized. But typically, once the program manager leaves who had the vision, their job is to get that implemented in their, in effect, DARPA lifetime. And if they haven't, then it's hard for DARPA to continue that, which we could view as a weakness of the program. On the other hand, you could view it as a strength.

MAX: Yeah, because then they make sure that things-- that if they are working, then they keep-- they may proceed with it, or they pass it off to someone else.

WILLIAM BONVILLIAN: Yes.

MAX: So speaking of Licklider, actually, while I was reading, I noticed that Licklider had mentioned that he heavily relied on standardized tests when picking candidates, so like that recruitment of great talent. Given that literally everyone in this room has experience with standardized testing, and at least I can't speak for everyone, but I don't like it very much. I wonder--

STUDENT: [INAUDIBLE] two tests.

MAX: Which two? It was ACTs.

STUDENT: One was like a psych test, and the other one was like a [INAUDIBLE].

STUDENT 1: GRE. We did the GRE, it said.

STUDENT: Well, there's another one, but it seemed like a psychology test. I'm not sure.

STUDENT 1: So I think that she mentioned that he was the only one who asked for test scores at all. And one was the GRE, which is kind of weird.

WILLIAM BONVILLIAN: Remember, he was a psychologist coming out of the testing world. So that's probably part of the reason.

MAX: I'll chalk it up to [INAUDIBLE], because there's only so much a standardized test can tell you. They won't really tell you how you work with people. They won't tell you anything about how you work on anything that's not related to, I don't know, math or literature.

WILLIAM BONVILLIAN: He did come up with a pretty talented team, though.

MAX: Yeah, that's true.

STUDENT: I mean, you get pretty far on psychology tests, especially when it comes to intertia. Because you have to figure out, for really good visionary leaders, what they'll figure out is they're intellectuals, introverts, extroverts, and they have a supporting side to their psyche. And they're kind of like the avatar of team groups.

So it's pretty good for if you want to find out good leaders. And I know it's been pretty effective in manageent theory when you use psych tests. It's also really good to figure out who is going to be a really good scientist, because it's usually highly introverted and comes up with new insights that doesn't follow the crowd.

So I can see why that's important. The GRE might be, but also, if your score is low, but if you've come out with a good body of work, then they can probably assume that you are pretty smart, even if you didn't do well on standardized tests.

STUDENT 2: Because I'm looking up her LinkedIn. And it says that she has-- her background is in communication. And then she got a Master's in Public Relations. And then she did a PhD in Mechanical Engineering Design.

But that must have been back in the day when the D School at Stanford accepted non-engineers and non-scientists, because now they don't. So it would be curious to think about the ways in which her background, in particular, might give her that insight that an engineer might not have about this particular field, going back to the question of multidisciplinary great groups. But I don't know to what extent, for example, someone like me might be accepted in maybe Lilly's lab.

LILLY: One of the most famous women in my field has an undergraduate degree in German. So you never know.

STUDENT: Lloyd Blankfein, the CEO of J P Morgan, I think-- his degree's in history. And he almost barely didn't pass. He said he still gets nightmares, because he didn't fulfill all the requirements.

WILLIAM BONVILLIAN: All right, so we're going to move out of the testing world back to the DARPA world.

MAX: Sure. So one of the things that Carleton pointed out was that as DARPA has aged, so, too, have its project managers. Even though they cycle between new managers every four or five years, the average age of the managers that they would pick was going up steadily. So I'm curious if anyone has thoughts on what the effect of that would be on their managerial ability, or on DARPA's direction as a whole.

STUDENT: Could you repeat that first part?

MAX: Yeah. So the average age of the project managers has been increasing, as DARPA has--

STUDENT: What's the average age?

MAX: I don't remember the numbers.

STUDENT: Is it 27?

WILLIAM BONVILLIAN: No, it's a lot older than that.

STUDENT: It's a lot older?

MAX: Yeah.

WILLIAM BONVILLIAN: One of the reasons, Max, may be that it's-- I mean, watching MIT, it's very hard for younger faculty to walk out of the place before they get tenure. It's hard enough to walk out once you get tenure, but it's impossible to walk out before you get tenure. And because you're establishing a whole-- your research portfolio and your whole researching system, and it's very hard to walk out of that.

In addition, when you go to DARPA, you've got a conflict. Once you've worked there, you have a conflict of interest with DARPA. You're not going to be able to deal with the agency for a certain period of time.

So if they happen to be funding your research before, they're not going to be able to continue funding your research once you've worked there. So it's gotten more complicated for DARPA to recruit at the university level. It's gotten a lot more difficult.

So the route between MIT and DARPA, frankly, used to be a lot easier to manage. And when I look at my colleagues here who have gone to DARPA, they are more typically fairly far along, and very assured about their career, and their research foundation, and their writing. They've done that stuff, and they can take a three or five year break, and get that really big, visionary thing that they've wanted to work on forever done when they go to DARPA.

But it's harder when you're trying to administer a whole research portfolio. Again, with the IPA authority, you've got a certain freedom to come back to your organization and keep the day-to-day stuff going. But it's still not simple. So that may be one part of the reason that you tend to get somewhat older faculty than they probably used to at the beginning.

STUDENT 1: Licklider wasn't especially young when he was the director.

WILLIAM BONVILLIAN: Right, he wasn't young.

STUDENT 1: Or the project manager. So I think age does affect the perspective that you bring in. But having older people isn't a bad thing. I think in the tech world, it's seen as like if you're not young and hip, you're going to be out of touch with what's going to work. But I don't know if that's a-- I think the points that Bill made about having to kind of establish yourself and figure out what your vision is, you have more time when you're older to have a vision.

LILLY: Yeah, and perspective as well. Just thinking about advisors I've had over the years, and younger or newer advisors versus more established advisors-- they definitely have a different perspective. And yeah, NASA is actually having the same phenomenon, with having directors who are progressively older and older. So that's very sim-- they can't-- it's very difficult to draw someone away from a mid-range career if they're being very successful, and having success getting grants. And it's hard to get them into administration or directorships or management.

STUDENT: What would the difference between young and older directors be?

LILLY: In general, actually, the more established and older advisors that I've had have had a lot more perspicacity with respect to what projects are worthwhile, actually, or--

WILLIAM BONVILLIAN: Through long experience, you're suggesting, because they've seen a lot.

 

LILLY: Yeah, that's the impression I get.

WILLIAM BONVILLIAN: Right. Next, how about another question?

MAX: Sure. All right. So Carleton mentioned that DARPA, in order to save time when they are trying to develop these projects, they try to discourage consensus, which I didn't really see as a-- I think it was the second, the next slide you have. Yeah.

So I understand why, because then it ensures that whatever you're trying to develop, it gets done quickly. But just because it gets done quickly doesn't really mean that it gets done right. So I guess I was curious what people thought on that, specifically that decision.

WILLIAM BONVILLIAN: It's a very interesting point, Max. And actually, ARPA-E-- and we'll talk about it when we get to ARPA-E and the energy class, but ARPA-E does have a process. It's a much smaller agency, so it's 300 million, not 3 billion like DARPA. So it's like a big DARPA office, sizable DARPA office, one of its five or six offices.

So ARPA-E does have a consensus process. They bring in-- they call them "project directors" but they're program managers. And that community, really, you have to present your vision to that community.

And that's a very tough-minded group, and a very sophisticated group. And the director is quite sophisticated at ARPA-E historically, as well. So they did a variation on this.

At DARPA, it's much more, convince your office director, and then the office director will work with you on convincing the director of DARPA. It's a simpler, more straightforward process. You do get support from other program managers, who will have a lot of advice and ideas for you.

So it's a pretty supportive operation. Nobody's in competition with each other. They're there to help each other, so that's a positive, although again, they're competing to achieve their vision.

And everybody kind of knows who's getting their vision done, and kind of who is in trouble with their vision. So that's kind of known and understood. There's a certain kind of competition in that way, but it's not a direct one-on-one competition with each other. But at ARPA-E, there is more of a consensus decision making.

And Luyao when she was telling us about Xerox PARC, remember, your whole discussion about the dealer process and having to bring your idea to the whole team sitting around on beanbags, who would tear it apart-- that's kind of a consensus process. It's a definite, tough-minded review process that DARPA doesn't really quite have. So there may be strengths and weaknesses in both models. Max, what do you think on that?

MAX: Yeah, I could see there'd be strengths and weaknesses to both. But it just feels if you have less consensus, than you have-- if you have fewer people that are trying to discuss these ideas, then there might be some critical flaw that you might not-- that you might just miss, just because there are fewer people with more diverse experiences. Of course, if you have a decent idea of the science behind it, and it's not something as untested as fusion, for example, then I could see it working pretty well.

I mean, if you had to do that for-- I could see it having problems with something untested like stealth. But apparently, they made it work. So maybe they're passing my expectations.

STUDENT: [INAUDIBLE] Silicon Valley perspective, like it's a pretty common mantra to say you want to have an idea that people kind of say is not a good idea. Because the idea is, if you're coming up with something that's pretty innovative, most likely people won't agree with it, or it doesn't make sense, or you have an insight that other people don't get. Also, if you're an expert, you probably know some things that other people don't know, that you'd have to bring them up to speed on, especially if it's in a completely new area.

And so the history has been, they're just saying that for new fields, who is the expert? It wouldn't be a PhD, because the field hasn't been invented yet. It's this person who just started working on it. So Bill Gates was an expert on software for personal computers before that even became a thing. So it's very common to-- consensus is actually really bad, because you put down these ideas based on new insights. So.

LILLY: Yeah, and another issue with consensus is you have to assume, or you have to have a group in which all members have the personality type that they will concede to someone else's idea to get consensus, even if it's not their pet project or their favorite. You know what I mean? Some people aren't disposed to do that, no matter what.

WILLIAM BONVILLIAN: About a closing thought on this, Max?

MAX: Sure. So overall, I feel that DARPA seems to have been a pretty great success. They've made lots of really cool projects. They do it very quickly. Not sure how cheaply they do it, but they get it done.

And because of that continued success, I feel that that's probably why they continue to get so much funding. I really try to emphasize that technology transfer aspect, because it ensures that the technology that they develop isn't just it's a nice, new toy for the military to have. It ensures that whatever they make, from the internet, GPS, et cetera, that everyone can use it, and that it makes our entire society better. So I guess that's pretty much just why I like DARPA.

WILLIAM BONVILLIAN: Great. There's a lot of MIT folks and people saying, what do you do? All right, let me do a quick wrap up of today's class. So we did Mitch Waldrop's book.

And I really wanted to kind of portray the governmental role in supporting the earlier stage of the evolution of computing. It was a proving ground for new concepts, and designs, and architectures. The Defense Department created an initial market for a lot of the new products, and services, and in fact, whole industries.

This greatly expanded university research capabilities, the computing revolution. And it was done to further governmental missions. But it had, obviously, dramatic societal effects.

So it featured governmental agencies playing a pretty central role. DARPA, as we discussed was critical, but some of the others were necessary and needed. And the governmental agency sponsored these industry-university collaborations that led to great strength in the university side, and focus areas around them.

So in the reading that we did from the textbook, the central point here was that DARPA was uniquely able to combine an innovation institutional role, as well as a great groups innovation at the face-to-face level kind of role. That's a remarkable accomplishment. And then DARPA's ability, as we talked about in that reading as well, to innovate in a legacy sector, to do a frontier sector like IT, but also innovate within a legacy sector, and do things like stealth, and UAVs-- that's a pretty fascinating organization, that's flexible, and interesting, and able to do great stuff.

Glenn Fong's piece was really about DARPA playing the central institutional mobilization role for the IT revolution. And Tammy Carleton's piece really told us about this kind of central importance of technology visioning as a key modus operandi, that DARPA been able to operate in. Any closing questions? Good luck with your paper outlines. Let me know if you have questions. And I look forward to seeing you on Tuesday.

Free Downloads

Video

• Internet Archive (MP4 - 64MB)

Caption

• English-US (SRT)