Class 6, Part 2: Crossing the "Valley of Death" Between Research and Development & The Public-Private Partnership Approach

• Download English-US transcript (PDF)

The following content is provided under a Creative Commons license. Your support will help MIT open courseware continue to offer high quality educational resources for free. To make a donation, or to view additional materials from hundreds of MIT courses, visit MIT OpenCourseWare at ocw.mit.edu.

WILLIAM BONVILLIAN: So I'm just putting In-Q-Tel out there as probably the most extreme example of governmental willingness to intervene in territories we thought the government really wouldn't do. Right? And, again, it's an example of this national security state doctrine that David Hart told us about. Whatever is needed for national security, we're going to do, however interventionist it might be. And this is a good example.

So Ruth David, a very talented former CIA Director for Science and Technology, comes up with the idea of creating this entity to sponsor IT research. Because the CIA realized it was falling behind. It wasn't part of the IT revolution. It didn't understand, wasn't a participant, and yet it was going to be absolutely critical. And, of course, virtually no day passes without reading in the newspapers about some latest cyber security set of issues.

So In-Q-Tel is set up as a government venture capital firm, a completely unique model. And it's aimed at early stage technology development. So the CIA is not new to technology development. I mean, in aerospace, the U2 and SR-71 blackbird are two of the most famous and spectacular aircraft ever put together. They were put together by Lockheed skunk works. But In-Q-Tel really is a very different model.

So Norm Augustine, who had been CEO of Lockheed-- and is the winner of the president's National Medal of Technology, by the way, so a great technologist in his own right-- he helps in the founding of this. And they pick a CEO right out of Silicon Valley. And they've got a board with ties to IT innovators, as well as economic researchers, and VC firms, and tech companies.

So In-Q-Tel is allowed to enter into joint ventures, i.e. In effect, it can co-own the companies that it's nurturing to bring along IT technologies that the CIA is going to be interested in. It can use sole source grants. It can set up open competitions. It can award sole source development contracts. It's got a set of powers here that are really quite interventionist in the model. And the CIA itself doesn't review its business deals. It's because it's a bit offshore.

So it collaborates with the CIA, but it's independent of it. It has this set of characteristics. And we're going to talk more about these kinds of characteristics when we do the great groups class, the seventh class. But agile, problem-driven, solutions-focused, team-oriented, technology aware, output measured, innovative. These are all kind of characteristics that In-Q-Tel likes to say that it's capable of. And these are some of the technology areas that In-Q-Tel, in its earlier stages, worked on. So a whole suite of IT-related and software-related tech capabilities. So that's the model, and it's different. So, Chris, do you want to bring us through it quickly.

CHRIS: So this is a pretty unique situation. Basically, it's kind of posed as a win-win. Like the CIA gets new tech that they are kind of behind in producing. And then, these tech companies, or In-Q-Tel, has the opportunity to get exposure to potential big next problems that they could develop commercial products for. So one question that was posed was, are there other such private enterprises funded by the government? And if not, what is preventing these partnerships from happening.

I think Bill mentioned that the Supreme unique case so, probably, I don't think there's any other such partnerships. But why is that, maybe?

AUDIENCE: They're still around, are they?

 

WILLIAM BONVILLIAN: Oh yeah.

AUDIENCE: They re-branded as IQT.

AUDIENCE: Ah, thank you. I do remember there was one, maybe it was similar to DARPA, I remember in DARPA, at least in the next reading, there was something that was similar. Oh, that's right. So I remember that one of the points they mentioned in this reading, there was some sort of licensing that corporations needed to work with the government. Do you remember what it was? Like, FRC licensing.

AUDIENCE: I think we were just talking about that. Like, level of strictness of the contracts. And how In-Q-Tel works to restructure how that contract works so that they could encourage smaller companies that didn't want to deal with overwhelming bureaucracy to join the party.

WILLIAM BONVILLIAN: Thanks. [INAUDIBLE]. Good.

 

AUDIENCE: I was going to say, I think agencies similar to In-Q-Tel might be a hard sell, because if you think about-- they probably have a pretty large employee base, and they're privy to information leak. Also, it's going-- kind of directly flying in the face of our aversion to industrial policy. So we are picking winners, in this case. So those two things I think make it really difficult to have other agencies like this ourselves.

WILLIAM BONVILLIAN: Right, because really, in effect, In-Q-Tel is a venture capital firm. It's not doing its own research. It's a venture capitalist, and it's taking positions in companies and helping drive those companies towards technology work that's going to be relevant to concerns that the CIA has got. Sorry, Chris. Go ahead.

CHLOE: Oh, no, I--

WILLIAM BONVILLIAN: Oh, Chloe.

CHLOE: I think it's interesting to think about why this would or wouldn't work for other agencies, because personally, I'm surprised that it has been such a successful model for the CIA, of all government agencies, on several levels. The article says that they didn't deal with any classified work or contracts, but I don't understand how such a firm could exist in partnership with an agency that does any classified work. I don't see how-- I'm surprised that that happened. I am surprised, also, a bit that that was able to-- yeah. Start with that.

AUDIENCE: Which part are you surprised by-- the fact that they work so much on non-classified stuff?

CHLOE: No, no. They didn't. In-Q-Tel, at least--

WILLIAM BONVILLIAN: Yeah, they didn't work on classified stuff.

CHLOE: So I'm surprised that-- I think, of all agencies that I think this model could work for, I'm surprised that it works--

AUDIENCE: For someone who--

CHLOE: --the CIA.

AUDIENCE: --who compartmentalizes their information.

CHLOE: Yeah, compartmentalizes information. They were probably-- they mentioned also, oh, there will definitely be people who will raise questions about ethics concerns and the government meddling in funding technologies that they might simply be interested in. And feel like the CIA is just the worst possible--

AUDIENCE: Customer?

CHLOE: Yeah. It's really surprising to me.

AUDIENCE: I think-- yeah, it's really cool that it works, and also kind of shows that private enterprise work with government. Private enterprise has a strong incentive to move quickly, but I also think it's a case study in terms of, it was an IT revolution. Their business is information like information in general. And if anybody's kind of better than us, then we're screwed. So they're like, we really need to win. We really need to win. So I think there's a strong stress to actually make it work.

So there's probably a lot of issues. And I'll look from a management perspective, a lot of times, you don't say about the bad things when they write stories about it. So their-- I feel like, yeah, it was really stressful probably, the people from the-- because it's going to be an old wing who's like, yo, we don't want to do this. And there was a new wing who were like, no, we need to do this. So I think it might have been a pretty stressful thing. I can see how it works, but I can also see how they made it work.

CHLOE: Yeah.

AUDIENCE: That's my point.

AUDIENCE: Which means that if it's such a difficult situation, and if they were still able to make it work just out of necessity, then [INAUDIBLE] probably any other government agency could do something similar.

WILLIAM BONVILLIAN: So in order that we get through everything, I'm going to push this ahead on this one. But first, just a closing thought, Chris, on this one?

CHRIS: I think the central idea here, that you can maybe outsource what you may be a little bit behind in or aren't so good at, is really interesting, so that the CIA can effectively focus on its core business, which is gathering and maintaining intel. And this goes back to our previous discussion about the government choosing winners and losers, and it's really interesting that this case has emerged as the real case that they've chosen a clear winner here, and they're actually forging a pretty strong public-private partnership that seems to work. So yeah, this is a pretty interesting article.

WILLIAM BONVILLIAN: And I wanted to have Sarah Jane Maxted-- that is, our distinguished visitor in our class today-- just introduce herself briefly. And maybe, Sarah Jane, you can tell the team some things you're working on.

SARAH JANE MAXTED: Sure. I'm Sarah Jane Maxted, also SJ. [INAUDIBLE] refer to me. I might pop in a few other times as well. I work at MIT. I run a program called the Regional Entrepreneurship Acceleration Program. We work with global regions all over the world to accelerate their innovation ecosystem.

So my background-- I'm not that old, but my specialization is in innovation ecosystems. Specifically, my background's most directly related to the energy space. So I worked in the US Department of Energy, and I worked for a US senator, and I worked in the private sector as well, all in relation to energy and innovation ecosystems. So how do you cultivate-- how do you deal with this valley of death? And so it was so interesting being in the beginning of this conversation, talking about not using market terms. And when I worked in the federal government, we were not allowed to say commercialization. It was the death word. It was ridiculous. I was like, why can't we just use it? To your point, that helps clarify. So it was very frustrating. It's been really great to hear.

AUDIENCE: What words did you use as an alternative?

SARAH JANE MAXTED: Tech to market.

[LAUGHTER]

 

We also weren't allowed to say marketing. Marketing was absolutely not OK.

AUDIENCE: What about tech transfer?

SARAH JANE MAXTED: Yeah.

[INTERPOSING VOICES]

SARAH JANE MAXTED: That was OK. Congressmen didn't love it. But it's really fascinating to hear each of you guys. You guys are very well-versed. I'm quite impressed. But yeah, any questions on that front. And what I'm doing here at MIT, I also work with the Innovation Initiative. I think somebody mentioned earlier some of the educational things related to innovation entrepreneurship and business, and being able to have multiple skill sets. And I am interested at some point to hear what you guys think about the minor in innovation and entrepreneurship, which was launched last year. Maybe some of you are part of it. But anyways, that's what I [INAUDIBLE].

WILLIAM BONVILLIAN: Thank you.

SARAH JANE MAXTED: You're welcome. Thank you for having me.

WILLIAM BONVILLIAN: All right. Now we're going to crash through the last couple of readings. This is a piece I did in 2014. And essentially, I began to realize that there were different periods at which point we began to reorient in the US our understanding and the organization of our innovation system towards, heaven forbid, commercialization. And I roughly delineated five periods-- the immediate postwar, the Sputnik period, the 1980s competitiveness period, and then more recently, these new potential innovation waves that are coming in-- energy for sure, I think, but also potentially advanced manufacturing. And the US essentially had to rethink the Vannevar Bush disconnected model that it brought into place in the early postwar, and create a more connected model. And each one of these stages presents movement towards more connected kinds of systems.

So the postwar period you're familiar with. That's Vannevar Bush himself. And he creates a highly connected model during the war, as we've discussed, and then disconnects it as we've gone through and talked about-- particularly from the Stokes reading-- at the close of World War II. So this is where, as we said before, our great, strong, and certainly very important basic research organizations come from. And as we've talked before, we get a lot out of that model.

But Sputnik is an attempt-- led by the Defense Department, but embracing other pieces like NASA and NSF-- where the Defense Department pushes on that disconnected model. The Defense Department can't take a disconnected model. It just won't work. It's got to have fairly rapid movement towards actual technology development. And to an extent a system is disconnected, it's not going to serve DoD well.

So DoD, in the course of the Cold War-- and certainly, this is rampant by 1957-- is rebuilding its much more connected, World War II-like system, with deep ties between industry, university research systems and efforts, and the defense lab system. So DoD plays a pervasive role at every stage of the pipeline, as we've talked about previously. So if that's the innovation pipeline, our classic basic research agencies are playing, at this stage, research and maybe invention of early-stage prototypes, maybe into early-stage development, but that's it.

Whereas the Defense Department, as we've talked about, will perform the research, the development, fund the prototype, fund the demonstration, fund the test bed, and move into actual production, often creating initial markets for new technologies. That's the IT story that we've begun to tell. We'll tell a little more when we talk more about DARPA. So DoD has got a much more connected system, because of its potential-- and it doesn't always play a role at every one of these stages, but it potentially plays a role at all of them. So it's a very different organizational model, as we've been talking about today.

DARPA comes right out of the Sputnik challenge. It's set up in '58. Sputnik is '57. President Eisenhower had been completely frustrated that the three separate service space programs were not talking to each other and not sharing ideas. Each of the services was pursuing a rival effort to capture what was then known as the higher ground, and put that higher ground project into its own procurement program. So the Army and the Air Force and Navy all had their own separate missile programs. Eisenhower wakes up, stuck with Sputnik, appropriately blames the services for not having undertaken collaboration. Eisenhower decides, I'm taking space away from you. We're going to create this DARPA thing, and it's going to report to the Secretary of Defense, and we're going to run it as a unified program.

That only lasts a few months, because there's a big movement to have a civilian agency led in space, not a military agency. So then NACA-- which was the Aeronautical Research Agency that, actually, Vannevar Bush led in the 1930s, which is where he learns his innovation organization lessons-- that becomes NASA, and becomes aeronautics and space, and is essentially created around the Apollo mission. Max?

MAX: Quick question regarding the previous slide. You mentioned that the DoD is basically involved in all of these steps, whereas places like the DoE are not involved. Why isn't the DoE involved?

WILLIAM BONVILLIAN: The Department of Energy doesn't have a-- Sarah Jane can tell us about this, but it doesn't have a procurement budget, so it doesn't buy stuff. So DoD buys its stuff. It will buy the technology advances that it's supporting. It will buy aircraft. It will buy IT systems. It will create the initial market for the integrated circuit for its own computer needs. The Department of Energy doesn't have a procurement budget, therefore it's limited in its ability to do this creation of initial markets and undertake support of the production stage.

It's hard-- look, in the end, the energy technology-- we're going to talk about it. We'll do a whole case study on energy, which I hope will sort of satisfy you. But in the end, we're going to stand up an energy technology revolution in the private sector. There's no pretending that a government contract model is going to achieve that revolution. It's gotta get stood up in the private sector.

So I don't think it solves our problem. I think that we have to recognize the reality that 12% of the economy is in energy. It's going to be a civilian sector. Let's figure out how to stand those technologies up in that civilian sector, because that's the lasting model. I mean, it wouldn't hurt. The DoE would buy certain stuff. But Congress is just not going to give it the power. And Sarah Jane's comments about the unwillingness to talk about, God forbid, commercialization, is symptomatic-- much less initial market creation. So it's a challenge in other organizations.

Third period is around competitiveness. So from 1973 to 1991-- and we talked about this in the first class-- the US economy significantly slowed. And this is where the Rust Belt comes from. This is the first great job disruption in manufacturing. This is Japan's brilliant quality production model coming to bear against an earlier model of US mass production, and quality production proved the better model. And US GDP growth historically is around the 3% level. It came down to the 2% level. US productivity growth, historically around the 2% level, comes down to the 1% level. Lower productivity growth means a lower GDP growth. This is not a happy time. This is a tough time in the United States. By the way, current growth rate in the United States is 2%. Productivity rate is 1%. Comparable economic problem. Economists now call this secular stagnation.

We had that period for an extended period of time. and the US had organized its economy around frontier innovation, but it missed an innovation wave. And when you organize your economy around frontier innovation and you miss a wave, it's a problem. So when Japan-- and, to some extent, Germany as well-- leads this manufacturing revolution, a real innovation wave around quality production, the US misses the wave and has to catch up. So in this time period, particularly in the '80s It's trying to rethink its model and figure out how to be more competitive, with particularly the economies in Japan and Germany.

And the models that it came up with were the following. So a lot of interesting things happened that are, in effect, more connected. And the government is going to be pushing further down the pipeline. The Bayh Dole Act. The government used to own the intellectual property rights to federal research. The government is not a company, so it just went and sat on the shelf.

Bayh Dole Act gives intellectual property ownership to the universities that are conducting the research, and they in turn share it with their researchers. It makes the university an economic actor. So universities have done education and research for a long time. Suddenly they've got an economic stake in the outcomes of their research. So increasingly, we are drawing universities in to the economic system and having them play an economic role. Whole new development. And the Bayh Dole Act is not the only thing going on here, but it's one significant part of the story.

We create the Manufacturing Extension Program to bring the latest manufacturing technologies and processes to small manufacturers. So the majority of US production is done by small firms and mid-sized firms that have less than 500 employees. They don't do any R&D. They have a lot of trouble-- as we talked about in the manufacturing classes-- keeping up with technology advances. So the MEP program is an attempt to bring the latest technology advances and process advances to small manufacturers. And here, there is an explicit attempt to understand what Japan is undertaking in quality manufacturing and translate that model back to small and mid-sized manufacturers.

We create the advanced technology program NIST. That's now over, on precisely the reasons we've been talking about. That was viewed as too interventionist. The Commerce Department was supporting company research at the applied level, and it was viewed as too interventionist. Eventually, Congress shuts it down.

The Small Business Innovation and Research Program. As we've talked before, when you all set up your startups, this is the first place you're going to go for money. This is where you go, and you can get up to about a million dollars in two different phases to really do startup efforts. And the number of startups that have used SBIR money-- they all use it. It's very critical. That program comes out of this general era. Sematech at DoD, that we've talked about briefly earlier today, is another program-- also quite interventionist-- where essentially, the government attempts an advanced manufacturing project in the semiconductor sector.

So these are all program elements that began developing in this period. Interestingly, we don't have to follow through, because Japan misses the IT revolution that really begins to hit at about 1990. The US leads it. We did very well very quickly starting in that 1990s time period-- one of the most successful economic periods in US 20th century, 21st century economic history. So we forgot the problems we had. And of course, those manufacturing problems are now right back on us, because we didn't fully follow through.

Period 4 is the problem of energy innovation. We're going to do a whole class on this. I'm not going to spend much time on it. But energy is driving the organizational model, and an organization like ARPA-E-- a DARPA within DoE-- explicitly has what Sarah Jane reminded us of-- a tech to market outlook that really very much considers the possibilities of commercialization before it selects the R&D projects it's going to fund. Is there a pathway to commercialization? There's a conscious attempt there. So this is not a curiosity-driven, Vannevar Bush basic research agency. ERE explicitly works with industry for the great bulk of its research portfolio.

Advanced manufacturing. We've talked about this one as well. This may be an evolving innovation movement. We'll see what the current administration wants to do with it. It's not clear yet.

So those are five periods. I'll just recap. Period 1, the postwar. We move from a wartime connected system to a disconnected system. And then slowly over time we've been, creating more and more connections in our system. So Sputnik creates DARPA, and we'll talk a good bit about DARPA. I'm just laying the groundwork here on several topics, including this one. But a reconnection of the whole Defense innovation system.

Period 3, around competitiveness. We do a series of experiments-- which, with one exception, all of which remain in place and have been pretty successful over time-- to attempt better connections between industry and the commercialization efforts and university research. And then the energy challenge and advanced manufacturing challenge push us further down that pipeline. The advanced manufacturing institutes and manufacturing that we talked about, those bring together collaborations of private sector firms, government, and university researchers for a collaborative effort around developing new advanced technologies. It's not early-stage technology. It's very applied. It's quite late-stage. A lot of development. So these are much-- this is the government supporting an effort to move much further down the pipeline towards commercialization of these new technologies. Who's got this one?

RASHEED: Me.

WILLIAM BONVILLIAN: Good.

RASHEED: So bringing together these five periods, I thought these were pretty-- these held up pretty well with all the readings that we've done so far. In particular, I think the postwar and the Sputnik one we've kind of gone over in spades. But a little bit newer, I think, is the period 3 through 5. But particularly, if we focus on '80s competitiveness, we see that that's spurred on a little bit towards-- with the advent of Sematech and semiconductors, and missing its innovation wave with Japan.

And is there sort of this-- so what we want to find out here is, is there an idea that globalization brings about new forms? So now that we have this '80s competitiveness, these periods get a lot shorter. So the '80s, and the period 4 is in the 2000s, and now we're in advanced manufacturing. So is there this opportunity for globalization to make these innovation waves maybe shorter, or add more and make them more compact so more people have this opportunity?

WILLIAM BONVILLIAN: Rasheed, I think that's an important point, because actually, each one of these stages are driven by external developments landing on the US economic situation and forcing change. So they're just not materializing from within. These are all external developments that force a US response. So you make an important point.

RASHEED: But is this something that you guys also saw, maybe, with the idea that globalization will bring more innovation to the forefront-- maybe not just in the US, but elsewhere as well, and make other people respond in the way that we have?

MAX: At least intuitively it does, because if you have more people that are talking, more people that are exchanging ideas through globalization, then by extension, you would think that would mean that more ideas can get produced and people have access to more resources, which means that whatever ideas that come up with, they can implement better. So at least intuitively.

AUDIENCE: I would challenge the resource point, though, because that would trigger [INAUDIBLE].

AUDIENCE: No, I was going to talk about everyone talking, if that's a good thing, because usually, the best decisions are made when everybody's following along the same path, and it's also not mainly just one person thinking for themselves. If you think about Wall Street and speculation-- because you get an echo chamber. And this was shown by the whole Facebook case after the election, where everyone was saying the same things. So it's usually-- a lot of evasion happens when an individual thinks of a different perspective and then echoes back to the community. So if you can have individual figures, it's really good. But usually what happens is, because of dogma or just the way groups work, there's very few innovative ideas, and it's more of an echo chamber.

AUDIENCE: Well, that's true, but that's been true always. So my thought is that having a greater access to more communities that have these different ideas, then you have more of those Zuckerbergs who are thinking of something different.

AUDIENCE: Perhaps.

AUDIENCE: Perhaps.

AUDIENCE: I would say, getting back to Rasheed's original question or point, I think that we probably are seeing a contraction of technology waves, or these periods, probably due to just the speed at which information can be transferred. That might explain it.

RASHEED: Also, I'll argue that financing and getting a reputation for things, that's easier now, because back in the day during Edison or during Ford's day, getting money was a huge problem for an entrepreneur. Nowadays, it's very easy to get money. And also, you get to keep your equity, because back in the day, too, you would create-- you could create one of the best technologies, and you would give up 1/3 your company or the full company, and then they'd kick you out. So you came up with this innovation. You understand technology, how to bring it to market. And now, because this person wants to make more money, they kick you out. So that's really a good opportunity, because now it's really the technologist who has more power. So that's why these cycles are [INAUDIBLE].

AUDIENCE: But see, that's exactly where-- the point at which I would challenge Max, because just because we have a globalized spread of ideas doesn't mean that we have a globalized spread of resources. And often our innovation, especially the United States, is built on imperialism. Dependency theory, borrowing from [INAUDIBLE] economics, is one of the reasons why the United States and why colonial governments were able to make such incredible gains. In particular, we saw this with the Dutch East India Company. Their exploitation of Southeast Asia was exactly what enabled them then to promote, I guess, a Marxist society in the Netherlands, and then enabled the spread of colonization in the New World.

And so what we're seeing now, especially with the innovate here, produce there paradigm is, I think-- and the sort of detriments that we talked about maybe three classes ago, about the ways in which countries are now understanding how to innovate there as well-- is that we may be sharing all of our ideas, but there's not going to be a capture of value economically or in terms of the practical gain of technological advances, because those resources are not local. We're going to have to purchase them from other countries. And if the United States doesn't have access to those resources and production and manufacturing, then we're going to miss out on the benefits of having created those ideas domestically.

And that's what I think is so treacherous about the international relations landscape that the Trump administration is creating for us, and why it's really important for us to be wary of our reputation on the international plane, because we don't have all of the resources necessary in order to produce, and if we destroy our reputation abroad, people won't want to work with us. And if they now have the innovation capacity as well, then what is the future for American innovation? Imperialism hegemony does not exist for Americans anymore.

AUDIENCE: But I would argue two points on what you just said. One is, the interesting thing about Trump is, he became such a polarizing figure that abroad, people know we don't like him. So it's like, that's just him. That's not the US. And then two, I think you're thinking about-- your framework is based on the past, in terms of what's happened. And I think in the past, it's been very much-- we're a scarcity society. So there's a scarce amount of resources, so we need to take from somebody else.

So there's been-- there's a philosopher who talked about how we have so many enemies, and the US creates so many enemies because we used to take, take, take. But I think what will change in the next 100 years, 200 years is that we will get closer to like a post-scarcity society so that we have more resources. Say people discover a better form of energy, so we have-- assume infinite energy, even though that's ideal. Assume we have water, because we've figured out desalination, so we can get enough food. Assume that now we have automation, so people don't have to work with their hands and more people can think. And then assume we're expanding, and we go into space. So now we're also-- it's like the American period of growth, where people are expanding other places.

So the game before was to take and take and take. And if the game becomes more to, how do you give? What becomes valuable once you already-- to have money wouldn't be valuable. It would be respect. It'd be, this person got a Nobel Prize. This person did a social initiative. That's just an evolution of what you just said.

WILLIAM BONVILLIAN: So you guys are really leading us out of here.

[LAUGHTER]

Go off into utopia land.

AUDIENCE: Yeah, really.

RASHEED: [INAUDIBLE] That was four individual points.

[INTERPOSING VOICES]

AUDIENCE: But the thing is--

AUDIENCE: Bring it back.

AUDIENCE: --from a startup perspective, you don't think linearly. You think exponentially. So we're talking about these contractions. That could be a reality in terms of-- if you think about how the past used to be, that's not the game we're playing anymore.

RASHEED: Yeah. I don't think that's the game we're playing anymore. You could probably look at these, and that's pretty already-- you're already looking exponential.

AUDIENCE: It could be bioengineering, energy, and physics all at once.

RASHEED: Yeah, which I think offers the opportunity to be pretty cool. And I think ARPA-E-- we can focus on them a little bit now, on period 4-- is looking at these interdeparmental collaborative mechanisms to spur these short-term, three-to-five-year, very industry-focused targeted growth periods. And so is that an idea that we want to kind of take back, is can we look at-- now that we're so interconnected in all of this globalization, [INAUDIBLE] try to create is, can we focus all of these things in these targeted areas with any degree of success? Or are we just playing the game, gambling [INAUDIBLE]?

MAX: So you're saying to focus the defense-- for example, the Department of Energy, you would say, focus it only on one form of energy. Is that what you're asking?

RASHEED: Let me take this back.

SARAH JANE MAXTED: Hear about the mechanism.

RASHEED: Yeah. So as a mechanism-- does the mechanism work? But the mechanism is basically, you have this idea for advanced materials, and you take a couple of industry partners-- the Department of Energy, for example, and maybe even an international partner-- and you make them all work together on an advanced materials project for three to five years. Do you think all of these people can get together and work effectively in the short time span to produce any degree of success?

MAX: Well, obviously, I have a bit of a pension for fusion, so my first thought's going to be here-- the International Thermonuclear Experimental Reactor-- partially because science and engineering for plasma is just really hard, and partially because I feel that, at least in my opinion, it hasn't been managed as effectively as it could have been. It was supposed to be the best example that I can think of international collaboration trying to solve a very big problem. And it's more than double its first budget, and it's, what, 10, 20 years behind schedule? Of course, that's not to say that the approach itself-- the fact that you're using international resources-- that's not to say that that is inherently flawed. But I can't really think of any counterexamples.

AUDIENCE: The way we think about it as a businessperson is-- this a pretty common problem-- is, you stress people. I gave the case example before of Silicon Valley, and how there's not minorities and there weren't that many women. So sometimes what you do is, you actually make the problem be very apparent. And so the CEO of a company knows, there's a problem in my company. They won't announce it, but they'll leak a paper. And somebody will be like, oh, there's a real problem. And then everyone's like, oh, we need to solve this. Oh my god, there's all this press. And you use the tools at your disposal. And so, yeah, you can-- you have to stress them just the right way, because people would stay by themselves in their own little bubble. But if you stress them so it's like, oh no, we need to do this, we need this now, that's how I think about it.

MAX: Actually-- oh, sorry.

SARAH JANE MAXTED: No, no. I was just going to say, I think the question that gets beyond the mechanism itself of the ARPA-E is, how do you monitor or evaluate success? So I think-- to your point, some technologies take a lot longer. So is three to four or five years a good time horizon? And if we keep it at that, that's fine, but how do we bundle in all these different energy technologies? For instance, for some, maybe more building technologies are going to be a lot quicker, and then some, like fusion, are going to take much longer. And how do you identify the right technologies or applied research to do in that capacity, and how do you evaluate the success? And that's the tech to market question as well-- what are those points of evaluation?-- which is, I think, [INAUDIBLE].

WILLIAM BONVILLIAN: Sarah, you're driving us, I think, to some additional perceptions here given your basic question, Rasheed. What are the innovation organization models that we're going to use if we're trying to take on these bigger tasks? In other words, it was straightforward enough to do a curiosity-driven research project and fund one PI. But if we're moving the model to move much further down the pipeline, which is what all these stages represent, what are the organizational mainstay models we're going to use here?

And then you come to a model like the manufacturing institutes, which can typically have more than 100 companies; four or five major research universities; a comparable number of community colleges, if not more; and, potentially, one or more federal agencies and five or six states and state economic development organizations. As some of the Defense Department described this to me, standing up a manufacturing institute is like standing up a country, compared to funding a single PI researcher.

So these call for-- and I think is what Rasheed is driving us to and Sarah is driving us to-- they call for a much more complex organizational model, because the tasks have gotten considerably larger, and the number of actors that need to be involved is getting considerably larger. So these are all lessons learned that I think we're going to have to pull out of the experiments were running now on energy and potentially advanced manufacturing, and really consider the models we're using. Rasheed, do you want to give us a closing thought about this reading?

RASHEED: Yeah. I thought this reading was pretty comprehensive, and it seemed very Bill, I would say, because it did a good job of starting us from the beginning at this postwar system and walking us all the way through now to advanced manufacturing and the [INAUDIBLE] problems that we have today. So it was pretty-- I felt like I was in class, reading through and--

[LAUGHTER]

AUDIENCE: I felt like, I've done this.

RASHEED: Yeah, yeah. And then you get to the new stuff at the end, and you're--

WILLIAM BONVILLIAN: Rasheed, I had to write this because I needed it in class, because there wasn't anything else that covered this entire panoply. So that's really, frankly, the reason why I wrote it.

All right. We're now going to jump to another Bill reading, and now we're going to take a deep dive into this whole concept of legacy sectors. And this will be foundational as we do some of the case studies, like in energy, like in health care.

AUDIENCE: If I could just ask you, at the very end of this last [INAUDIBLE] just talking about, you have-- I guess your driving conclusion is that the new model innovation organizations discussed here deserve ongoing scrutiny and pragmatic evaluation of their performance to refine these models. Obviously, you published this three years ago, and were probably thinking of this five years ago. So I was curious if you could talk a little bit about how you've maybe developed that already, or if you've been thinking about what a model might look like.

WILLIAM BONVILLIAN: Well, I learned a lot in working on the advanced manufacturing initiative-- the Advanced Manufacturing Partnership that the President set up, and the Production in the Innovation Economy project at MIT, because it really forced a lot of thinking about what the organizational model was going to be. And in turn, I've learned a tremendous amount about looking at what the Department of Energy has been doing.

So we'll dive into this in the energy case study, but essentially, DoE, at the front end of the innovation, the R&D stage, has created a whole set of new models. It is a very different agency than it was 10 years ago. And it's created a host of new pieces on the innovation front that enable it to do a lot more things. So yes, they still have a Vannevar Bush basic research agency-- the Office of Science, a $5 billion basic research organization-- but they've got all these other pieces on the table now which make the mix quite interesting. So we'll have a chance, actually, Steph, to dive into what some of those models look like. We'll take a particular look at ARPA-E, which is actually maybe on the chopping block at the moment. So it may turn out to be an historical lesson. But a pretty interesting organizational model.

But let me do another driver here that forces us to think about innovation organization, which is innovation in legacy sectors. So the US is pretty good at doing the next big thing and standing up new technologies in frontier territories. In other words, in effect-- to use a metaphor-- we take our new technologies, we put them into covered wagons, we go across the mountain, and we open up a new technology territory. We do frontier innovation, and we're pretty good at this. There was nothing really like computing before computing. So that would be a good example of opening up a whole new territory.

The US is not good at bringing innovation into legacy sectors. The new innovations often have to parachute in, and they get shot at on the way down. So the US tends to do the next big thing, but doesn't tend to go back and fix the old. That's why the-- driving from Kennedy Airport into Manhattan often is kind of a third-world experience. Sometimes you think you could lose an axle on your cab before you get into downtown. We don't tend to maintain a lot of our existing systems. We push on to the next big thing. And that's not bad. That's not a bad American tendency. But we do pay a price for it if we're going to have to bring innovation into these legacy sectors.

Just think of the health care debate we've been having in recent months. The United States does biotech. It does not go back and fix the health care delivery system. That's a legacy sector. That would be hard. We'd rather open a new thing. So this is a problem in our society. Look, every society has got these problems. It's hard for anybody to innovate in the legacy sectors. But it's particularly tough here, I would argue.

And the problem is that a lot of our big societal problems are tied to these legacy sectors. We can't avoid the innovation challenge in these sectors-- climate and energy, food and water, the problem we've got of quality job creation, health care delivery, improving education to address inequality problems. These are all profound societal challenges, and they all have to be fixed within legacy sectors. And to do this, we've got to confront these sectors. Frankly, I was-- Chuck Weiss and I, my co-author, were amazed we wrote this book. There's no literature on this. There's no literature in the innovation policy field about innovating in legacy sectors. It's a problem that's just hidden in plain sight.

So how would we do this? Because we need to do it. One example is the revolution in military affairs in the defense sector in the 1990s, where we bring a raft of new technologies to bear even though the Defense Department is one of the most notorious legacy sectors around. How is it the DoD is able to organize to bring these technology advances in?

But we can start to see a whole set of innovations that are starting to move in what we can consider legacy sectors. So advanced manufacturing. Manufacturing is definitely a legacy sector. New energy technologies. Intelligent cars. Higher education for sure is a legacy sector. We're starting to see online education. Commercial space. So there are possibilities here. We're starting to see what some of the possibilities might be. So learning how to innovate in these sectors is critical.

What are some of the take-home lessons? There are obstacles to innovation in the legacy sectors. Innovation, by the way, is not just the shiny new lights and only the cutting edge. There's a lot of innovation that needs to land in these legacy sectors that are not quite as glamorous. Legacy sectors have a lot in common, both in the US and abroad. So there are definitional aspects of a legacy sector that apply to manufacturing, but also to health care delivery. We can find commonalities here. And legacy sectors in countries here and abroad have commonalities.

And the context of innovation can be every bit as important as the innovation system itself. So this class is focused on the innovation system, but it's within a larger context. And I'll explain what that means in a minute.

Entrenched legacy sectors resist innovation. For example, we often provide incentives to producers that don't align with societal needs. The legacy sectors are defended by a technological, economic, political, social, and maybe even cultural paradigm. So a legacy sector is a castle, and it's got a rich defense system. That is, it defends the technology advances that it's encouraged. It will defend the economic models that it's pursued. It will use the political system to block change. And it uses societal and educational systems to ensure that it gets staffed up and has a social outlook that tends to protect it. So these legacy sectors tend to share this common paradigm. They will resist disruptive innovation, and they will be more accepting of innovations that fit their own business models. And we'll talk about that in a second.

The features of the legacy sector, they get a little definitional, and I don't want to go into a lot of details on these. But you get the idea. And these are characteristics that-- a legacy sector will share all of these characteristics, but it will typically share many. In the characteristics side, there are often perverse prices that don't reflect externalities-- i.e., like the low cost of gasoline. There will be an established infrastructure-- gas stations. Massive infrastructure system that's got to get rethought and overcome if you're going to make change. There are strong public expectations of low cost of energy in the United States. It's very hard to overcome these.

There are often regulatory requirements that block entry of new technology-- again, in the energy space. Good luck in connecting your out of state new solar system with the transmission system across a series of states. You've going to have to get the states' regulatory systems lined up. It's very tough. There are career pathways and curricula at universities that tend to support the existing system.

All of these, there's typically very limited R&D compared to revenue. So in the energy sector, for example, less than 1% of annual revenues is spent on research and development in the energy private sector. An astonishingly low number for a major industry. And there are powerful vested interests behind these legacy sectors that defend them.

On terms of-- I'm not going to go into detail about these kind of market imperfections, but often perverse subsidies, network economies, non-appropriability where the developer of the technology can't recapture its value. There's often problems of collective action. So for example, it's tough to introduce technology advance into the building sector because it's so totally decentralized. There are not major actors. There's just lots and lots and lots of little actors. So you can't get collective action together in a sector that's organized like that. There's a short time horizon for financing, and that limits the ability to scale up new technology advances.

So these are-- now, in contrast, if you're developing a technology that is compatible with the legacy sector's paradigm of technology and business model and so forth, then you can do it. You can do it in relatively short order. So fracking was adapted within, really, a 15-year period. The original technology advances came out of the national labs around things like 3D seismic imaging. And they moved very quickly because fracking fit the existing fossil fuel economy so nicely. If you're doing solar, good luck to you. It's a much more complicated adaptation process. So if you're doing disruptive technologies, that's where the problem really comes in.

So we have a series of models of the dynamics of innovation, and we've been struggling to see these in the course of the classes so far. But as you think about it, there's really five ways that our economy undertakes innovation. One, we've spent a lot of time last class and this class-- the pipeline. Again, the model-- dump basic research in the end of the pipeline, mysterious things happen, great products emerge. That's technology push. Federal government is, in effect, pushing a new kind of technology or scientific advance into the pipeline, and hopefully it will emerge from the pipeline. It's a technology push model, a technology supply model. That's the dominant model underlying US innovation policy.

But there's also induced innovation. Remember, Vernon Ruttan came up with this theoretical framework. Industry does the majority of innovation in our economy. Industry will typically see a market niche or a market opportunity and will move to fill that opportunity-- typically with incremental advances, not radical advances. So that is a demand pull. Industry is identifying a potential demand that can pull the technology into the marketplace. That's typically what industry does.

But then there's other models. The extended pipeline, we've just been talking about today. That's what the Defense Department does. It operates at every stage of the pipeline. So in addition to the pipeline model in the US system, we have an extended pipeline model in the US system. A fourth model is manufacturing-led innovation. US doesn't do this one. We didn't build this one. Countries like Germany and Japan do this model. Korea, Taiwan do this model. China now jumped on this model. In other words, their innovation system is led by the creative work, the new engineering, the new science that can lead to produce products. It's that product design stage that's central in a manufacturing-led innovation system. US never built one of these. So by creating these manufacturing institutes, that's an attempt in the US to get this piece into our system.

Overall, entry into legacy sectors is such a complex effort that you're not going to do one of these innovation organization models. You're going to have to do probably all of them. When you think about a sector as complex as energy, you've probably got to tackle all of these dynamics to get the changes you need. So that puts a premium on innovation organization, the way in which you organize your innovation system. That becomes critical when you're trying to tackle a legacy sector. Just as we were having that conversation, Steph, that you pushed for, what's the new ways of organizing around innovation? What are the new models? If you're trying to innovate in a legacy sector like energy or manufacturing, you're going to need new models, and you're going to need to think about the innovation dynamics at each one of these points.

Oh, I don't want to forget. You need change agents, because legacy sectors will not change themselves. You're going to have to have people prepared to push these legacy sectors to adapt change. And maybe it's political leadership. Maybe it's new companies and startups that are going to be the change agents. Maybe it's a new R&D agency. Maybe it's a mix of all of the above. But you're going to need change agents that are prepared to move things in these legacy sectors.

So those are the five models. Some of this calls for us to think about what the governmental role is. Do we need a more activist governmental role in this kind of context? What's the process? And this is particularly important, because this is going to be important in your papers. So the stuff we're talking about here is going to be-- in terms of launching new technologies, this multi-step, five-step process is going to be something that I'm going to ask you to use as you consider the topics you develop in your papers.

Step one-- you can't innovate without innovation. So you're going to need stuff. So you're going to have to have a front-end innovation system that's giving you creative stuff to innovate with. Here's some ideas here. Form critical innovation institutions. Build thinking communities so that you can have a-- you got have a community that's big enough to really get a lot of ideas on the table for innovation to occur. It's Romer. It's the prospective theory here. You've got to have a thinking community that's at a big enough scale that they can really work on creating innovations in a legacy sector.

You need to link the technologists to the operators. They can't be in isolation from each other if you want to implement this stuff. And you can use that island and bridge model, and we'll talk more about this next week when we talk about innovation by great groups. But your innovators need to be connected decision-makers that can effectuate the changes they're going to propose.

Some other steps, and then I'll quit and we can do some Q&A. Step two is, each technology is going to have a different launch path. The launch path for batteries for grid storage is probably going to be a very different launch path than batteries for transport. Launching into the utility sector versus launching into cars, those are different launch pathways. So you're going to have to think about what the launch pathway is for your technology that you want to have enter the marketplace. And then you're going to need to tie support packages to those launch pathways that fit, that are relevant to the particular launch pathways.

A fourth step is, look at the innovation system and the gaps in the innovation system. What are the holes in the system? Where are things dying? For sure it's the valley of death, but what are the other gaps in that system? Because again, the valley of death is only between research and development. You've got all kinds of other stages to get through, too, all the way up into production. What are the gaps in that overall system? So think about what the gaps are, and then think about institutional fixes-- public or private or both-- that fill those gaps.

So this is kind of a menu of how to think about bringing innovation into legacy sectors. And if you all are going to work in the energy field, that's the king of legacy sectors. Right if you want to pursue advanced manufacturing, that is a big-time legacy sector. So we're going to have to have a much more sophisticated innovation organization approach. We're not going to solve these problems by individual awards to individual investigators. That's not going to work. It's not going to get us there. We're going to have to have a whole new delivery system for the innovation system.

Let's see if there's anything else I want to cover. Change agent orchestrates the activity here, and therefore is a pretty critical player. So again, these chapters here are pretty important for your paper, and that five-step process and understanding the dynamics of innovation are pieces that you're going to need in your paper. So Rasheed, it's all yours.

RASHEED: Great. That's actually a great place to start, because I think a lot of us were pretty much in line with what was going on, understanding what a legacy sector is and identifying them. We're pretty much all OK with that. Where I think people started to diverge was deciding on who would play the role of these change agents. A lot of people were looking in their questions to figure out who would be the best poised in certain situations to play that role of a change agent. Would it change for different initiatives? And I think one of the interesting things was, we tried to identify of this community of change agents first. We were all looking for this thinking community for different individuals. So just to start off with, who do you think might be able to play a role, in any particular case, to be the change agent for a legacy sector? And it can be anyone.

AUDIENCE: I'll start it off. This is a complex question. There's different answers. But I think for energy, I think the big thing would be having a breakthrough innovation. Because the main issue right now is, we are worried about climate change, but solar and-- well, solar in general is expensive, and doesn't really do what we need to do, because if you look at the energy market, how much is solar, it's rather small versus our energy needs and also the cost. It's hard to innovate there. But if you can come up with a much, much better innovation-- so maybe fusion. Maybe there's something there. It'd be the difference between if you had to come up with a whole new phone line system versus if you create the cell phone. You might have a huge opportunity there.

And the thing that I think is important this issue is-- Steve Case, the guy who made AOL, has this thin called the third-wave entrepreneur, where it's like-- the thing about innovating in this sector is that, you kind of Uberize it in terms of, there's already an existing society. There's already all these institutions that people-- there's these pathways for this legacy sector. So when you innovate in it, the issue is going to be that you tear those down. And these people will spend their lives dedicating themselves to being part of the sector, and you just cannibalized it.

So it's like, how do you really work with it? And that's a policy issue, but it's also a whole social issue. And I think Uber has been a good preliminary basic example, because-- it's pretty much an appetizer, or a warm-up example. But if somebody did come up with an amazing innovation in energy, like fusion that is relatively cheap and won't take like 10 years to build a reactor, then maybe something like that could happen. Any other thoughts?

AUDIENCE: Maybe another good example, probably almost more disruptive. You mentioned gasoline. It would be-- that's part of what makes Elon Musk a really interesting change agent. He's not just developing the electrics that drive the car, but the charging stations that will go in the house, and then even connected to the solar roofing that will charge those stations. I think you put a lot of thought into, basically, the launch pathway for that technology.

STEPH: That's a really good point, because I think often about the energy value chain in the oil and gas industry-- like upstream, midstream, downstream-- and the flow of value within the industry, and I feel like-- I hadn't thought of that, that Elon is essentially creating not only upstream change, but also midstream change and downstream change, in that he's comprehensively trying to transform the industry and capture all of the value for himself. But I definitely think that perhaps-- or I don't know, maybe I'll pose this question. do you feel like change happens more-- or change is more feasible on the midstream or downstream sector? Because I feel like upstream is kind of that disruption. It's trying to create fusion, which can be challenging, and it can be difficult to have people adopt. But then you get stuck at the midstream or downstream. Do you want to change infrastructure, or do you want to change consumer patterns, is my question.

RASHEED: Steph, quickly define upstream, midstream, downstream for me.

STEPH: Oh, yeah. So upstream is just the production aspects of it-- so the extraction of natural gas and oil. So the derrick systems. Midstream is the infrastructure that's created in order to transport that-- so, say, the railways, the trailers, the trucks, et cetera. And then downstream is the consumer aspects of it-- so the people buying it in their car at scale.

RASHEED: So I think what's interesting about this-- and maybe particularly in energy, but in other fields and other legacy sectors-- creating these thinking communities of change agents, which is the first step, is something that you can't really-- is there a way to organically do this? Because I don't see getting together the heads of all of the big energy corporations together in a room. They're not going to come up with anything super radical. And so there's no-- and it's way different from-- before you could just get the Comptons, the Vannevar Bush in the room, and you're going to get something crazy out. But now, if you're looking to innovate in a sector, you can't really call upon the greatest minds pulled here and then get something out.

AUDIENCE: Can you give me an example [INAUDIBLE] just to [INAUDIBLE]?

AUDIENCE: Well, isn't MIT trying to create a thinking group, or a space where thinking groups can occur, with The Engine? Isn't that one of--

WILLIAM BONVILLIAN: Yeah, or Mighty could be and the Energy Initiative could be seen as a thinking community. It's not simply MIT-based. It brings in all kinds of people from various sectors and places to think through these big challenges. And the big reports that MIT does are the products of these thinking communities as they attempt to think through a new technology area, like solar or the future of the grid.

STEPH: [INAUDIBLE] did you interact at all with the American Petroleum Institute?

SARAH JANE MAXTED: Well, not really. It was there. It was affiliated-- I don't even know how. I think it was out of Office of Science? Or was it Fossil?

WILLIAM BONVILLIAN: Fossil.

SARAH JANE MAXTED: Fossil. I think the Office of Fossil Energy has the relationship. But we didn't interact with them at all.

STEPH: So I think an interesting example for Rasheed within the legacy sector of oil and gas, because I know API does a lot of work in ensuring that the regulatory framework is there for all of the oil companies. And the heads of all the oil companies get together as a group, or provide representatives at meetings, and then they go through and talk about the potential outcomes for the energy industry, and then think collaboratively about what that industrial framework might look like. And obviously, there's a lot of those industries that are trying to create disruptive innovations, because that's the only way they're going to win in the market in the long run.

SARAH JANE MAXTED: Well, I think there-- so there are-- I think something to think about. Most applied programs in government have-- what do you call them, working sessions where they hear industry-- what do they call them? Oh my god. Peer reviews? Oh, not peer reviews. I don't know what they call them. But it's basically where you bring together the community of practice of all of the people that make sense to bring together, in solar and batteries or whatever, and you have a conversation to identify pain points. And all of the ERE does this. The Energy Efficiency [INAUDIBLE] office and I know most other applied programs do as well, to kind of do that.

But I actually think-- to your point about, does it really make sense to get all these execs in a room? Are they really going to move the ball forward? Is actually a really interesting question, because I think we're starting to see more of the exevs in the room with the midstream and the downstream, and all of the-- to actually be innovative, it can't just be the CEOs of these companies. It has to actually be different layers as well. And I think that goes to this-- they can do this kind of model. I don't know if that answered the question.

WILLIAM BONVILLIAN: So I'm going to need to-- if we're going to get out of here, I'm going to need to do a wrap-up of today. And then I want to talk about next week's class, because seven of you will be presenting, and I just want to go through the dynamics of-- that's going to be a different kind of class, so I just want to talk about that. But before we do that, Rasheed, why don't you lay out you know some closing thoughts for us on this piece?

RASHEED: Yeah. So I didn't get to it, and I was hoping to, actually, another question. But another idea that everybody had was identifying policy and policymakers to give this impetus towards making these changes, and making these changes more politically favorable and apparent. And it's difficult in these legacy sectors because, I think, you're facing and you're up against powerful vested interests. And I think it's just, to Martin's point, making the case for, maybe I don't have fusion yet, but it's making it socially applicable or politically palatable to say, if we invest here and get this fusion, we can make this change that can be politically favorable for everybody. And so I think looking at policy incentives, you have to think a little bit bigger than just what is scientifically or technologically possible.

But I think this is a great piece, and this is was a great piece for us because it encourages us to not only think about innovation systems as being differentiating-- you can do different things but achieve the same result-- but also, a lot of people have to buy into and get on board and be coordinated in their actions to achieve feasible and long-term change, because we have a lot bigger problems than we did 50 or 100 years ago.

WILLIAM BONVILLIAN: And a lot of them are tied to these legacy sectors. Thank you. So I'm going to do a quick wrap-up of this class. So our backdrop was Donald Stokes and Pasteur's Quadrant, and that led us into Branscomb and Auerswald. And their underlying-- the focus of innovation policy, frankly, for the last 25 years has been on the problem that they write about, this valley of death problem. And how do you get across it?

But one thing I want to be clear about today is, the valley of death is only one piece of a larger innovation system which stretch a lot further than just research and development, and we've got to think about the gaps in that system overall. But Branscomb and Auerswald really lead us into this foundational problem. And a valley of death is essentially a linear view, and they also point out that, hey, the system is really not linear. It's more complex than that. It's this Darwinian sea.

Ruttan describes for us this parallel universe. We have a set of civilian basic research agencies, and then we have a very differently organized set of defense R&D agencies and mission agencies. And they are very different systems, and one is, from an organizational point of view purposely disconnected. The other is purposely connected, and they are different worlds. We have gotten a lot out of that defense innovation system. We don't normally think about it or talk about it, but the results are quite powerful.

Glenn Fong told us about the models that DoD uses when it's actually going to play a role in civilian sectors, and laid out the byproduct model, the spin-off model, the dual-use model, the industrial base model. So these are all models the DoD uses in practice. We looked at In-Q-Tel, which in some ways is the most extreme model of governmental intervention into the private sector, because it will actually play the role of being a venture capital firm and take ownership positions in companies that it wants to encourage innovation in.

We talked about the new model innovation agency piece. Essentially, the idea here is that the system isn't static. It isn't standing still from the time Vannevar Bush created it. It has been changing pretty dynamically over time. And these outside crises-- the competitiveness challenge in the 1980s, the Sputnik challenge of the later part of the '50s, the energy technology challenge, now this advanced manufacturing challenge-- these are all external elements that have been driving change in the US system and pushing its model to be more of a connected model.

And then the final reading was about innovation in legacy sectors. As we discussed just now, many of our deep societal problems are tied up in these legacy sectors. And these legacy sectors need innovation, and they resist it inherently in their model. And how do you think about innovation organization that's going to start to bring new technology advances, new innovative advances into these legacy sectors that they will be disruptive of their business models? How do we start to think about that? Because we're now into that project, and we need to much more systematically organize our innovation organization approaches to cope with it.

So that takes us through class 6. Next week is going to be a different kind of class, because you guys are going to own it much more than I'm going to own it. So seven of you will be presenters. I'm asking that-- and I've got pretty detailed directions in describing the class. If you've got questions, let me know. But divvy up the innovation organization groups you want to present on.

And as we discussed a few weeks ago, I'd love to have innovation groups that were based on other than dead white males in the group. So if somebody can find some, fine. And talk to me about them. I may be able to help you on sources. If you can't, that's fine, too. And these are very interesting groups, so it's going to be-- whichever one you pick, these are very interesting stories. I think we'll have fun with them.

And the great group model takes us-- we've been talking about institutional innovation. We talked a lot about that today. Innovation at the institutional level. But innovation belongs to people, and they organize themselves in groups, and we have to understand the dynamics of those groups as well. So that's what the object of the class is.

So I'm going to ask the presenters to do three or four slides on your group, and tell us what they accomplished, what they did. Tell us what-- the rules that they followed from the Bennis and Biederman framework, which is one of the several foundational readings that everybody should read. What were the rules that they followed? And then what is a new rule or two that you see them doing that's different in the Bennis and Biederman rule? What's different about your particular group that contributes to our thinking about how great groups operate?