Dr Finkel gave an introductory address at theInnovation Metrics Review International Workshop on 13 March 2019 in Canberra. A full version of the speech is below or you can download a pdf here.

I acknowledge the Ngunnawal people who are the traditional custodians of the land on which we meet and pay my respects to their Elders past and present. I extend this respect to all Aboriginal and Torres Strait Islander peoples in attendance today.

***

You all know the old joke about a police officer who sees a drunk searching for something under a streetlight and asks what the man has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the officer asks the drunk if he is sure he lost them here, and the man replies, no, he lost them in the park. The officer asks why he is searching here, and the man replies, “the light is much better here”.

The moral: we look where it’s easy, not necessarily where it’s useful.

And that’s where the story ends.

But I say it’s where the real story begins.

Because the police officer could shake her head and walk away in frustration…

…or she could persuade the man to get a torch and go to the park…

… or even better: she could persuade the local council to move the streetlight.

How do we, the police officers, achieve the right result?

To start, we need to focus on the outcome.

In our case, it’s simple: what we all want is increased productivity and higher living standards.

Innovation is the key that unlocks them – and metrics are the light with which we find the key.

So that’s why Chief Economist, Mark Cully, and I teamed up – as Good Cop, and Bad Cop, I’ll let Mark decide which is which – to help this country to move to the park and find the damn keys.

My own journey into the police force began several years ago.

Like most people in my field, I’d always accepted that innovation was hard to define and even harder to measure, but the measures we had were no doubt the best we’d got.

I began to suspect that something wasn’t right when I was President of the Australian Academy of Technology and Engineering, and somehow was made to feel guilty for Australia coming up in last place on the measurement of collaboration between universities and innovation intensive companies.

As Chief Scientist, colleagues expected me to travel around the country berating our research institutions about our woeful record.

But it was also my job to travel around the country launching business-university collaborations.

And I discovered at the first university I visited that they had lots of collaborations with industry. So I asked the Vice-Chancellor how he explained the discrepancy – and he told me that the problem must lie in all the other universities.

Funny, at the next university I visited, I made the same observation, asked the same question and got the same answer!

Something wasn’t right. I discussed the problem extensively with Mark Cully. Eventually, I called some colleagues at two of our leading universities and each of them had nearly as many collaborations as we reported to the OECD for the whole country. So Australia was coming up as infeasibly low, dead last in the list, at about 3% of innovation active companies. It didn’t seem plausible.

And I must say that at a gut level I am equally surprised that the leading countries on this particular metric, at the other end of the spectrum, have apparently achieved a collaboration rate of nearly 70%.

This dead-last collaboration statistic for Australia was driving a frenzy of negative commentary. All the while, our economy is outperforming most of the OECD…

…we have had 27 years of recession free growth – not achieved by any other country since GDP records began…

…we have a world-class health-care system, and we’re a world-class exporter of minerals, agricultural products and educational services…

… and still, we were convinced that we were somehow devoid of innovation.

None of the policy measures we adopted seemed to make a measurable difference.

As Chief Scientist, I felt that the discrepancy between what the data were saying and what the Australian innovation system was actually achieving could no longer be ignored.

We were stubbing our toe on the streetlight that was supposed to be helping us find something useful.

Worse, we were starting to believe that the keys didn’t actually exist.

It all came to a head for me in my role as Deputy Chair of Innovation and Science Australia.

We were asked by the Prime Minister for a comprehensive review of the Australian innovation system.

This request was for the obvious reason that in order for governments to implement innovation policy they need to be able to measure innovation, to decide where to intervene, and to determine whether their interventions have been successful.

Inherent in the purpose of the review is that our audience is government rather than business, because published indicators are generally too broad for management purposes.

It is obviously important to have meaningful measures of performance – a scorecard of useful metrics. Not too many and not too few.

Instead, we were constantly frustrated by measures that were incomplete, likely to be affected by erroneous or non-comparative data, or wrongly adapted to our economy.

My pet peeve is the Australian mining industry. Every industry insider, here and globally, will tell you that this country is a world leader in mining innovation, with remotely controlled underground drilling machines, possibly the largest autonomous vehicle fleet in the world, algorithmically determined process quality control and remote control rooms to optimise the overall operations.

And now they are adopting artificial intelligence approaches to make their operations even more efficient.

And yet, in most innovation metrics, the mining industry is basically invisible. Why? Because a lot of their innovation is in house, and even more comes from the R&D buried in supply contracts.

Even worse, on minor metrics such as the percentage of high tech exports, since the mining industry’s actual exported product hasn’t changed in ten million years and is regarded by many as 'dirt’, our mining exports do not contribute to the top 'high-tech’ line in the ratio. However, they do contribute to the bottom 'total exports’ line of the ratio, which means that every time our mining industry innovates and captures a greater share of the world market this particular measure of innovation gets worse, not better.

I started to use the phrase “hidden innovation” to refer to important innovation that is fundamentally invisible to the existing innovation metrics.

I’ve already mentioned mining, but what about education? International education is reported as bringing in $30 billion of revenue to Australia. The industry was developed by innovative Vice-Chancellors, but I can’t see where its growth shows up in any of the innovation metrics.

The problem is probably because, in part, the existing innovation metrics focus on the linear process of research and development leading to new products. That works well for countries with strong manufacturing and high tech industries, but in Australia only 7% of our workforce is employed in manufacturing.

Another problem we encountered is that the methodology used for business surveys is so different between countries. Some are compulsory, while others are voluntary. The surveys are administered at different intervals and they use different reference periods. These differences contribute to statistical noise that sometimes dwarfs the signal.

So, in one of its recommendations, Innovation and Science Australia called for a review of the existing innovation metrics for accuracy and adequacy.

And I became a cop.

***

There are several goals for this Review.

First, in the short term, to improve data sources and metrics that are not quite fit for purpose, or are in some way inaccurate, or do not allow direct country comparisons.

Second, to identify and fill measurement gaps, so that innovation is measured in the hitherto invisible, or perhaps difficult to see, sectors of our economy such as mining, education and hundreds of thousands of small businesses.

Third, to build a short list of metrics – what I call a scorecard – that will be of policy relevance to government.

It is a task for Australia, but at the same time we aim for this to be a project for the world: our measures have to be comprehensible, credible and comparable to our global partners.

To start, we have to think about what, in a nutshell, is innovation.

You all know the formal definition, but my simplest definition is doing things differently and doing them better. I am attracted to this simple definition for a few reasons.

First, it is not locked into the linear definition of research being the starting place of all innovation. Instead, in addition to evolving from research, innovation arises from an idea in the middle of the night or the creative outputs from a brainstorming meeting.

Second, this definition eliminates consideration of the trivial.

Third, my definition is short enough that it is easy to remember!

This definition of innovation arguably applies to this international workshop and the Innovation Metrics Review. If we are going to be innovative, we need to do innovation measurement differently and we need to do it better.

Dare I say it? – we need to be innovative in our approach to innovation measurement.

A lot of attention internationally is focused on advanced manufacturing and high tech.

And so it should be, because these are important.

But there is so much more to our economy.

If we get it right, we will make visible the innovation in traditional industries such as mining, health, education, banking and agriculture. These sectors have a major impact on people’s lives, and they are critical to the economy.

I want to stress that this is not an exercise in making Australia look better than it is.

It is an exercise in giving us useful information.

That includes the problems we’re not seeing.

I also want to stress that we are not blind to the limitations of data when it comes to capturing a complex phenomenon like innovation in policy–relevant terms.

That is why, for example, in Australia we have started a process to try to understand the research relationship between universities and end users such as industry and government departments.

A few years ago, work began on a fair and credible metric for university impact – first through a pilot program led by the Australian Academy of Technology and Engineering, called Research Engagement for Australia; and then through our national research funding body, the Australian Research Council.

The new ARC Engagement and Impact metric is now a compulsory data gathering exercise for all Australian universities, collected last year, with results expected soon.

One thing to note is that after a lot of design the ARC decided that data alone would not be enough and that a series of short impact statements would be required. These will be evaluated by expert panels. This will be difficult and expensive but the conclusion was that impact statements will provide insights that would otherwise not be available.

Perhaps there is a role for impact statements, evaluated by expert panels, in innovation measurement. This would be hard work and fraught with risks, but if that is the only way to measure innovation in some sectors we should be open minded about the possibility.

It could be another important step to moving the streetlight – and finding the keys.

The Innovation Metrics Review taskforce, my co-chair Mark Cully, the Academy of Technology and Engineering, the Steering Committee and the Expert Reference Panel have done a lot of excellent work to get us to this point.

But we don’t have a solution in hand yet.

The purpose of this workshop is to bring into the open innovative thinking about innovation measurement.

We need to come to meaningful conclusions so that we can finish our report by the end of June.

I urge you not to be incremental. Our goal must be to go beyond tweaking.

We must avoid doing things differently for the sake of it, but be prepared to recommend new ways to do it better.

Above all, whatever we recommend must go beyond the academic and be useful for policy formulation.

I thank every one of you for what you have contributed so far and I thank you in advance for what you will contribute to the remainder of this workshop.

And, for the sake of all of us, may the Force be with you.

THANK YOU