Tag: stanford

Why Do Tech Companies Need Policy Teams More Than Ever? (An Evening with X, Andressen Horowitz & 23AndMe)

On By karentay1 Comment

The Stanford Business School just launched a new Policy Innovation Initiative. Earlier this week, I attended their launch event featuring Sarah Hunter, Policy Director of X (previously Google X), Kathy Hibbs, Chief Regulatory Officer, 23&Me, and Ted Ullyot, Partner of Policy and Regulatory Affairs, Andreessen Horowitz.

Why the need for policy and regulatory thinking within the tech world?

The motivation is simple. In the past decade or so, software innovations have dominated. We’ve seen how great software platforms – sometimes built by tiny crack teams – can scale rapidly in way that completely changes markets. Think Amazon and Ebay for commerce, Facebook, Snapchat, Instagram for social networking, Box, Salesforce, Workday and Slack for enterprise solutions (decode: software that helps us manage our HR, customer relations and intra-office discussions with much less pain). The market is increasingly saturated with software solutions for almost every area of life. Hence, while will continue to see gains in productivity and efficiency in these systems because of Artificial Intelligence, pure software will no longer be the area of rapid technological innovation.

Instead, technological innovation in the next decade will be dominated by technologies spanning hardware (things that are in our physical world) and software (the virtual world). Examples include self-driving cars and surgical robots, which are performing physical functions but controlled by algorithms in the virtual world. A term often used to describe this general area is “cyber-physical systems”.

Here comes the challenge: objects in the physical world are more directly risky to human life than software systems. Furthermore, these objects can harm people who don’t choose to use them – I choose to download Facebook if I can stomach the risks to my personal privacy. On the other hand, even if I never buy my own Self Driving Car, my life could be at risk if someone else owns a faulty one. There is a more acute need to manage risks to the general public.

Hence, the regulatory landscape is stacked against emerging tech. First, Legacy regulations abound to protect consumers from death or physical harm, such as long Food and Drug Administration (FDA) and vehicle/driver-licensing processes. Second, Because of potential harm to human life, regulators are likely to approach the emerging technology from the perspective of ‘mitigating every risk’ (read: adding even more new conditions and clauses). Third, regulations and legislation are typically based on precedent, and are hence biased towards incremental (as opposed to disruptive) improvements in incumbent tech and business models.

Regulatory risk will be the major Go-to-Market hindrance for most emerging tech companies in the next decade; if they fail to address regulations, a company could be dead in the water before they even begin. Policy teams within tech companies exist to minimize this regulatory risk. They advise companies on questions such as:

  1. Who do we need to influence so that regulations fall in our favour? Policy teams often go above regulators to paint visions for politicians: how the emerging technology will solve social problems and create new economic opportunities.
  2. Should we work with regulators to co-create new regulations, or break the regulations? The risk of breaking regulations varies – if you’re able to get widespread support from users (think Uber+AirBnB), you may be able to force regulators into certain positions. It’s more difficult to take this approach for hardware solutions.
  3. If we desire to co-create new regulations, what approach should we take? One company designed their own set of self-driving car regulations, which never came to pass because the technology was pivoting so quickly.
  4. How early should we engage regulators? Generally, it isn’t good to give regulators surprises, but sometimes engaging too quickly before there are good answers on how to mitigate the risks will scare them into coming down hard
  5. Is it even worth trying to enter this market, or should we start where regulations/Governments are more relaxed? For example, most successful drone companies tested outside the U.S.

The role of policy teams in tech companies can be likened to master chess players. They get to know the kings, queens, knights and pawns who influence the regulatory system, and appeal to a range of motivations to move the pieces in their company’s favour.

Each speaker pointed out that regulators aren’t technological dinosaurs who intentionally regulate technology to death (though they are often caricatured this way). They simply have a different bottom line, which is to minimize risks and externalities. Put this way, regulators and innovators can provide a healthy check and balance to each other.

Areas I Hope They Address

I’m excited about this initiative by Stanford Business School and would love to see it be a neutral place for tech and policy to folks to discuss the best approaches to regulating emerging tech. Here are some areas I hope they will address.

How do companies manage the competitive vs collaborative dynamic in lobbying for regulatory change?

On one hand, there are great advantages to be the first mover and defining the regulations in your favour. Kathy from 23&Me shared that if you are able to set precedent, all your competitors have to follow your standards. This locks is a certain competitive advantage. There are other circumstances where working collaboratively is more productive. For example, on the same issue, politicians and regulators might be far more willing to listen to a group of local start-up founders than large multinationals like Google. Smaller companies sometimes have to work through trade associations because they lack the scale needed for lobbying.

Will start-ups lose out as this policy/regulatory expertise becomes more critical to success, yet is dominated by large players?

Small companies are often unable to recruit for the policy/regulatory function because of their resource constraints. This is why VCs like Andreessen Horowitz have policy teams that advise their stable of start-ups. Will more of such advisory services become available to start-ups? Who will provide them?

We are also starting to see coalitions such as the “Partnership in AI” by Google, Facebook, Amazon, IBM and Microsoft – no doubt one of the objectives is to lobby Governments on AI-related policies. How do start-ups fit in? Is there a risk that the agenda is overly swayed by large companies?

An idea for policy teams in tech companies: Go beyond lobbying for regulation; work with Governments to support widespread adoption of emerging tech

One of the themes of the night was how tech companies need to paint a vision to politicians on the benefits of the emerging technology, so that they support favourable regulatory change.

I think we have to go further than persuading politicians to get to the point of favourable regulation. Widespread adoption of emerging technology especially in areas of healthcare, transport and education is hindered by more than regulation. For example, change can’t take place if you don’t inspire, resource, and manage the morale of teams on the ground who are accustomed to existing ways of work and will not change just because a new technology exists. I saw it first hand when I worked at the 40,000-strong Ministry of Education in Singapore.

Without this, politicians will find it difficult to move, even if they agree strongly with tech companies’ visions for the future. Singapore’s Prime Minister, who takes a personal interest in Smart Nation, recently lamented that the whole effort is moving too slowly. This, coming from one of the most efficient Governments in the world, suggests that there are deep-seated issues in achieving widespread adoption of technology.

Here are some things that are essential for widespread adoption of emerging tech, but that Governments/Politicians will not be able to tackle alone:

  • Painting a vision that shows implementers and constituents how emerging tech will exponentially improve their reality;
  • Tying concrete benefits to these emerging technologies such as creating new local jobs;
  • Actively advocating for programs that help people deal with the downsides disruptive technology, such as re-training for displaced workers.

These are areas that policy teams within tech companies can consider as they seek to move chess pieces in their favour: not just to achieve favourable regulations, but to see widespread adoption of their technologies in regulated sectors.

<if you work in a policy/reg team within a tech company and already work on these areas, I would love to hear from you>

In the coming posts, I will also feature folks who work in policy/regulatory teams within Silicon Valley and Singaporean companies. Look out for those!

“What I Wish They Knew”: 5 Answers from a Government Data Scientist

On By karentay2 Comments

One of the goals of this blog is to bridge the worlds of tech and government: I believe we can do so much more by working together, yet we often don’t understand each other deeply enough to begin. I will be starting the “What I Wish They Knew” series, which features people who are familiar with both these communities.

The first person to kick off this series is none other than my husband, Kenneth, who worked in Singapore’s Government Data Science unit before pursuing a PhD in Statistics at Stanford. By the way, you can find out more about Singapore’s Government Data Science unit on their blog: https://blog.gds-gov.tech/ – highly recommended.

kenneth handsome.jpg
Kenneth at The Hive, where the Government Digital Services team is located.

  1. How did you get into Government Data Science? 

I’ve had a strong interest in mathematics and related quantitative fields like statistics and computer science for as long as I can remember, and studied math at Princeton as an undergrad. [NB: Kenneth’s mathematics blog can be found here.]

I began my career in the Singapore public service, hoping to give back to society in the small ways I could. While I was at the Ministries of Defence and Environment, most of my work did not involve any advanced math or data analysis.

I missed the intellectual challenge of quantitative thinking, and started taking online courses on the side. Andrew Ng’s “Machine Learning” course on Coursera first piqued my interest in data science. I learnt how we can use a small toolbox of algorithms to extract a whole lot of information from data, and I thought to myself, “How cool would it be if I could use some of these techniques in my work?”

Fortunately, a unit in Government was being set up to do just that: use state-of-the-art data analysis techniques to inform policy decision-making. I joined the Government Data Science unit in 2015 as a consultant. My work experience in policy and my quantitative undergraduate training put me in a rare position to understand the mindsets of both the policy officer and the data scientist. As such, I felt that I was an effective translator between the 2 parties.

  1. How about an example – what is one meaningful thing you did in Government Data Science?

An agency was very concerned about congestion during peak hours at the checkpoint between Singapore and Malaysia. Understanding cargo traffic patterns could help them design policies to reduce congestion. All they had was tens of millions of “permit data” entries, which captured the time that the cargo truck carrying the permit passed through a checkpoint, the industry code and value of the goods carried. I worked with the agency to define useful problem statements to shape the direction of analysis. One example: what are the top 5 industries moving cargo during peak hours? (Since policy interventions would be done at an industry-level).

Next, since each truck could hold multiple permits and the number of trucks was what we cared about, we went through a (non-trivial) process of turning “permit data” into “truck data”. We were able to identify the top industries moving cargo during peak hours, and further narrowed this group to those who were moving cargo on the busiest roads. We were also able to develop hypotheses on what influenced industry behaviour.

After completing the analysis, I shaped the narrative of our presentation in a way that delivered impactful policy insights, ruthlessly cutting down on unnecessary details. This is often the most painful part of the process for a data scientist – it’s so tempting to want to show ALL the great analysis we did.

After dozens of hours of work, there was nothing more satisfying in seeing the audience’s facial expressions saying, in effect, “before I was blind, now I see”! They never had a picture of the cargo traffic patterns until our analysis was done, and could now act upon it to improve congestion.

  1. What is one thing you wish non-data scientists knew about working with data scientists? 

That good data analysis requires significant collaboration between the data scientist and you, the domain expert.

Some people view the relationship between the domain expert and the data scientist as follows:

  1. Domain expert gives data scientist a bunch of Excel files.
  2. Data scientist crunches the numbers and churns out a report or presentation 3 months later. After all, the data scientist knows everything about data and that’s what we are paying them to do, right?

Nothing could be further from the truth! Domain expertise can speed up the data analysis process tremendously and direct it meaningfully, resulting in greater value from the project. Let me give two examples of this.

First, explaining the data to the data scientist, down to what each column means and how the data was collected, will save him/her much second-guessing angst. For example, if the patient check-out time was 18:00:00, does it mean that the person checked out at 6pm, or does it mean that the clinic closed at 6pm, and so everyone who hadn’t checked out yet was given a standard check-out time? Explaining the data will also give the data scientist a better sense of which variables are of greater importance and deserve more attention. In the example above, what does “checking-out” mean anyway, and is it significant?

Second, domain experts can pick up on insights that would escape data scientists. For example, a data scientist finds that Chromosome 21 seems to have an impact on a health outcome. Is that expected? Does it confirm some of the other hunches that we have? Or is it something completely unexpected, that suggests that the model is wrong? These are questions that a data scientist is unlikely to have any intuition about. However, with feedback from the domain expert, the data scientist can quickly decide to pursue or drop lines of inquiry.

  1. As a policy-maker, what is one thing you wish more data scientists paid attention to?

That data analysis is not for the data scientist, but for the policy-maker (or client). As such, good data analysis always puts findings and insights in the proper context.

Consider the sentence: “Our prediction model for which patients will be re-admitted over the next 6 months is 34.56% more accurate than the existing model.” Upon seeing this sentence, several questions come to mind:

  • What do you mean by accuracy? Is the measure for accuracy that you are using appropriate? (See this https://en.wikipedia.org/wiki/Confusion_matrix for a whole zoo of accuracy measures.)
  • 56% seems overly precise: can we really compare performance down to 2 decimal places? (35% might be better.)
  • Does 35% more accuracy translate to a meaningful difference? For example, will this allow us to tailor our services better to 100,000 patients, or 10 patients? (If possible, relate the finding to something the policy-maker cares about, like dollars or man-hours saved.)
  • Is this even a meaningful thing to predict?? (Hopefully, the domain expert would have said so. See answer to question 2.)

Good data analysis also provides enough detail to illuminate, but not too much till it confuses. For example, saying nothing about the modeling process could make me wonder whether you did your homework in choosing the most appropriate model (and whether I should have spent all that money hiring you). At the other extreme, I will not appreciate going through slide after slide of raw SAS/R output.

At this point I cannot overstate the importance of appropriate data visualisation. These visualisations have to be thought through: good charts clarify, poor ones confuse. Unfortunately, it’s a lot easier to make the latter. (See this for examples not to follow.)

kenneth-graph
Truly, a (well-designed) picture is worth a thousand words

 

5. What is your hope for the field of data science?

The economist Ronald Coase famously said “Torture the data, and it will confess to anything.” In an era of subjective reporting and “fake news”, this concern is more pertinent than ever.

My hope is that the general population will have enough statistical knowledge so that they can call a bluff when they see one, and demand quantitative evidence for decisions their leaders make. To this end, I hope to see reforms in statistics education at the high-school level so that it becomes a subject that people feel is relevant and interesting, rather than abstract and theoretical (which is often how it is taught today).

Thanks for reading! Know anyone who should be featured in this series? Do let me know at karentay@gmail.com.