Author Archives: Sarah O'Rourke

Expanding our work with the open source security community

Posted by Eduardo Vela, Vulnerability Collector, Google 

At Google, we’ve always believed in the benefits and importance of using open source technologies to innovate. We enjoy being a part of the community and we want to give back in new ways. As part of this effort, we are excited to announce an expansion of our Google Vulnerability Rewards Program (VRP) to cover all the critical open-source dependencies of Google Kubernetes Engine (GKE). We have designed this expansion with the goal of incentivizing the security community to work even more closely with open source projects, supporting the maintainers whose work we all rely on.

The CNCF, in partnership with Google, recently announced a bug bounty program for Kubernetes that pays up to $10,000 for vulnerabilities discovered within the project. And today, in addition to that, we are expanding the scope of the Google VRP program to also include privilege escalation bugs in a hardened GKE lab cluster we've set up for this purpose. This will cover exploitable vulnerabilities in all dependencies that can lead to a node compromise, such as privilege escalation bugs in the Linux kernel, as well as in the underlying hardware or other components of our infrastructure that could allow for privilege escalation inside a GKE cluster.

How it works
We have set up a lab environment on GKE based on an open-source Kubernetes-based Capture-the-Flag (CTF) project called kCTF. Participants will be required to:
  • Break out of a containerized environment running on a Kubernetes pod and,
  • Read one of two secret flags: One flag is on the same pod, and the other one is in another Kubernetes pod in a different namespace.
Flags will be changed often, and participants need to submit the secret flag as proof of successful exploitation. The lab environment does not store any data (such as the commands or files used to exploit it), so participants need the flags to demonstrate they were able to compromise it.

The rewards will work in the following way:
  • Bugs that affect the lab GKE environment that can lead to stealing both flags will be rewarded up to 10,000 USD, but we will review each report on a case-by-case basis. Any vulnerabilities are in scope, regardless of where they are: Linux, Kubernetes, kCTF, Google, or any other dependency. Instructions on how to submit the flags and exploits are available here.
  • Bugs that are 100% in Google code, qualify for an additional Google VRP reward.
  • Bugs that are 100% in Kubernetes code, qualify for an additional CNCF Kubernetes reward.
Any vulnerabilities found outside of GKE (like Kubernetes or the Linux kernel) should be reported to the corresponding upstream project security teams. To make this program expansion as efficient as possible for the maintainers, we will only reward vulnerabilities shown to be exploitable by stealing a flag. If your exploit relies on something in upstream Kubernetes, the Linux Kernel, or any other dependency, you need to report it there first, get it resolved, and then report it to Google. See instructions here.

The GKE lab environment is built on top of a CTF infrastructure that we just open-sourced on GitHub. The infrastructure is new, and we are looking forward to receiving feedback from the community before it can be actively used in CTF competitions. By including the CTF infrastructure in the scope of the Google VRP, we want to incentivise the community to help us secure not just the CTF competitions that will use it, but also GKE and the broader Kubernetes ecosystems.

In March 2020, we announced the winner for the first Google Cloud Platform (GCP) VRP Prize and since then we have seen increased interest and research happening on Google Cloud. With this new initiative, we hope to bring even more awareness to Google Cloud by experienced security researchers, so we can all work together to secure our shared open-source foundations.

Introducing portability of Google Authenticator 2SV codes across Android devices

Posted by Dongjing He, Software Engineer; Teddy Katz, Software Engineer; Christiaan Brand, Product Manager

Today is World Password Day, and we found it fitting to release an update that'll make it even easier for users to manage Google Authenticator 2-Step Verification (2SV) codes across multiple devices. We are introducing one of the most anticipated features - allowing users to transfer their 2SV secrets, the data used to generate 2SV codes across devices that have Google Authenticator installed. For instance, when upgrading from an old phone to a new phone. This feature has started rolling out and is available in the latest version (5.10) of Google Authenticator on Android.



Transferring accounts from one device to another with Google Authenticator

Using 2SV, 2-Factor Authentication (2FA) or Multi-Factor Authentication (MFA) is critical to protecting your accounts from unauthorized access. With these mechanisms, users verify their identity through their password and an additional proof of identity, such as a security key or a passcode.

Google Authenticator makes it easy to use 2SV on accounts. In addition to supplying only a password when logging in, a user also enters a code generated by the Google Authenticator app on their phone. This is a safer alternative, used by millions of users, compared to passcodes via text message.

Users place their trust in Google Authenticator to keep their accounts safe. As a result, security is always a high priority. We made several explicit design decisions to minimize the attack surface while increasing the overall usability of the app. 
  • We ensured that no data is sent to Google’s servers during the transfer -- communication is directly between your two devices. Your 2SV secrets can’t be accessed without having physical access to your phone and the ability to unlock it.
  • We implemented a variety of alerting mechanisms and in-app logs to make sure users are aware when the transfer function has been used.

You can find more information about the Google Authenticator and its usage guide here.

Research Grants to support Google VRP Bug Hunters during COVID-19

Posted by Anna Hupa, Senior Strategist, Trust & Safety at Google

In 2015, we launched our Vulnerability Research Grant program, which allows us to recognize the time and efforts of security researchers, including the situations where they don't find any vulnerabilities. To support our community of security researchers and to help protect our users around the world during COVID-19, we are announcing a temporary expansion of our Vulnerability Research Grant efforts.

In light of new challenges caused by the coronavirus outbreak, we are expanding this initiative by  creating a COVID-19 grant fund. As of today, every Google VRP Bug Hunter who submitted at least two remunerated reports from 2018 through April 2020 will be eligible for a $1,337 research grant. We are dedicating these grants to support our researchers during this time. We are committed to protecting our users and we want to encourage the research community to help us identify threats and to prevent potential vulnerabilities in our products.

We understand the individual challenges COVID-19 has placed on the research community are different for everyone and we hope that these grants will allow us to support our Bug Hunters during these uncertain times. Even though our grants are intended to recognize the efforts of our frequent researchers regardless of their results, as always, bugs found during the grant are eligible for regular rewards per the Vulnerability Reward Program (VRP) rules. We are aware that some of our partners might not be interested in monetary grants. In such cases, we will offer the option to donate the grant to an established COVID-19 related charity and within our discretion, will monetarily match these charitable donations.

For those of you who recently joined us or are planning to start, it’s never too late. We are committed to continue the Vulnerability Research Grant program throughout 2020, so stay tuned for future announcements and follow us on @GoogleVRP!

Introducing our new book “Building Secure and Reliable Systems”

Posted by Royal Hansen, VP of Security Engineering, Google


For years, I’ve wished that someone would write a book like this. Since their publication, I’ve often admired and recommended the Google Site Reliability Engineering (SRE) books—so I was thrilled to find that a book focused on security and reliability was already underway when I arrived at Google. Ever since I began working in the tech industry, across organizations of varying sizes, I’ve seen people struggling with the question of how security should be organized: Should it be centralized or federated? Independent or embedded? Operational or consultative? Technical or governing? The list goes on and on.

Both SRE and security have strong dependencies on classic software engineering teams. Yet both differ from classic software engineering teams in fundamental ways:
  • Site Reliability Engineers (SREs) and security engineers tend to break and fix, as well as build.
  • Their work encompasses operations, in addition to development.
  • SREs and security engineers are specialists, rather than classic software engineers.
  • They are often viewed as roadblocks, rather than enablers.
  • They are frequently siloed, rather than integrated in product teams.
For many years, my colleagues and I have argued that security should be a first-class and embedded quality of software. I believe that embracing an SRE- inspired approach is a logical step in that direction. As my understanding of the intersection between security and SRE has deepened, I’ve become even more certain that it’s important to more thoroughly integrate security practices into the full lifecycle of software and data services. The nature of the modern hybrid cloud—much of which is based on open source software frameworks that offer interconnected data and microservices—makes tightly integrated security and resilience capabilities even more important.

At the same time, enterprises are at a critical point where cloud computing, various forms of machine learning, and a complicated cybersecurity landscape are together determining where an increasingly digital world is going, how quickly it will get there, and what risks are involved.

The operational and organizational approaches to security in large enterprises have varied dramatically over the past 20 years. The most prominent instantiations include fully centralized chief information security officers and core infrastructure operations that encompass firewalls, directory services, proxies, and much more—teams that have grown to hundreds or thousands of employees. On the other end of the spectrum, federated business information security teams have either the line of business or technical expertise required to support or govern a named list of functions or business operations. Somewhere in the middle, committees, metrics, and regulatory requirements might govern security policies, and embedded Security Champions might either play a relationship management role or track issues for a named organizational unit. Recently, I’ve seen teams riffing on the SRE model by evolving the embedded role into something like a site security engineer, or into a specific Agile scrum role for specialist security teams.

For good reasons, enterprise security teams have largely focused on confidentiality. However, organizations often recognize data integrity and availability to be equally important, and address these areas with different teams and different controls. The SRE function is a best-in-class approach to reliability. However, it also plays a role in the real-time detection of and response to technical issues—including security- related attacks on privileged access or sensitive data. Ultimately, while engineering teams are often organizationally separated according to specialized skill sets, they have a common goal: ensuring the quality and safety of the system or application.

In a world that is becoming more dependent upon technology every year, a book about approaches to security and reliability drawn from experiences at Google and across the industry is an important contribution to the evolution of software development, systems management, and data protection. As the threat landscape evolves, a dynamic and integrated approach to defense is now a basic necessity. In my previous roles, I looked for a more formal exploration of these questions; I hope that a variety of teams inside and outside of security organizations find this discussion useful as approaches and tools evolve. This project has reinforced my belief that the topics it covers are worth discussing and promoting in the industry—particularly as more organizations adopt DevOps, DevSecOps, SRE, and hybrid cloud architectures along with their associated operating models. At a minimum, this book is another step in the evolution and enhancement of system and data security in an increasingly digital world.

The new book can be downloaded for free from the Google SRE website, or purchased as a physical copy from your preferred retailer.

Announcing our first GCP VRP Prize winner and updates to 2020 program

Posted by Harshvardan Sharma, Information Security Engineer, Google


Last year, we announced a yearly Google Cloud Platform (GCP) VRP Prize to promote security research of GCP. Since then, we’ve received many interesting entries as part of this new initiative from the security research community. Today, we are announcing the winner as well as several updates to our program for 2020.

After careful evaluation of all the submissions, we are excited to announce our winner of the 2019 GCP VRP prize: Wouter ter Maat, who submitted a write-up about Google Cloud Shell vulnerabilities. You can read his winning write-up here.

There were several other excellent reports submitted to our GCP VRP in 2019. To learn more about them watch this video by LiveOverflow, which explains some of the top submissions in detail.

To encourage more security researchers to look for vulnerabilities in GCP and to better reward our top bug hunters, we're tripling the total amount of the GCP VRP Prize this year. We will pay out a total of $313,337 for the top vulnerability reports in GCP products submitted in 2020. The following prize amounts will be distributed between the top 6 submissions:
  • 1st prize: $133,337
  • 2nd prize: $73,331
  • 3rd prize: $73,331
  • 4th prize: $31,337
  • 5th prize: $1,001
  • 6th prize: $1,000

Like last year, submissions should have public write-ups in order to be eligible for the prize. The number of vulnerability reports in a single write-up is not a factor. You can even make multiple submissions, one for each write-up. These prizes are only for vulnerabilities found in GCP products. If you have budget constraints regarding access to testing environments, you can use the free tier of GCP. Note that this prize is not a replacement of our Vulnerability Reward Program (VRP), and that we will continue to pay security researchers under the VRP for disclosing security issues that affect Google services, including GCP. Complete details, terms and conditions about the prize can be found here.


Thank you to everyone who submitted entries in 2019! Make sure to nominate your VRP reports and write-ups for the 2020 GCP VRP prize here before December 31, 2020 at 11:59 GMT.

How Google does certificate lifecycle management

Posted by Siddharth Bhai and Ryan Hurst, Product Managers, Google Cloud 

Over the last few years, we’ve seen the use of Transport Layer Security (TLS) on the web increase to more than 96% of all traffic seen by a Chrome browser on Chrome OS. That’s an increase of over 35% in just four years, as reported in our Google Transparency Report. Whether you’re a web developer, a business, or a netizen, this is a collective achievement that’s making the Internet a safer place for everyone.


Percentage of pages loaded over HTTPS in Chrome by platform (Google Transparency Report)

The way TLS is deployed has also changed. The maximum certificate validity for public certificates has gone from 5 years to 2 years (CA/Browser Forum), and that will drop to 1 year in the near future. To reduce the number of outages caused by manual certificate enrollments, the Internet Engineering Task Force (IETF) has standardized Automatic Certificate Management Environment (ACME). ACME enables Certificate Authorities (CAs) to offer TLS certificates for the public web in an automated and interoperable way. 

As we round off this exciting tour of recent TLS history, we’d be remiss if we didn’t mention Let’s Encrypt - the first publicly trusted non-profit CA. Their focus on automation and TLS by default has been foundational to this massive increase in TLS usage. In fact, Let’s Encrypt just issued their billionth (!) certificate. Google has been an active supporter of Let’s Encrypt because we believe the work they do to make TLS accessible is important for the security and resilience of the Internet's infrastructure. Keep rocking, Let’s Encrypt!

Simplifying certificate lifecycle management for Google’s users

These are important strides we are making collectively in the security community. At the same time, these efforts mean we are moving to shorter-lived keys to improve security, which in-turn requires more frequent certificate renewals. Further, infrastructure deployments are getting more heterogeneous. Web traffic is served from multiple datacenters, often from different providers. This makes it hard to manually keep tabs on which certificates need renewing and ensuring new certificates are deployed correctly. So what is the way forward? 

With the adoption numbers cited above, it’s clear that TLS, Web PKI, and certificate lifecycle management are foundational to every product we and our customers build and deploy. This is why we have been expanding significant effort to enable TLS by default for our products and services, while also automating certificate renewals to make certificate lifecycle management more reliable, globally scalable, and trustworthy for our customers. Our goal is simple: We want to ensure TLS just works out of the box regardless of which Google service you use.

In support of that goal, we have enabled automatic management of TLS certificates for Google services using an internal-only ACME service, Google Trust Services. This applies to our own products and services, as well as for our customers across Alphabet and Google Cloud. As a result, our users no longer need to worry about things like certificate expiration, because we automatically refresh the certificates for our customers. Some implementation highlights include:

  • All Blogger blogs, Google Sites, and Google My Business sites now get HTTPS by default for their custom domains.
  • Google Cloud customers get the benefits of Managed TLS on their domains. So:
    • Developers building with Firebase, Cloud Run, and AppEngine automatically get HTTPS for their applications.
    • When deploying applications with Google Kubernetes Engine or behind Google Cloud Load Balancing (GCLB), certificate management is taken care of if customers choose to use Google-managed certificates. This also makes TLS use with these products easy and reliable.
Performance, scalability, and reliability are foundational requirements for Google services. We have established our own publicly trusted CA, Google Trust Services to ensure we can meet those criteria for our products and services. At the same time, we believe in user choice. So even as we make it easier for you to use Google Trust Services, we have also made it possible across Google’s products and services to use Let’s Encrypt. This choice can be made easily through the creation of a CAA record indicating your preference.

While everyone appreciates TLS working out of the box, we also know power users have specialized needs. This is why we have provided rich capabilities in Google Cloud Load Balancing to let customers control policies around TLS termination. 

In addition, through our work on Certificate Transparency in collaboration with other organizations, we have made it easier for our customers to protect their and their customers’ brands by monitoring the WebPKI ecosystem for certificates issued for their domains or those that look similar to their domains, so they can take proactive measures to stop any abuse before it becomes an issue. For example, Facebook used Certificate Transparency Logs to catch a number of phishing websites that tried to impersonate their services. 

We recognize how important security, privacy, and reliability are to you and have been investing across our product portfolio to ensure that when it comes to TLS, you have the tools you need to deploy with confidence. Going forward, we look forward to a continued partnership to make the Internet a safer place together.

Improving Malicious Document Detection in Gmail with Deep Learning

Posted by Elie Bursztein, Security & Anti-Abuse Research Lead; David Tao, Software Engineer; Neil Kumaran, Product Manager, Gmail Security 


Gmail protects your incoming mail against spam, phishing attempts, and malware. Our existing machine learning models are highly effective at doing this, and in conjunction with our other protections, they help block more than 99.9% of threats from reaching Gmail inboxes.

One of our key protections is our malware scanner that processes more than 300 billion attachments each week to block harmful content. 63% percent of the malicious documents we block differ from day to day. To stay ahead of this constantly evolving threat, we recently added a new generation of document scanners that rely on deep learning to improve our detection capabilities. We’re sharing the details of this technology and its early success this week at RSA 2020.

Since the new scanner launched at the end of 2019, we have increased our daily detection coverage of Office documents that contain malicious scripts by 10%. Our technology is especially helpful at detecting adversarial, bursty attacks. In these cases, our new scanner has improved our detection rate by 150%. Under the hood, our new scanner uses a distinct TensorFlow deep-learning model trained with TFX (TensorFlow Extended) and a custom document analyzer for each file type. The document analyzers are responsible for parsing the document, identifying common attack patterns, extracting macros, deobfuscating content, and performing feature extraction.


Strengthening our document detection capabilities is one of our key focus areas, as malicious documents represent 58% of the malware targeting Gmail users. We are still actively developing this technology, and right now, we only use it to scan Office documents.

Our new scanner runs in parallel with existing detection capabilities, all of which contribute to the final verdict of our decision engine to block a malicious document. Combining different scanners is one of the cornerstones of our defense-in-depth approach to help protect users and ensure our detection system is resilient to adversarial attacks.

We will continue to actively expand the use of artificial intelligence to protect our users’ inboxes, and to stay ahead of attacks.

Titan Security Keys – now available in Austria, Canada, France, Germany, Italy, Japan, Spain, Switzerland, and the UK

Posted by Christiaan Brand, Product Manager, Google Cloud 

Security keys provide the strongest protection against phishing attacks. That’s why they are an important feature of the Advanced Protection Program that provides Google’s strongest account protections for users that consider themselves at a higher risk of targeted, sophisticated attacks on their personal or work Google Accounts.

Last year, we made the Titan Security Key bundle with USB-A/NFC and Bluetooth/USB/NFC keys available in Canada, France, Japan, the UK, and the US. Starting today, USB-C Titan Security Keys are available in those countries, and the bundle and USB-C Titan Security Keys are now available on the Google Store in Austria, Germany, Italy, Spain, and Switzerland.

Titan Security Keys are now available in 10 countries

Security keys use public-key cryptography to verify your identity and URL of the login page so that an attacker can’t access your account even if they have your username or password. Unlike other two-factor authentication (2FA) methods that try to verify your sign-in, security keys support FIDO standards that provide the strongest protection against automated bots, bulk phishing attacks, and targeted phishing attacks.

We highly recommend users at a higher risk of targeted attacks (e.g., political campaign teams, activists, journalists, IT administrators, executives) to get Titan Security Keys and enroll into the Advanced Protection Program (APP). If you’re working in a federal political campaigns team in the US, you can now request free Titan Security Keys via Defending Digital Campaigns and get help enrolling into the APP. Bulk orders are also available for enterprise organizations in select countries.

You can also use Titan Security Keys for any site where FIDO security keys are supported for 2FA, including your personal or work Google Account, 1Password, Bitbucket, Bitfinex, Coinbase, Dropbox, Facebook, GitHub, Salesforce, Stripe, Twitter, and more.

Say hello to OpenSK: a fully open-source security key implementation

Posted by Elie Bursztein, Security & Anti-abuse Research Lead, and Jean-Michel Picod, Software Engineer, Google 


Today, FIDO security keys are reshaping the way online accounts are protected by providing an easy, phishing-resistant form of two-factor authentication (2FA) that is trusted by a growing number of websites, including Google, social networks, cloud providers, and many others. To help advance and improve access to FIDO authenticator implementations, we are excited, following other open-source projects like Solo and Somu, to announce the release of OpenSK, an open-source implementation for security keys written in Rust that supports both FIDO U2F and FIDO2 standards.

Photo of OpenSK developer edition: a Nordic Dongle running the OpenSK firmware on DIY case

By opening up OpenSK as a research platform, our hope is that it will be used by researchers, security key manufacturers, and enthusiasts to help develop innovative features and accelerate security key adoption.

With this early release of OpenSK, you can make your own developer key by flashing the OpenSK firmware on a Nordic chip dongle. In addition to being affordable, we chose Nordic as initial reference hardware because it supports all major transport protocols mentioned by FIDO2: NFC, Bluetooth Low Energy, USB, and a dedicated hardware crypto core. To protect and carry your key, we are also providing a custom, 3D-printable case that works on a variety of printers.

“We’re excited to collaborate with Google and the open source community on the new OpenSK research platform,” said Kjetil Holstad, Director of Product Management at Nordic Semiconductor. “We hope that our industry leading nRF52840’s native support for secure cryptographic acceleration combined with new features and testing in OpenSK will help the industry gain mainstream adoption of security keys.”

While you can make your own fully functional FIDO authenticator today, as showcased in the video above, this release should be considered as an experimental research project to be used for testing and research purposes.


Under the hood, OpenSK is written in Rust and runs on TockOS to provide better isolation and cleaner OS abstractions in support of security. Rust’s strong memory safety and zero-cost abstractions makes the code less vulnerable to logical attacks. TockOS, with its sandboxed architecture, offers the isolation between the security key applet, the drivers, and kernel that is needed to build defense-in-depth. Our TockOS contributions, including our flash-friendly storage system and patches, have all been upstreamed to the TockOS repository. We’ve done this to encourage everyone to build upon the work.


How to get involved and contribute to OpenSK 

To learn more about OpenSK and how to experiment with making your own security key, you can check out our GitHub repository today. With the help of the research and developer communities, we hope OpenSK over time will bring innovative features, stronger embedded crypto, and encourage widespread adoption of trusted phishing-resistant tokens and a passwordless web.

Acknowledgements

We also want to thank our OpenSK collaborators: Adam Langley, Alexei Czeskis, Arnar Birgisson, Borbala Benko, Christiaan Brand, Dirk Balfanz, Dominic Rizzo, Fabian Kaczmarczyck, Guillaume Endignoux, Jeff Hodges, Julien Cretin, Mark Risher, Oxana Comanescu, Tadek Pietraszek

Have an iPhone? Use it to protect your Google Account with the Advanced Protection Program

Posted by Christiaan Brand, Product Manager, Google Cloud and Kaiyu Yan, Software Engineer, Google


Phishing—when an online attacker tries to trick you into giving them your username and password—is one of the most common causes of account compromises. We recently partnered with The Harris Poll to survey 500 high-risk users (politicians and their staff, journalists, business executives, activists, online influencers) living in the U.S. Seventy-four percent of them reported having been the target of a phishing attempt or compromised by a phishing attack.

Gmail automatically blocks more than 100 million phishing emails every day and warns people that are targeted by government-backed attackers, but you can further strengthen the security of your Google Account by enrolling in the Advanced Protection Program—our strongest security protections that automatically help defend against evolving methods attackers use to gain access to your personal and work Google Accounts and data.

Security keys are an important feature of the Advanced Protection Program, because they provide the strongest protection against phishing attacks. In the past, you had to separately purchase and carry physical security keys. Last year, we built security keys into Android phones—and starting today, you can activate a security key on your iPhone to help protect your Google Account.

Activating the security key on your iPhone with Google’s Smart Lock app

Security keys use public-key cryptography to verify your identity and URL of the login page, so that an attacker can’t access your account even if they have your username or password. Unlike other two-factor authentication (2FA) methods that try to verify your sign-in, security keys are built with FIDO standards that provide the strongest protection against automated bots, bulk phishing attacks, and targeted phishing attacks. You can learn more about security keys from our Cloud Next ‘19 presentation.


Approving the sign-in to a Google Account with Google’s SmartLock app on an iPhone

On your iPhone, the security key can be activated with Google’s Smart Lock app; on your Android phone, the functionality is built in. The security key in your phone uses Bluetooth to verify your sign-in on Chrome OS, iOS, macOS and Windows 10 devices without requiring you to pair your devices. This helps protect your Google Account on virtually any device with the convenience of your phone.

How to get started

Follow these simple steps to help protect your personal or work Google Account today:
  • Activate your phone’s security key (Android 7+ or iOS 10+)
  • Enroll in the Advanced Protection Program
  • When signing in to your Google Account, make sure Bluetooth is turned on on your phone and the device you’re signing in on.
We also highly recommend registering a backup security key to your account and keeping it in a safe place, so you can get into your account if you lose your phone. You can get a security key from a number of vendors, including Google, with our own Titan Security Key.

If you’re a Google Cloud customer, you can find out more about the Advanced Protection Program for the enterprise on our G Suite Updates blog.

Here’s to stronger account security—right in your pocket.