Tag Archives: federated learning

More ways we’re making every day safer with Google

Every day, we focus on making sure you’re in control of your data by building products that are secure by default and private by design. At this year’s I/O, we’re introducing new features and technologies to keep you safer with Google. 


Putting you in control of your data 


Privacy is personal. That's why we make it easy for you to choose the settings that are right for you — whether that’s one place to manage settings in your Google Account, Auto-Delete options, or controls that appear in context when you’re using our products. We announced a number of new controls today: 


  • Quick delete in Search. We’re introducing a new, “quick delete” option to delete the last 15 minutes of your Search history with a single tap from the Google Account Menu. 

  • A passcode protected Locked Folder in Photos. Have you ever handed your phone to show someone a photo, but worried they might scroll to a personal or sensitive image — like a photo of your passport or a surprise gift? “Locked Folder” is a new feature in Google Photos —  a passcode-protected space where select photos can be saved separately. These photos won’t show up as you scroll through your grid or in shared albums. This feature is coming to Google Pixels first, and more Android devices throughout the year.

  • Location History reminders in your Maps Timeline. Now, when you see places you've visited in your Timeline, we'll remind you that it's because you turned on Location History — which you can easily turn off right there in your Timeline. 

1. New “quick delete” option in Search.  2. The new Locked Folder in Photos. 3. Location History reminders in your Maps Timeline. 


We’re also introducing new, industry-leading transparency and permission features on Android 12. The new OS includes a Privacy Dashboard where you will see a timeline of when apps accessed your camera, microphone, or device location. We’ve also added indicators that show when your camera or microphone are in use, as well as easy toggles to disable access to both across your device. And you can now choose to share your approximate location with an app instead of a precise one.  


Building products that are secure by default


As recent high-profile third-party security incidents show, your information isn’t private if it’s not secure. With AI-driven technologies that protect billions of users around the world, our products are secure by default: every day, we block 100 million phishing attempts and 15 billion spam messages in Gmail and encrypt 4 billion photos. And Safe Browsing on Chrome and most other browsers helps keep the rest of the Internet secure, automatically protecting more than 4 billion devices.


One of the biggest security risks is still the continued reliance on passwords — they’re often easy to crack, used across multiple sites, or stolen in phishing attacks. That’s why we’ve been working towards a password-free future — focusing on safer ways to authenticate your identity and building multiple layers of protection into your Google Account, like automatic enrollment in 2-step verification. 


But because passwords are still required for most online accounts, we’ve also continued to improve our Password Manager, built directly into Chrome, Android and now iOS, to help you create, remember, save and auto-fill passwords across the web. Today, we announced new enhancements to Password Manager:

  • A new tool that makes it easy to import passwords from other password managers

  • Deeper integrations with Chrome and Android to seamlessly fill your passwords across sites and apps, regardless of whether you’re on desktop or on mobile 

  • Password Alerts that automatically warn you if we detect one of your saved passwords has been compromised via a third party breach.

  • A smart way to fix compromised passwords in Chrome with a simple tap. For supported sites and apps, whenever Password Manager finds a password that may have been compromised, you’ll see a "change password" button from Assistant. When you tap the button, the Assistant will not only navigate to the site, but also go through the entire process of changing your password. This feature is available on Android devices and will be rolling out to more sites and apps in the future.


1. A new way to fix compromised passwords in Chrome. 2. A new tool to import passwords from other password managers to Password Manager. 3. Password Alerts. 


Making our products private by design


We’ve pioneered new computing technologies like Federated Learning (invented by Google researchers in 2016) that make it possible to deliver helpful experiences while protecting individual data and privacy. We’ve also led on Differential Privacy, which powers some of our most helpful features and products, from our COVID-19 Community Mobility Reports to traffic predictions in Maps, without revealing individual user data. And this expertise guides our work on broader industry initiatives, like the open-source Privacy Sandbox. 


Now, we’re continuing that work with Android's Private Compute Core, which keeps your information safe and private for a number of popular AI-driven features like Live Caption (which displays captions based on audio), Now Playing (which tells you the song that’s playing) and Smart Reply (which suggests short responses to messages and emails). For these features, the audio and language processing happens exclusively on your device. Like the rest of Android, Private Compute Core is open source — it’s fully inspectable and verifiable by the security community. 


We’ll continue our work to make every day safer with Google with new controls, advanced security, and privacy-preserving technologies.


Posted by Jen Fitzpatrick, SVP, Core

Privacy-Preserving Smart Input with Gboard

Posted by Yang Lu, Software Engineer, Angana Ghosh, Group Product Manager, and Xu Liu, Director of Engineering, Gboard team

Google Keyboard (a.k.a Gboard) has a critical mission to provide frictionless input on Android to empower users to communicate accurately and express themselves effortlessly. In order to accomplish this mission, Gboard must also protect users' private and sensitive data. Nothing users type is sent to Google servers. We recently launched privacy-preserving input by further advancing the latest federated technologies. In Android 11, Gboard also launched the contextual input suggestion experience by integrating on-device smarts into the user's daily communication in a privacy-preserving way.

Before Android 11, input suggestions were surfaced to users in several different places. In Android 11, Gboard launched a consistent and coordinated approach to access contextual input suggestions. For the first time, we've brought Smart Replies to the keyboard suggestions - powered by system intelligence running entirely on device. The smart input suggestions are rendered with a transparent layer on top of Gboard’s suggestion strip. This structure maintains the trust boundaries between the Android platform and Gboard, meaning sensitive personal content cannot be not accessed by Gboard. The suggestions are only sent to the app after the user taps to accept them.

For instance, when a user receives the message “Have a virtual coffee at 5pm?” in Whatsapp, on-device system intelligence predicts smart text and emoji replies “Sounds great!” and “?”. Android system intelligence can see the incoming message but Gboard cannot. In Android 11, these Smart Replies are rendered by the Android platform on Gboard’s suggestion strip as a transparent layer. The suggested reply is generated by the system intelligence. When the user taps the suggestion, Android platform sends it to the input field directly. If the user doesn't tap the suggestion, gBoard and the app cannot see it. In this way, Android and Gboard surface the best of Google smarts whilst keeping users' data private: none of their data goes to any app, including the keyboard, unless they've tapped a suggestion.

Additionally, federated learning has enabled Gboard to train intelligent input models across many devices while keeping everything individual users type on their device. Today, the emoji is as common as punctuation - and have become the way for our users to express themselves in messaging. Our users want a way to have fresh and diversified emojis to better express their thoughts in messaging apps. Recently, we launched new on-device transformer models that are fine-tuned with federated learning in Gboard, to produce more contextual emoji predictions for English, Spanish and Portuguese.

Furthermore, following the success of privacy-preserving machine learning techniques, Gboard continues to leverage federated analytics to understand how Gboard is used from decentralized data. What we've learned from privacy-preserving analysis has let us make better decisions in our product.

When a user shares an emoji in a conversation, their phone keeps an ongoing count of which emojis are used. Later, when the phone is idle, plugged in, and connected to WiFi, Google’s federated analytics server invites the device to join a “round” of federated analytics data computation with hundreds of other participating phones. Every device involved in one round will compute the emoji share frequency, encrypt the result and send it a federated analytics server. Although the server can’t decrypt the data individually, the final tally of total emoji counts can be decrypted when combining encrypted data across devices. The aggregated data shows that the most popular emoji is ? in Whatsapp, ? in Roblox(gaming), and ✔ in Google Docs. Emoji ? moved up from 119th to 42nd in terms of frequency during COVID-19.

Gboard always has a strong commitment to Google’s Privacy Principles. Gboard strives to build privacy-preserving effortless input products for users to freely express their thoughts in 900+ languages while safeguarding user data. We will keep pushing the state of the art in smart input technologies on Android while safeguarding user data. Stay tuned!