In the most recent issue of Apple’s Machine Learning Journal, titled “Learning with Privacy at Scale,” the team working on differential privacy shares details on exactly how its systems work. While much of the article is highly technical in nature, it concludes by sharing results from several real-life applications. Regarding emoji:
The data shows many differences across keyboard locales. In Figure 6, we observe snapshots from two locales: English and French. Using this data, we can improve our predictive emoji QuickType across locales.
The referenced chart is featured above, showing the popularity of certain emoji in different parts of the world.
The results regarding QuickType words aren’t presented in a chart, but the article does mention words in several specific categories that Apple has been able to learn about thanks to differential privacy.
The learned words for the English keyboard, for example, can be divided into multiple categories: abbreviations like wyd, wbu, idc; popular expressions like bruh, hun, bae, and tryna, seasonal or trending words like Mayweather, McGregor, Despacito, Moana, and Leia; and foreign words like dia, queso, aqui, and jai. Using the data, we are constantly updating our on-device lexicons to improve the keyboard experience.
Another category of words discovered are known words without the trailing e (lov or th) or w (kno). If users accidentally press the left-most prediction cell above the keyboard, which contains the literal string typed thus far, a space will be added to their current word instead of the character they intended to type. This is a key insight that we were able to learn due to our local differentially private algorithm.
Though the article doesn’t mention it, presumably the latter example of accidentally-tapped QuickType suggestions might lead to Apple adjusting sensitivity for its touch targets related to the ‘e’ button and the left-most prediction cell. It’s interesting to consider what other unexpected lessons may be learned from differential privacy data.
