Back in the last year, Google announced a Live Captions feature as one of the most promising features of last year’s Google I/O. The system makes use of three on-device machine learning parameters. The feature can detect English-language speech in media and generates captions automatically. The feature came along with Pixel 4, however, its availability was expanded to Pixel 3a, Pixel 3, Pixel 2, Galaxy S20, OnePlus 7T, and OnePlus 8. After its launch, Google didn’t release any update to change functionality. However, the folks at XDA-Developers have found evidence that this feature will make way to phone calls.
The developers were able to dig into the code of Android 11 Developer Preview 3 for Google Pixel 4. There, pieces of evidence about this new feature were present at the new version of Device Personalization settings. That’s the application behind the Live Captions feature. According to the report, the feature will become available and allow users to turn on Live Caption over a phone call.
If you enable this feature in a phone call, a message will pop-out for all participants. It will announce that Live Captions is transcribing the call. Moreover, an audio file will be heard during the phone call. The audio says the following message:
“Hi, the person you’re about to speak with has call captions turned on. They’ll see captions of what you say to help them listen along.”
Currently, Live Captions follows AudioPlaybackCaptureConfiguration API that does not allow for capturing voice call audio. However, the newest developer preview has new system permission that allows it to bypass this restriction. We’ll have to wait for more details from Google regarding this new feature.