At Made by Google 2024, Google unveiled some big new AI features. For example, if you buy a Pixel 9 or Pixel 9 Pro Fold, you can try Add Me, a feature that makes it possible to get a whole group in a shot without an extra photographer. It’s a cool feature, but requires a brand new Pixel.
Not everything Google announced yesterday requires upgrading your smartphone, however. In fact, the company announced four new AI changes to accessibility features that apply to phones from three or four years ago. If you have a Pixel 5 or later, you’re going to see at least some AI features on your phone.
Guided Frame updates (Pixel 6 and later)
Google rolled out Guided Frame back in 2022 as a feature to help users with blindness and low vision take photos. Guided Frame will speak instructions out loud to users to position the camera correctly, whether you’re taking a selfie or a photo of someone else.
This year, Google is adding two key updates to Guided Frame: The company says Guided Frame is better at recognizing objects, has “intelligent face filtering” in group photos, and can focus better on subjects in busy shots. The end result should be an improved Guided Frame, that can better help users frame their photos.
Second, Guided Frame is now available from your camera’s settings. Before, the feature was exclusive to TalkBack, the screen reader built into Android. If you don’t use screen readers, or you just want easier access to Guided Frame, it’s now located in the camera app.
Changes to Magnifier (Pixel 5 and later)
Google’s Magnifier app is designed for users who have trouble seeing small details up close: You use the app to zoom in on these subjects, and make adjustments to the image quality to make things easier to see and read.
Following Made by Google, the company announced four new AI-based changes to the app. Going forward, you’ll be able to search for words within the frame: If you’re looking for something specific, like an item on a menu, you can search for “hamburger” while keeping the menu in Magnifier’s frame. If the menu has a hamburger, it’ll pop up.
There’s also now a picture-in-picture mode, which lets you keeps tabs on the larger frame while zooming in close. You’ll see an outline in the larger frame of where you’re zoomed in, which seems like useful context so you don’t get lost while scrolling through the fine details of an image.
The app now lets you use multiple lenses for different purposes: If you want to use your Pixel’s macro lens to really zoom in on the details, or switch to the wide-angle lens to see more of your surroundings, you can. Finally, Magnifier now supports the front camera, so you can use the app as a quick mirror.
A new Live Transcribe mode for foldables
When enabled, Live Transcribe will write out what you and a partner say during a conversation. It’s a great tool for users who are hard of hearing, since they can follow what the other party says in real time through these captions.
If you have a foldable phone, there’s a new Live Transcribe mode you can try: Now, you can use the feature across both screens of your foldable: Each speaker’s transcriptions appear on both screens, while means you can follow along both ways.
New languages for Live Caption and Live Transcribe
In a similar vein to Live Transcribe, Live Caption adds captions for any audio on your Android device. Google says they’re now adding support for Korean, Polish, Portuguese, Russian, Chinese, Turkish and Vietnamese to both Live Caption and Live Transcribe. If you want a video in one of these languages, you should now see captions appear in real time; if you have a conversation in one of these languages, Live Transcribe should have no problem documenting it.
While Live Caption works on Pixel 2 and later, you need a Pixel 6 or later to try out the new languages.
