The recent rollout of iOS 18.2 finally brings many of the promised Apple Intelligence features, like Genmoji and Image Playground. One such long-awaited tool is Visual Intelligence, a feature currently reserved for the iPhone 16 Pro and Pro Max that was first introduced at the company’s September event.
What is Visual Intelligence?
Visual Intelligence is Apple’s answer to Google Lens. It leverages the camera system and AI to analyze images in real-time and provide useful information. This can help people learn more about the world around them and is particularly handy for shopping, looking up details about a restaurant or business, translating written text, summarizing text or having something read aloud. It can also integrate with Google Image Search and ChatGPT.
Are there any caveats?
There are two caveats. The Apple Intelligence rollout has been something of a convoluted mess, and this trend continues with Visual Intelligence. For now, the tools only work with the iPhone 16 Pro and Pro Max, which are the beefiest of the company’s recent handsets. Apple has indicated that the feature could eventually become available for older models. Google Lens, after all, has been around since 2017, which was when the Pixel 2 was the hottest handset on the block.
There’s also a wait list, which is true of all Apple Intelligence features. To join the list, head to settings and look for “Apple Intelligence & Siri.” Then click on “Join Waitlist.” Once approved, the software will be ready to use.
How to use Visual Intelligence
As of this writing, the only way to launch Visual Intelligence is to long-press the Camera Control button. That’s the new control interface on the bottom right side of the handset. Once pressed, the Visual Intelligence interface will open up.
Now the fun begins. Just point your phone at something and select ChatGPT, via the bottom left icon, or Google Image Search, via the bottom right icon. Alternatively, if the visual field includes text, tap the circle at the bottom of the screen. The phone can also be pointed at a business to obtain useful information.
How to interact with text
Hover the phone in front of the text, activate Visual Intelligence and tap the circle at the bottom of the screen. This will analyze the text. Once analyzed, there are a few options. Tap “Translate” at the bottom of the screen to translate the text into another language. Tap “Read Aloud” if you want the text to be read aloud by Siri. Tap “Summarize” for a quick summary of the copy.
The tool will also identify contact information in the text, like phone numbers, email addresses and websites. Users can take action depending on the type of text. For instance, tap on the phone number to give it a ring. Other actions include starting an email, creating a calendar event or heading to a website. Tap the “More” button to see all of the available options. Tap “Close” or swipe up to end the session.
How to interact with a business
Visual Intelligence can provide details about a business that’s directly in front of you. Just open up the tool and point the camera in front of the signage. The name of the business should appear at the top of the screen. Tap “Schedule” to see the hours of operation or tap “Order” to buy something. View the menu or available services by tapping “Menu” and make a reservation by touching “Reservation.” To call the business, read reviews or view the website, tap “More.”
Swipe up or tap “Close” to end the session. This feature is currently only available to US customers.
What to do with ChatGPT
Start by pointing the camera at an object. Activate Visual Intelligence and tap the ChatGPT icon on the bottom left side of the screen. Tap the “Ask” button for information about the object. We used it on a bottle of hand cream, which it properly identified. After that, a text field will appear for follow-up questions. Users can ask whatever they want, but results may vary. We asked ChatGPT where to buy the hand cream and how much it costs. It performed admirably at this task. Yay shopping.
Tap the "Close" button or swipe up to remove all fields, which will also shut down Visual Intelligence.
What to do with Google Image Search
Choosing Google Image Search will bring up a Safari dialog box that contains similar photos pulled from the web. A good use case here is finding deals. We took a photo of a bottle of hand cream and the Safari results had plenty of different price points to choose from. However, users have to find the best deal and complete a purchase on their own.
Tap the "Close" button to eliminate these results and then swipe up from the bottom of the screen to shut down the tool.
This article originally appeared on Engadget at https://www.engadget.com/mobile/smartphones/how-to-use-visual-intelligence-apples-take-on-google-lens-150039141.html?src=rss
Leave a Reply