Summary of Apple's WWDC24 Developer Conference: iOS 18 has a major update, Apple's "AI" makes a surprise appearance

Today, Apple’s WWDC24 Global Developers Conference was held as scheduled.

In the keynote speech, Apple focused entirely on software, releasing a new generation of operating systems, including iOS 18, iPadOS 18, macOS 15, watchOS 11, and visionOS 2. And as previously mentioned by various sources, Apple talked a lot about AI.

Next, let’s take a look at the details of WWDC24.

1. visionOS

Vision Pro was the One More Thing at last year’s WWDC, and its new operating system, visionOS 2, was first unveiled at this WWDC.

Officials said that visionOS 2 has ushered in several major updates.

The core change of visionOS 2 lies in the photo app. Ordinary 2D photos in the iPhone gallery can be converted into 3D spatial photos with a natural sense of depth.

Apple said that this is done by spatialising ordinary photos through “advanced machine learning”, pushing spatial computing a step further and completely changing the way users appreciate photos.

In addition, visionOS 2 also introduces intuitive gesture control features. Mac users can simply raise their hands and tap two fingers to open the home screen view and can flip their palms to call up the control centre. After connecting to a Mac, it supports widescreen mode, which can be further extended to a surround ultra-wide screen.

Moreover, visionOS 2 uses advanced volume API technology to enable multiple complex 3D applications to run in parallel, improving multitasking capabilities. The use of travel mode is no longer limited to airplanes, and now supports trains.

In addition, Apple also announced a partnership with Canon to launch a new space lens for the Canon EOS R7 camera.

It is worth mentioning that Cook announced that Vision Pro will officially land in the Chinese mainland market on June 28.

Currently, the Apple Vision Pro sold in China is now on sale and will be accepted for pre-order starting from June 14. It will be officially released on June 28. The 256GB version starts at 29,999 yuan, the 512GB version starts at 31,499 yuan, and the 1TB version starts at 32,999 yuan.

2. iOS

Following the release of visionOS 2 is the most anticipated iOS 18. Earlier, many whistleblowers have bluntly stated that iOS 18 will be the “biggest update in history.”

The biggest highlight of iOS 18 is the new home screen customisation function – users can freely set the app icon theme, freely place app icons, and add an option to automatically fill app icons with dark colours in dark mode.

At the same time, the control centre has also been significantly changed, adopting a multi-page layout and becoming more extensible. You can slide up and down to display different controls, and you can also customise the arrangement and size. Users can also add new control functions to the lock screen to replace the original flash and camera.

The Messages app has been significantly adjusted, with new features for bolding, underlining, and italicising text, and support for underlining and striking through text messages. The “tap back” function has been expanded to support any emoji or sticker, and users can also write messages and set a time to send them later.

With the satellite communication function, you can send messages via satellite without a network, and the other party can receive messages even if they do not have satellite service. iOS 18 also supports RCS when sending messages to contacts who do not have Apple devices.

The magnitude of the photo app’s revision is also visible to the naked eye. It is known as “the most significant design innovation to date”. The top is a photo grid, and the bottom is classified according to themes.

There are more buttons at the bottom, a new function to hide screenshots, a new function to recall journeys, and the ability to group images by face.

The Mail app adds “summary” emails and automatic classification, the Maps app adds detailed terrain maps, and the Wallet and Notes apps have also been upgraded.

In terms of security, iOS 18 adds an App Lock feature that supports locking individual apps with Face ID and hiding apps. Users can also choose certain apps to see certain contacts, and there are new controls to limit network and accessory access.

It is worth noting that iOS 18 has also become good news for gamers. It supports game mode. Once it is turned on, the system will optimise the game scenes, minimise background activities, and ensure the highest frame rate performance.

3. Audio & Home

During the keynote, Apple also showcased upgraded audio and home experiences.

One of the most eye-catching is the new feature of AirPods, which will introduce head movement recognition technology. Thanks to this technology, users can achieve an interactive experience of “nodding yes and shaking head no”.

Specifically, users only need to nod to answer a call and shake their heads to hang up. Apple said that this experience is very suitable for use in situations where it is inconvenient to operate the phone with both hands.

In addition, AirPods has added a call voice separation function, which effectively distinguishes and emphasises the user’s speaking voice while reducing the noise of the surrounding environment.

There is also a new tvOS, which adds InSight, which can recognise people and background music when seeing the scene.

4. watchOS

watchOS 10, launched last year, has been well received by users for its significant updates.

In contrast, the new watchOS 11, although less updated, is more focused, with Apple focusing on innovating health and fitness tracking functions.

First, watchOS 11 adds the “Training Load” function to estimate the exercise load for users and score the effect after the training by integrating heart rate, altitude, pace, duration, age, weight and other dimensions. If the exercise exceeds the body’s load capacity, the watch will issue a reminder in time.

Secondly, watchOS 11 Ring Challenge supports more flexible settings, including leave and “rest days”. If the user is unwell or has no training arrangements, the Ring Challenge can be paused without affecting the progress of the winning streak.

Again, watchOS 11 introduced a new health data app “Vitality”, which can intuitively present the user’s health data, making it easier to understand their own health status and make decisions that are conducive to a healthy lifestyle.

Finally, watchOS 11 upgrades menstrual tracking for women’s health and adds pregnancy detection to display pregnancy status.

In addition, watchOS 11 upgrades also include automatic stacking function, automatic recommendation of suitable wallpapers, real-time translation, and the “real-time activity” function will be launched soon.

5. iPadOS

Epic update! In the new iPadOS 18, Apple finally brought a native calculator to the iPad.

Judging from the official demonstration, the iPadOS 18 calculator interface is the same as the iPhone calculator and has been optimised for the size of the iPad.

It should be noted that the iPad calculator will be more powerful than the iPhone. It not only supports the same digital calculation functions but also supports using Apple Pencil to directly write formulas and perform calculations in the same way as daily calculations. The official name is “Math Notes”.

In addition, written calculations support real-time adjustment of variables. The calculator will update the results in real-time based on the written variables, display the results based on the user’s handwriting, and support the automatic generation of curve graphs based on expressions. It also supports all the new mathematical functions of the calculator used in memos.

In addition to the native calculator, the upgrades of iPadOS 18 are basically in line with iOS 18, such as home screen personalisation, control centre functions, game mode, etc.

However, on these foundations, iPadOS 18 also has a new floating bar that can customise the commonly used function bar within the app.

In addition, it also supports automatic optimisation of handwritten notes, helping users’ handwriting fonts look better and write faster.

6. macOS

Apple named the new macOS “Sequoia” and brought a new feature “iPhone mirroring”.

This function is similar to Huawei’s multi-screen collaboration. Users can directly control the iPhone on the Mac, and the iPhone interface will be mirrored on the Mac.

Not only can users use iPhone apps on Mac, but iPhone sounds will also be played through the Mac, and the iPhone will remain locked to ensure privacy and security.

In addition, the new macOS also has five new features:

First, a new intelligent multi-window management feature has been added, which automatically sizes windows to tile and fills the screen, making full use of the maximum screen space.
Second, a new independent “Password” application has been added, which supports cross-platform use and allows users to manage Wi-Fi passwords, application and website passwords, verification codes, etc., making credential management easier and more convenient.
Third, add a background beautification function for video conferences to help users cover up cluttered environments.
Fourth, the Safari browser has become more intelligent, introducing a new “Highlights” feature that uses AI to extract useful information on web pages.
Fifth, the game development kit Game Porting Toolkit has been upgraded to version 2.0. With the support of Xcode, the game porting process will be further simplified.

七・Apple Intelligence

In the last part of the keynote, Cook talked about Apple’s use of artificial intelligence and machine learning for many years and unveiled its big move – Apple Intelligence.

Apple said that Apple intelligence has five characteristics: powerful, intuitive, complete, personal, and private, putting generative models at the core of iPhone, iPad and Mac.

The official gave a detailed introduction to several outstanding features of Apple smartphones.

First, it supports natural language understanding, which can automatically identify the importance of notifications and re-sort them. It can not only automatically polish emails and summarise memos but also generate generative images.
Secondly, it supports global analysis of content in different applications and adjustment of data and information across applications. Apple emphasises that all information adjustments have the highest level of privacy protection, and does not collect data, but only processes data.
Third, private cloud computing technology is used to achieve chip-level data protection. If the local computing power is insufficient, the calculation will be sent to the private cloud for calculation.

With the birth of Apple’s smartphone, Siri has been completely renewed and its practicality has reached a new level.

According to Apple, Siri’s icon has been updated, the human-computer interaction interface has been innovated, and a luminous effect will appear around the phone when it’s running, making it more natural, more appropriate, and more personalised.

Just double-tap at the bottom of the screen to pop up the keyboard to talk to Siri, and the response to the user’s command will be presented in the form of a content-rich card.

The new Siri will have an enhanced ability to understand contextual conversations, helping users understand and discover content like never before.

It will also have a “screen perception” function that can recognise the screen and perform operations such as adding the address sent in the message to the contacts, obtaining flight details from emails, and automatically extracting and filling in ID numbers.

In addition, it will have a richer product knowledge base to help users solve equipment usage problems.

Apple emphasised that the new features will be rolled out in stages and will be gradually pushed to all users in the “next year”, and more powerful features will be added to Siri over the life cycle of iOS 18.

In addition, thanks to Apple’s intelligence, the iPhone will have more practical operations.

For example, it supports the rewrite function, which allows you to rewrite emails, cover letters, etc., adjust the tone, proofread, recognise the content, and reply with one click.

For example, in the enhanced focus mode, in the Do Not Disturb focus mode, if you receive an urgent notification, AI will remind you on the screen.

For example, to implement Genmoji, you can use text descriptions to create a freely customised Genmoji emoticon package.

For example, the new imagewand function in the memo can automatically generate an illustration based on the text content in the user’s memo, and generate an image based on the text content.

For example, with Image Wand, you can transform a rough sketch into a refined image, and you can also erase and repair the image with one click.

For example, it supports call recording function, so when you are making a call, you can turn on the recording and transcribe it directly into text.

It is worth noting that Apple also announced a partnership with OpenAI to add ChatGPT to Siri in the future.

Siri will leverage GPT-4o for free, without having to create an account. Requests will not be recorded. ChatGPT users will also be able to access paid features within the experience.

Apple’s integration of GPT-4o will be available on iOS 18, iPadOS 18, and macOS, with plans to support other AI models in the future.

However, there are some requirements for using it. Apple smartphones have hardware limitations. For iPhone, only the iPhone 15 Pro/Max equipped with A17 Pro can use it.

Apple’s smartphone is still in a “pie” state. It is still under development and will not be open to developers immediately. It is planned to be open to the public this fall, but some functions, more languages and platform support will have to wait until next year.

The above is the content of Apple’s WWDC24 keynote speech. Are you satisfied with the updates of the new system?

Judging from the feedback from netizens on social platforms, five new features have been well received: iOS 18’s new customisation options, the ability to lock apps, call recording, iPad’s built-in native calculator, and Apple’s efforts in AI.

Of course, as usual, some old devices will miss out on the new system. Apple has now announced the compatible models, details are as follows: