{"id":132419,"date":"2023-03-13T14:30:44","date_gmt":"2023-03-13T14:30:44","guid":{"rendered":"https:\/\/developer.apple.com\/news\/?id=0vz78ua8"},"modified":"2023-03-13T14:30:44","modified_gmt":"2023-03-13T14:30:44","slug":"spotlight-on-spatial-audio","status":"publish","type":"post","link":"https:\/\/sickgaming.net\/blog\/2023\/03\/13\/spotlight-on-spatial-audio\/","title":{"rendered":"Spotlight on: Spatial Audio"},"content":{"rendered":"
<\/div>\n

When designing soundscapes for apps and games, the right notes can make all the difference. And when those notes are built to support multichannel audio, they might even turn someone\u2019s head. (Literally.)<\/p>\n

Endel<\/em> and Odio<\/em> are just two of the many apps and games taking advantage of Spatial Audio. They use multichannel mixes, Core Audio, and AVFoundation to add texture and dimensionality, creating resonating surround-sound experiences that further immerse listeners into the world within their apps. <\/p>\n

\n
\n
<\/a> <\/section>\n
<\/p>\n

Design for spatial interaction<\/h4>\n

Discover the principles for creating intuitive physical interactions between two or more devices, as demonstrated by Apple designers who worked on features for iPhone, HomePod mini, and AirTag. Explore how you can apply these patterns to your own app when designing features for Apple platforms, and…<\/p>\n

<\/a> <\/section>\n<\/section>\n<\/section>\n

Endel<\/em> (pictured above) conjures up personalized and adaptive soundscapes based on biometrics and environment to help people focus and get better sleep. Its inaugural Spatial Audio soundscape \u2014 one with the satisfyingly otherworldly name of Spatial Orbit \u2014 brings the app\u2019s remarkable mix of art and AI to a new dimension.<\/p>\n

\u201cIt feels like you\u2019re inside a vast, glittery space,\u201d says Dmitry Evgrafov, Endel<\/em> cofounder and chief sound officer. \u201cIt\u2019s almost like the sonic equivalent of pointillism, where the small dots create a structure themselves and you kind of drown in the thing. It\u2019s a very beautiful state, and it\u2019s not something you can reproduce in stereo.\u201d<\/p>\n

\"Endel\u2019s<\/div>\n

When bringing Spatial Audio into their ecosystem, the Endel<\/em> team\u2019s first task was determining if the technology was compatible with their ever-changing, generative soundscape. That job fell largely to Kyrylo Bulatsev, cofounder and chief technology officer. \u201c[Spatial Audio] meant we had to add one more dimension to the non-static element,\u201d he says. \u201cBesides choosing what sound to play and when, we had to think about where the sound would be and how it would move around you.\u201d<\/p>\n

That soundscape also had to hit the \u201cthin line between augmenting an experience and making it distracting,\u201d Evgrafov says. That\u2019s because while most apps (and games and movies and songs) are designed for active engagement, Endel<\/em> aims to be a perfect background companion \u2014 enhancing your experience without pulling from your focus. \u201cOur use case is different from other products that utilize the technology,\u201d says Evgrafov (whom fellow cofounder Oleg Stavisky credits with \u201call the beautiful sounds in the app\u201d).<\/p>\n

\n

It\u2019s almost like the sonic equivalent of pointillism.<\/p>\n

Dmitry Evgrafov, *Endel* cofounder and chief sound officer<\/cite><\/p><\/blockquote>\n

A pianist and musician with 10 albums to his credit, Evgrafov certainly knows his way around stereo. \u201cBut randomization of the position of audio in the space? That\u2019s a whole different beast,\u201d he says.<\/p>\n

The first serious prototype of Spatial Orbit was earthbound, set to a realistic jungle scene. \u201cThe idea was you\u2019d walk around this magical Garden of Eden and exotic tropical animals would sing around you,\u201d he says. \u201cWe had a harp playing by the water, a creek, birds that don\u2019t exist in the real world, stuff like that.\u201d<\/p>\n

Similar ideas kept coming: a Gregorian choir that slowly shuffled past you while chanting, field recordings from inside a cave. Although the concepts were cool and the prototypes sounded great, the team kept running up against the same problem. \u201cThey weren\u2019t Endel,\u201d says Evgrafov. \u201cThey transported you to a place, but that meant people were using the app consciously. They didn\u2019t match what we stood for.\u201d <\/p>\n

\"Like<\/div>\n

The final version of Spatial Orbit does match what Endel<\/em> stands for \u2014 and achieves the synthesis of art and technology that Endel strives for. \u201cThe rain [in our soundscape] is almost metaphorical,\u201d says Evgrafov. \u201cIt\u2019s got this slightly augmented feel that allows you to just drown a little and be with your thoughts, focus on your book, or whatever you\u2019re doing.\u201d<\/p>\n

Tweaking the soundscape was an adventure in itself. \u201cWatching people test Endel<\/em> is kind of a funny exercise,\u201d laughs Stavitsky. That\u2019s because there\u2019s really not an established way to test an personalized auto-generated soundscape for a group of people all at once. <\/p>\n

\n

[The rain has] this slightly augmented feel that allows you to just drown a little and be with your thoughts, focus on your book, or whatever you\u2019re doing.<\/p>\n

Dmitry Evgrafov<\/cite><\/p><\/blockquote>\n

\u201cWe invented the process and the toolset,\u201d says Evgrafov. It involved a lot of people wandering Endel\u2019s Berlin offices… and elsewhere. \u201cIt was also a lot of me in public spaces just staring at nothing, like a cat.\u201d<\/p>\n

In the end, Spatial Orbit captures that elusive mix of innovative technology and artistic resolve. \u201cWhen we realized the science was there and that it still checked all the Endel boxes, it was a big relief,\u201d says Evgrafov. \u201cWe thought, \u2018OK, we can be non-intrusive and Spatial at the same time.\u2019\u201d<\/p>\n

Download Endel from the App Store<\/a><\/p>\n

Odio<\/em> also focuses on creating great ambient soundscapes \u2014 but with a sci-fi twist. \u201cI want our composers to imagine inventing planets and filling them up with sound,\u201d says Joon Kwak, the app\u2019s Seoul-based cofounder. \u201cWe want to walk people through these new planets.\u201d<\/p>\n

The app\u2019s soundscapes, which can evoke anything from a crashing waterfall to a buzzy digital backdrop to the spooky calm of the deep sea, use head tracking and multichannel audio to create a truly mesmerizing mix. (The app is also a visual feast, with each soundscape accompanied by ever-shifting techno-tinged art.) <\/p>\n

But you\u2019re no passive listener in these audio realms. The individual elements that make up each soundscape can be manipulated through an imaginative, playful UI that lets you reposition each audio element (like that waterfall) anywhere you like. <\/p>\n

\"*Odio*'s<\/div>\n

Befitting its futuristic feel, Odio’s<\/em> backstory is one of serendipitous meetings, well-timed hardware and software releases, and a stroke of good fortune. Kwak conceived the app\u2019s initial version as a graduation project at the Design Academy Eindhoven. Originally known as Virtual Sky, the prototype contained the bones of what would become Odio<\/em>, but was largely grounded in real-world sounds. It also required a mess of hardware and special equipment \u2014 all of which was rendered pretty much irrelevant once AirPods with Spatial Audio arrived. <\/p>\n

\u201cI was depressed for a while,\u201d laughs Kwak. \u201cI was like, \u2018I\u2019ve been working on this for months, and now it\u2019s pointless!\u2019 But then I thought about it more deeply and realized, \u2018Oh, this just means I don\u2019t need to provide hardware,\u2019 and it was actually great.\u201d<\/p>\n

Kwak partnered with Volst, a company that was interested in a 3D soundscape app. With the building blocks in place, Odio’s<\/em> UI developer and designer, Rutger Schimmel, took on the challenge of bringing Kwak\u2019s project to life \u2014 a process that went much faster than expected.<\/p>\n

\n

I want our composers to image inventing planets and filling them up with sound.<\/p>\n

Joon Kwak, *Odio* cofounder<\/cite><\/p><\/blockquote>\n

\u201cWe knew the AirPods had [surround sound] support, but we were skeptical,\u201d he says. \u201cWe thought, \u2018OK, they have head tracking, but it\u2019s probably just for first-party stuff.\u2019 But we were still excited, so we quickly set up an Xcode project to get the data from the AirPods to the device.\u201d <\/p>\n

They had a prototype up and running on the headphones within minutes. \u201cWe were blown away by how easy it was,\u201d Schimmel says. \u201cAnd in about an hour we decided on excellent 3D audio frameworks from Apple that were the perfect foundation for what we were working on.\u201d Coding began in January. By April, the team had a Swift-built demo ready to go.<\/p>\n

To build an Odio<\/em> soundscape, composers like Kwak, Odio<\/em> sound designer Max Frimout, and a team of outside musicians collaborate \u2014 generally in Logic Pro \u2014 by blending ambient sounds, synthetic bells and whistles, and music. <\/p>\n

\"Odio\u2019s<\/div>\n

After the soundscapes are completed and duly field-tested in coffee shops, parks, and subways, the artists hand their files over to Schimmel. For a role that involves cutting-edge design, immersive audio, and incredible degrees of customization, Schimmel\u2019s toolbox is surprisingly uncluttered: AVAudioEnvironmentNode (AVKit) for creating the 3D audio environment, CMHeadphoneMotionManager (Core Motion) to access headphone motion data, and Sentry for error tracking and QA. <\/p>\n

\u201cEverything else in Odio is created from scratch in Swift \u2014 from data management to interacting with soundscapes to real-time buffering the interactive sound files,\u201d Schimmel says. <\/p>\n

The result is a remarkable example of the power and simplicity of designing for Spatial Audio. \u201cHonestly,\u201d Schimmel says, \u201cmost of the hard work is done by the composer.\u201d <\/p>\n

Download Odio from the App Store<\/a><\/p>\n

\n
\n
<\/a> <\/section>\n
<\/p>\n

Discover geometry-aware audio with the Physical Audio Spatialization Engine (PHASE)<\/h4>\n

Explore how geometry-aware audio can help you build complex, interactive, and immersive audio scenes for your apps and games. Meet PHASE, Apple\u2019s spatial audio API, and learn how the Physical Audio Spatialization Engine (PHASE) keeps the sound aligned with your experience at all times \u2014 helping…<\/p>\n

<\/a> <\/section>\n<\/section>\n<\/section>\n

\n
\n
<\/a> <\/section>\n
<\/p>\n

Immerse your app in Spatial Audio<\/h4>\n

Discover how spatial audio can help you provide a theater-like experience for media in your apps and on the web. We\u2019ll show you how you can easily bring immersive audio to those listening with compatible hardware, and how to automatically deliver different listening experiences depending on…<\/p>\n

<\/a> <\/section>\n<\/section>\n<\/section>\n","protected":false},"excerpt":{"rendered":"

When designing soundscapes for apps and games, the right notes can make all the difference. And when those notes are built to support multichannel audio, they might even turn someone\u2019s head. (Literally.) Endel and Odio are just two of the many apps and games taking advantage of Spatial Audio. They use multichannel mixes, Core Audio, […]<\/p>\n","protected":false},"author":2,"featured_media":132420,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[55],"tags":[],"class_list":["post-132419","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-apple-developer-news"],"_links":{"self":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/132419","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/comments?post=132419"}],"version-history":[{"count":0,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/posts\/132419\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media\/132420"}],"wp:attachment":[{"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/media?parent=132419"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/categories?post=132419"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sickgaming.net\/blog\/wp-json\/wp\/v2\/tags?post=132419"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}