Posted on Leave a comment

Challenge: Build an app using built-in Sound Classification

Speaker symbol badging a folder symbol

With Sound Classification, you can create experiences for camera, video, productivity, and game apps on all Apple platforms — and for this challenge, we’re inviting you to explore a sample project and build your own.

When you use the built-in sound classifier in Sound Analysis you have access to over 300 different sound classes trained on a massive amount of data to ensure great model performance. The model won’t predict just a single sound at once: It returns multiple labels with individual confidence scores for each so that you can understand all the sounds being heard at a given time.

The API in Sound Analysis provides a full list of all trained classes, you decide which ones you care about, ignoring all others. You could use the speech detector, for example, to identify when someone has finished speaking. Even further, you have control over the sampling window for the prediction and can apply sound-specific confidence thresholds to greatly improve the real world accuracy of the features you will create.

Begin the challenge

For this challenge, we invite you to brainstorm how you could incorporate Sound Classification into an existing app or a brand new app idea. You can use sound classifiers on all Apple platforms — Mac, iPhone, iPad, Apple Watch, or Apple TV — allowing you to explore a variety of different ideas and situations. For example, a camera app could enable people to quickly locate the precise moment in personal videos where things like laughter occurred, or listen for specific sounds during video capture to trigger special effects and overlays in the camera frame. A video editing or productivity app could leverage sound classification to help someone quickly organize media assets based on the sounds that are in them. Or an interactive game experience could be developed that leverages recognized sounds in the environment as a trigger for unlocking special modes where characters mimic what they’re hearing.

We’ve provided the “Classifying live audio input with a built-in sound classifier” project to help you get started. From here, we invite you to come up with an app of your own that uses the microphone or another audio source to listen to and identify sounds. What will you make? Show off the creative ways you can apply this built-in capability.

Discover built-in sound classification in SoundAnalysis

Explore how you can use the Sound Analysis framework in your app to detect and classify discrete sounds from any audio source — including live sounds from a microphone or from a video or audio file — and identify precisely in a moment where that sound occurs. Learn how the built-in sound…

Classifying Live Audio Input with a Built-in Sound Classifier

Need support, or want help from the community as you explore Sound Classification? You can share your progress in the Developer Forums.

Visit the Apple Developer Forums

Resources

Sound Analysis

Read the WWDC21 Challenges Terms and Conditions

Posted on Leave a comment

Challenge: Create your first 3D model with Object Capture

Symbol signifying a camera focusing on an object that looks like a cube

Object capture provides a quick and easy way to create lifelike 3D models of real-world objects with only a few images. You can capture just about any object to incorporate directly into your apps and games. For this challenge, we’re inviting you to use the new Object Capture Swift API and build your very own 3D model from scratch.

What object you scan is entirely up to you, though we’ve supplied a few themes to get you started:

  • Toys and figurines
  • Items you only find in nature, like pine cones or leaves
  • Items you might only find in a kitchen

Begin the challenge

To help you get started, download our sample iOS app to capture the images of your object.

Taking Pictures for 3D Object Capture

Some tips:

  • You’ll get the best experience when you capture your object on a turntable with your iPhone mounted to a tripod
  • If you don’t have a turntable, find a table in your space for your object, then slowly move around it to take photos while maintaining overlap between adjacent images
  • No matter how you capture your object, make sure you’re in a well-lit space

Discover more best practices for capturing images of your object in “Create 3D models with Object Capture” from WWDC21.

Create 3D models with Object Capture

Object Capture provides a quick and easy way to create lifelike 3D models of real-world objects using just a few images. Learn how you can get started and bring your assets to life with Photogrammetry for macOS. And discover best practices with object selection and image capture to help you achieve…

Once you’ve taken photographs of the item, collect them into a folder on your Mac. Then open the HelloPhotogrammetry macOS app from “Creating a Photogrammetry Command-Line App”, add your folder of images, and the app will transform them into a fully-functioning 3D model.

Creating a Photogrammetry Command-Line App

After your 3D model is complete, export your creation to a .usdz file and preview it with AR Quick Look. Want to share what you’ve made with the community? Post a link to your model in the Developer Forums.

Resources

Visit the Apple Developer Forums

Read the WWDC21 Challenges Terms and Conditions

Posted on Leave a comment

Get the most out of WWDC21

Five rows of unique memojis gaze in amazement at three iMessage texts. A blue text bubble in the center reads “#WWDC21”. A white text bubble above it displays three party popper emojis. A white text bubble below it displays the clapping hands emoji in all six skin tones. This text has been “loved” with a pink heart.

There’s so much to explore and learn during WWDC21: It’s nice to have a place to start. We’ve put together a few resources we know will be useful for this week.

Rewatch the Keynote and Platforms State of the Union

Monday’s Keynote brought an in-depth look at the future of Apple platforms. And don’t miss the Platforms State of the Union and dive deep into the latest tools, technologies, and advances.

Keynote

Keynote (ASL)

Platforms State of the Union

Platforms State of the Union (ASL)

Meet pavilions

Visit the WWDC tab in the Developer app to take a peek at what’s in store for you this week. Browse a full list of all sessions, labs, and special activities, or — new for 2021 — check out pavilions. Within a pavilion, you can browse a customized schedule for sessions and labs within that area, discover daily featured content, participate in coding challenges, explore activities and session collections, and find related documentation.

There are more than 200 sessions, technology and design labs, special activities, and so much more arriving throughout the week. Bookmark sessions you’re interested in and sign up for labs directly within the app. Don’t want to miss a thing? Make sure you’ve signed up for WWDC notifications, available on both iOS and Mac.

Connect in the labs, lounges, and forums with Apple engineers and designers

Members of the Apple Developer Program, Apple Developer Enterprise Program, or 2021 Swift Student Challenge winners can request a lab appointment with an Apple specialist, engineer, or designer for one-on-one guidance and conversation. You can explore the entire labs schedule on the WWDC tab or within a specific pavilion, request appointments directly within the app, and manage your schedule.

You can also connect with Apple engineers and designers around select topics in the Digital Lounges. Hosted by the Developer Tools, SwiftUI, Accessibility, and Machine Learning teams, the lounges invite you to participate in text-based discussions, ask questions about the latest technologies, get help with coding challenges, meet presenters, and more. Space is limited. Sign-ups are open now and available to members of the Apple Developer Program, Apple Developer Enterprise Program, or 2021 Swift Student Challenge winners.

And you can always ask questions about the latest technologies on the Apple Developer Forums.

Explore developer labs at WWDC21

Discover the Apple Developer forums

Sign up for Digital Lounges

Read developer documentation

Check out the latest sample code projects, API reference and article updates, and more on the Developer Documentation website. Explore the latest updates to Apple frameworks on the Technologies page, check out the Human Interface Guidelines, and get a primer on interface resources that integrate beautifully with Apple platforms.

Find the right documentation

Apple Developer Documentation

Explore WWDC20 sample code

Download developer betas

Itching to explore the latest developer betas for macOS, iOS, iPadOS, watchOS, Xcode, and more? Anyone with an active Apple Developer account can visit developer.apple.com after the Keynote for download links and more.

Download the latest developer beta software

Learn more about Feedback Assistant

How to file great bug reports

Enhance bug reports with debug profiles and logging

How to test your app on beta software

Explore what’s new

While there are many great sessions that cover all the latest technologies and frameworks in detail during WWDC week, for a broader view of our platforms and technologies — as well as release notes for each platform — be sure to check out developer.apple.com.

Posted on Leave a comment

Challenge: Animated artistry in SwiftUI

Put trigonometry to good use and show us your artistic side by creating a stunning graphical animation in SwiftUI.

Begin the challenge

The SwiftUI canvas is a powerful new way to represent and animate graphics, while drawingGroup remains a great way to flatten and render views offscreen to create beautiful graphics. Explore both canvas and drawingGroup and create an awesome graphical animation that you can animate with animations or the new TimelineView. Add your creation to a view in your existing app project, or start an entirely new project in Xcode.

  • WWDC21

Add rich graphics to your SwiftUI app

Learn how you can bring your graphics to life with SwiftUI. We’ll begin by working with safe areas, including the keyboard safe area, and learn how to design beautiful, edge-to-edge graphics that won’t underlap the on-screen keyboard. We’ll also explore the materials and vibrancy you can use…

Don’t forget to consider the accessibility and design implications of your concept. Have you made sure your animation is fully accessible to everyone? Do your awesome graphics follow the Human Interface Guidelines with regard to safe areas and contrast ratios? We’re excited to see what you can make in just a day using platform best practices!

Want to share your work-in-progress or collaborate with other developers? You can show off your finished animation in the Developer Forums.

Visit the Apple Developer Forums

Resources

Read the WWDC21 Challenges Terms and Conditions

Posted on Leave a comment

Get ready for WWDC21

Digital Lounge sign-ups are now open. Join Apple engineers and designers from Developer Tools, SwiftUI, Accessibility, and Machine Learning and participate in text-based discussions, ask questions about the latest technologies, get help with coding challenges, meet presenters, and more. Space is limited and available for members of the Apple Developer Program and Apple Developer Enterprise Program, as well as Swift Student Challenge winners.

In addition to WWDC, a variety of other exciting developer events will take place throughout the week. Find even more opportunities for learning, networking, and fun.

Learn more about Digital Lounges

View developer community events

Posted on Leave a comment

2021 Apple Design Award finalists announced

A silver-colored metallic cube sits on a black background. It’s engraved on the left side with the Apple logo and on the right side with the words “Apple Design Award For Outstanding Design and Innovation”.

The Apple Design Awards honor excellence in innovation, ingenuity, and technical achievement in app and game design. We’re thrilled to announce this year’s finalists — an impressive collection of apps and games produced by some of the most talented developers in the world. Finalists were chosen across six categories and winners will be announced on June 10.

View the finalists

Posted on Leave a comment

New Apple Developer Forums features now available

The Apple Developer Forums offer a trusted space to find answers, ask questions, and share thoughts on a variety of code-level topics with fellow developers and Apple engineers. New features now make it even easier to keep track of activity, search for content, post replies, and more.

  • Post comments on questions or answers to provide context or ask for clarification.
  • Search for content across multiple tags.
  • Add and manage favorite tags.
  • Upload images to your question or answer to provide supporting visual details.
  • See tag descriptions when choosing tags for your question so you can quickly select the most appropriate ones.
  • Subscribe to RSS feeds for tags you’re interested in.
  • See your authored and watched content, favorite tags, and trending tags on the newly designed home page.

Learn more about the forums

Visit the forums

Posted on Leave a comment

Updates to age rating settings in App Store Connect

To help you describe your app’s content more precisely, the age rating settings in App Store Connect have been refined. The Gambling and Contests setting is now split into two settings, allowing you to indicate these content types separately. They’re also indicated separately on the App Store.

If your answer to the Gambling and Contests setting was Yes, the update automatically reflects that your app includes instances of Gambling. Contests will be marked as Frequent/Intense. If your answer was No, the update automatically reflects that your app does not include instances of Gambling or Contests.

In addition, you can access age rating settings directly from the App Information section in App Store Connect. Apps that offer Universal Purchase display the same age rating on the App Store across all platforms. You can make changes to your age rating settings when submitting a new version of your app and your app’s availability in certain regions will be updated based on local laws.

Learn more about age rating settings

Posted on Leave a comment

App Store stopped more than $1.5 billion in potentially fraudulent transactions in 2020

Apple helps keep the App Store a safe and trusted place for users to discover apps by detecting and taking action against fraudulent developers and users.

Threats have been present since the first day the App Store launched on iPhone, and they’ve increased in both scale and sophistication in the years since. Apple has likewise scaled its efforts to meet those threats, taking relentless steps forward to combat these risks to users and developers alike.

It takes significant resources behind the scenes to ensure these bad actors can’t exploit users’ most sensitive information, from location to payment details. While it’s impossible to catch every act of fraud or ill intent before it happens, thanks to Apple’s industry-leading antifraud efforts, security experts agree the App Store is the safest place to find and download apps.*

In 2020 alone, Apple’s combination of sophisticated technology and human expertise protected customers from more than $1.5 billion in potentially fraudulent transactions, preventing the attempted theft of their money, information, and time — and kept nearly a million risky and vulnerable new apps out of their hands.

App Review

The App Review team is an essential line of defense, carefully reviewing every app and every update to ensure they adhere to the App Store’s strong guidelines on privacy, security, and spam. The guidelines have changed over time to respond to new threats and challenges, with the goal of protecting users and providing them with the very best experience on the App Store.

Apple’s goal is always to get new apps onto the store. In 2020, the team assisted more than 180,000 new developers in launching apps. Sometimes this takes a few tries. An app might be unfinished or not functioning properly when it’s submitted for approval, or it might not yet have a sufficient mechanism for moderating user-generated content. In 2020, nearly 1 million problematic new apps, and an additional nearly 1 million app updates, were rejected or removed for a range of reasons like those.

A smaller but significant set of these rejections was for egregious violations that could harm users or deeply diminish their experience. In 2020 alone, the App Review team rejected more than 48,000 apps for containing hidden or undocumented features, and more than 150,000 apps were rejected because they were found to be spam, copycats, or misleading to users in ways such as manipulating them into making a purchase.

Some developers perform a bait and switch: fundamentally changing how the app works after review to evade guidelines and commit forbidden and even criminal actions. When such apps are discovered, they’re rejected or removed immediately from the store, and developers are notified of a 14-day appeals process before their accounts are permanently terminated. In 2020, about 95,000 apps were removed from the App Store for fraudulent violations, predominantly for these kind of bait-and-switch maneuvers.

In just the last few months, for example, Apple has rejected or removed apps that switched functionality after initial review to become real-money gambling apps, predatory loan issuers, and pornography hubs; used in-game signals to facilitate drug purchasing; and rewarded users for broadcasting illicit and pornographic content via video chat.

Another common reason apps are rejected is they simply ask for more user data than they need, or mishandle the data they do collect. In 2020, the App Review team rejected over 215,000 apps for those sorts of privacy violations. Apple believes privacy is a fundamental right, and this commitment is a major reason why users choose the App Store.

Even with these stringent review safeguards in place, with 1.8 million apps on the App Store, problems still surface. Users can report problematic apps by choosing the Report a Problem feature on the App Store or calling Apple Support, and developers can use either of those methods or additional channels like Feedback Assistant and Apple Developer Support.

Fraudulent Ratings and Reviews

App Store ratings and reviews help many users make decisions about which apps to download, and developers rely on them to incorporate new features that respond to user feedback. Apple relies on a sophisticated system that combines machine learning, artificial intelligence, and human review by expert teams to moderate these ratings and reviews to help ensure accuracy and maintain trust. Since 2020, Apple has processed over 1 billion ratings and over 100 million reviews, and over 250 million ratings and reviews were removed for not meeting moderation standards.

Apple also recently deployed new tools to verify rating and review account authenticity, to analyze written reviews for signs of fraud, and to ensure that content from deactivated accounts is removed.

Account Fraud

Unfortunately, sometimes developer accounts are created entirely for fraudulent purposes. If a developer violation is egregious or repeated, the offender is expelled from the Apple Developer Program and their account terminated. Apple terminated 470,000 developer accounts in 2020 and rejected an additional 205,000 developer enrollments over fraud concerns, preventing them from ever submitting an app to the store.

Despite fraudsters’ sophisticated techniques to obscure their actions, Apple’s aggressive monitoring means these accounts are terminated, on average, less than a month after they are created.

Apple’s work to ensure the safety of users who download apps extends even beyond the App Store. Over the last 12 months, Apple found and blocked nearly 110,000 illegitimate apps on pirate storefronts. These storefronts distribute malicious software often designed to resemble popular apps — or that modify popular apps without their developers’ authorization — while circumventing the App Store’s security protections.

And in just the last month, Apple blocked more than 3.2 million instances of apps distributed illicitly through the Apple Developer Enterprise Program. The program is designed to allow companies and other large organizations to develop and privately distribute internal-use apps to their employees that aren’t available to the general public. Fraudsters attempt to distribute apps via this method to circumvent the rigorous App Review process, or to implicate a legitimate enterprise by manipulating an insider to leak credentials needed to ship illicit content.

In addition to fraudulent developer accounts, Apple works to identify and deactivate fraudulent user accounts. In 2020 alone, Apple deactivated 244 million customer accounts due to fraudulent and abusive activity. In addition, 424 million attempted account creations were rejected because they displayed patterns consistent with fraudulent and abusive activity.

Payment and Credit Card Fraud

Financial information and transactions are some of the most sensitive data that users share online. Apple has invested significant resources in building more secure payment technologies like Apple Pay and StoreKit, which are used by more than 900,000 apps to sell goods and services on the App Store. For example, with Apple Pay, credit card numbers are never shared with merchants — eliminating a risk factor in the payment transaction process.

With online data breaches frustratingly common, these protections are an essential part of keeping users safe. But users may not realize that when their credit card information is breached or stolen from another source, fraudsters may turn to online marketplaces like the App Store to attempt to purchase digital goods and services that can be laundered or used for illicit purposes.

Apple focuses relentlessly on this kind of fraud as well. In 2020 alone, the fusion of sophisticated technology and human review prevented more than 3 million stolen cards from being used to purchase stolen goods and services and banned nearly 1 million accounts from transacting again. In total, Apple protected users from more than $1.5 billion in potentially fraudulent transactions in 2020.

From App Review, to fraudulent account detection, to prevention of financial crimes, Apple works around the clock and behind the scenes to keep the App Store a safe and trusted place for users and developers alike.

* nokia.com/networks/portfolio/cyber-security/threat-intelligence-report-2020; media.defense.gov