Category: News

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

---

*All ideas mentioned are from my memory and shouldn’t be considered official.
*All names mentioned are internal to be renamed if the game had ever been released.
*All material is considered Pre-Alpha.

Now that Arkane Studio’s version of Prey is out and living its own life, I rummaged through some old notes I had on Human Head Studio‘s Prey 2 world.

What you’ll be looking at are top-down designs from the 3D environments. Areas the player could not reach in the distance are not included. So don’t interpret these layouts as small city islands. Central City on Exodus was a vast urban sprawl surrounding the player.

The planet Exodus was tidal locked, so regions had a fixed amount of light for the whole day. Only the weather conditions changed. Spanning the equator was Central City at the edge of night. The Bowery was one of the many districts found there. Undercity was below the raised Central City, so it was naturally dark surrounded by massive caverns. The Depot was just south at the edge of Central City in the colder night. And Locktown was out in the baking hot sun.

The Bowery’s the entertainment, neon-noir hub the public saw at E3 in 2011. It’s filled with loads of vice and wares. This city layout went through 3-4 different versions.

Here’s some scraps from an early version I did.  Various paths and population density provided diversity to different neighborhoods.

The further north you went in this district the more elevated and opulent it became. The northern edge featured glitzy casinos and blood sport arenas. The southern edge featured seedy theaters and questionable gambling.

Solar energy collected on the light side of Exodus beyond Locktown fueled various Energy Hubs throughout Central City. These arched over the sky and were sometimes subject to raids. One of these anchored itself on the eastern edge of the district.

The Bowery was also the main marketplace for Central City. This huge hive of activity was right in the middle of the district.

As with any district, it featured a lot of verticality. Primary streets tracked the player to different points-of-interest while more challenging routes shortcutted across, over, and under the neighborhoods.

More playtesting and ways to improve navigation prompted another version. This in turn spawned Version 3 which was showcased at E3.

This Bowery was like a big block of swiss cheese. Where most games at the time provided some verticality, they relied a lot on moving horizontally across the game to get to new content. In the Bowery, events could be found on platforms layered above and below one another.

You could sprint over local rooftops, still dwarfed in the shadows of larger suspended buildings out of reach above. You could slip underneath the main promenades to avoid detection. Some characters couldn’t be reached unless you had the right gadget or knew the right climbing path to encounter them.

Looking down, here’s some of the larger skybox of buildings and skyroads overlapping the district.

Below is the variety of things you would discover in the Bowery. You had various ways to travel to other locations on the planet. Trains and Taxis were common ways.  Some where unique, though.  You could take an elevator down to the mining zone or you could Portal jump to a larger casino beyond the western edge of the Bowery.  Both of these were “dungeons” you experienced in a more authored and linear way, contrasting the sandbox nature of the city hubs.

The portal jump to the west side Casino…

It should be no surprise Blade Runner inspired several locations throughout the game.  Several could be found in the Bowery…

City Interiors

In addition to the player’s apartment, several medium-sized interiors were spaced around the district including several clubs, a casino, and a marketplace.

Beyond that, a dozen shop types provided items to purchase while other nooks had alien contacts to help you track down info.  Blue labeled locations were managed by the authoritarian Regime.  Those along with city-system outlets could be sabotaged to cause chaos, distraction, and draw out enemies.  Some vendors gave you specific upgrades, some gave you credits for things you found or captured. Some simply let you mingle with the inhabitants, gamble, or drink. 

Later in the project, we were told Prey 2 suddenly had a lot of quality issues. One of the ideas discussed was changing the beginning of the game. One had you start out at the plane crash site from the first Prey. Another one had you working with Tommy sabotaging a location before he appeared to sacrifice himself saving you.

One that I favored was getting the player in as quickly as possible without a series of tutorials. A bombastic James Bond opening.  You play what you saw in Blur’s Cinematic trailer made with Human Head’s direction. The video did a great job at introducing all the core mechanics.  At the time it was a rare trailer showcasing what you could actually do in the game instead of a story cinematic.

Like a classic id Software game, just drop players in and let them figure it out. Similar to Temple Run or one of Uncharted’s running sequences, you quickly chase down a bounty dodging enemies and obstacles. (Of course this idea would need to be playtested!)

A more radical idea I did in the final days was to unravel the Bowery. Neighborhoods were easy to readjust because how they had been built and designed. Below is a comparison image to understand how areas were shifted between version 3 and 4.  This was able to be playtested in less than a week.

The Bowery was no longer a big block of swiss cheese. It was now a ridge of buildings spiraling down to slums. Just beyond the slums was a massive Regime tower looming over the citizens.

Before − players were traveling to points-of-interests right in front or above them and anything far away was only in an unreachable skybox.

Now − this gave players longer vistas to locations they could reach. They were no longer in a box surrounded by other boxes.  This was my favorite layout because it let the Bowery breathe and showcase all the diverse areas the team created.

This post also appears on my website at CuriousConstructs.com where you can find larger versions of these images.

Giving players the power to move around your VR game without making themselves sick is tricky business, but Neat Corporation seem to pulled it off pretty well with the “portal” locomotion in its VR sneaker Budget Cuts.

As part of VRDC @ GDC 2018 earlier this year, Neat Corp.’s Joachim Holmer gave an insightful talk about how the studio conceived of and implemented the solution.

It was a fun, concise talk that shed light on the ins and outs of the game’s portal translocator device, the technical implementation of the system, and Neat’s solutions to making it performant.

Holmer’s presentation was very interesting, especially if you’re a VR dev or a fan of Budget Cuts, so don’t miss your chance to now watch his talk completely free via the official GDC YouTube channel! 

In addition to this presentation, the GDC Vault and its accompanying YouTube channel offers numerous other free videos, audio recordings, and slides from many of the recent Game Developers Conference events, and the service offers even more members-only content for GDC Vault subscribers.

Those who purchased All Access passes to recent events like GDC or VRDC already have full access to GDC Vault, and interested parties can apply for the individual subscription via a GDC Vault subscription page. Group subscriptions are also available: game-related schools and development studios who sign up for GDC Vault Studio Subscriptions can receive access for their entire office or company by contacting staff via the GDC Vault group subscription page. Finally, current subscribers with access issues can contact GDC Vault technical support.

Gamasutra and GDC are sibling organizations under parent UBM Americas.

Location: Plymouth, Michigan

Stardock released the first DirectX 12 game as well as the first game to support both DirectX 12 and Vulkan. We are currently seeking a full-time Graphics Engineer to work on our next generation game engine for both announced and unannounced titles to build up our team of dedicated and experienced talent!  This is a salaried, full-time position at our Plymouth, Michigan office.

 Primary Responsibilities Include:

• Develop graphics technologies to enable aesthetics and gameplay.
• Create and optimize shaders and materials to run on a variety of platforms and machine specifications. 
• Develop systems for animations and special effects.
• Creating and maintaining game art pipeline.

 Education and/or Experience Desired:

• Bachelor’s Degree in Computer Science, Engineering, Mathematics or equivalent work experience.
• 5+ years of DirectX, Open GL, or Metal experience.
• Strong proficiency in Math including Linear Algebra, Trigonometry and Quaternions.
• A passion for optimization and familiarity with optimization tools.
• Self-motivated, likes to identify and solve problems. 
• Familiarity with Vulkan and excitement about its potential. 
• Very strong C++ background.

Interested? Apply now.

Whether you’re just starting out, looking for something new, or just seeing what’s out there, the Gamasutra Job Board is the place where game developers move ahead in their careers.

Gamasutra’s Job Board is the most diverse, most active, and most established board of its kind in the video game industry, serving companies of all sizes, from indie to triple-A.

Looking for a new job? Get started here. Are you a recruiter looking for talent? Post jobs here.

Steam Spy creator Sergey Galyonkin has opened up about how Valve’s recent changes to Steam interfered with his unofficial Steam sales tracker, noting in a Medium post that he’s working on getting everything up and running again despite the setback. 

In its previous form, Steam Spy was be a useful tool for developers looking get a read on trends in video game markets, so the return of the service could stand to once again give developers an information boost.

Right now, Galyonkin says he has brought Steam Spy back online in a limited capacity. Following a second change to the Steam API and hundreds of emails to developers, he says he decided to set the new algorithm powering the utility live and get basic features like the ownership ranges, playtime distribution, and related games up and running once again. 

But he notes that, in this new form, Steam Spy is by no means as accurate as its earlier version. Galyonkin says that the margin of error is within 10 percent for a good chunk of the games he has verifiable data for, but that he’s spotted some “crazy outliers” as well.

Galyonkin’s full post has more details on the issues Steam Spy encountered when Valve switched things up and a full explanation of what’s different between the old and new utility.

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

---

Originally posted on compoundtheory.com.

This is part four of a fivefour-part series on scaling game servers with Kubernetes.

In the previous three posts, we hosted our game servers on Kubernetes, measured and limited their resource usage, and scaled up the nodes in our cluster based on that usage. Now we need to tackle the harder problem: scaling down the nodes in our cluster as resources are no longer being used, while ensuring that in-progress games are not interrupted when a node is deleted.

On the surface, scaling down nodes in our cluster may seem particularly complicated. Each game server has in-memory state of the current game and multiple game clients are connected to an individual game server playing a game. Deleting arbitrary nodes could potentially disconnect active players — and that tends to make them angry! Therefore, we can only remove nodes from a cluster when a node is empty of dedicated game servers.

This means that if you are running on Google Kubernetes Engine (GKE), or similar, you can’t use a managed autoscaling system. To quote the documentation for the GKE autoscaler “Cluster autoscaler assumes that all replicated Pods can be restarted on some other node…” — which in our case is definitely not going to work, since it could easily delete nodes that have active players on them.

That being said, when looking at this situation more closely, we discover that we can break this down into three separate strategies that when combined together make scaling down a manageable problem that we can implement ourselves:

1. Group game servers together to avoid fragmentation across the cluster
2. Cordon nodes when CPU capacity is above the configured buffer
3. Delete a cordoned node from the cluster once all the games on the node have exited

Let’s look at each of these detail.

Grouping Game Servers Together in the Cluster

We want to avoid fragmentation of game servers across the cluster so we don’t end up with a wayward small set of game servers still running across multiple nodes, which will prevent those nodes from being shut down and reclaiming their resources.

This means we don’t want a scheduling pattern that creates game server Pods on random nodes across our cluster like this:

But instead want to have our game server Pods scheduled packed as tight as possible like this:

To group our game servers together, we can take advantage of Kubernetes Pod PodAffinity configuration with the PreferredDuringSchedulingIgnoredDuringExecution option. This gives us the ability to tell Pods that we prefer to group them by the hostname of the node that they are currently on, which essentially means that Kubernetes will prefer to put a dedicated game server Pod on a node that already has a dedicated game server Pod on it already.

In an ideal world, we would want a dedicated game server Pod to be scheduled on the node with the most dedicated game server Pods, as long as that node also has enough spare CPU resources. We could definitely do this if we wanted to write our own custom scheduler for Kubernetes, but to keep this demo simple, we will stick with the PodAffinity solution. That being said, when we consider the short length of our games, and that we will be adding (and explaining) cordoning nodes shortly, this combination of techniques is good enough for our requirements, and removes the need for us to write additional complex code.

When we add the PodAffinity configuration to the previous post’s configuration, we end up with the following, which tells Kubernetes to put pods with the labels sessions: game on the same node as each other whenever possible.

apiVersion: v1
kind: Pod
metadata: generateName: "game-"
spec: hostNetwork: true restartPolicy: Never nodeSelector: role: game-server containers: - name: soccer-server image: gcr.io/soccer/soccer-server:0.1 env: - name: SESSION_NAME valueFrom: fieldRef: fieldPath: metadata.name resources: limits: cpu: "0.1" affinity: podAffinity: # group game server Pods preferredDuringSchedulingIgnoredDuringExecution: - podAffinityTerm: labelSelector: matchLabels: sessions: game topologyKey: kubernetes.io/hostname

Cordoning Nodes

Now that we have our game servers relatively well packed together in the cluster, we can discuss “cordoning nodes”. What does cordoning nodes really mean? Very simply, Kubernetes gives us the ability to tell the scheduler: “Hey scheduler, don’t schedule anything new on this node here”. This ensures that no new Pods get scheduled on that node. In fact, in some places in the Kubernetes documentation, this is simply referred to as marking a node unschedulable.

In the code below, if you focus on the section s.bufferCount < available you will see that we make a request to cordon nodes if the amount of CPU buffer we currently have is greater than what we have set as our need. We’ve stripped some parts out for brevity, but you can see the original here.

// scale scales nodes up and down, depending on CPU constraints
// this includes adding nodes, cordoning them as well as deleting them
func (s Server) scaleNodes() error { nl, err := s.newNodeList() if err != nil { return err } available := nl.cpuRequestsAvailable() if available < s.bufferCount { finished, err := s.uncordonNodes(nl, s.bufferCount-available) // short circuit if uncordoning means we have enough buffer now if err != nil || finished { return err } nl, err := s.newNodeList() if err != nil { return err } // recalculate available = nl.cpuRequestsAvailable() err = s.increaseNodes(nl, s.bufferCount-available) if err != nil { return err } } else if s.bufferCount < available { err := s.cordonNodes(nl, available-s.bufferCount) if err != nil { return err } } return s.deleteCordonedNodes()
}

As you can also see from the code above, we can uncorden any available cordoned nodes in the cluster if we drop below the configured CPU buffer. This is faster than adding a whole new node, so it’s important to check for cordoned nodes before adding a whole new node from scratch.  Because of this we also have a configured delay on how long before a cordoned node is deleted (you can see the source here) to limit thrashing on creating and deleting nodes in the cluster unnecessarily.

This is a pretty great start. However, when we want to cordon nodes, we want to cordon only the nodes that have the least number of game server Pods on them, as in this instance, they are most likely to empty first as game sessions come to an end.

Thanks to the Kubernetes API, it’s relatively straightforward to count the number of game server Pods on each Node, and sort them in ascending order.  From there we can do arithmetic to determine if we still remain above the desired CPU buffer if we cordon each of the available nodes. If so, we can safely cordon those nodes.

// cordonNodes decrease the number of available nodes by the given number of cpu blocks (but not over),
// but cordoning those nodes that have the least number of games currently on them
func (s Server) cordonNodes(nl *nodeList, gameNumber int64) error { // … removed some input validation ... // how many nodes (n) do we have to delete such that we are cordoning no more // than the gameNumber capacity := nl.nodes.Items[0].Status.Capacity[v1.ResourceCPU] //assuming all nodes are the same cpuRequest := gameNumber * s.cpuRequest diff := int64(math.Floor(float64(cpuRequest) / float64(capacity.MilliValue()))) if diff <= 0 { log.Print("[Info][CordonNodes] No nodes to be cordoned.") return nil } log.Printf("[Info][CordonNodes] Cordoning %v nodes", diff) // sort the nodes, such that the one with the least number of games are first nodes := nl.nodes.Items sort.Slice(nodes, func(i, j int) bool { return len(nl.nodePods(nodes[i]).Items) < len(nl.nodePods(nodes[j]).Items) }) // grab the first n number of them cNodes := nodes[0:diff] // cordon them all for _, n := range cNodes { log.Printf("[Info][CordonNodes] Cordoning node: %v", n.Name) err := s.cordon(&n, true) if err != nil { return err } } return nil
}

Removing Nodes from the Cluster

Now that we have nodes in our clusters being cordoned, it is just a matter of waiting until the cordoned node is empty of game server Pods before deleting it. The code below also makes sure the node count never drops below a configured minimum as a nice baseline for capacity within our cluster.

You can see this in the code below, and in the original context:

// deleteCordonedNodes will delete a cordoned node if it
// the time since it was cordoned has expired
func (s Server) deleteCordonedNodes() error { nl, err := s.newNodeList() if err != nil { return err } l := int64(len(nl.nodes.Items)) if l <= s.minNodeNumber { log.Print("[Info][deleteCordonedNodes] Already at minimum node count. exiting") return nil } var dn []v1.Node for _, n := range nl.cordonedNodes() { ct, err := cordonTimestamp(n) if err != nil { return err } pl := nl.nodePods(n) // if no game session pods && if they have passed expiry, then delete them if len(filterGameSessionPods(pl.Items)) == 0 && ct.Add(s.shutdown).Before(s.clock.Now()) { err := s.cs.CoreV1().Nodes().Delete(n.Name, nil) if err != nil { return errors.Wrapf(err, "Error deleting cordoned node: %v", n.Name) } dn = append(dn, n) // don't delete more nodes than the minimum number set if l--; l <= s.minNodeNumber { break } } } return s.nodePool.DeleteNodes(dn)
}

Conclusion

We’ve successfully containerised our game servers, scaled them up as demand increases, and now scaled our Kubernetes cluster down, so we don’t have to pay for underutilised machinery — all powered by the APIs and capabilities that Kubernetes makes available out of the box. While it would take more work to turn this into a production level system, you can already see how to take advantage of the many building blocks available to you.

Before we finish, I would like to apologise for the delay in producing the fourth part in this series. If you saw the announcement, you may have guessed that a lot of my time got taken up developing and releasing Agones, the open source, productised version of this series of posts on running game servers on Kubernetes.

On that note, this will also be the last installment in this series. I had already completed the work to implement scaling down before starting on Agones, and rather than build out new functionality for global cluster management on Paddle Soccer, I’m going to focus those efforts building out awesome new features for Agones and bring it up from its current 0.1 alpha release, to a full 1.0, production-ready milestone.

I’m very excited about the future of Agones, and if my series of blog posts have inspired you, watch the GitHub repository, join the Slack, follow us on Twitter and get involved the mailing list. We’re actively seeking more contributors, and would love to have you involved.

Lastly, I welcome questions and comments here, or reach out to me via Twitter. You can also see my presentation at GDC and GCAP from 2017 on this topic, as well as check out the code in GitHub.

All posts in this series:

1. Containerising and Deploying
2. Managing CPU and Memory
3. Scaling Up Nodes
4. Scaling Down Nodes
5. Running Globally

The game industry’s enthusiasm for eSports is clear as day, but what’s less clear for many game developers is how, exactly, you design and tune a game to be played competitively at a professional level for years at a time.

Blizzard Entertainment did just that with StarCraft II, and at GDC 2011 Blizzard’s Dustin Browder took the stage to deconstruct the game’s design and examine how its competitive multiplayer was tuned to serve the needs of a global eSports community.

It was a unique look inside Blizzard’s design process, and shed light on the development and maintenance of one of the world’s early eSports success stories. If you missed it in person, it’s well worth going back to watch the entire talk for free over on the official GDC YouTube channel.

In addition to this presentation, the GDC Vault and its new YouTube channel offers numerous other free videos, audio recordings, and slides from many of the recent Game Developers Conference events, and the service offers even more members-only content for GDC Vault subscribers.

Those who purchased All Access passes to recent events like GDC, GDC Europe, and GDC Next already have full access to GDC Vault, and interested parties can apply for the individual subscription via a GDC Vault subscription page. Group subscriptions are also available: game-related schools and development studios who sign up for GDC Vault Studio Subscriptions can receive access for their entire office or company by contacting staff via the GDC Vault group subscription page. Finally, current subscribers with access issues can contact GDC Vault technical support.

Gamasutra and GDC are sibling organizations under parent UBM Tech

Following earlier conversations about the role the 3DS would play in a post-Switch industry, Nintendo now says that it has plans to support the 3DS into and beyond 2019.

Currently, the 7-year-old handheld platform has sold 72 million units across its various models and Nintendo says it plans to continue to leverage that install base in the coming years. During a presentation to investors, Nintendo president Tatsumi Kimishima said that the company is still looking at ways it can drive hardware and software sales, noting that Nintendo is currently “preparing new software for release in 2019 and beyond.”

Right now, the company has a handful of games like Captain Toad: Treasure Tracker, WarioWare Gold, and a Luigi’s Mansion rerelease due out for the system in 2018, with Mario & Luigi: Bowser’s Inside Story + Bowser Jr’s Journey slated for a 2019 release.

Kimishima also notes that the 3DS has a library of over 1,000 games between both digital and physical releases and that Nintendo is currently looking at ways to drive both hardware and software sales going forward.

“We will continue the Nintendo 3DS business by leveraging its installed base and rich software library,” said Kimishima. “We believe it is important to leverage the rich library of Nintendo 3DS series titles to drive sales with existing Nintendo 3DS owners as well as with consumers who recently purchased the Nintendo 3DS hardware, and are investigating strategies for doing so.”

Earlier this year, Nintendo leadership addressed concerns about how the 3DS would fit into its company following the Switch’s success since the portable nature of the Switch gave it some level overlap with Nintendo’s own dedicated handheld system.

At the time, Kimishima noted that the 3DS was still profitable and expressed hopes that “consumers will embrace it together with Nintendo Switch,” something that now looks to be reflected in the company’s release calendar.