James

gRPC is a modern open source remote procedure call framework. There are many exciting features in gRPC: real-time streaming, end-to-end code generation, and great cross-platform support to name a few. The most exciting to me, and consistently mentioned by developers who are interested in gRPC, is performance.

Last year Microsoft contributed a new implementation of gRPC for .NET to the CNCF. Built on top of Kestrel and HttpClient, gRPC for .NET makes gRPC a first-class member of the .NET ecosystem.

In our first gRPC for .NET release, we focused on gRPC’s core features, compatibility, and stability. In .NET 5, we made gRPC really fast.

gRPC and .NET 5 are fast

In a community run benchmark of different gRPC server implementations, .NET gets the highest requests per second after Rust, and is just ahead of C++ and Go.

This result builds on top of the work done in .NET 5. Our benchmarks show .NET 5 server performance is 60% faster than .NET Core 3.1. .NET 5 client performance is 230% faster than .NET Core 3.1.

Stephen Toub discusses dotnet/runtime changes in his Performance Improvements in .NET 5 blog post. Check it out to read about improvements in HttpClient and HTTP/2.

In the rest of this blog post I’ll talk about the improvements we made to make gRPC fast in ASP.NET Core.

HTTP/2 allocations in Kestrel

gRPC uses HTTP/2 as its underlying protocol. A fast HTTP/2 implementation is the most important factor when it comes to performance. Our gRPC server builds on top of Kestrel, a HTTP server written in C# that is designed with performance in mind. Kestrel is a top contender in the TechEmpower benchmarks, and gRPC benefits from a lot of the performance improvements in Kestrel automatically. However, there are many HTTP/2 specific optimizations that were made in .NET 5.

Reducing allocations is a good place to start. Fewer allocations per HTTP/2 request means less time doing garbage collection (GC). And CPU time “wasted” in GC is CPU time not spent serving HTTP/2 requests.

The performance profiler above is measuring allocations over 100,000 gRPC requests. The live object graph’s sawtooth shaped pattern indicates memory building up, then being garbage collected. About 3.9KB is being allocated per request. Lets try to get that number down!

dotnet/aspnetcore#18601 adds pooling of streams in a HTTP/2 connection. This one change almost cuts allocations per request in half. It enables reuse of internal types like Http2Stream, and publicly accessible types like HttpContext and HttpRequest, across multiple requests.

Once streams are pooled a range of optimizations become available:

There are many smaller allocation savings. dotnet/aspnetcore#19783 removes allocations in Kestrel’s HTTP/2 flow control. A resettable ManualResetValueTaskSourceCore<T> type replaces allocating a new object each time flow control is triggered. dotnet/aspnetcore#19273 replaces an array allocation with stackalloc when validating the HTTP request path. dotnet/aspnetcore#19277 and dotnet/aspnetcore#19325 eliminate some unintended allocations related to logging. dotnet/aspnetcore#22557 avoids allocating a Task<T> if a task is already complete. And finally dotnet/aspnetcore#19732 saves a string allocation by special casing content-length of 0. Because every allocation matters.

Per-request memory in .NET 5 is now just 330 B, a decrease of 92%. The sawtooth pattern has also disappeared. Reduced allocations means garbage collection didn’t run at all while the server processed 100,000 gRPC calls.

Reading HTTP headers in Kestrel

A hotpath in HTTP/2 is reading and writing HTTP headers. A HTTP/2 connection supports concurrent requests over a TCP socket, a feature called multiplexing. Multiplexing allows HTTP/2 to make efficient use of connections, but only the headers for one request on a connection can be processed at a time. HTTP/2’s HPack header compression is stateful and depends on order. Processing HTTP/2 headers is a bottleneck so has to be as fast as possible.

dotnet/aspnetcore#23083 optimizes the performance of HPackDecoder. The decoder is a state machine that reads incoming HTTP/2 HEADER frames. The approach here is good, the state machine allows Kestrel to decode frames as they arrive, but the decoder was checking state after parsing each byte. Another problem is literal values, the header names and values, were copied multiple times. Optimizations in this PR include:

• Tighten parsing loops. For example, if we’ve just parsed a header name then the value must come afterwards. There is no need to check the state machine to figure out the next state.
• Skip literal parsing all together. Literals in HPack have a length prefix. If we know the next 100 bytes are a literal then there is no need to inspect each byte. Mark the literal’s location and resuming parsing at its end.
• Avoid copying literal bytes. Previously literal bytes were always copied to an intermediary array before passed to Kestrel. Most of the time this isn’t necessary and instead we can just slice the original buffer and pass a ReadOnlySpan<byte> to Kestrel.

Together these changes significantly decrease the time it takes to parse headers. Header size is almost no longer a factor. The decoder marks the start and end position of a value and then slices that range.

private HPackDecoder _decoder = CreateDecoder();

private byte[] _smallHeader = new byte[] { /* HPack bytes */ };

private byte[] _largeHeader = new byte[] { /* HPack bytes */ };

private IHttpHeadersHandler _noOpHandler = new NoOpHeadersHandler(); [Benchmark]

public void SmallDecode() => _decoder.Decode(_smallHeader, endHeaders: true, handler: _noOpHandler); [Benchmark]

public void LargeDecode() => _decoder.Decode(_largeHeader, endHeaders: true, handler: _noOpHandler);

Once headers have been decoded, Kestrel needs to validate and process them. For example, special HTTP/2 headers like :path and :method need to be set onto HttpRequest.Path and HttpRequest.Method, and other headers need to be converted to strings and added to the HttpRequest.Headers collection.

Kestrel has the concept of known request headers. Known headers are a selection of commonly occuring request headers that have been optimized for fast setting and getting. dotnet/aspnetcore#24730 adds an even faster path for setting HPack static table headers to the known headers. The HPack static table gives 61 common header names and values a number ID that can be sent instead of the full name. A header with a static table ID can use the optimized path to bypass some validation and quickly be set in the collection based on its ID. dotnet/aspnetcore#24945 adds extra optimization for static table IDs with a name and value.

Adding HPack response compression

Prior to .NET 5, Kestrel supported reading HPack compressed headers in requests, but it didn’t compress response headers. The obvious advantage of response header compression is less network usage, but there are performance benefits as well. It’s faster to write a couple of bits for a compressed header than it is to encode and write the header’s full name and value as bytes.

dotnet/aspnetcore#19521 adds initial HPack static compression. Static compression is pretty simple: if the header is in the HPack static table then write the ID to identify the header instead of the longer name.

Dynamic HPack header compression is more complicated, but also provides bigger gains. Response header names and values are tracked in a dynamic table and are each assigned an ID. As a response’s headers are written, the server checks to see if the header name and value are in the table. If there is a match then the ID is written. If there isn’t then the full header is written, and it is added to the table for the next response. There is a maximum size of the dynamic table, so adding a header to it may evict other headers with a first in, first out order.

dotnet/aspnetcore#20058 adds dynamic HPack header compression. To quickly search for headers the dynamic table groups header entries using a basic hash table. To track order and evict the oldest headers, entries maintain a linked list. To avoid allocations, removed entries are pooled and reused.

Using Wireshark, we can see the impact of header compression on response size for this example gRPC call. .NET Core 3.x writes 77 B, while .NET 5 is only 12 B.

Protobuf message serialization

gRPC for .NET uses the Google.Protobuf package as the default serializer for messages. Protobuf is an efficient binary serialization format. Google.Protobuf is designed for performance, using code generation instead of reflection to serialize .NET objects. There are some modern .NET APIs and features that can be added to it to reduce allocations and improve efficiency.

The biggest improvement to Google.Protobuf is support for modern .NET IO types: Span<T>, ReadOnlySequence<T> and IBufferWriter<T>. These types allow gRPC messages to be serialized directly using buffers exposed by Kestrel. This saves Google.Protobuf allocating an intermediary array when serializing and deserializing Protobuf content.

Support for Protobuf buffer serialization was a multi-year effort between Microsoft and Google engineers. Changes were spread across multiple repositories.

protocolbuffers/protobuf#7351 and protocolbuffers/protobuf#7576 add support for buffer serialization to Google.Protobuf. This is by far the biggest and most complicated change. Three attempts were made to add this feature before the right balance between performance, backwards compatibility and code reuse was found. Protobuf reading and writing uses many performance oriented features and APIs added to C# and .NET Core:

• Span<T> and C# ref struct types enables fast and safe access to memory. Span<T> represents a contiguous region of arbitrary memory. Using span lets us serialize to managed .NET arrays, stack allocated arrays, or unmanaged memory, without using pointers. Span<T> and .NET protects us against buffer overflow.
• stackalloc is used to create stack-based arrays. stackalloc is a useful tool to avoid allocations when a small buffer is required.
• Low-level methods such as MemoryMarshal.GetReference(), Unsafe.ReadUnaligned() and Unsafe.WriteUnaligned() convert directly between primitive types and bytes.
• BinaryPrimitives has helper methods for efficiently converting between .NET primitive types and bytes. For example, BinaryPrimitives.ReadUInt64LittleEndian reads little endian bytes and returns an unsigned 64 bit number. Methods provided by BinaryPrimitive are heavily optimized and use vectorization.

A great thing about modern C# and .NET is it is possible to write fast, efficient, low-level libraries without sacrificing memory safety. When it comes to performance, .NET lets you have your cake and eat it too!

private TestMessage _testMessage = CreateMessage();

private ReadOnlySequence<byte> _testData = CreateData();

private IBufferWriter<byte> _bufferWriter = CreateWriter(); [Benchmark]

public IMessage ToByteArray() => _testMessage.ToByteArray(); [Benchmark]

public IMessage ToBufferWriter() => _testMessage.WriteTo(_bufferWriter); [Benchmark]

public IMessage FromByteArray() => TestMessage.Parser.ParseFrom(CreateBytes()); [Benchmark]

public IMessage FromSequence() => TestMessage.Parser.ParseFrom(_testData);

Adding support for buffer serialization to Google.Protobuf is just the first step. More work is required for gRPC for .NET to take advantage of the new capability:

• grpc/grpc#18865 and grpc/grpc#19792 adds ReadOnlySequence<byte> and IBufferWriter<byte> APIs to the gRPC serialization abstraction layer in Grpc.Core.Api.
• grpc/grpc#23485 updates gRPC code generation to glue the changes in Google.Protobuf to Grpc.Core.Api.
• grpc/grpc-dotnet#376 and grpc/grpc-dotnet#629 updates gRPC for .NET to use the new serialization abstractions in Grpc.Core.Api. This code is the integration between Kestrel and gRPC. Because Kestrel’s IO is built on top of System.IO.Pipelines, we can use its buffers during serialization.

The end result is gRPC for .NET serializes Protobuf messages directly to Kestrel’s request and response buffers. Intermediary array allocations and byte copies have been eliminated from gRPC message serialization.

Wrapping Up

Performance is a feature of .NET and gRPC, and as cloud apps scale it is more important than ever. I think all developers can agree it is fun to make fast apps, but performance has real world impact. Lower latency and higher throughput means fewer servers. It is an opportunity to save money, reduce power use and build greener apps.

As is obvious from this tour, a lot of changes have gone into gRPC, Protobuf and .NET aimed at improving performance. Our benchmarks show a 60% improvement in gRPC server RPS and a 230% improvement in gRPC client RPS.

.NET 5 RC2 is available now, and the official .NET 5 release is in November. To try out the performance improvements and to get started using gRPC with .NET, the best place to start is the Create a gRPC client and server in ASP.NET Core tutorial.

We look forward to hearing about apps built with gRPC and .NET, and to your future contributions in the dotnet and grpc repos!

Watch as developers share a deeper look into our plans for Year 4 with Beyond Light, Stasis, and Season of the Hunt.

Destiny 2: Beyond Light kicks off on November 10 with Guardians traveling to Europa to investigate the threats, mysteries, and power residing there. Season of the Hunt also begins on November 10 — you will be able to start earning ranks and rewards from the Season Pass, claim your Artifact, and begin to customize it as you power it up. This Season’s story mission and new Wrathborn Hunts activity will kick off the following week on Nov 17.

Starting with Season of the Hunt, most of the Seasonal content and all of the sweet gear will be sticking around for all of year 4. We hope this alleviates some of the FOMO that has been present with past Seasons. Now you can jump back in and experience past Seasons’ story, activities, and loot anytime during year 4. 

Here is a look at the calendar for Beyond Light and Season of the Hunt for a glimpse of what we have in store for you!

When was the last time you went on a nice long trip? Thanks to one thing and another this year, it’s probably been quite a while, although it’s hard to believe that three years have now passed since we first grabbed our caps and went on a kingdom-hopping jaunt in Super Mario Odyssey.

It is, however, entirely true: Nintendo UK even tweeted about it to remind us:

If that hypnotically looping gif above featuring the principal characters boogie-ing looks familiar, that’s because it’s adapted from a video put out to celebrate the game’s first anniversary in 2018.

Looking back on the game, it’s perhaps a little odd that it never got any meaty DLC. Yes, it was periodically updated with free costumes, some new snapshot filters, and it got that nugget of Labo VR content (not to mention Luigi’s Balloon World), but of all the games that do get paid DLC, a new locale or two to explore is something we would definitely have coughed up for. So many memories!

Perhaps there’s something in that, though. Could Nintendo be channelling all those ideas into a straight sequel, a-la Super Mario Galaxy 2? After all, everyone could do with getting away from it all again — now more than ever, no?

We can but dream (and cross our fingers, legs and other limbs until we look quite silly). While we wait for Mario’s next adventure — one that we didn’t already enjoy on Wii U, that is — let’s raise a glass to his last truly epic journey.

Apologies for mentioning Galaxy 2 (we know it still smarts). Feel free to share your own memories from three years of Super Mario Odyssey — and your hopes for the future — below.

Hyrule Warriors: Age of Calamity is out on Switch in less than a month, but a tweet from Kelios on Twitter (thanks Game Revolution) — which features apparent screen-grabs of a page from Nintendo’s Korean website suggests that a playable demo of the game is incoming.

According to the page (which now leads to an error message), the demo will be released alongside a new trailer for the game and a bunch of new details about the return to the Hyrule of the past — one we first visited in The Legend of Zelda: Breath of the Wild.

According to online translations and further investigation, you’ll be able to play through the title’s prologue and carry your progress over to the full game should you decide that, yes, I do want to explore Hyrule again, thank you. Take a look at the relevant tweets below, along with the rather amusing (and potentially spoiler-y) translation courtesy of Google:

While none of this has been confirmed officially, it certainly seems a solid sales tactic to employ and would likely hook people back into the world of Breath of the Wild who may have been sitting on the fence until Koei Tecmo’s game is out. People seemed to enjoy the Pikmin 3 Deluxe demo which became available without warning — and this is Zelda!

If a demo for HWAOC (the acronym that nobody’s using) is going to drop before release, we’d expect to see it soon; Age of Calamity is launching in less than a month’s time on 20th November.

Would you be up for dipping your toe into Age of Calamity with a demo? Is a taster pointless to you because you’ve had your pre-order in since it was first announced? Let us know with a comment below.

In game development, understanding player behavior is the key to tuning fun mechanics, creating compelling art direction, and smoothing out programming hiccups that might manifest as roadbumps to the player experience.

It’s a complicated process to understand the human mind, and in game development, it’s usually bundled under the field of “User Experience” (otherwise known as UX). One expert in the field, Celia Hodent, wants to take the time to educate you on the best practices for augmenting human psychology into your UX practices in her December 3rd GDC Masterclass.

Hodent’s instruction will go beyond her numerous GDC talks—she’ll be taking the day to teach you what she’s learned about human psychology in a classroom setting, mixing one-on-one instruction and exercises with the professional insight she’s picked up as a consultant and working at Ubisoft, LucasArts, and Epic Games.

As an attendee, here are some of the takeaways you can expect to gain from Hodent’s course:

• Learn how to significantly improve the experience of the game you are developing as perceived by your target audience.
• Improve usability and engagement.
• Understand the main characteristics and limitations of the human brain (in terms of perception, attention, memory, motivation, and emotion), and how they impact game development.
• Learn what makes a game engaging (in terms of motivation, emotion, and game flow) and how to foster these components in your game.
• Have a clear and concrete understanding of what having a UX mindset and developing a UX strategy entail.
• Course includes a complimentary copy of The Gamer’s Brain: How Neuroscience and UX Can Impact Video Game Design

But that’s not all. Hodent wants to make sure you have the best tools to apply what you’ve learned when you return to the field, so she’ll be including a copy of her book The Gamer’s Brain: How Neuroscience and UX Can Impact Video Game Design.

Space is limited–only a handful of talented game designers will be able to participate in this exceptionally useful class. Register today and claim your seat before it’s too late!

For more information on the GDC Masterclass program, be sure to visit our website or subscribe to regular updates via Facebook, Twitter, or RSS.

<!– –>

<!–

Calendar

View All     Submit Event

---

–>

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

---

---

  

---

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

---

[The GameDiscoverCo game discovery newsletter is written by ‘how people find your game’ expert Simon Carless, and is a regular look at how people discover and buy video games in the 2020s.]

Welcome to the final GameDiscoverCo newsletter of the week, since we’ve stored ‘like a billion’ interesting links in our newsletter template. And now we have to disgorge them, like a squirrel unhiding its nuts from a tree trunk. (Totally like that.)

Before we get started, just wanted to point out this video interview with me on game discoverability as part of a mini-series by Rocket Jump Events over on the Etch Play YouTube channel. I talk about how people find your games! Some of you care about that! Go check it out…

The new PlayStation (web) store gets.. mixed reviews

Ahead of the PlayStation 5’s launch, Sony has rolled out a brand new web version of the PlayStation Store, which definitely needed an upgrade. (Though the removal of digital support for older Sony platforms is not always popular.)

There’s a very long ResetEra forum thread (which started before the store was available to all) including a lot of good/interesting player feedback in it. Thus far, it seems like the design is way cleaner and mobile friendly.

But there’s a number of obvious usability & discovery issues with the new version of the store, which is ‘bare bones’ in a variety of ways. I thought Toumari’s ResetEra comments (scroll down to post #200) were fair and helpful. Some of the suggestion headings – which he goes into more detail on – are as follows:

“Add titles below the thumbnails on the category view… Make it obvious what is DLC and what is a game… Show DLC of a game on their respective pages… Add filters on the category/search pages… Show details about the different game editions.”

In addition, he noted some of the features that are missing in the new store that were included in the old store: “No screenshots/gameplay videos on the game pages, No download file-size listed on the game pages, No wishlist support, Can no longer add games/products to the cart from the list views.”

I’m sure the store will be a work in progress. And it’s unclear how closely its usability or layout mirrors that of the PlayStation 5 UI itself, since the UI demos so far haven’t looked closely at the PlayStation Store. But there’s clearly some work to be done here that will help more people buy games, if correctly addressed.

New perspectives on genre viability

The subject of Steam/game genre tags and viability has come up a lot recently. So kudos to Chris Zukowski for his latest article/newsletter, which includes this gem of a graph:

Basically, select genres with a higher median earning on Steam are towards the left, and lower towards the right. And then each genre also has a bar chart listing the number of games categorized within it (often by players).

There’s definitely some nuance here – particularly because a) your game can have multiple tags/genres and b) some large-tagged categories can refer to game elements – e.g. everything classified as puzzle is probably not a traditional puzzler. And this is Steam-specific – the PlayStation 4 might see different styles of game being popular.

But the point stands. It’s not solely about mechanically picking the right genres, but some game styles have less competition than others. Danny Weinbaum’s Steam tag data Google Doc (original blog post) clearly shows this. I recommend that everyone keeps considering it.

The game discovery news round-up..

And here’s the final final for the week, with a whole bunch of cloud gaming updates, as that space continues to hot up, plus updates on yesterday’s newsletter and more besides:

• There’s a new interview with Xbox’s Phil Spencer (originally at Stratechery, excerpted at The Verge) where he notes: “I think you’re going to see lower priced hardware as part of our ecosystem when you think about streaming sticks and other things that somebody might want to just go plug into their TV and go play via xCloud.” So all kinds of possibilities for Game Pass playing in the future, eh?

• Couple of interesting Stadia things – the company revealed Pac-Man Mega Tunnel Battle, a 64-person Pac-Man melee, with a cloud-streamable demo linked from the YouTube page. It’s part of an attempt to make ‘instant demos’ a better onramp to the service. (Tunnel Battle itself will be $20 when released in November on Stadia.)

• So, a lot of stuff went down on Twitch this week, since the company “informed streamers across its platform on Tuesday that it has deleted content violating music copyright laws after receiving “a sudden influx of DMCA takedown requests” back in June.” This was probably (mainly?) commercial music, not game OSTs, but definitely make sure your game’s music licenses don’t screw your streamers. (They don’t like it!)

• A couple of follow-ups from my Steam x China article that ran yesterday. Firstly, this EGM article talks about government licensing woes from a Chinese indie dev perspective. Secondly, I did want to reference this profanity-laced quote Tweet stream about the article (lol), since the author makes a good ethical point: “that requirement for text filtering? you’re engaging in the erasure of Uighurs, Taiwan and Hong Kong.” I don’t think we know that yet, since this is all hypotheticals. But I also don’t think we should ignore the concept.

• Amazon is now rolling out early access to its Amazon Luna streaming game service in the US, after “hundreds of thousands of requests”. The Luna+ and Ubisoft game channels are hyped, and it’s also noted: “We will continue to add great games, channels, and features to ensure there is something for everyone to play.” Looking forward to see who else steps up with a ‘channel’.

• A fan published a (slightly veiled) app that made Stadia streaming on iOS possible using a modified browser app. (“You can basically trick Stadia’s website into thinking you’re a Chrome browser. Thus, Stadia streaming then works.”) Then they went to update it, Apple noticed and it’s been pulled. It’s fun times in the cloud streaming wars when the fans get involved!

• Microlinks: there’s a LOT of investment activity in games right now, here’s a great summary for 2020 so far; on the way smaller side of things, here’s a Reddit post about a Steam game with 1200 pre-launch wishlists and 35 sales in Week 1, showing it can be tricky out there.

And finally for this week, throwing in a bonus video about game… shows? John Teti’s piece on the contrasting host entrances for famed U.S. game show The Price Is Right’ is a tremendous example of ‘design in detail’, and thought provoking for all creatives:

[This newsletter is handcrafted by GameDiscoverCo, a new agency based around one simple issue: how do players find, buy and enjoy your premium PC or console game? We’ll be launching a ‘Plus’ paid newsletter tier with lots of extra info/data – watch out for it soon.]

---

Related Jobs

---

<!–

Extra Div –>

October 27, 2020: We now have a release date for Crash Bandicoot: On the Run.

The mobile game publisher, King, has finally revealed when we are likely to see the newest Crash Bandicoot game arrive on mobile. Crash Bandicoot: On the Run is set to release in Spring 2021, and if the App Store listing is anything to go by, March 25 is the likely release date.

We first heard about the new Crash Bandicoot mobile game back in February, when rumours appeared that it was in development by King, the famous publisher of Candy Crush. The game was later confirmed and named as Crash Bandicoot: On the Run, alongside a reveal trailer, and ever since then we’ve been waiting to hear news about release.

Fans of the original Crash Bandicoot games can expect this endless runner to involves sprinting, jumping, sliding, and just generally avoiding obstacles as you complete levels to earn rewards. There’s also set to be boss battles with some of the series most iconic baddies, such as Scorporilla, Nitrus Brio, Nina Cortex, Dingodile, and Fake Crash.

If you want to see what gameplay looks like, check out the brand new trailer below:

Crash Bandicoot: On the Run is available to pre-order now from Google Play and App Store, and if you do you get the exclusive blue hyena skin at launch, made famous by Crash Team Racing: Nitro Fueled. You can also stay up to date with the release schedule by following the official Crash Bandicoot: On the Run Twitter account.

If you just can’t wait to play On the Run, we’ve got some other great recommendations for endless runners in our lists of the best iOS games and the best Android games!