Digital board game ports are a wonderful thing, but they can be at times confusing. As an inherently license’-based activity (with one or two exceptions), it can create weird situations like what happened with the digital adaptation of Carcassonne.

There in fact two different versions of Carcassonne you can buy on the market right now, one developed by The Coding Monkeys that’s only available on iOS, and then a slightly shinier version that came later developed by Asmodee Digital, which is only available on Steam and Android. That’s about to change come March 1st, 2020.

The Coding Monkeys announced over the weekend that their deal with the Carcassonne license holder, Hans im Glück, is coming to an end and won’t be renewed. Their version of the game will be removed from the App store come March, and you won’t be able to buy it again. Here’s what you need to bear in mind:

• If you already own the game and it’s installed on your device, you’ll be able to play it beyond March 1st, 2020.
• If you already own the game but it’s not installed, in theory you should still be able to download it regardless but this can be an inconsistent principle at times – best to put it on a device for safe keeping.
• TCM have said they will keep their servers running for at least a year, so multiplayer will still work for a time. What happens after that will depend on whether the studio can afford to keep them running.
• Even if they turn off the servers, as far as we know the game didn’t need to connect to an online server to play so solo/pass-and-play should still work.

iOS users need not fear that they’ll lose access to Carcassonne forever – Asmodee Digital’s own port will finally be coming to the Apple App Store come March. I imagine this is something they’ve been wanting ever since they made their own version of the game a couple of years ago.

The Coding Monkey’s version of the game is held in pretty high esteem and is a poster-child example of board game ports on mobile. Asmodee’s version is fine and is in 3D, giving it more appealing visuals, but i’d be hard-pressed to say it was the ‘better’ version. Still, it’s probably better to have a single, unified app for things like this.

As a final farewell, The Coding Monkeys are running a sale on their version of the game plus everything else they’ve made to date, so be sure to check it out.

What are your thoughts on this and the two versions of Carcassonne? Let us know in the comments!

 

By AppleInsider Staff
Monday, January 20, 2020, 04:16 pm PT (07:16 pm ET)

Apple CEO Tim Cook on Monday said the company is investigating technology that could help identify health risks at an early stage, similar to heart monitoring features introduced with Apple Watch.

Cook commented on Apple’s contributions to the healthcare space during a panel, suggesting what started with heart health tracking on Apple Watch could soon branch out into other areas of interest.

Current Apple Watch models are equipped with sensors capable to detecting atrial fibrillation, or AFib, a common heart arrhythmia that can lead to stroke in some patients. Apple Watch Series 4 and Series 5 go a step further and include an FDA-approved electrocardiogram function for more accurate readings.

As the first FDA-approved consumer device to incorporate an ECG, Apple Watch is an early entrant in what appears to be a burgeoning crossover sector that joins consumer tech with healthcare.

“I’m seeing that this intersection has not yet been explored very well. There’s not a lot of tech associated with the way people’s healthcare is done unless they get into very serious trouble.”” Cook said in a Q&A session with IDA Ireland CEO Martin Shanahan, according to Silicon Republic. IDA on Monday presented Cook with the inaugural Special Recognition Award for Apple’s 40 years of investment in Ireland

Most Apple Watch heart monitoring features, like AFib detection, are inherently preventative and can potentially reduce healthcare fees or even save lives.

“I think you can take that simple idea of having preventive things and find many more areas where technology intersects healthcare, and I think all of our lives would probably be better off for it,” Cook said. He added that the cost of healthcare can “fundamentally be taken down, probably in a dramatic way” by integrating common healthcare technologies in consumer devices.

“Most of the money in healthcare goes to the cases that weren’t identified early enough,” Cook said. “It will take some time but things that we are doing now — that I’m not going to talk about today — those give me a lot of cause for hope.”

Apple is known to be at work on multiple health-focused initiatives, though none have been formally announced. A recent patent filing from December, for example, suggests the company is developing methods of using Apple Watch to detect Parkinson’s Disease and diagnose tremor symptoms. Similar initiatives, like the sound monitoring Noise app and menstrual cycle tracking Cycle app, were announced and subsequently released with watchOS 6.

The Apple chief also touched on AR, once again calling it the “next big thing” in tech. Cook has long been bullish on the prospects of AR, which are being borne in iOS app releases.

“I think it’s something that doesn’t isolate people. We can use it to enhance our discussion, not substitute it for human connection, which I’ve always deeply worried about in some of the other technologies.”

Operating system distributions try to craft a one size fits all partition layout for their file systems. Distributions cannot know the details about how your hardware is configured or how you use your system though. Do you have more than one storage drive? If so, you might be able to get a performance benefit by putting the write-heavy partitions (var and swap for example) on a separate drive from the others that tend to be more read-intensive since most drives cannot read and write at the same time. Or maybe you are running a database and have a small solid-state drive that would improve the database’s performance if its files are stored on the SSD.

The following sections attempt to describe in brief some of the historical reasons for separating some parts of the file system out into separate partitions so that you can make a more informed decision when you install your Linux operating system.

If you know more (or contradictory) historical details about the partitioning decisions that shaped the Linux operating systems used today, contribute what you know below in the comments section!

Common partitions and why or why not to create them

The boot partition

One of the reasons for putting the /boot directory on a separate partition was to ensure that the boot loader and kernel were located within the first 1024 cylinders of the disk. Most modern computers do not have the 1024 cylinder restriction. So for most people, this concern is no longer relevant. However, modern UEFI-based computers have a different restriction that makes it necessary to have a separate partition for the boot loader. UEFI-based computers require that the boot loader (which can be the Linux kernel directly) be on a FAT-formatted file system. The Linux operating system, however, requires a POSIX-compliant file system that can designate access permissions to individual files. Since FAT file systems do not support access permissions, the boot loader must be on a separate file system than the rest of the operating system on modern UEFI-based computers. A single partition cannot be formatted with more than one type of file system.

The var partition

One of the historical reasons for putting the /var directory on a separate partition was to prevent files that were frequently written to (/var/log/* for example) from filling up the entire drive. Since modern drives tend to be much larger and since other means like log rotation and disk quotas are available to manage storage utilization, putting /var on a separate partition may not be necessary. It is much easier to change a disk quota than it is to re-partition a drive.

Another reason for isolating /var was that file system corruption was much more common in the original version of the Linux Extended File System (EXT). The file systems that had more write activity were much more likely to be irreversibly corrupted by a power outage than those that did not. By partitioning the disk into separate file systems, one could limit the scope of the damage in the event of file system corruption. This concern is no longer as significant because modern file systems support journaling.

The home partition

Having /home on a separate partition makes it possible to re-format the other partitions without overwriting your home directories. However, because modern Linux distributions are much better at doing in-place operating system upgrades, re-formatting shouldn’t be needed as frequently as it might have been in the past.

It can still be useful to have /home on a separate partition if you have a dual-boot setup and want both operating systems to share the same home directories. Or if your operating system is installed on a file system that supports snapshots and rollbacks and you want to be able to rollback your operating system to an older snapshot without reverting the content in your user profiles. Even then, some file systems allow their descendant file systems to be rolled back independently, so it still may not be necessary to have a separate partition for /home. On ZFS, for example, one pool/partition can have multiple descendant file systems.

The swap partition

The swap partition reserves space for the contents of RAM to be written to permanent storage. There are pros and cons to having a swap partition. A pro of having swap memory is that it theoretically gives you time to gracefully shutdown unneeded applications before the OOM killer takes matters into its own hands. This might be important if the system is running mission-critical software that you don’t want abruptly terminated. A con might be that your system runs so slow when it starts swapping memory to disk that you’d rather the OOM killer take care of the problem for you.

Another use for swap memory is hibernation mode. This might be where the rule that the swap partition should be twice the size of your computer’s RAM originated. Ideally, you should be able to put a system into hibernation even if nearly all of its RAM is in use. Beware that Linux’s support for hibernation is not perfect. It is not uncommon that after a Linux system is resumed from hibernation some hardware devices are left in an inoperable state (for example, no video from the video card or no internet from the WiFi card).

In any case, having a swap partition is more a matter of taste. It is not required.

The root partition

The root partition (/) is the catch-all for all directories that have not been assigned to a separate partition. There is always at least one root partition. BIOS-based systems that are new enough to not have the 1024 cylinder limit can be configured with only a root partition and no others so that there is never a need to resize a partition or file system if space requirements change.

The EFI system partition

The EFI System Partition (ESP) serves the same purpose on UEFI-based computers as the boot partition did on the older BIOS-based computers. It contains the boot loader and kernel. Because the files on the ESP need to be accessible by the computer’s firmware, the ESP has a few restrictions that the older boot partition did not have. The restrictions are:

1. The ESP must be formatted with a FAT file system (vfat in Anaconda)
2. The ESP must have a special type-code (EF00 when using gdisk)

Because the older boot partition did not have file system or type-code restrictions, it is permissible to apply the above properties to the boot partition and use it as your ESP. Note, however, that the GRUB boot loader does not support combining the boot and ESP partitions. If you use GRUB, you will have to create a separate partition and mount it beneath the /boot directory.

The Boot Loader Specification (BLS) lists several reasons why it is ideal to use the legacy boot partition as your ESP. The reasons include:

1. The UEFI firmware should be able to load the kernel directly. Having a separate, non-ESP compliant boot partition for the kernel prevents the UEFI firmware from being able to directly load the kernel.
2. Nesting the ESP mount point three mount levels deep increases the likelihood that an intermediate mount could fail or otherwise be unavailable when needed. That is, requiring root (/), then boot (/boot), then efi (/efi) to be consecutively mounted is unnecessarily complex and prone to error.
3. Requiring the boot loader to be able to read other partitions/disks which may be formatted with arbitrary file systems is non-trivial. Even when the boot loader does contain such code, the code that works at installation time can become outdated and fail to access the kernel/initrd after a file system update. This is currently true of GRUB’s ZFS file system driver, for example. You must be careful not to update your ZFS file system if you use the GRUB boot loader or else your system may not come back up the next time you reboot.

Besides the concerns listed above, it is a good idea to have your startup environment — up to and including your initramfs — on a single self-contained file system for recovery purposes. Suppose, for example, that you need to rollback your root file system because it has become corrupted or it has become infected with malware. If your kernel and initramfs are on the root file system, you may be unable to perform the recovery. By having the boot loader, kernel, and initramfs all on a single file system that is rarely accessed or updated, you can increase your chances of being able to recover the rest of your system.

In summary, there are many ways that you can layout your partitions and the type of hardware (BIOS or UEFI) and the brand of boot loader (GRUB, Syslinux or systemd-boot) are among the factors that will influence which layouts will work.

Other considerations

MBR vs. GPT

GUID Partition Table (GPT) is the newer partition format that supports larger disks. GPT was designed to work with the newer UEFI firmware. It is backward-compatible with the older Master Boot Record (MBR) partition format but not all boot loaders support the MBR boot method. GRUB and Syslinux support both MBR and UEFI, but systemd-boot only supports the newer UEFI boot method.

By using GPT now, you can increase the likelihood that your storage device, or an image of it, can be transferred over to a newer computer in the future should you wish to do so. If you have an older computer that natively supports only MBR-partitioned drives, you may need to add the inst.gpt parameter to Anaconda when starting the installer to get it to use the newer format. How to add the inst.gpt parameter is shown in the below video titled “Partitioning a BIOS Computer”.

If you use the GPT partition format on a BIOS-based computer, and you use the GRUB boot loader, you must additionally create a one megabyte biosboot partition at the start of your storage device. The biosboot partition is not needed by any other brand of boot loader. How to create the biosboot partition is demonstrated in the below video titled “Partitioning a BIOS Computer”.

LVM

One last thing to consider when manually partitioning your Linux system is whether to use standard partitions or logical volumes. Logical volumes are managed by the Logical Volume Manager (LVM). You can setup LVM volumes directly on your disk without first creating standard partitions to hold them. However, most computers still require that the boot partition be a standard partition and not an LVM volume. Consequently, having LVM volumes only increases the complexity of the system because the LVM volumes must be created within standard partitions.

The main features of LVM — online storage resizing and clustering — are not really applicable to the typical end user. Most laptops do not have hot-swappable drive bays for adding or reconfiguring storage while the system is running. And not many laptop or desktop users have clvmd configured so they can access a centralized storage device concurrently from multiple client computers.

LVM is great for servers and clusters. But it adds extra complexity for the typical end user. Go with standard partitions unless you are a server admin who needs the more advanced features.

Video demonstrations

Now that you know which partitions you need, you can watch the sort video demonstrations below to see how to manually partition a Fedora Linux computer from the Anaconda installer.

These videos demonstrate creating only the minimally required partitions. You can add more if you choose.

Because the GRUB boot loader requires a more complex partition layout on UEFI systems, the below video titled “Partitioning a UEFI Computer” additionally demonstrates how to install the systemd-boot boot loader. By using the systemd-boot boot loader, you can reduce the number of needed partitions to just two — boot and root. How to use a boot loader other than the default (GRUB) with Fedora’s Anaconda installer is officially documented here.

ONE PUNCH MAN: A HERO NOBODY KNOWS is Now Available for Pre-Purchase on Steam!

The first “One Punch Man” game finally makes its debut! Dive into a dynamic fighting game experience with beloved characters from the first Season of One Punch Man! Play as your favorite hero – or become one! YOU are the hero!

One of the most memorable features of the recent PlayStation 4 title Horizon Zero Dawn are the sophisticated robots, known as Machines, that wander the game world like a natural part of the landscape.

They are clearly automatons, with all their inner workings plainly visible. But they also exhibit unmistakable animal-like behaviors and movements. These Machines are a key feature of the game’s unique primordial futurist milieu.

How were these distinctive robot/creatures conceived of and designed? We talked with several devs from Guerrilla Games, the studio behind HZD, to see just what went into the making of the Machines.

At the start of the design process, the team queried local Holland universities, including the Delft University of Technology’s robotics department, for assistance. (Among the insights gleaned — far-robots would potentially be 3D printed.) One question the developers posed: Are there areas of nature that could be improved upon?

“Skeletons, they told us,” says Jan-Bart van Beek, studio art director. “Skeletons are kind of shit, because they’re on the inside,  surrounded by very soft tissue. And it’s a single point of failure — if your leg breaks, you’re pretty much dead as an animal.”

Tallneck

Instead, the profs suggested the possibility of something more like exoskeletons to the team–think lobsters, not humans, and you’ve got the right idea.

“If you look at some of our robot designs, you can see the outside has sort of a metal framework, and there is soft tissue on the insides, which also creates a convenient soft spot for arrows to be pumped into, ” van Beek says.

“All these things came from the inspiration of the robotics engineer explaining to us how they would build a robotic T-Rex if they had to do that,” he adds.

Thunderjaw

Originally, the machines weren’t intended to have such animal-like movements. The team initially made the machines behave in a way that was more, well, mechanical. “It felt too much like a bug, like it wasn’t moving the way it should be moving,” says Richard Oud, lead creature animator.

To bring the designs into the game — instead of waiting for a perfected model — the team would take unpolished models directly from ZBrush, slap a skeleton in, and bring it right in. Even if it was running at a slow FPS, it would give them some idea of the creature.

“You had to have big tough ones that were fast and deadly, and you also wanted ones that were easier at the beginning of the game,” says Blake Politeski, machine designer.

“And at the same time, we needed to find ways that these would fit in with the fiction of these sort of robots that are maintaining nature. They’re part of nature, so it had to fit with that narrative as well.”

Watchers

The modeling was in and of itself a colossal task. It took “man years” of work, and van Beek believes that it took five modelers around eight months for the T-Rex-like Thunderjaw. And that was just on the modeling end: it was around 18 months from the initial sketch to get it working and enjoyable in-game.

Despite the work, the team had a “realization that if we would nail the Thunderjaw, we would sort of know how to make the game,” van Beek says.

And to dial in how these robotic animals should move, the team even pulled in Dr. Stuart Sumida, who has done anatomical consulting on myriad projects including films The Lion King, Dinosaur, Hercules, and Harry Potter, and Disney World’s Countdown to Extinction and Expedition Everest rides. Oud also took a several month long course focused on animal and creature animation, as well.

Stormbird

Balancing the design of the animals with their in-game functions was another challenge, especially in terms of the visual complexity of the machines.

“There was always a risk maybe that it would just overload everything, you’d end up with a Christmas tree problem, where you have all these kinds of blinking lights and different colors and you’d actually have no idea what’s going on anymore,” van Beek says.

Animation and design worked together on this issue, so that all of the various machines’ gameplay elements were kept intact, while also keeping it clear to the players what they were there for.

Weight was also a challenge. The bigger things are, the harder they fall, as they say. But it also tends to mean the slower they move. Dennis Zopfi, lead machine designer, mentioned this difficulty in the Behemoth in particular.

Behemoth 

“From [an] AI and animation point, it was very hard to get right … you can’t make the thing turn too fast, because then it loses its weight,” Zopfi says.

“But if it cannot turn fast enough then, because the player’s very agile and athletic, so the players run by very fast and we had a lot of issues with making it look right but also making it responsive enough to deal with the player.”

Conveying weight to the player was a combination of factors, including the sound, animation, particles, and how the camera shakes.

“There’s lots of subtle little things maybe that can make a bunch of polygons colliding with a bunch of other polygons sort of look like a real thing,” van Beek says.

Guerrilla Games shared a series of images and videos with us that show the process of creating one particular type of Machine. Specifically it’s the Longleg, a flightless avian robot that’s somewhat reminiscent of the extinct apex predator known as “terror birds.”

1. Visual concept art of the Longleg

2. Final model of the Longleg

[embedded content]

3. Animation rig of the Longleg

[embedded content]

4. Reference footage: a real flightless bird

[embedded content]

5a. Blocking (CTRLs)

[embedded content]

5b. Block (No CTRLs)

6. Longleg animation network

[embedded content]

7. Longleg final polish

[embedded content]

8. The end result: Longleg in the game

Looking back at the team’s past work — like the giant spider-like MAWLR in KillZone 3 – brings into perspective just how quickly gaming technology continues to improve.

“That was incredibly complex at the time,” Politeski says. “And I remember thinking, ‘Oh my God we can’t do anything bigger than this.’ And then I look back now and it’s like, even our most basic robots overshadow that a lot. The amount that goes into each of these and how technically detailed they are and how complex they are is not really going to be seen by the players. But  to me, it’s actually amazing how big and complex these things are, and how we somehow managed to get it all working together.”

The Gamasutra Job Board is the most diverse, active and established board of its kind for the video game industry!

Here is just one of the many, many positions being advertised right now.

Location: Champaign, Illinois

Volition is searching for a Principal Writer for the Saints Row franchise. The ideal candidate is adept at interactive writing and storytelling for AAA open world action games, while also being able to direct, inspire, and mentor writers and other game developers. This is a rare opportunity to become an integral part of Volition’s creative team.

About Volition:

Deep Silver Volition has created original, smash hit games for over 20 years, including the Saints Row series. The studio has prospered by making games no one else can make, focusing on work-life balance, and developing a culture of collaboration, continuous growth, and learning. Volition is located in the heart of Champaign, Illinois – the best kept secret in the Midwest. Champaign has all the activities, amenities and diversity of much larger cities, but without the traffic and high cost of housing.

Responsibilities:

• Drive the narrative vision for the Saints Row franchise.
• Ensure narrative vision is consistent with the creative vision for projects throughout production.
• Lead a team of internal and remote writers to produce high quality writing.
• Collaborate across disciplines to identify and deliver on writing needs.
• Write and edit dialogue and other in-game text to be high-quality, snappy, and entertaining, and to match the Saints Row tone and style.
• Write or contribute high level direction for pitches, concepts, cinematics, style guides, documentation, and marketing material.
• Facilitate giving and receiving narrative feedback across the team.
• Share and represent the game’s narrative vision to the team and to the public.
• Conduct presentations, writing reviews, brainstorming sessions, and other meetings as needed.

Required Qualifications:

• 8+ years of experience as a game writer.
• In-depth understanding of how writing and gameplay work together to create fun, engaging experiences for the player.
• Exceptional talent for creating vivid characters, compelling narratives, and believable dialogue.
• Ability to lead, mentor, delegate, and provide clear direction to others.
• Open minded, with a knack for inspiring others and building consensus.
• Meticulous proofreading and editorial skills. Attention to detail.
• Positive, professional attitude. Disciplined and self-motivated with a strong work ethic.
• Strong communication and organizational skills.
• Passion for games and ability to articulate that passion clearly and analytically.
• Broad knowledge of pop culture, modern games, and entertainment.
• Ambition to create a Game of the Year title.

Preferred Qualifications:

• Previous experience as a Narrative Director, Lead Writer, Principal Writer or Creative Director.
• Proven track record with AAA open world games.
• Experience working with actors, casting, and directing VO recording sessions.
• Experience with narrative development tools.

Interested? Apply now.

Whether you’re just starting out, looking for something new, or just seeing what’s out there, the Gamasutra Job Board is the place where game developers move ahead in their careers.

Gamasutra’s Job Board is the most diverse, most active, and most established board of its kind in the video game industry, serving companies of all sizes, from indie to triple-A.

Looking for a new job? Get started here. Are you a recruiter looking for talent? Post jobs here.

The rise of microservices-oriented architecture brought us new development paradigms and mantras about independent development and decoupling. In such a scenario, we have to deal with a situation where we aim for independence, but we still need to react to state changes in different enterprise domains.

I’ll use a simple and typical example in order to show what we’re talking about. Imagine the development of two independent microservices: Order and User. We designed them to expose a REST interface and to each use a separate database, as shown in Figure 1:

We must notify the User domain about any change happening in the Order domain. To do this in the example, we need to update the order_list. For this reason, we’ve modeled the User REST service with addOrder and deleteOrder operations.

Solution 1: Queue decoupling

The first solution to consider is adding a queue between the services. Order will publish events that User will eventually process, as shown in Figure 2:

This is a fair design. However, if you don’t use the right middleware you will mix a lot of infrastructure code into your domain logic. Now that you have queues, you must develop producer and consumer logic. You also have to take care of transactions. The problem is to make sure that every event ends up correctly in both the Order database and in the queue.

Solution 2: Change data capture decoupling

Let me introduce an alternative solution that handles all of that work without your touching any line of your microservices code. I’ll use Debezium and Apache Camel to capture data changes on Order and trigger certain actions on User. Debezium is a log-based data change capture middleware. Camel is an integration framework that simplifies the integration between a source (Order) and a destination (User), as shown in Figure 3:

Debezium is in charge of capturing any data change happening in the Order domain and publishing it to a topic. Then a Camel consumer can pick that event and make a REST call to the User API to perform the necessary action expected by its domain (in our simple case, update the list).

Decoupling with Debezium and Camel

I’ve prepared a simple demo with all of the components we need to run the example above. You can find this demo in this GitHub repo. The only part we need to develop is represented by the following source code:

public class MyRouteBuilder extends RouteBuilder { public void configure() { from("debezium:mysql?name=my-sql-connector" + "&databaseServerId=1" + "&databaseHostName=localhost" + "&databaseUser=debezium" + "&databasePassword=dbz" + "&databaseServerName=my-app-connector" + "&databaseHistoryFileName=/tmp/dbhistory.dat" + "&databaseWhitelist=debezium" + "&tableWhitelist=debezium._order" + "&offsetStorageFileName=/tmp/offset.dat") .choice() .when(header(DebeziumConstants.HEADER_OPERATION).isEqualTo("c")) .process(new AfterStructToOrderTranslator()) .to("rest-swagger:http://localhost:8082/v2/api-docs#addOrderUsingPOST") .when(header(DebeziumConstants.HEADER_OPERATION).isEqualTo("d")) .process(new BeforeStructToOrderTranslator()) .to("rest-swagger:http://localhost:8082/v2/api-docs#deleteOrderUsingDELETE") .log("Response : ${body}"); } } 

That’s it. Really. We don’t need anything else.

Apache Camel has a Debezium component that can hook up a MySQL database and use Debezium embedded engine. The source endpoint configuration provides the parameters needed by Debezium to note any change happening in the debezium._order table. Debezium streams the events according to a JSON-defined format, so you know what kind of information to expect. For each event, you will get the information as it was before and after the event occurs, plus a few useful pieces of meta-information.

Thanks to Camel’s content-based router, we can either call the addOrderUsingPOST or deleteOrderUsingDELETE operation. You only have to develop a message translator that can convert the message coming from Debezium:

public class AfterStructToOrderTranslator implements Processor { private static final String EXPECTED_BODY_FORMAT = "{\"userId\":%d,\"orderId\":%d}"; public void process(Exchange exchange) throws Exception { final Map value = exchange.getMessage().getBody(Map.class); // Convert and set body int userId = (int) value.get("user_id"); int orderId = (int) value.get("order_id"); exchange.getIn().setHeader("userId", userId); exchange.getIn().setHeader("orderId", orderId); exchange.getIn().setBody(String.format(EXPECTED_BODY_FORMAT, userId, orderId)); } } 

Notice that we did not touch any of the base code for Order or User. Now, turn off the Debezium process to simulate downtime. You will see that it can recover all events as soon as it turns back on!

You can run the example provided by following the instructions on this GitHub repo.

Caveat

The example illustrated here uses Debezium’s embedded mode. For more consistent solutions, consider using the Kafka connect mode instead, or tuning the embedded engine accordingly.

The post Decoupling microservices with Apache Camel and Debezium appeared first on Red Hat Developer.

Why have regular expressions survived seven decades of technological disruption? Because coders who understand regular expressions have a massive advantage when working with textual data. They can write in a single line of code what takes others dozens!

This article is all about the re.compile(pattern) method of Python’s re library. Before we dive into re.compile(), let’s get an overview of the four related methods you must understand:

• The findall(pattern, string) method returns a list of string matches. Read more in our blog tutorial.
• The search(pattern, string) method returns a match object of the first match. Read more in our blog tutorial.
• The match(pattern, string) method returns a match object if the regex matches at the beginning of the string. Read more in our blog tutorial.
• The fullmatch(pattern, string) method returns a match object if the regex matches the whole string. Read more in our blog tutorial.

Equipped with this quick overview of the most critical regex methods, let’s answer the following question:

How Does re.compile() Work in Python?

The re.compile(pattern) method returns a regular expression object (see next section)

You then use the object to call important regex methods such as search(string), match(string), fullmatch(string), and findall(string).

In short: You compile the pattern first. You search the pattern in a string second.

This two-step approach is more efficient than calling, say, search(pattern, string) at once. That is, IF you call the search() method multiple times on the same pattern. Why? Because you can reuse the compiled pattern multiple times.

Here’s an example:

import re # These two lines ...

regex = re.compile('Py...n')

match = regex.search('Python is great') # ... are equivalent to ...

match = re.search('Py...n', 'Python is great')

In both instances, the match variable contains the following match object:

<re.Match object; span=(0, 6), match='Python'>

But in the first case, we can find the pattern not only in the string ‘Python is great’ but also in other strings—without any redundant work of compiling the pattern again and again.

Specification:

re.compile(pattern, flags=0)

The method has up to two arguments.

• pattern: the regular expression pattern that you want to match.
• flags (optional argument): a more advanced modifier that allows you to customize the behavior of the function. Want to know how to use those flags? Check out this detailed article on the Finxter blog.

We’ll explore those arguments in more detail later.

Return Value:

The re.compile(patterns, flags) method returns a regular expression object. You may ask (and rightly so):

What’s a Regular Expression Object?

Python internally creates a regular expression object (from the Pattern class) to prepare the pattern matching process. You can call the following methods on the regex object:

Method	Description
Pattern.search(string[, pos[, endpos]])	Searches the regex anywhere in the string and returns a match object or None. You can define start and end positions of the search.
Pattern.match(string[, pos[, endpos]])	Searches the regex at the beginning of the string and returns a match object or None. You can define start and end positions of the search.
Pattern.fullmatch(string[, pos[, endpos]])	Matches the regex with the whole string and returns a match object or None. You can define start and end positions of the search.
Pattern.split(string, maxsplit=0)	Divides the string into a list of substrings. The regex is the delimiter. You can define a maximum number of splits.
Pattern.findall(string[, pos[, endpos]])	Searches the regex anywhere in the string and returns a list of matching substrings. You can define start and end positions of the search.
Pattern.finditer(string[, pos[, endpos]])	Returns an iterator that goes over all matches of the regex in the string (returns one match object after another). You can define the start and end positions of the search.
Pattern.sub(repl, string, count=0)	Returns a new string by replacing the first count occurrences of the regex in the string (from left to right) with the replacement string repl.
Pattern.subn(repl, string, count=0)	Returns a new string by replacing the first count occurrences of the regex in the string (from left to right) with the replacement string repl. However, it returns a tuple with the replaced string as the first and the number of successful replacements as the second tuple value.

If you’re familiar with the most basic regex methods, you’ll realize that all of them appear in this table. But there’s one distinction: you don’t have to define the pattern as an argument. For example, the regex method re.search(pattern, string) will internally compile a regex object p and then call p.search(string).

You can see this fact in the official implementation of the re.search(pattern, string) method:

def search(pattern, string, flags=0): """Scan through string looking for a match to the pattern, returning a Match object, or None if no match was found.""" return _compile(pattern, flags).search(string)

(Source: GitHub repository of the re package)

The re.search(pattern, string) method is a mere wrapper for compiling the pattern first and calling the p.search(string) function on the compiled regex object p.

Is It Worth Using Python’s re.compile()?

No, in the vast majority of cases, it’s not worth the extra line.

Consider the following example:

import re # These two lines ...

regex = re.compile('Py...n')

match = regex.search('Python is great') # ... are equivalent to ...

match = re.search('Py...n', 'Python is great')

Don’t get me wrong. Compiling a pattern once and using it many times throughout your code (e.g., in a loop) comes with a big performance benefit. In some anecdotal cases, compiling the pattern first lead to 10x to 50x speedup compared to compiling it again and again.

But the reason it is not worth the extra line is that Python’s re library ships with an internal cache. At the time of this writing, the cache has a limit of up to 512 compiled regex objects. So for the first 512 times, you can be sure when calling re.search(pattern, string) that the cache contains the compiled pattern already.

Here’s the relevant code snippet from re’s GitHub repository:

# --------------------------------------------------------------------

# internals _cache = {} # ordered! _MAXCACHE = 512

def _compile(pattern, flags): # internal: compile pattern if isinstance(flags, RegexFlag): flags = flags.value try: return _cache[type(pattern), pattern, flags] except KeyError: pass if isinstance(pattern, Pattern): if flags: raise ValueError( "cannot process flags argument with a compiled pattern") return pattern if not sre_compile.isstring(pattern): raise TypeError("first argument must be string or compiled pattern") p = sre_compile.compile(pattern, flags) if not (flags & DEBUG): if len(_cache) >= _MAXCACHE: # Drop the oldest item try: del _cache[next(iter(_cache))] except (StopIteration, RuntimeError, KeyError): pass _cache[type(pattern), pattern, flags] = p return p

Can you find the spots where the cache is initialized and used?

While in most cases, you don’t need to compile a pattern, in some cases, you should. These follow directly from the previous implementation:

• You’ve got more than MAXCACHE patterns in your code.
• You’ve got more than MAXCACHE different patterns between two same pattern instances. Only in this case, you will see “cache misses” where the cache has already flushed the seemingly stale pattern instances to make room for newer ones.
• You reuse the pattern multiple times. Because if you don’t, it won’t make sense to use sparse memory to save them in your memory.
• (Even then, it may only be useful if the patterns are relatively complicated. Otherwise, you won’t see a lot of performance benefits in practice.)

To summarize, compiling the pattern first and storing the compiled pattern in a variable for later use is often nothing but “premature optimization”—one of the deadly sins of beginner and intermediate programmers.

What Does re.compile() Really Do?

It doesn’t seem like a lot, does it? My intuition was that the real work is in finding the pattern in the text—which happens after compilation. And, of course, matching the pattern is the hard part. But a sensible compilation helps a lot in preparing the pattern to be matched efficiently by the regex engine—work that would otherwise have be done by the regex engine.

Regex’s compile() method does a lot of things such as:

• Combine two subsequent characters in the regex if they together indicate a special symbol such as certain Greek symbols.
• Prepare the regex to ignore uppercase and lowercase.
• Check for certain (smaller) patterns in the regex.
• Analyze matching groups in the regex enclosed in parentheses.

Here’s the implemenation of the compile() method—it looks more complicated than expected, no?

def _compile(code, pattern, flags): # internal: compile a (sub)pattern emit = code.append _len = len LITERAL_CODES = _LITERAL_CODES REPEATING_CODES = _REPEATING_CODES SUCCESS_CODES = _SUCCESS_CODES ASSERT_CODES = _ASSERT_CODES iscased = None tolower = None fixes = None if flags & SRE_FLAG_IGNORECASE and not flags & SRE_FLAG_LOCALE: if flags & SRE_FLAG_UNICODE: iscased = _sre.unicode_iscased tolower = _sre.unicode_tolower fixes = _ignorecase_fixes else: iscased = _sre.ascii_iscased tolower = _sre.ascii_tolower for op, av in pattern: if op in LITERAL_CODES: if not flags & SRE_FLAG_IGNORECASE: emit(op) emit(av) elif flags & SRE_FLAG_LOCALE: emit(OP_LOCALE_IGNORE[op]) emit(av) elif not iscased(av): emit(op) emit(av) else: lo = tolower(av) if not fixes: # ascii emit(OP_IGNORE[op]) emit(lo) elif lo not in fixes: emit(OP_UNICODE_IGNORE[op]) emit(lo) else: emit(IN_UNI_IGNORE) skip = _len(code); emit(0) if op is NOT_LITERAL: emit(NEGATE) for k in (lo,) + fixes[lo]: emit(LITERAL) emit(k) emit(FAILURE) code[skip] = _len(code) - skip elif op is IN: charset, hascased = _optimize_charset(av, iscased, tolower, fixes) if flags & SRE_FLAG_IGNORECASE and flags & SRE_FLAG_LOCALE: emit(IN_LOC_IGNORE) elif not hascased: emit(IN) elif not fixes: # ascii emit(IN_IGNORE) else: emit(IN_UNI_IGNORE) skip = _len(code); emit(0) _compile_charset(charset, flags, code) code[skip] = _len(code) - skip elif op is ANY: if flags & SRE_FLAG_DOTALL: emit(ANY_ALL) else: emit(ANY) elif op in REPEATING_CODES: if flags & SRE_FLAG_TEMPLATE: raise error("internal: unsupported template operator %r" % (op,)) if _simple(av[2]): if op is MAX_REPEAT: emit(REPEAT_ONE) else: emit(MIN_REPEAT_ONE) skip = _len(code); emit(0) emit(av[0]) emit(av[1]) _compile(code, av[2], flags) emit(SUCCESS) code[skip] = _len(code) - skip else: emit(REPEAT) skip = _len(code); emit(0) emit(av[0]) emit(av[1]) _compile(code, av[2], flags) code[skip] = _len(code) - skip if op is MAX_REPEAT: emit(MAX_UNTIL) else: emit(MIN_UNTIL) elif op is SUBPATTERN: group, add_flags, del_flags, p = av if group: emit(MARK) emit((group-1)*2) # _compile_info(code, p, _combine_flags(flags, add_flags, del_flags)) _compile(code, p, _combine_flags(flags, add_flags, del_flags)) if group: emit(MARK) emit((group-1)*2+1) elif op in SUCCESS_CODES: emit(op) elif op in ASSERT_CODES: emit(op) skip = _len(code); emit(0) if av[0] >= 0: emit(0) # look ahead else: lo, hi = av[1].getwidth() if lo != hi: raise error("look-behind requires fixed-width pattern") emit(lo) # look behind _compile(code, av[1], flags) emit(SUCCESS) code[skip] = _len(code) - skip elif op is CALL: emit(op) skip = _len(code); emit(0) _compile(code, av, flags) emit(SUCCESS) code[skip] = _len(code) - skip elif op is AT: emit(op) if flags & SRE_FLAG_MULTILINE: av = AT_MULTILINE.get(av, av) if flags & SRE_FLAG_LOCALE: av = AT_LOCALE.get(av, av) elif flags & SRE_FLAG_UNICODE: av = AT_UNICODE.get(av, av) emit(av) elif op is BRANCH: emit(op) tail = [] tailappend = tail.append for av in av[1]: skip = _len(code); emit(0) # _compile_info(code, av, flags) _compile(code, av, flags) emit(JUMP) tailappend(_len(code)); emit(0) code[skip] = _len(code) - skip emit(FAILURE) # end of branch for tail in tail: code[tail] = _len(code) - tail elif op is CATEGORY: emit(op) if flags & SRE_FLAG_LOCALE: av = CH_LOCALE[av] elif flags & SRE_FLAG_UNICODE: av = CH_UNICODE[av] emit(av) elif op is GROUPREF: if not flags & SRE_FLAG_IGNORECASE: emit(op) elif flags & SRE_FLAG_LOCALE: emit(GROUPREF_LOC_IGNORE) elif not fixes: # ascii emit(GROUPREF_IGNORE) else: emit(GROUPREF_UNI_IGNORE) emit(av-1) elif op is GROUPREF_EXISTS: emit(op) emit(av[0]-1) skipyes = _len(code); emit(0) _compile(code, av[1], flags) if av[2]: emit(JUMP) skipno = _len(code); emit(0) code[skipyes] = _len(code) - skipyes + 1 _compile(code, av[2], flags) code[skipno] = _len(code) - skipno else: code[skipyes] = _len(code) - skipyes + 1 else: raise error("internal: unsupported operand type %r" % (op,))

Don’t worry, you don’t need to understand the code. Just note that all this work would have to be done by the regex engine at “matching runtime” if you wouldn’t compile the pattern first. If we can do it only once, it’s certainly a low-hanging fruit for performance optimizations—especially for long regular expression patterns.

How to Use the Optional Flag Argument?

As you’ve seen in the specification, the compile() method comes with an optional third ‘flag’ argument:

re.compile(pattern, flags=0)

What’s the purpose of the flags argument?

Flags allow you to control the regular expression engine. Because regular expressions are so powerful, they are a useful way of switching on and off certain features (for example, whether to ignore capitalization when matching your regex).

Syntax	Meaning
re.ASCII	If you don’t use this flag, the special Python regex symbols w, W, b, B, d, D, s and S will match Unicode characters. If you use this flag, those special symbols will match only ASCII characters — as the name suggests.
re.A	Same as re.ASCII
re.DEBUG	If you use this flag, Python will print some useful information to the shell that helps you debugging your regex.
re.IGNORECASE	If you use this flag, the regex engine will perform case-insensitive matching. So if you’re searching for [A-Z], it will also match [a-z].
re.I	Same as re.IGNORECASE
re.LOCALE	Don’t use this flag — ever. It’s depreciated—the idea was to perform case-insensitive matching depending on your current locale. But it isn’t reliable.
re.L	Same as re.LOCALE
re.MULTILINE	This flag switches on the following feature: the start-of-the-string regex ‘^’ matches at the beginning of each line (rather than only at the beginning of the string). The same holds for the end-of-the-string regex ‘$’ that now matches also at the end of each line in a multi-line string.
re.M	Same as re.MULTILINE
re.DOTALL	Without using this flag, the dot regex ‘.’ matches all characters except the newline character ‘n’. Switch on this flag to really match all characters including the newline character.
re.S	Same as re.DOTALL
re.VERBOSE	To improve the readability of complicated regular expressions, you may want to allow comments and (multi-line) formatting of the regex itself. This is possible with this flag: all whitespace characters and lines that start with the character ‘#’ are ignored in the regex.
re.X	Same as re.VERBOSE

Here’s how you’d use it in a practical example:

import re text = 'Python is great (python really is)' regex = re.compile('Py...n', flags=re.IGNORECASE) matches = regex.findall(text)

print(matches)

# ['Python', 'python']

Although your regex ‘Python’ is uppercase, we ignore the capitalization by using the flag re.IGNORECASE.

Where to Go From Here?

You’ve learned about the re.compile(pattern) method that prepares the regular expression pattern—and returns a regex object which you can use multiple times in your code.

Learning Python is hard. But if you cheat, it isn’t as hard as it has to be:

Download 8 Free Python Cheat Sheets now!