Python’s regex capturing groups allow you to extract parts of a string that match a pattern.

• Enclose the desired pattern in parentheses () to create a capturing group.
• Use re.search() to find matches, and access captured groups with the .group() method or by indexing the result.

For example: match = re.search(r'(\d+)', 'abc123') captures the digits, and match.group(1) returns '123'.

---

One of the powerful aspects of Python’s regular expression capabilities is the use of capturing groups. By using capturing groups, you can easily extract specific portions of a matching string and efficiently process and manipulate data that meets a particular pattern.

I like to use capturing groups to isolate and extract relevant data from a given text. To define a capturing group, I simply place the desired regex rule within parentheses, like this: (rule). This helps me match portions of a string based on the rule and output the captured data for further processing.

Tip: An essential technique I employ while working with capturing groups is using the finditer() method, as it finds all the matches and returns an iterator yielding match objects that match the regex pattern. Subsequently, I can iterate through each match object and extract its value.

Before I’ll teach you everything about capturing groups, allow me to give some background information on Python regular expressions. If you’re already an expert, you can jump directly to the “capturing groups” part of the article.

Understanding Regular Expressions

As someone who works with Python, I often find myself using regular expressions.

Recommended: Python Regex Superpower [Full Tutorial]

They provide a powerful tool for dealing with strings, patterns, and parsing text data. In this section, I’ll guide you through the basics of regular expressions and shed some light on capturing groups, which can be extremely helpful in many situations.

Basic Syntax

Regular expressions, or regex, are patterns that represent varying sets of characters. In Python, we can use the re module to perform various operations with regular expressions. A key component of regex is the set of metacharacters, which help define specific patterns.

Some common metacharacters are:

• . – matches any single character except a newline
• \w – matches any word character (letters, digits, and underscores)
• \d – matches any digit (0-9)
• \s – matches any whitespace character (including spaces, tabs, and newlines)

It’s important to remember that these metacharacters must be preceded by a backslash to represent their special meanings.

Special Characters

There are several special characters in regex that have specific meanings:

• * – matches zero or more occurrences of the preceding character
• + – matches one or more occurrences of the preceding character
• ? – matches zero or one occurrences of the preceding character
• {n} – matches exactly n occurrences of the preceding character
• {n,m} – matches a minimum of n and a maximum of m occurrences of the preceding character

These special characters can be combined with metacharacters and other characters to create complex patterns. My experience with Python’s regex capturing groups has been incredibly useful in extracting and manipulating specific parts of text data. Once you get the hang of it, you’ll find many ways to leverage these tools for your projects.

Python Regex Module

In this section, I will share my knowledge on importing the regex module and some useful common functions when working with Python regex capturing groups.

Importing the Module

Before I can use the regex module, I need to import it into my Python script. To do so, I simply add the following line of code at the beginning of my script:

import re

After importing the re module, you can start using regular expressions to perform various text searching and manipulation tasks.

Common Functions

The Python regex module has several helpful functions that make working with regular expressions easier. Some of the most commonly used functions include:

• re.compile(): Compiles a regular expression pattern into an object for later use. The pattern can then be applied to various texts using the object’s methods. Example:

pattern = re.compile(r'\d+')

• re.search(): Searches the given string for a match to the specified pattern. Returns a match object if a match is found, and None if no matches are found. Example:

result = re.search(pattern, "Hello 123 World!")

• re.findall(): Returns a list of all non-overlapping matches of the pattern in the target string. If no matches are found, an empty list is returned. Example:

result = re.findall(pattern, "My number is 555-1234, and my friend's number is 555-5678")

• re.finditer(): Returns an iterator containing match objects for all non-overlapping matches in the target string. Example:

result = re.finditer(pattern, "I have 3 cats, 2 dogs, and 1 turtle")

By using these functions, I can effectively search and manipulate text data using regular expressions. Python regex capturing groups make it even simpler to extract specific pieces of information from the text.

Capturing Groups

As I dive into Python regex, one concept that has consistently come up is capturing groups. These groups simplify the process of isolating parts of a matched string for further use. In this section, I’ll discuss creating capturing groups, referencing captured groups, and the concept of non-capturing groups. Let’s dive in!

Creating Capturing Groups

Creating a capturing group is as simple as encasing a part of a regular expression pattern in parentheses. For instance, if I have the pattern (\d+)-(\d+), there are two capturing groups: one for each set of digits.

You can see this in action using the Python regex library like this:

import re pattern = re.compile(r'(\d+)-(\d+)')

match = pattern.search('Product: 123-456')

Now, the match object contains two captured groups : one for '123' and another for '456'.

Referencing Captured Groups

After capturing groups, you might want to reference them for various operations. Using the group() method, you can obtain the values captured. You can access them by their index, where group(0) represents the entire matched string, and group(1), group(2), etc., correspond to the subsequent captured groups.

In my previous example, I can quickly access the captured groups like this:

first_group = match.group(1) # '123'

second_group = match.group(2) # '456'

Pretty straightforward, right?

Non-Capturing Groups

Sometimes, you want a group only for the regex pattern, without capturing its content. This can be achieved by using non-capturing groups. To create one, add ?: following the opening parenthesis: (?:...).

Here’s an example:

import re pattern = re.compile(r'(?:ID: )(\d+)')

match = pattern.search('User ID: 789')

In this case, the 'ID: ' portion is within a non-capturing group, and only the digits afterwards are captured. Now, if I reference the captured group, I only get the user ID:

user_id = match.group(1) # '789'

And there you have it! I hope this illustrates the basics of Python regex capturing groups, including creating captures, referencing them, and when to use non-capturing groups. Happy regex-ing!

Advanced Techniques

In this section, I will discuss some advanced techniques for working with capturing groups in Python regular expressions. These techniques, such as named capturing groups and conditional matching, can make your regex patterns more powerful and easier to read. Let’s dive in!

Named Capturing Groups

Named capturing groups allow you to assign a name to a specific capturing group. This makes your regex patterns more readable and easier to understand. In Python, you can define a named capturing group using the following syntax: (?P<name>...), where “name” is the desired name for the group, and “…” represents the pattern you want to capture.

For example, let’s say I want to extract dates with the format “MM/DD/YYYY“. Here’s how I can use named capturing groups:

import re pattern = r"(?P<month>\d\d)/(?P<day>\d\d)/(?P<year>\d\d\d\d)"

date_string = "12/25/2020"

match = re.search(pattern, date_string) if match: print('Month:', match.group('month')) print('Day:', match.group('day')) print('Year:', match.group('year'))

This will output:

Month: 12
Day: 25
Year: 2020

As you can see, using named capturing groups made our regex pattern more readable, and accessing the captured groups is much simpler.

Recommended: Named Capturing Groups Made Easy

Conditional Matching

Conditional matching in regex allows you to match different patterns based on the existence of specific capturing groups. In Python, you can use the following syntax for conditional matching: (?(id)yes|no), where “id” is the identifier for a capturing group, and “yes” and “no” are the patterns to match if the specified group exists, respectively.

For example, let’s say I want to find all occurrences of the word "color" or "colour" in a text. I can use conditional matching to achieve this:

import re pattern = r"col(ou)?r(?(1)u|o)r"

text = "I like the color red. My favourite colour is blue."

matches = re.findall(pattern, text) for match in matches: print(match[0])

This will output:

o
ou

Here, we used conditional matching to identify both the American and British spellings of "color/colour" and print the captured group responsible for the difference.

I hope you find these advanced techniques useful in your Python regex adventures. Good luck exploring even more regex possibilities!

Practical Examples

In this section, I’ll demonstrate a couple of practical examples using Python regex capturing groups, focusing on email validation and URL parsing.

Email Validation

Validating email addresses is a common task in many applications. Using capturing groups, I can create a regex pattern to match and validate email addresses. Let’s get started. First, here’s the regex pattern:

'^([a-zA-Z0-9._%+-]+)@([a-zA-Z0-9.-]+)\.([a-zA-Z]{2,})$'

In this pattern, I’ve used several capturing groups:

• The first group ([a-zA-Z0-9._%+-]+) captures the username part of the email address. It includes letters, numbers, and some special characters.
• The second group ([a-zA-Z0-9.-]+) captures the domain name, which consists of letters, numbers, and some special characters.
• The third group ([a-zA-Z]{2,}) captures the top-level domain, consisting of at least two letters.

Now, let’s use this regex pattern in a Python function to validate an email address:

import re def validate_email(email): pattern = r'^([a-zA-Z0-9._%+-]+)@([a-zA-Z0-9.-]+)\.([a-zA-Z]{2,})$' if re.match(pattern, email): # I match the input email against the pattern return True else: return False

URL Parsing

In this example, I’ll show you how to use capturing groups to parse and extract components from a URL. Let’s start with the regex pattern:

'^(https?)://([^\s/:]+)(:\d+)?(/)?(.*)?$'

In this pattern, I’ve used several capturing groups:

• The first group (https?) captures the protocol (http or https).
• The second group ([^\s/:]+) captures the domain name.
• The third group (:\d+)? captures the optional port number.
• The fourth group (/)? captures the optional slash after the domain and port.
• The fifth group (.*)? captures the remaining URL path, if any.

Now, let’s create a Python function to extract the components from a URL:

import re def parse_url(url): pattern = r'^(https?)://([^\s/:]+)(:\d+)?(/)?(.*)?$' match = re.match(pattern, url) # I match the input URL against the pattern if match: return { 'protocol': match.group(1), 'domain': match.group(2), 'port': match.group(3), 'slash': match.group(4), 'path': match.group(5) } else: return None

With this parse_url function, I can now extract and analyze various components of a URL.

I recently started a new website and to rank up in the search engines I hired an SEO expert. He suggested several activities out of which one was building lots of backlinks. We agreed on the budget and the timeframe in which he delivers 200 links in a month. These links are a mix of social bookmarks, profile pages, and guest articles. After he delivers the Links, I have to verify the work done and give feedback. The process of checking a URL containing the backlink is as follows –

• Open a URL from a list of URLs in the sheet
• Search for my “mydomain.com” in the source code or click on the visible link pointing to my domain
• In the sheet, mark it as LINK FOUND or LINK NOT FOUND

Only if it was this easy.

The link-building project will go on for several months. He will be building several hundred links so I must have some tool to do the checking for me. So, I came up with the idea of asking ChatGPT to give me a code in Python. Here is my prompt:

Prompt

Give me a Python code that scrapes all given URLs and looks for any link that contains "mydomain.com" and prints out the URL and the link it found

ChatGPT reply:

import requests

from bs4 import BeautifulSoup # list of URLs to scrape

urls = ['https://example.com', 'https://example.net', 'https://example.org'] # loop through each URL

for url in urls: # send a request to the URL response = requests.get(url) # parse the HTML content with BeautifulSoup soup = BeautifulSoup(response.content, 'html.parser') # find all links in the page links = soup.find_all('a') # loop through each link for link in links: # check if the link contains "mydomain.com" if 'partymap.in' in link.get('href', ''): # print the URL and link found print(f"{url} - {link.get('href')}")

Choosing Google Colab as Web-Based Python Interface

Now I have to run this code in a web interface. The interface should have an input box and an output box.

I find Google Collab to be the best option to do this in a quick, easy, and time-efficient manner. But I need the code to read the list of URLs from an input widget box. So I returned to ChatGPT and defined my environment along with many improvisations. There were several trials and errors.

Here are some of the prompts

Improvisation Prompt 1:

Also add the following Display domains that are duplicate
Display unique list of domains in which the string was not found

Improvisation Prompt 2:

I got this error ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)

Improvisation Prompt 3:

Check for Redirection, if the URL redirects, print "E:REDIRECTED" and skip iteration

Improvisation Prompt 4:

I got a mod_security error in request.get, how can I fix it

Improvisation Prompt 5:

Add a try catch block around request and beautiful soup

Improvisation Prompt 6:

If there are no Links found, print "E:ZERO LINKS" and skip iteration

Improvisation Prompt 7:

The list of URLs will come from a google collab input box can you make the change

And there were many more prompts to achieve the final results. But, since I am a Python coder, I could exit the back and forth with ChatGPT and change the code my way.

ERROR/STATUS CODES

Explanation of error codes is as follows

Errors found in URL given in the sheet

• UNRESOLVED – The URL in the sheet is malformed
• DUPLICATE DOMAIN – There are multiple URLs from the same domain
• REDIRECTED – The URL redirected to another URL, if this happens ask the SEO analyst to post the final URL in the sheet

Errors found in Links found in the source code of the URL

• FOUND – Our domain backlink was found
• NOT FOUND – Our domain backlink was not found
• BAD LINK – Our domain backlink was not found
• ZERO LINKS – No links were found in the source code

I begin each error code with ‘E:’ to easily identify them in sheet for conditional formatting process.

So here is the final code:

The Code

This goes in the first code cell of Google Colab

from IPython.display import display

import ipywidgets as widgets url_box = widgets.Textarea( placeholder='Enter URLs here', description='URLs:', layout=widgets.Layout(width='70%')

) # display the text box widget

display(url_box)

This goes in the second code cell of Google Colab

/enl

import requests

from bs4 import BeautifulSoup

from urllib.parse import urlparse # disable SSL certificate verification

requests.packages.urllib3.disable_warnings() # get the input URLs as a list

urls = url_box.value.split()

# create lists to store URLs and domains

scraped_urls = []

unique_domains = []

duplicate_domains = []

notfound_domains = []

inputstring = "" headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3'

} # loop through each URL

for url in urls: parsed_url = urlparse(url) domain = parsed_url.netloc

# add the domain to the list of unique domains if domain not in unique_domains: unique_domains.append(domain) else: # add the domain to the list of duplicate domains if domain not in duplicate_domains: duplicate_domains.append(domain) print("Duplicate domains:", len(duplicate_domains))

print(duplicate_domains)

print() # loop through each URL and check if the backlink exists

for url in urls: inputstring = "" parsed_url = urlparse(url) domain = parsed_url.netloc if not domain: print('E:UNRESOLVED',',',domain) continue if domain in duplicate_domains: print("E:DUPLICATE DOMAIN") continue # send a request to the URL try: response = requests.get(url, headers=headers, verify=False) except Exception as e: print('REQ:',str(e)) # check if the URL is redirecting to "mydomain.com" # check if the response is a redirect if hasattr(response, 'is_redirect') and response.is_redirect: print("E:REDIRECTED",',',domain) continue # parse the HTML content with BeautifulSoup try: soup = BeautifulSoup(response.content, 'html.parser') except Exception as e: print('BS:',str(e)) # find all links in the page links = soup.find_all('a') # print(links) #if no links found if len(links) == 0: print('E:ZERO LINKS',',',domain) continue # loop through each link for link in links: # Get the domain name from the link parsed_url = urlparse(link.get('href', '')) domain_name = parsed_url.netloc # print(domain_name) # domain_name = link.get('href', '') if domain_name: # Check if the domain name is "mydomain.com" if 'mydomain.com' in domain_name: # print(domain_name) inputstring = "FOUND" break else: inputstring = "E:NOT FOUND" # if domain not in notfound_domains: # notfound_domains.append(domain) else: inputstring = "E:BAD LINK" # add the URL to the list of scraped URLs # scraped_urls.append(inputstring) print(inputstring,',',domain)

See the CELL setup in the image. Press play in the first cell. You will get a URL input box. Paste your URLs in it.

Input Box:

https://sketchfab.tld/mydomain https://30seconds.tld/mydomain/
https://speakerdeck.tld/mydomainus
https://www.ted.tld/profiles/<some page>/about
https://dzone.tld/users/mydomainindia.html
https://www.reddit.tld/user/mydomainusa
https://medium.tld/@mydomainusa/about

https://www.intensedebate.tld/people/mydomainusa
https://www.growkudos.tld/profile/<some page>
https://www.universe.tld/users/<some page>
https://www.dostally.tld/post/<some page>
https://www.socialbookmarkzone.info/<some page>
https://app.raindrop.io/my/-1/item/<somepage>/web
https://www.tamaiaz.tld/posts/<somepage>
https://www.socialbookmarkzone.info/<some page>/
https://gab.tld/mydomain/posts/<some page>

Now press Play in the second cell and watch output panel

Output:

Duplicate domains: 5
['www.socialbookmarkzone.tld, 'www.reddit.tld', 'www.instapaper.tld', 'www.wibki.tld', 'diigo.tld'] FOUND , sketchfab.tld
E:BAD LINK , 30seconds.tld
FOUND , speakerdeck.tld
E:BAD LINK , www.ted.tld
FOUND , dzone.tld
E:DUPLICATE DOMAIN
FOUND , medium.tld
FOUND , www.pinterest.tld
FOUND , www.intensedebate.tld
FOUND , www.growkudos.tld
E:ZERO LINKS , www.universe.tld
FOUND , www.dostally.tld
E:DUPLICATE DOMAIN
E:ZERO LINKS , app.raindrop.io
FOUND , www.tamaiaz.tld
E:DUPLICATE DOMAIN
E:NOT FOUND , gab.tld

INPUT BOX CODE [GOOGLE COLLAB]

GOOGLE COLLAB CODE CELL SETUP

PASTE THE OUTPUT IN YOUR SEO TRACKER SHEET in the same line as the URLs & APPLY SPLIT TEXT TO COLUMN

STEPS TO APPLY CONDITIONAL FORMATTING

FINAL OUTPUT

Based on the above output the SEO analyst can rework on the links or drop these sites completely.

If you like the code leave a comment and I am available on Upwork for Prompt Engineering, AI Art jobs. I use ChatGPT, Midjourney, Python and many more tools for my client jobs.

My Upwork profile is https://www.upwork.com/freelancers/~018645334d3b757e4d

---

Recommended: 7 Effective Prompting Tricks for ChatGPT