Windows PowerShell

The Case for Regular Expressions

Don Jones

A Regular Puzzle

Working with regular expressions can be a lot like solving a puzzle. Just as if you were staring at one of those three-dimensional posters, squinting helps. You have to view the information in the form of a pattern, rather than as individual characters. Squinting can help blur the characters so you can focus on the larger pattern. Consider these four hyperlinks: https://concentratedtech.com. Click here to visit.

<a name="data">Data Sheet</a>

<a target="_blank" href="https://microsoft.com">Microsoft</a>

<a href="search.aspx" target="_top">Search</a>

I want to capture links that have these common elements:

• They all start with <a
• They all end with </a>
• They all contain href=" somewhere after the <a and before the >

I don’t want to capture a link that doesn’t contain all of these elements. Ignoring the bits I don’t care about, and squinting, the links look like this:

<a_href=”xxxxxxxxx">xxxxxxx</a>

<a_xxxxxxxx>xxxxxxx</a>

<a_xxxxxxxxxx_href="xxxxxxxxx">xxxxxxxx</a>

<a_href="xxxxxxxxx"_xxxxxxxxxx>xxxxxxxx</a>

Notice that I’ve replaced the space character with an underscore just to make it stand out a bit more, and replaced the stuff I don’t care about with “x.” Suddenly, these start to look a lot more similar, and a pattern emerges.

A Regular Pattern

Patterns are the whole point of regular expressions. Using the regular expression language, you describe the pattern of text for which you’re looking. You can get a pretty comprehensive description of that language by running Help about_regular_expressions in Windows PowerShell.

In my case, my regular expression might look something like this:

(<a\s.{0,}?href=".+?".{0,}?>.+?</a>)

I know—it’s crazy. Getting it right took me about an hour, with a huge amount of help from the Web site RegExTester.com. Let me break it down:

• The (parentheses) define a pattern for a single hyperlink, and tell the shell to capture the matches (more on this later).
• The <a is a literal match; the shell will look for those two characters.
• \s means to match a single whitespace character, like a space, which always should follow <a in HTML.
• .{0,}? means I want to see zero or more characters of any kind. The period means “any character,” and the {0,} means zero or more. The trailing question mark is special. It makes the match non-greedy (more on that in a second).
• Next, I want to see the literal characters href="
• Next, I want to see one or more of any character. The plus sign means “one or more,” and, once again, a trailing question mark makes it non-greedy.
• Following that, I want to see a closing quotation mark.
• Then I want to see zero or more characters and the closing >.
• Lastly, I want to see one or more characters before the </a>. Here, too, the question mark after the plus sign makes this a non-greedy match.

This non-greedy business is tricky. Let’s say I have the following text string in my HTML page:

This is a <a href="test">link</a> and this is a <a href="test">link</a> but this is not.

And let’s say I’m using this regular expression, which does not use the question marks to create non-greedy matches:

(<a\s.{0,}href=".+".{0,}>.+</a>)

The shell will match the first <a and space, and then look for zero or more characters—ending only when it finds the last </a>. I’ll use boldface to show what it matches:

This is a <a href="test">link</a> and this is a <a href="test">link</a> but this is not.

That’s because the first .{0,} match is greedy: The shell consumes as many characters as possible while making the regular expression work. By making it non-greedy, I tell it to consume as few characters as possible while making the regular expression work:

This is a <a href="test">link</a> and this is a <a href="test">link</a> but this is not.

A Regular Solution

To put this to work in the shell, I started by defining a test variable and filling it with HTML. I then used the -match operator to match the test HTML against my regular expression.

PS C:\> $html = 'This is a <a href="test">test</a> but <a name="anchor">This</a> is not and <a target="_top" href="something">definitely is</a> a link.'

PS C:\> $html -match '(<a\s.{0,}?href=".+?".{0,}?>.+?</a>)'

True

The “True” result simply tells me that the shell found one or more matches; it doesn’t tell me what it matched. However, after using the -match operator, the shell automatically creates an array, called $matches. This array contains all of the captured matches:

PS C:\> $matches

I can access individual matches by using normal array syntax:

PS C:\> $matches[0]

<a href="test">test</a>

I can easily send those to a file, too:

PS C:\> $matches | out-file c:\matches.txt

Once you’ve mastered the syntax, as bizarre as it is, regular expressions provide a powerful and valuable way of matching even complex, variable patterns in a large body of text. You can extract the text that matches your pattern and work with it independently of the main body of text. This is especially useful for parsing log files, HTML files or any other kind of semi-structured text.

Don Jones is a contributing editor for TechNet Magazine*, and publishes Windows PowerShell tips and news at www.ConcentratedTech.com. He is the co-author of “Windows PowerShell: TFM” (Sapien Press, 2009), now in its third edition and covering Windows PowerShell version 2.*

Related Content

• Windows PowerShell: Filter Left, Format Right
• Windows PowerShell: Stay Seated!
• Windows PowerShell: What Not to Script