Advanced Search and Replace With RegEx

Dan Beeston talks regular expressions, explains what they are and demonstrates -- with practical examples -- why they are useful and how powerful they can be.

---

Closed Caption:

today it's Dan based in here for Mac
tut's plus
and today we can learn how to do search
and replace using regular expressions
now most everyone knows how to do a
simple search in our place on text
here we have a missive from the
department of government affairs
you can see that we've abbreviated
department he it but we want to change
that throughout the entire document
so that's a simple as finding and
replacing
the abbreviation with the entire word
do a search in our place and every
instance a bed abbreviation
gets changed but in this case all the
government departments being renamed
throughout the entire structure the
department government is being turned
into the
government department the Department of
Defense is being turned into the Defense
Department
it wastes a lot of time but at least it
keeps the business card printers in the
money
now account to a simple search in our
place here so we can have to use
regular expressions now I'm using
Sublime Text
to you to do my search and replace see a
here we can see the regular expression
button
a number of text editors provide this
facility
but while we're learning I'd like to use
a program that makes it a little bit
easy to see
exactly what's going on so we gonna cut
that text
and we're gonna download a program cold
rage
exe you can download rejects from
G Skinner dot com slash reg exe
slash desktop you can use the web
version if you like
I prefer the desktop version he's the
desktop version
we get a price to information in there
and now we can play with it to our
hearts content
now the first thing we want to do
is we wanna find every instance that
department all love
and then we will find the word that
follows it and then we want to take that
word and
add the word Department after it here
I'll show you here we have
the phrase the Department of Defense
it's made up of basic text
out dynamic word and the end of that
word
what we want to do is remove
Department all keep how dynamic word
and then add Department at the end of it
so if we open up rejects a we can type
in
Department all spice and you can see
that it's detected
Department all in every instance then
what we want to do
is we wanna grab and alphanumeric
character we do that by hitting
backslash W forward a word character
and he'll see that is detected the first
alphanumeric character
after Department golf
now what we wanna do is we want to get
as many
alphanumeric characters until they run
out so
we had a plus sign he'll see it's
detected
the entirety of the word following
Department all now this way is the thing
that we want to keep
so what we gonna do is get iraq that in
a token
that token is assigned to the value
dollar sign one
so if we wrap thats
in a token and then we replace it
with dollar sign one we can see that
every instance have Department
all has been replaced
see you could see the department
Government has been replaced with simply
government the Department of Defense has
been replaced with defense
the department agriculture has been
replaced with agriculture
so he just put Department after its
and the ego Department government has
been replaced with government department
the Department of Defense with Defense
Department
that works great except here we see that
apartment
agriculture and farming and the
Department of Wildlife and Fisheries
have both been muddled up so it's now
the Agriculture Department and farming
that doesn't make any sense now how do
we select the
and farming in this method
well we could put spice ampersand space
farming that selects all farming man
but not fisheries so of course we change
that to
a wood backslash W plus
and that selects the word San out
selecting agriculture and farming
and Wildlife and Fisheries but it's no
longer selecting
one word wouldn't it be great if we
could allocate everything after the
first word
to be optional well we can
here we see the phrase the Department
agriculture and farming
that simple text that is out this
dynamic word
and that is the text that comes
afterwards
what we're gonna do his work in Iraq bat
element
inside a token but we're gonna add a
question mark that makes an
optional token then we can wrap the
whole thing
inside another token
this is out optional token we put a
question mark after it
main we rap both words
and the ampersand inside a token
naked see that department have
government doesn't have an ampersand
and it's a place with government
department the Department of Defense is
replaced with defense department
the department agriculture and farming
optionally has an ampersand
and a second word and so the entire
thing
goes into the token and is replaced with
agriculture and farming Department
our regular expression he looks like
complete gibberish
but we know it's simply a breakdown
above normal text
a dynamic word inside a token plus an
optional element
of normal text and a second dynamic word
let's grab that regular expression take
it to Sublime Text 2
paced game at out token
dollar one plus Department
and replace old and maybe go
welcome to the government department
Defense Department
even the agriculture and farming
Department
let's move on
here we have some text from an open
source book by Eric Raymond
the weather the book was originally for
matted makes every double quote
a succession to apostrophe characters
but I don't wanna print it like that
what I want to do is on a replace
all those pays apostrophes with a double
quite mock
and that's easy to do simply finding
your place
to apostrophes with double quotes
replace all a nadie go but this simple
generic
quotation marks wouldn't it be great to
change the opening quotation mark
to the special character
left quotation mock
and the right one to write quotation
marks
does not look much nicer but I don't
wanna go through and change every single
one by hand
and I don't have to be because got
regular expressions
at my fingertips
okay with back without double apostrophe
marks let's take all this text
and move it into rage exe
okay let's ever think about exactly what
we're looking for
we wanna find to apostrophes
the text in the center that we want to
put inside a token
and the final set of apostrophes
well that shouldn't be too hard let's
get a Reg exe
we're gonna put two apostrophes and then
inside a token
we're gonna look for a word as may pat
is
I'll but what we've discovered is we're
only selecting a word
up until the spice and we want to select
all the way
to the double quotes at the end
so we're looking for either a word
or a spice well we want to define
character set so will make square
brackets on
either side about word character I me
wanna say
either getaway character or
get a space and then it'll select
to apostrophes and then either awake
character
or spice as many as possible of them
then two more pasta fees at the end and
we've selected out quotation
but the problem is that down HEA we've
got a quotation that has a question mark
in it
so we can add a question mark and that
selects that one
and now we see dan here at the open
source movement there's a hyphen
so we need to at a hyphen as well
and we just can't get you in check
everything in the document
we need a better plane to figure out how
we're going to define
out quote so let's get rid everything in
token
having our place all that with the
simple full stop which means
any character at all
then hit plus as many as possible
then outside the token grabbing out
double apostrophes
now the issue here is that we found a
double apostrophe
and then we found any characters up into
the double apostrophes but it's found
those double apostrophes
and kept going and then found a second
set of apostrophes
and kept going and found the final set
up apostrophes
if we put any info here at the end it
wouldn't selected
it's making the biggest selection at
kean between the front
apostrophes and the end apostrophes
it's been greedy now we don't want to be
greedy
we're gonna force it to be like easy so
we're gonna put a question mark Bay
so that will look for apostrophes it'll
8 as much as they can
but then as soon as I possibly can. it's
gonna stop at those apostrophes
hey that's looking pretty good isn't it
its elected the first quite
its elected the second quote it selected
the third quote here
but down at the fourth quite it's gotten
confused it seems that the original
document has
open quotes for these are the people but
it hasn't closed those quotes
its mistake in the country lani on at
text to beat mistake-free
now the problem here is that if I put
more quotes
in we can see that our open quotes an
app closed quotes
gonna be back to front we need to
understand that sometimes
out text is gonna have mistakes we need
to have a think about what patten's
exist within at quotations
that we can always rely on happening
well
let's have a look at our quotations each
one
Stutz with the spice before the
quotation marks
soggy if we put in spice character
and then to apostrophes we can see that
it's only selected
the opening quotes each element
except for that very first one which
doesn't have a spice
it has the beginning at the line so
let's replace that spice
with backslash s which means space bit
also includes any white-space character
which includes tabs all line breaks
being
we open up out token
we go for any character as many as
possible
make it lazy and then put a final
apostrophes on me
this allows us to select out quotes
and then replace it with an hour
left quotation mark out token
an hour right quotation marks so let's
working
right its selecting all the quotes that
we want
but down here at stripping out but space
that we want
before the quotation marks
now we could replace that with the spice
that opens up
every instance hiya
so we're taking away the space hiya and
replacing it here
but Dane it replaces the spice at
beginning at lines
as well what we really want to do is
select that first set
double apostrophes but only if it's got
a space in front of it
but we don't really wanna take that
spice as well
what we can do today is used a look
behind
now a look behind is a special taken
it's not set
to string1 all string2 what asks is
height he is question mark
left arrow before law this element is
there
a spice
so let's put that into ray Jackson
will put special taken around me and
will say
is left arrow
there is space
before these double quotes it only finds
double apostrophes
if there is a white space character
preceding them
but it doesn't actually select them so
we can get rid of that space
they and now all about quotations
are correct her memory cause that we
have that
stake in dropped double quotes
here in the text but we can do with that
in a moment
let's take that regular expression
do how search and replace using
regular expressions it's found the mall
then it's replacing them with left
double quotes the token and right double
quotes
let's replace all and now what we can do
is we can do a find
and we can find any leftover double
quotes here we find
leftover double quotes at the end of
hackers
and we can see that we've gotta write
quite the beginning which is not what we
want
so let's move that to where it's
supposed to be
and let's fix that by hand which we
would have had to do
anyway because it was our mistake
initially
apparently the formatting at this
document didn't allow for a lip sees
so the writer just typed in 3 periods
and in fact down HEA that put in four
periods
what we want to do is replace those with
the ellipsis character
now we can do a search and replace some
three dots but we'd meese
any instance where there's been a
mistake made four dots
what two dots we want to be consistent
throughout the document
so
we're gonna get back into rejects a
we gonna do a search for periods now
simple period socking away because we
already know I with that
he's a regular expression 10 meaning any
character
so we put backslash period
and it selects all the periods in a
document but what we want to do is
select
every instance way there are three
periods
now that's a bit verbose and it's only
selecting sets a
three periods and not for periods
what we'll do is will set some curly
braces
and will say we wanna find sets have
three periods
now wherever there are three dots it
will find them
and now what we can do is we can replace
them
with an ellipsis
so it's replacing it hiya and down HEA
but it's leaving that extra period on me
and
so we want to take care this and say if
it's between two dots
and four dots we know that that's
probably going to be an ellipsis
and its gonna replace those with an
ellipsis
but the problem is if we keep talking
dots it finds groups have four as the
maximum
and replaces all them with ellipse sees
what we want to do is replace all those
dots
with a single ellipsis character so
will eat the last one of which means two
dots
until as many as possible so it's found
all have those dots
and replace it with one ellipsis so
let's take that
regular expression into Sublime Text
do a search in our place
replace it with how ellipsis
and replace all doesn't look much better
that's gonna print beautifully okay so
that's doing such a place in
text but what if I wanted to do a search
in her place in a list of files
here we have forklift this is a great
FTP program but it also allows for
number
really interesting file manipulation
tools
and here we have a directory where I've
put some photographs
be to some photos that I took at Fiji
you can see that I took them on
different cameras
they have different numbering
conventions and different naming
conventions
you can see that a also taking out some
images that I didn't like
and left only the ones that I do what I
want to do is that I want to rename them
so that they're all the same
set them up so that they numbered
correctly and I wanna order them by the
date that they were created
now first things first if you rename
files
you cannot undo them so let's make a
backup
there's a backup nice insight now
welcome to mess up fr names if we want
we can always go back to a backup
select open that up where now Fiji pics
directory
first thing we want to do is we won
order them by the date that they were
created
currently we can order them by name this
size
when they were modified let's go to our
view
show our view options and add date
created:
now we can click a mass and order it's
by the date that it was created
now let's select the mall and he enter
and add a sequence to them we can see
but it's applying it to the name
and its adding a sequential number see
can see it says one
to 34 but saying it after the rest
of the name so that's but a perfect
sunny and right
nom and we can see in the Live View
that separates those numbers from an new
sequence 23 456 up to 14
was gonna put a bit depending on the
number three digits so it goes
001 up to zero forcing that's gonna be
helpful same
so gonna rename the mall
select them all again and now we're
going to use our regular expression
where applying it just to the name not
to the extension
so let's have a look at fr names each
one has
a string of alphanumeric characters
men in underscore been some digits
then the characters in %um their numbers
that we want to keep
and put inside a token so let's open
this in rejects a and have a closer look
at it
what we're looking for in our list he's
San alphanumeric characters slash W
forward then multiple love them
then an underscore we selected the first
P then we're looking for digits
other set of three digits are set up
four digits so let's put
slash def digit and then in brackets
three digits of four digits
an input nom
and then inside a token
as many digits as we can
and then we gonna replace it with
Fiji trip under school
and then the token then you can see Fiji
trip
underscore and then the token so let's
copy that regular expression
open that info clift
and under replace regular expression put
our regular expression in
and replace it with Fiji
trip underscore token
and he see all those files there in the
created order
and then numbered 12 14 using the same
file name:
and because that's in a token we can
move that
to the beginning the file name or even
in the center the farming
you can see just how powerful these
tools can be
you can really move elements a text
around like magic
let's change that back and rename out
files
and the ego Fiji trip from 1 to 14
now hopefully this is giving you an idea
for what situations regular expressions
can be used for
there are a whole bunch about the
regular expressions that we haven't even
touched on
if you're interested if your curiosity
has been piqued
I highly recommend the net touts
introducing regular expressions
up and running by Jeffrey Way this
course
is fantastic and well worth signing up
to touch plus
I've seen great program is turn and run
in the face regular expressions
they look like gibberish when you first
see them but now it's exactly have a
break apart in today components
and how easy it is to understand them
once you know what you're looking for
as always if you have any questions
please do ask them in the comments
this is Dan based in for Mac touts plus