Keyword Research

The Advanced
Guide to SEO

Chapter 06

By Neil Patel
and
Sujan Patel

You've all used Google AdWords keyword tool before right? We're not going to cover that here though. We're going to go beyond the common keyword research tools and explore some less talked about ways to find the best keywords to target for your site.

Beyond Google AdWords Keyword Research

Many people get keyword data from AdWords. But there's other sources of keywords and search volume. We're going to walk through a few of them now!

  1. Bing

    The newer Bing keyword research tool, although still in development, is a good place to check.

    Sign in (or create an account)

    Beyond Google AdWords Keyword Research #1

    Start with the basic settings. Bing's keyword tool is a little different than Google's. If set to 30 days, it give you the exact amount of search from those 30 days.

    Beyond Google AdWords Keyword Research #2 Beyond Google AdWords Keyword Research #3

    You can export the data

    Beyond Google AdWords Keyword Research #4

    Use the benefit of exact search numbers to judge long tail:

    Beyond Google AdWords Keyword Research #5

    You can bring in Google AdWords data for a comparison of search volume metrics:

    Beyond Google AdWords Keyword Research #6

    Now, let's get a little more refined. Set the language to English (assuming that's your target language) — and expand the date range to 6 months. This is going to give you a large data set to make good decisions:

    Beyond Google AdWords Keyword Research #7

    Set to "strict mode" for basically Bing's version of "exact match"

    Beyond Google AdWords Keyword Research #8
  2. Wordtracker and SEO Book's Keyword Research Tools

    These versions are free. You can also register and pay for the pro versions.

    Beyond Google AdWords Keyword Research #9

    Run the report in SEO Book to export CSV's

    Beyond Google AdWords Keyword Research #10

    Use the overall daily estimates for measure potential traffic across the entire web:

    Beyond Google AdWords Keyword Research #11
  3. Wordpot

    Wordpot is another source of keywords you may not have thought of.

    Beyond Google AdWords Keyword Research #12

    It also shows definitions, synonyms, related words and associated words

  4. YouTube Keyword Tools

    Of course if you're doing optimization for YouTube you'll be interested in the search volume — but if not, this is still a fantastic tool very generating more keyword ideas.

    Beyond Google AdWords Keyword Research #13 Beyond Google AdWords Keyword Research #14

    Go to the bottom, click "add all 100" and then click export to CSV or the file of your choice

    Beyond Google AdWords Keyword Research #15-16

    Also — let's say you want to rank in youtube for "search marketing".

    1. Search for the term "search marketing" in YouTube

    2. Find the top ranking video

    3. Enter that video in the keyword tool — this give you the top keywords for that video

      Beyond Google AdWords Keyword Research #17

      You can add these to the existing report or start over

      Beyond Google AdWords Keyword Research #18
    4. Local Marketing Source — Local Keyword Tool

      This is a handy tool for generating local keyword variations. It doesn't give you search volumes, just a list of keywords.

      Start with a zip code, radius and keyword(s)

      Beyond Google AdWords Keyword Research #19

      Generate your list and export to CSV

Use Search Suggest Scrapers For Fast Keyword Ideas

The web collects data every day. Data that people are actually typing — they are telling us what they are looking for.

Search suggestions are a great place to look for words people are actually typing. This can often bring immediate content ideas as well as keyword prospects. The important part in this process is collecting as many ideas as possible.

Many SEO's out there miss some of the most useful tools. Let's walk through some now!

First, a list of suggest scrapers

All are great (there's a lot out there on ubersuggest) but I'll show you a few tricks with Soovle and KeywordBuzz

Soovle

There's two ways we're going to use Soovle. First, we're going to use it as a general suggest scraper.

  1. Set it is 15 engines. We want as many keywords as possible!

    Use Search Suggest Scrapes For Fast Keyword Ideas #1
  2. Start typing the root keyword surrounding the topic you're researching

    Use Search Suggest Scrapes For Fast Keyword Ideas #2
  3. Use Scraper for Chrome to Key Them in a Google Doc

    1. Who wants to cut and paste all of those words? We can get them in a Google Doc with a small tweaking to the Scraper xPath

      Use Search Suggest Scrapes For Fast Keyword Ideas #3
    2. You'll see the default results don't include every keyword.

      Use Search Suggest Scrapes For Fast Keyword Ideas #4
    3. Eliminate the [4]

      Use Search Suggest Scrapes For Fast Keyword Ideas #5
    4. Scrape again (or hit enter)

    5. Export the results to Google Docs.

      Use Search Suggest Scrapes For Fast Keyword Ideas #6
  4. Filter and sort the raw data

    1. You'll notice there's a little extra stuff in there to clean up. There's some data from others divs and duplicate keywords.

      Use Search Suggest Scrapes For Fast Keyword Ideas #7
    2. Delete the extra two rows at the bottom

      Use Search Suggest Scrapes For Fast Keyword Ideas #8
    3. Sort by keyword alphabetically

      Use Search Suggest Scrapes For Fast Keyword Ideas #9
    4. Now we need to filter duplicates! In cell B2 put this formula;

      =if(A2=A3,true,false)

      This will put true if the items are the same and false if they are different.

      Use Search Suggest Scrapes For Fast Keyword Ideas #10
    5. Then hover over the corner (wait for the cross to appear) and double click.

      Use Search Suggest Scrapes For Fast Keyword Ideas #11

      This marks every items as a duplicate or not.

      Before we do the next step, copy the true/false results and paste as values onto its self. This won't visually change the true/false results - but will remove the formulas and leave just the words.

      Use Search Suggest Scrapes For Fast Keyword Ideas #12
    6. Sort descending by column B

      Use Search Suggest Scrapes For Fast Keyword Ideas #13

      This should put all the TRUE's at the top. These are your duplicates.

    7. Delete all the "TRUE" rows.

      Use Search Suggest Scrapes For Fast Keyword Ideas #14

    You're left with about 100 of unique and diverse keywords!

Soovle To Catch Trends

Next, we'll use Soovle to capture trending keywords if you want to create quickly and catch a trend.

  1. Click on the Star

    This will bring you to the trending words for the day.

    Use Search Suggest Scrapes For Fast Keyword Ideas #15
  2. Do a control-f in your browser. Search for words surrounding your niche

    Use Search Suggest Scrapes For Fast Keyword Ideas #16
  3. Then — enter THAT word back into the suggest scraper.

    Use Search Suggest Scrapes For Fast Keyword Ideas #17

KeywordBuzz

I just want to show you one thing with keywordbuzz.net — I don't know of any other scrapers to do this.

  1. First — arrive at the search screen and enter your keyword.

    Use Search Suggest Scrapes For Fast Keyword Ideas #18
  2. Then — not only can you download as a text file. You can instantly check domain availability for .com .net and .org

    Use Search Suggest Scrapes For Fast Keyword Ideas #19
  3. Just click 'check' — and you'll instantly have some great niche domain name ideas.

    Use Search Suggest Scrapes For Fast Keyword Ideas #20
  4. You could even get ideas to latch onto a brand's traffic with an auxiliary tutorial or informational site.

    Use Search Suggest Scrapes For Fast Keyword Ideas #21

Use Google Correlate

Google correlate is another not often talked about tool for keyword research, especially seasonal trends. http://www.google.com/trends/correlate/

Here's an example — if you enter 'red wine' this is the result:

Use Google Correlate #1

You can see it gives us keywords where search volume across time has a strong correlation. This is a useful tool for exploring trends and coming up with some outside of the box keyword ideas — let's explore a few!

Additional Topic Ideas

Let's say you're a food blogger — and you like to do an article on red wine every once and awhile. Wouldn't you like to know some other topics you could attach to red wine?

The correlate result for red wine is:

Use Google Correlate #2

You can see that a lot of people are searching for cheese related things around the same time they are searching for wine. This is somewhat expected, but there are many targeted keyword ideas you may not have thought of. And you know these trend right along with red wine - so you could time your content to go with the seasonal trends a bit better too.

You can show more words and also do a CSV export

Use Google Correlate #3

Ideas By Location

The location feature in Google Correlate is great for targeting an audience in a particular location, with a parallel interest. This works great with seasonal items, or interests that fluctuate in search patterns through the year.

For example, let's say you're an eCommerce shop that sells grills etc. You might be interested in people searching for "outdoor party". You know this search pattern matches the seasonality and interests of your customers.

An initial correlation shows:

Use Google Correlate #4

This shows similar search patterns for "poison oak" and "boat registration". But let's get more targeted and get correlations for the location.

Use Google Correlate #5

And the results are:

Use Google Correlate #6

Here you have people in the states of Connecticut and South Carolina interested in an Antigua all inclusive vacation.

So what do you do?

You do a vacation giveaway to the Antiguas for your customers from those states at the time of year people begin having outdoor parties (and buying grills).

Tracking Internal Site Search

Many site owners often overlook the most obvious place to get keywords. Your own website. I'm not just referring to keywords people use to arrive at your site. Rather, searches users are making within your site.

This especially works if you have a larger site — ecommerce or a large editorial site. You will have a lot of data to work with and many opportunities to learn about how people are using your site and keyword ideas you'd never think of otherwise.

I'm assuming you already have a search function in your site. I'll show you how to set it up in analytics and use the data effectively.

Check How Your Site Does a Site Search

You're going to need the URL parameter your site uses to track site search.

For example, the site I'm demonstrating this on, looks like the following;

Tracking Internal Site Search #1

See the search paameter?

/search.php?search=search+example

where "search example" is the keyword typed.

The parameter is "search" (hint: if in php its after the question mark)

Set It Up in Analytics

Head over to your analytics profile.

Go to the Admin area. Click on Profile settings

Tracking Internal Site Search #2

In profile settings, you're going to scroll down to "Site Search Settings"

Type the query parameter we found in the first step in the field. Also, select to strip query parameters from the URL.

How To Use The Data

I'm going to show you how to best access this data for keyword research. We're going to do this within analytics but you can also export the data for further use.

First - View the site search keywords

Tracking Internal Site Search #3

If you've never done this before, its best to zoom out at least six months.

Tracking Internal Site Search #4

We're going to apply a few different filters to the data. (You could do this with advanced segments too for a more "permanent" filter).

This first filter says "show me words that get searched more often but then most people immediately leave my site. (You may have to play with the numbers specific to your metrics). These are the words you want to build new content around.

Tracking Internal Site Search #5

And may get something like this;

Tracking Internal Site Search #6

OR you can do sort of the opposite. Say "show me words that get searched a lot but where people stay on the site. Again, the exact number will depend on the behavior of people on your site.

Tracking Internal Site Search #7 Tracking Internal Site Search #8

Take Action

Either way, you'll have a list of keywords you can take immediate action on.

  • Highly searched words where people leave = create the content!
  • Highly searched words where people stay = keyword research and optimize around those words.

Scraping Search Suggest From the Command Line

Extract Initial JS File

Google runs a script when computing its search suggest words. You can use the following URL in your browser, and in turn generate a JS file.

Paste the following into your browser.

http://www.google.com/s?sugexp=pfwl&cp=15&q=<search term here>

Replace <search term here> with your keyword. Let's do marketing.

http://www.google.com/s?sugexp=pfwl&cp=15&q=marketing

Paste in your browser

Scraping Search Suggest From the Command Line #1

It will automatically create and download a JS file

Scraping Search Suggest From the Command Line #2

Open it and you'll see your raw suggest terms

Scraping Search Suggest From the Command Line #3

That's not very much practical use, so there's more!

Create an .sh File

Simply copy and paste this code into your text editor

#!/bin/bash
q=$(echo "$1" | sed 's/ /%20/g')
curl -s "http://www.google.com/s?sugexp=pfwl&cp=15&q=$q" | sed 
's/\[/\n\[/g' | cut -d'"' -f2 | tail -n +4
Scraping Search Suggest From the Command Line #4

Save it as an .sh file. You can simply name it suggest.sh

Run The Code In Terminal (Mac)

Run the following command in terminal

./suggest.sh "marketing"

Which will produce the follow results;

  • marketing jobs in wa
  • marketing solutions
  • marketing plan
  • marketing solutions wa
  • marketing jobs
  • marketing strategies
  • marketing drive
  • marketing mix
  • marketing internships

(note that some results may be localized).

Running Your Own Keyword Frequency Analysis

There's many tools available out there to run a keyword frequency analysis. Some are free, some are paid. But there may be reason and benefit to having the control to run your own reports. I'll show you one way to do this!

Create your .sh Code

Start by cutting and pasting this into your test editor;

#!/bin/bash

input=$1

IFS=$'\n'
# for keyword files...
if [ -e "$input" ]; then
	cat "$input" | sed 's/ /\n/g' >> tmp
	for word in $(cat "$input" | sed 's/ /\n/g' | sort | uniq); do
		count=$(grep -c "^$word\$" tmp)
		echo -e "$count\t$word" >> tmp_o
	done
	cat tmp_o | sort -hr
	rm tmp tmp_o
# for web pages...
elif [[ "$input" =~ "http://" ]]; then
	# check if links is installed. found solution at 
	http://stackoverflow.com/questions/592620/check-if-a-program-exists-
	from-a-bash-script
	type links >/dev/null 2>&1 || { echo >&2 "This script requires 
	'links' but it's not installed.  Aborting."; exit 1; }
	links -dump "$input" | sed -e 's/ /\n/g;s/\(.*\)/\L\1/' | sed 's/\W//g' >> tmp
	for word in $(cat tmp | sort | uniq); do
		count=$(grep -c "^$word\$" tmp)
		echo -e "$count\t$word" >> tmp_o
	done
	cat tmp_o | sort -hr
	rm tmp tmp_o
fi

Save this as a file — anything will work, such as keyword.report.sh

Run the Script

Open up Terminal in your Mac

You can run the script in two ways — either by file or by URL.

  • By File

    bash keyword.report.sh keyword.file.txt

    'keyword.report.sh' is the name of your .sh file and 'keyword.file.txt' is the name of your file with the content in it you're analyzing.

  • By URL

    bash keyword.report.sh http://quicksprout.com

    Again, 'keyword.report.sh' is the .sh filename and then you just put the URL! Easy!

I bet you have a killer list of keywords and data now! Now your site is fast, crawlable, semantically correct, and you've got the best keywords. Time to target and create content that will attract links while you sleep.

Well done! You made it through chapter six! Are you ready for chapter seven:
Link Building with Content?