The Complete Guide to Keyword Research For SEO

All content campaigns begin with the same thing:

keyword research.

And there’s a reason for this. Targeting keywords will lead to long-term and, possibly, short-term organic search traffic for your website.

The bad news is that 80% of SEOs and marketers do keyword research wrong.

They plug in a main keyword into Google’s Keyword Planner, download the results, and then start sorting through them in a spreadsheet.

But here’s the thing: literally thousands of other marketers and SEOs have already searched that exact same keyword.

Unless you’re very lucky, you won’t find anything resembling a low-competition keyword to target.

Now, if you have a site with the authority of Quick Sprout, that’s not a big deal. But if you’re working with a less authoritative domain, or a brand new one, you need to find realistic keywords to rank for.

The good news is that the keyword planner only shows a very small portion of keywords you can target.

If you can find “hidden” keywords that have a decent search volume, but less competition, your organic search traffic will grow rapidly.

I’m going to walk you through 5 modern keyword research methods that you can use to uncover keywords that haven’t been targeted by hundreds or thousands of your competitors.

You’ve all used Google AdWords keyword tool before right? We’re not going to cover that here though. We’re going to go beyond the common keyword research tools and explore some less talked about ways to find the best keywords to target for your site.

Understand what keyword research really is

Keyword research is an art, not a science.

While there are some good tools out there, they should be the start of your research, not the end.

Most SEOs use keyword research to figure out how much search traffic they could potentially attract for free.

Experienced SEOs use keyword research to understand their readers.

Keywords, and their search volumes, tell you a few things:

  1. What your audience’s main problems are – if 5,000 people are searching for a particular keyword, it shows that it’s a common and important problem in your niche. Create your content around the most popular problems.
  2. The language they use – people describe problems in different ways. If you can understand which words your reader uses to describe problems, you can write more compelling content.
  3. Which topics are becoming more important or less important over time – as niches evolve, keywords gain and lose popularity. Good keyword research will help you identify which searches are becoming more common so that you can create more content around those.

The methods in this post aren’t meant to help you find basic keyword phrases like “best toaster.” They are meant to help you understand what’s important to your reader and help you create a reader persona.

image24

Once you understand how your reader searches for help, you’ll be able to find untapped keywords that are perfect to target.

Students are your best audience

You do online marketing to sell something.

Most often, it’s knowledge.

Not everyone will pay for knowledge; many are happy to try to soak up everything for free.

But your most valuable readers are those that know that paying for high quality training is worth it. They get the highest chance of success and will save a lot of time as well.

This ideal type of reader mainly hangs out in a few different places:

  • Large e-commerce stores – they buy books and audiobooks from marketplaces like Amazon
  • Blogs – they follow influencers in a niche and buy training courses directly from them
  • Education marketplaces – the demand for high quality online training is rising fast. Sites like Udemy and Skillshare sell thousands of courses to eager students.

With this method, we’re looking at the third location listed above. Users of Udemy are highly motivated learners and are ready to pay significant amounts for good training.

Even though you’re not selling anything yet, you want to create content around keywords that will attract this audience.

That way, when you are ready to sell, you have a highly receptive audience.

There are two main ways that you can do keyword research on Udemy.

Option #1 – Look at what students already like: The top courses in each category on Udemy have a few important things in common.

First, they are created by experts in a niche. These are the people who have been writing and teaching for years, so they know the language and phrases that the audience typically uses.

Secondly, we can sort courses by popularity. By looking at the most popular courses, we can see all of the most taken courses. What this means for our keyword research is that the course participants relate strongly to the copy for the course. We can generate keyword ideas from these sales pages.

Let’s go through this process step by step.

First, click on “browse courses” after you have created an account or logged in (free).

This will bring up a menu on the left side with all the different categories of courses:

image34

In this post, I’ll use a marketing blog as an example site to conduct keyword research. So, from the list above, I would choose the “Marketing” category.

Once you pick a main category, you’ll get a list of featured courses. You can choose to drill down a bit more. I could pick digital marketing, SEO, or social media marketing if my blog had a more specific audience:

image29

Whether or not you pick a sub-category, the courses you see aren’t necessarily the most popular courses, so we don’t want to look at them.

Instead, pick “All Courses” from the top menu, and change the “sort by” option to “popularity” on the next page:

image31

You’ll notice that you have additional filtering options on the side.

The most interesting option to me is “level,” which describes the difficulty of a course. If I created content for beginner marketers on my blog, I would want to look at the most popular courses for beginners.

But if I wanted to create content for experts, I’d want to filter the results down to “expert” level courses. As audiences get more experienced and advanced, they typically use different keywords to describe their problems.

It’s important because you don’t want to create content that solves a problem that experts have using the language a beginner would use. Your content won’t attract any expert marketers, and even if it did, they wouldn’t be impressed by your writing (i.e., it would not resonate with them).

Once you’ve filtered down the courses to only the top courses your audience would be interested in, we can finally find some keywords.

Here’s the start of the course description for the first top course I chose:

image03

You’ll notice that the author already bolded most of the keyword phrases important to this audience. He knows the phrases potential students are looking for (from years of experience) and makes them stand out.

Not all course authors will do this, but this is an area where you can find several keywords.

At this point, you want to start recording your keywords. You may directly target them later, or you may use them to generate additional keyword ideas to target. For now, just start a spreadsheet with them.

Note that you don’t have to copy them down exactly as they appear. For example, I doubt that anyone is searching for “how to attract ultra targeted traffic.” But a phrase such as “how to attract targeted traffic” is likely a fairly common one.

Once you’ve finished extracting keywords from the course description, scroll down to the “curriculum.” It has a list of each “lecture” in the course.

These will typically be on the most important (and alluring) topics for potential customers:

image14

From these first three sections, I would add the following keywords to my spreadsheet of keywords:

  • What is list building
  • Why you must build a list
  • Tools to build a (email) list
  • How to create a sales funnel

Again, take out any word(s) that aren’t going to be used very often by searchers.

What I just showed you was part of the sales page for a single course. You can often generate dozens of quality keyword ideas for your list from a single course. Go through as many courses as you like until you start seeing too many duplicate keyword ideas.

Option #2 – Look at what students are searching for: Any large website that sells something typically has autocomplete suggestions because having this feature usually increases conversions.

Udemy is one of those sites that provide suggestions in its search bar.

When you start typing a word, it will show you the most searched terms containing what you’ve typed so far:

image05

I would write down all of these keyword suggestions on my spreadsheet for later. These are very useful because they are already ranked from most searched to least searched.

Finally, don’t just stop with your main keyword. If you start typing in any of those suggestions, you may be able to get more suggestions.

For example, as I started typing “marketing free,” a suggestion for “marketing freelance” came up, which would make a good addition to my list.

Find the questions your readers are asking

The best keywords are the ones that come from your readers themselves.

Years ago, it was difficult to find them, but now you have quite a few options at your disposal.

Here are three different ways to find questions that your readers are asking so that you can extract additional keywords.

Option 1 – Faqfox: Faqfox is a handy little tool that can be used to generate content ideas or keyword ideas.

Based on a keyword you enter, the tool will scrape a list of threads on various forums and aggregators.

To start with, enter a keyword in the first text box. Then, you can either enter a forum (which may or may not work) or choose one of the pre-made categories of starter sites.

Once you click on one of the categories, URLs of relevant forums and subreddits will be loaded into the tool automatically. If you can’t find an appropriate category, choose “generic,” which will search all the biggest sites such as Reddit and Yahoo Answers.

image01

Once you click the search button, you’ll get a long list of thread titles (with links) relevant to your keyword.

From this list, you can start picking out keyword phrases and full questions that your audience has already asked in other places:

image12

You get a list of at least 100 threads for your keyword. Just from the results in the picture above, I would write down:

  • Which email marketing service should I use
  • My Gmail email is going to junk folders
  • Video marketing platform
  • How to get a job in advertising
  • How to take over a webmastering job
  • Free online marketing tools

All of these are potential keywords to target in future content.

Option 2 – Scrape a forum: A tool like Faqfox covers all the biggest forums and Q&A sites—but not all of them.

If you find a forum in your niche, you can look at the threads just like we did above to extract keywords for your list.

In addition, you can learn more about the kind of language your readers use.

To do so, we’re going to copy thread titles and then feed them into a keyword density tool. By looking at the words and short phrases that are used most often, you will begin to see a picture of what matters most to your audience.

For example, let’s say you wanted to learn more about the audience in the main Internet marketing section on Warrior Forum.

First, highlight all the thread titles:

image08

Copy and paste this into a blank spreadsheet, and then delete all the junk data:

image11

Copy the remaining data (first column), which should be all of the thread titles.

Paste them into a free keyword density analyzer tool—just make sure you select the “text” option:

image27

All these tools work the same, so use any you like.

Once you run the tool, you’ll get a long list of single words, 2-word phrases, and 3-word phrases listed in order from most used to least used:

image22

To be honest, my results aren’t great in the screenshot above. That’s because I only scraped one page of forum results. Ideally, you want at least a few hundred so that trends start to become more obvious.

Still, you can learn a little bit about your audience from a brief analysis like this. For example, they likely use the word “best” instead of “top.” In addition, they are interested in “finding” answers to their problems.

Option 3 – Q&A sites: The last method I’ll describe here to find questions from your readers is by going on Q&A sites such as Quora.

Quora is probably the biggest, but there are other question and answer sites you could use for research:

The reason that Q&A sites are often more useful than forums for keyword research is that unlike forums, which are ordered by the last reply, Q&A sites have algorithms to show you popular content, both past and present.

The first step is to enter your main niche into the main search bar.

Use the filter on the left to narrow down the results to “Topics.” Topics are basically collections of all the questions in a particular niche.

image30

You can follow the topic, which will make it easier to monitor new questions in the future. Or you can just click the link to the main topic for now.

When I clicked “Marketing”, which is a huge topic (1.7 million followers), it gave me several smaller categories to choose from.

Right from these small category previews, you can already find a few good keywords:

image33

Once you’re done mining that page for ideas, click one of the sub-topics to dig in a little further.

In my case, I clicked the main “Digital Marketing” header:

image09

As you scroll down, you’ll see a mixture of new and old questions.

You can look at the number of upvotes each question has (bottom left corner of any question) to see if it’s popular. Ideally, look for questions with at least a few upvotes.

image13

Go through as many questions as you like to find keywords. You can easily find 50-100 to add to your list with this method.

Steal your competitor’s accidental keywords

Large sites, the leading sites in your niche that produce a lot of content, target a lot of keywords without doing it on purpose.

They may mention one in a headline of an article and/or a few times throughout the article.

You can’t compete with these sites for the main keywords they target. However, if they rank for any of these other keywords, you can try to overtake them.

If you create great content around these keywords, you will have better content for those particular keywords because it is more targeted.

This is a difficult method to use, but it can provide some useful results. It’s best used as a second or third option to the other methods in this article.

Step 1 – Use Google’s Keyword Planner: Go to the keyword planner and paste the URL of a competitor in the space where it asks for “your landing page”. Leave the keyword textbox blank. The main blog URL typically works best.

image26

Google will return up to 800 keywords that it associates with the site, sorted by relevance.

You can also input a specific URL of a blog post into the landing page area. For example, if my new hypothetical blog was going to focus on social media marketing, I could enter posts about social media from Quick Sprout into the tool:

image25

You will get some keyword ideas that wouldn’t have come up if you just typed “marketing” (or related) into the keyword box of the tool.

I recommend downloading all the results into a spreadsheet (“Download” button in the top right corner). Copy any keywords that seem good into your main list.

Step 2 – Plug these keywords into a rank tracking tool: One of the main ideas behind this strategy was to find keywords that are easy to rank for. If a competitor can rank for a keyword without even trying, it should be easier for you to rank highly for it.

Just because the tool associates those keywords with the site doesn’t mean the site ranks highly for them. You’ll need to plug them into a rank tracking tool or check them in Google manually (time-consuming).

For the sake of this example, I used Pro Rank Tracker. The free plan will give you 20 keyword results, but you’ll obviously need a premium account if you’re taking this seriously.

Add in your competitor’s domain along with the keywords from the previous step:

image02

Click the “Save” button, and give the tool a minute to check the ranking for those keywords.

When you go to the URL view in the side menu, you’ll see all the keywords you just entered under your competitor’s URL:

image21

Ideally, you’re looking for keywords that are ranking in the first two pages and aren’t fully included in the title. Those are typically the best candidates.

Twitter chats are a gold mine

All marketers know that Twitter is a powerful marketing platform, but few know how to use all its features effectively.

Twitter chats are one of those features. When people want to talk about a subject on Twitter, the only way to do it is to include a hashtag in their tweets. Then, other users can search for that hashtag to see the latest results and interact with them.

A Twitter chat uses a specific hashtag so that groups of users can discuss topics, live. They are usually hosted or moderated for a short time in order to stimulate discussion if needed.

To take advantage of Twitter chats fully, you’ll need to use a tool designed specifically for them. Here are a few options:

All of these tools work fairly similarly. You’ll need to enter a hashtag for a particular chat (shown on Tweetchat below) so that you can see Tweets from other users:

image06

Technically, you can enter any hashtag. For example, “#SEO” will bring up recent Tweets that include “#SEO” in them:

image17

That’s not very useful for our keyword research though.

Here are two options that are.

Option 1 – Observe established Twitter chats: By far, the best way to take advantage of Twitter chats for keyword research is to actually participate in them in real time.

The idea is to observe the chats, which typically consist of followers asking influencers questions. These questions often contain useful keywords, just like in forums.

When you enter a hashtag for an old chat, you’ll typically only see the most popular (re-tweeted and shared) posts. Unfortunately, these aren’t usually the questions, but rather the answers.

So, if you really want to take advantage of chats, plan to attend a few upcoming ones.

Here are four places where you can find Twitter chats:

  1. http://twubs.com/twitter-chats
  2. https://docs.google.com/spreadsheet/ccc?key=0AhisaMy5TGiwcnVhejNHWnZlT3NvWFVPT3Q4NkIzQVE&hl=en#gid=52
  3. http://www.gnosisarts.com/index.php?title=Tweetchat_Wiki/By_Day
  4. http://tweetreports.com/twitter-chat-schedule/

None are particularly well organized, but you can use the “find” command of your browser (ctrl + f) and type in a common niche keyword to find good chats:

image23

Make a list of chats you’d be interested in attending from each of the four sources as well as the times they run.

When you enter the hashtag into one of the chat tools I gave you at the start of the section while the chat is actually running, you’ll get to see many questions from followers. Write those down.

In addition, you can type in the hashtag in the Twitter search bar, although it’s not as effective as being in a live chat.

image32

You’ll have to scroll through some of the “answer” Tweets, but you should be able to dig out a few good questions and keywords:

image28

Option 2 – Observe old chats or any relevant hashtag: As I just showed you, you can observe parts of old chats.

In addition, you can type in other hashtags that aren’t necessarily Twitter chats.

There’s no easy way to find these, but you can typically guess them. Try combining:

“niche” + “questions”

For example, I’d try the hashtag #marketingquestions:

image20

If you type in an old chat, you may be able to find a few keywords if you’re willing to dig through a bunch of other tweets:

image19

Don’t put all your eggs in one basket

All keyword research methods and tools have their own limitations.

In order to get the most complete set of keywords, you need to use at least a few different tools.

Now that you have a large list of potential keywords to target, you should run them through a keyword tool to get even more results that might not show up on a standard Google keyword planner list.

There are a few different tools you can use for this.

Tool #1 – Keywordtool.io: I told you about how powerful the autocomplete suggestion feature on large sites can be—none more so than Google itself.

This keyword tool types in your keyword plus different letters, one at a time, and records the autocomplete suggestions.

image07

Type in any of your keywords (from your spreadsheet) into the tool, and click the search icon.

This will bring up a large list of keyword suggestions:

image10

Not all of them will be good, but you’ll be able to find quite a few keywords to target.

There are other tools that utilize this same method, e.g., Ubersuggest:

image04

Tool #2 – Term Explorer: This tool seems to use data from Google’s keyword planner but is able to give you more than the standard results for a keyword.

Once you create an account, select the bulk keyword tool either from the dashboard or the top menu:

image15

Give the job a name, and then enter one or more of your keywords.

Term explorer is a really powerful tool if you have a paid account. Free accounts are limited to tiny jobs (1,000 keywords), which is still better than a basic Keyword Planner search.

However, paid accounts can change the results size to up to 90,000 keywords, which will reveal all kinds of hidden keyword gems.

image16

After you run a job, a list of keywords will be generated, similar to the Adwords display.

You can filter the results with the left sidebar if needed:

image00

If you start with a broad term (such as “marketing” in my example above), don’t stop there.

Pick some of the best long-tail keywords, and create another job for them to reveal even more unique ideas.

I created a new one for the keyword “internet marketing strategy” and ended up with a few long-tail keywords that would probably be good to target.

image18

Keyword research and competition go hand in hand

Keyword research will allow you to compile a list of hundreds or thousands of relevant keywords.

However, this is useless until you assess the competition level of each keyword.

If several professionals are targeting a particular keyword, it’s typically better to pass on it and find an easier one to go after.

Analyzing the existing search engine results for a keyword and determining how competitive they are is a whole different topic.

Most keyword research or rank tracking tools have some sort of built-in competition metric. However, some are more accurate than others.

If you’ve used the methods in this article to find keywords, your chosen competition metric will be much lower for your keywords than the easy ones that Google’s Keyword Planner gives to everyone else.

Beyond Google AdWords Keyword Research

Many people get keyword data from AdWords. But there’s other sources of keywords and search volume. We’re going to walk through a few of them now!

  1. Bing

    The newer Bing keyword research tool, although still in development, is a good place to check.

    Sign in (or create an account)

    Beyond Google AdWords Keyword Research #1

    Start with the basic settings. Bing’s keyword tool is a little different than Google’s. If set to 30 days, it give you the exact amount of search from those 30 days.

    Beyond Google AdWords Keyword Research #2
    Beyond Google AdWords Keyword Research #3

    You can export the data

    Beyond Google AdWords Keyword Research #4

    Use the benefit of exact search numbers to judge long tail:

    Beyond Google AdWords Keyword Research #5

    You can bring in Google AdWords data for a comparison of search volume metrics:

    Beyond Google AdWords Keyword Research #6

    Now, let’s get a little more refined. Set the language to English (assuming that’s your target language) — and expand the date range to 6 months. This is going to give you a large data set to make good decisions:

    Beyond Google AdWords Keyword Research #7

    Set to “strict mode” for basically Bing’s version of “exact match”

    Beyond Google AdWords Keyword Research #8

  2. Wordtracker and SEO Book’s Keyword Research Tools

    These versions are free. You can also register and pay for the pro versions.

    Beyond Google AdWords Keyword Research #9

    Run the report in SEO Book to export CSV’s

    Beyond Google AdWords Keyword Research #10

    Use the overall daily estimates for measure potential traffic across the entire web:

    Beyond Google AdWords Keyword Research #11

  3. Wordpot

    Wordpot is another source of keywords you may not have thought of.

    Beyond Google AdWords Keyword Research #12

    It also shows definitions, synonyms, related words and associated words

  4. YouTube Keyword Tools

    Of course if you’re doing optimization for YouTube you’ll be interested in the search volume — but if not, this is still a fantastic tool very generating more keyword ideas.

    Beyond Google AdWords Keyword Research #13
    Beyond Google AdWords Keyword Research #14

    Go to the bottom, click “add all 100” and then click export to CSV or the file of your choice

    Beyond Google AdWords Keyword Research #15-16

    Also — let’s say you want to rank in youtube for “search marketing”.

    1. Search for the term “search marketing” in YouTube

    2. Find the top ranking video

    3. Enter that video in the keyword tool — this give you the top keywords for that video

      Beyond Google AdWords Keyword Research #17

      You can add these to the existing report or start over

      Beyond Google AdWords Keyword Research #18

    4. Local Marketing Source — Local Keyword Tool

      This is a handy tool for generating local keyword variations. It doesn’t give you search volumes, just a list of keywords.

      Start with a zip code, radius and keyword(s)

      Beyond Google AdWords Keyword Research #19

      Generate your list and export to CSV

Use Search Suggest Scrapers For Fast Keyword Ideas

The web collects data every day. Data that people are actually typing — they are telling us what they are looking for.

Search suggestions are a great place to look for words people are actually typing. This can often bring immediate content ideas as well as keyword prospects. The important part in this process is collecting as many ideas as possible.

Many SEO’s out there miss some of the most useful tools. Let’s walk through some now!

First, a list of suggest scrapers

All are great (there’s a lot out there on ubersuggest) but I’ll show you a few tricks with Soovle and KeywordBuzz

Soovle

There’s two ways we’re going to use Soovle. First, we’re going to use it as a general suggest scraper.

  1. Set it is 15 engines. We want as many keywords as possible!

    Use Search Suggest Scrapes For Fast Keyword Ideas #1

  2. Start typing the root keyword surrounding the topic you’re researching

    Use Search Suggest Scrapes For Fast Keyword Ideas #2

  3. Use Scraper for Chrome to Key Them in a Google Doc

    1. Who wants to cut and paste all of those words? We can get them in a Google Doc with a small tweaking to the Scraper xPathUse Search Suggest Scrapes For Fast Keyword Ideas #3
    2. You’ll see the default results don’t include every keyword.Use Search Suggest Scrapes For Fast Keyword Ideas #4
    3. Eliminate the [4]Use Search Suggest Scrapes For Fast Keyword Ideas #5
    4. Scrape again (or hit enter)
    5. Export the results to Google Docs.Use Search Suggest Scrapes For Fast Keyword Ideas #6
  4. Filter and sort the raw data

    1. You’ll notice there’s a little extra stuff in there to clean up. There’s some data from others divs and duplicate keywords.Use Search Suggest Scrapes For Fast Keyword Ideas #7
    2. Delete the extra two rows at the bottomUse Search Suggest Scrapes For Fast Keyword Ideas #8
    3. Sort by keyword alphabeticallyUse Search Suggest Scrapes For Fast Keyword Ideas #9
    4. Now we need to filter duplicates! In cell B2 put this formula;=if(A2=A3,true,false)This will put true if the items are the same and false if they are different.

      Use Search Suggest Scrapes For Fast Keyword Ideas #10

    5. Then hover over the corner (wait for the cross to appear) and double click.Use Search Suggest Scrapes For Fast Keyword Ideas #11This marks every items as a duplicate or not.

      Before we do the next step, copy the true/false results and paste as values onto its self. This won’t visually change the true/false results – but will remove the formulas and leave just the words.

      Use Search Suggest Scrapes For Fast Keyword Ideas #12

    6. Sort descending by column BUse Search Suggest Scrapes For Fast Keyword Ideas #13This should put all the TRUE’s at the top. These are your duplicates.
    7. Delete all the “TRUE” rows.Use Search Suggest Scrapes For Fast Keyword Ideas #14

    You’re left with about 100 of unique and diverse keywords!

Soovle To Catch Trends

Next, we’ll use Soovle to capture trending keywords if you want to create quickly and catch a trend.

  1. Click on the Star

    This will bring you to the trending words for the day.

    Use Search Suggest Scrapes For Fast Keyword Ideas #15

  2. Do a control-f in your browser. Search for words surrounding your niche

    Use Search Suggest Scrapes For Fast Keyword Ideas #16

  3. Then — enter THAT word back into the suggest scraper.

    Use Search Suggest Scrapes For Fast Keyword Ideas #17

KeywordBuzz

I just want to show you one thing with keywordbuzz.net — I don’t know of any other scrapers to do this.

  1. First — arrive at the search screen and enter your keyword.Use Search Suggest Scrapes For Fast Keyword Ideas #18
  2. Then — not only can you download as a text file. You can instantly check domain availability for .com .net and .orgUse Search Suggest Scrapes For Fast Keyword Ideas #19
  3. Just click ‘check’ — and you’ll instantly have some great niche domain name ideas.Use Search Suggest Scrapes For Fast Keyword Ideas #20
  4. You could even get ideas to latch onto a brand’s traffic with an auxiliary tutorial or informational site.Use Search Suggest Scrapes For Fast Keyword Ideas #21

Use Google Correlate

Google correlate is another not often talked about tool for keyword research, especially seasonal trends. http://www.google.com/trends/correlate/

Here’s an example — if you enter ‘red wine’ this is the result:

Use Google Correlate #1

You can see it gives us keywords where search volume across time has a strong correlation. This is a useful tool for exploring trends and coming up with some outside of the box keyword ideas — let’s explore a few!

Additional Topic Ideas

Let’s say you’re a food blogger — and you like to do an article on red wine every once and awhile. Wouldn’t you like to know some other topics you could attach to red wine?

The correlate result for red wine is:

Use Google Correlate #2

You can see that a lot of people are searching for cheese related things around the same time they are searching for wine. This is somewhat expected, but there are many targeted keyword ideas you may not have thought of. And you know these trend right along with red wine – so you could time your content to go with the seasonal trends a bit better too.

You can show more words and also do a CSV export

Use Google Correlate #3

Ideas By Location

The location feature in Google Correlate is great for targeting an audience in a particular location, with a parallel interest. This works great with seasonal items, or interests that fluctuate in search patterns through the year.

For example, let’s say you’re an eCommerce shop that sells grills etc. You might be interested in people searching for “outdoor party”. You know this search pattern matches the seasonality and interests of your customers.

An initial correlation shows:

Use Google Correlate #4

This shows similar search patterns for “poison oak” and “boat registration”. But let’s get more targeted and get correlations for the location.

Use Google Correlate #5

And the results are:

Use Google Correlate #6

Here you have people in the states of Connecticut and South Carolina interested in an Antigua all inclusive vacation.

So what do you do?

You do a vacation giveaway to the Antiguas for your customers from those states at the time of year people begin having outdoor parties (and buying grills).

Tracking Internal Site Search

Many site owners often overlook the most obvious place to get keywords. Your own website. I’m not just referring to keywords people use to arrive at your site. Rather, searches users are making within your site.

This especially works if you have a larger site — ecommerce or a large editorial site. You will have a lot of data to work with and many opportunities to learn about how people are using your site and keyword ideas you’d never think of otherwise.

I’m assuming you already have a search function in your site. I’ll show you how to set it up in analytics and use the data effectively.

Check How Your Site Does a Site Search

You’re going to need the URL parameter your site uses to track site search.

For example, the site I’m demonstrating this on, looks like the following;

Tracking Internal Site Search #1

See the search paameter?

/search.php?search=search+example

where “search example” is the keyword typed.

The parameter is “search” (hint: if in php its after the question mark)

Set It Up in Analytics

Head over to your analytics profile.

Go to the Admin area. Click on Profile settings

Tracking Internal Site Search #2

In profile settings, you’re going to scroll down to “Site Search Settings”

Type the query parameter we found in the first step in the field. Also, select to strip query parameters from the URL.

How To Use The Data

I’m going to show you how to best access this data for keyword research. We’re going to do this within analytics but you can also export the data for further use.

First – View the site search keywords

Tracking Internal Site Search #3

If you’ve never done this before, its best to zoom out at least six months.

Tracking Internal Site Search #4

We’re going to apply a few different filters to the data. (You could do this with advanced segments too for a more “permanent” filter).

This first filter says “show me words that get searched more often but then most people immediately leave my site. (You may have to play with the numbers specific to your metrics). These are the words you want to build new content around.

Tracking Internal Site Search #5

And may get something like this;

Tracking Internal Site Search #6

OR you can do sort of the opposite. Say “show me words that get searched a lot but where people stay on the site. Again, the exact number will depend on the behavior of people on your site.

Tracking Internal Site Search #7

Tracking Internal Site Search #8

Take Action

Either way, you’ll have a list of keywords you can take immediate action on.

  • Highly searched words where people leave = create the content!
  • Highly searched words where people stay = keyword research and optimize around those words.

Scraping Search Suggest From the Command Line

Extract Initial JS File

Google runs a script when computing its search suggest words. You can use the following URL in your browser, and in turn generate a JS file.

Paste the following into your browser.

http://www.google.com/s?sugexp=pfwl&cp=15&q=<search term here>

Replace <search term here> with your keyword. Let’s do marketing.

http://www.google.com/s?sugexp=pfwl&cp=15&q=marketing

Paste in your browser

Scraping Search Suggest From the Command Line #1

It will automatically create and download a JS file

Scraping Search Suggest From the Command Line #2

Open it and you’ll see your raw suggest terms

Scraping Search Suggest From the Command Line #3

That’s not very much practical use, so there’s more!

Create an .sh File

Simply copy and paste this code into your text editor

#!/bin/bash
q=$(echo "$1" | sed 's/ /%20/g')
curl -s "http://www.google.com/s?sugexp=pfwl&cp=15&q=$q" | sed 
's/\[/\n\[/g' | cut -d'"' -f2 | tail -n +4

Scraping Search Suggest From the Command Line #4

Save it as an .sh file. You can simply name it suggest.sh

Run The Code In Terminal (Mac)

Run the following command in terminal

./suggest.sh "marketing"

Which will produce the follow results;

  • marketing jobs in wa
  • marketing solutions
  • marketing plan
  • marketing solutions wa
  • marketing jobs
  • marketing strategies
  • marketing drive
  • marketing mix
  • marketing internships

(note that some results may be localized).

Running Your Own Keyword Frequency Analysis

There’s many tools available out there to run a keyword frequency analysis. Some are free, some are paid. But there may be reason and benefit to having the control to run your own reports. I’ll show you one way to do this!

Create your .sh Code

Start by cutting and pasting this into your test editor;

#!/bin/bash

input=$1

IFS=$'\n'
# for keyword files...
if [ -e "$input" ]; then
	cat "$input" | sed 's/ /\n/g' >> tmp
	for word in $(cat "$input" | sed 's/ /\n/g' | sort | uniq); do
		count=$(grep -c "^$word\$" tmp)
		echo -e "$count\t$word" >> tmp_o
	done
	cat tmp_o | sort -hr
	rm tmp tmp_o
# for web pages...
elif [[ "$input" =~ "http://" ]]; then
	# check if links is installed. found solution at 
	http://stackoverflow.com/questions/592620/check-if-a-program-exists-
	from-a-bash-script
	type links >/dev/null 2>&1 || { echo >&2 "This script requires 
	'links' but it's not installed.  Aborting."; exit 1; }
	links -dump "$input" | sed -e 's/ /\n/g;s/\(.*\)/\L\1/' | sed 's/\W//g' >> tmp
	for word in $(cat tmp | sort | uniq); do
		count=$(grep -c "^$word\$" tmp)
		echo -e "$count\t$word" >> tmp_o
	done
	cat tmp_o | sort -hr
	rm tmp tmp_o
fi

Save this as a file — anything will work, such as keyword.report.sh

Run the Script

Open up Terminal in your Mac

You can run the script in two ways — either by file or by URL.

  • By File

    bash keyword.report.sh keyword.file.txt

    ‘keyword.report.sh’ is the name of your .sh file and ‘keyword.file.txt’ is the name of your file with the content in it you’re analyzing.

  • By URL

    bash keyword.report.sh http://quicksprout.com

    Again, ‘keyword.report.sh’ is the .sh filename and then you just put the URL! Easy!

I bet you have a killer list of keywords and data now! Now your site is fast, crawlable, semantically correct, and you’ve got the best keywords. Time to target and create content that will attract links while you sleep.