Last update: 16th March 2019

How does it work?


We offer scraping large amounts of keywords for the Google Search Engine. Large means any number of keywords between 30 and 50000. Additionally, we support scraping the following search engines:

  • Duckduckgo - For Duckduckgo, results include all SERP data from search results pages.
  • Youtube - For YouTube, results include all videos and corresponding metadata on the first page for each keyword.
  • Baidu - You will get the links, titles and snippets of the Baidu SERP page.
  • Bing - SERP data (links, titles and snippets) of the results pages
  • Infospace - SERP data
  • Google Image Search - For image search, results include the link to the image result.
  • Google News Search - For News search, results include link and publication date to the article.

For all search engines, the results include links, titles and snippets from all results pages for each keyword.

In a first step, you must upload a text file that contains your keywords that you want to be scraped. Each line must have exactly one keyword. For example, a valid keyword file would be this file of the hundred largest cities in the US.

Then in a second step, after you have created the Scrape Job and paid the pending amount, you can download the scraping data as json or csv file. The complete scraping results in json and csv format for the hundred largest cities in the US can be downloaded here. The search engine used was Google.

Scraping Results JSON Scraping Results CSV

Uploading a CSV file as keyword file

Instead of uploading a plain .txt keyword file, you can also upload a CSV keyword file.

  1. The delimiter of your CSV file is must be a commata , and the quotechar must be a quote marks "
  2. The first row of your CSV file will always be ignored, because we assume that it is the header.
  3. Make sure that your CSV file is UTF8 encoded.
  4. Scrapeulous interprets the complete row of your CSV file as a single keyword.
  5. The separator character will be replaced with a single whitespace. For example: "," will be replaced with " "

Use Case - Market analysis for private investigator company

Consider this CSV file as input file. Scrapeulous will convert it to this keyword file. It has two columns: City Name and Business Type. The City Name column contains the 40 largest cities in the United States. The Business Type column contains the string cheating spouse investigator for each row. Consider the image below:

CSV file for market analysis
A CSV file containing two columns used for scraping on scrapeulous.com. Will be converted to this keyword file.

Story Line

Maybe you are a veteran and used to serve in intelligence operations overseas. Now you retired from the forces with a couple of 100k$ in the bank and you want to create a cheating spouse private investigator company, because your girlfriend cheated on you during your deployment.

Because you consider yourself smart, you want to run a market analysis on all existing private investigator companies before you attempt launching such a company. Therefore, you want to search each line of this CSV file on the first five pages of Google.

Analysis

From the search results, you want to figure out the following pieces of information:

  1. What keywords appear in the title an snippets of the search results?
  2. How many unique providers per city are in this niche? You can count the unique website urls to get this information.
  3. Which city has the largest competition (largest number of individual companies)?
  4. How does the landing page of the most ranked competitors look like? You can analyze this by following the url.

Results

To get this data, you submit the CSV file to scrapeulous.com and enter the appropriate settings.

After submitting the scrape job, you will receive the results and can start your analysis.

This is the results page for this scrape job

In the image below, you can see how the json results for the above CSV keyword file will look like.

JSON results for CSV file
The scraping results for the CSV file over 5 pages for each keyword. When there are less pages, then the search engine (in this case Google) didn't have more results for the specific keyword.

Testing for free!

You can test scraping up to 45 keywords. Just submit a scrape job with maximally 45 keywords and you can start scraping without paying for it. This allows you to see if scrapeulous.com offers you the service you actually need.

What do we offer?

Scrapeulous.com allows you to scrape many search engines fast and efficient.

  1. Upload a keyword file and calculate the price for scraping the keywords in your file
  2. Make a payment when you are happy with the price and scraping settings
  3. Wait until your scraping project is done and download the results as json or csv files

Advanced scraping projects

If you have a special requirement for your scraping project, you can write us an email at our contact mail and ask us if we find a solution for your needs.