Frequently Asked Questions

What is our hourly rate for customized scraping services?

The minimal rate for creating custom scrapers is set to 30$ per hour.

Can I use special search operators like intitle:, site: or filetype:?

We do not agree that scraping with keywords that excessively use those search parameteres will provide best results.

If you want to make sure that you will get proper results, please submit a manual scrape job to our contact page

What do I get as results?

Depends on the search engine. Usually you will get the link, title and description of the results on the SERP page.

For example, this are the results for the keyword Europe News with the search engine Google scraped from Germany:

{ 'Europe News':
   { time: 'Wed, 31 Oct 2018 19:28:40 GMT',
     num_results: 'Ungefähr 1.800.000.000 Ergebnisse (0,46 Sekunden) ',
     no_results: false,
     effective_query: '',
     results:
      [ { link: 'https://www.bbc.com/news/world/europe',
          title: 'Europe - BBC News - BBC.com',
          snippet:
           'Get the latest European news from BBC News in Europe: headlines, features and analysis from BBC correspondents across the European Union, EU, and the ...',
          visible_link: 'https://www.bbc.com/news/world/europe',
          rank: 1 },
        { link: 'https://de.euronews.com/nachrichten/europe',
          title:
           'Europa | euronews - internationale Nachrichten zum Thema Europa',
          snippet:
           'Dänemark 31/10/2018. Dänemark vermutet Iran hinter Anschlagsversuch · Im September wurde Dänemark durch eine großangelegte Polizeiaktion für Stunden ...',
          visible_link: 'https://de.euronews.com/nachrichten/europe',
          rank: 2 },
        { link: 'https://www.theguardian.com/world/europe-news',
          title: 'Europe | World | The Guardian',
          snippet:
           'Europe now I live among the neo-Nazis in eastern Germany. And it\'s terrifying. Anonymous ... Europe now Andrzej Krauze on the armistice centenary – cartoon.',
          visible_link: 'https://www.theguardian.com/world/europe-news',
          rank: 3 },
        { link: 'https://de.europenews.dk/',
          title: 'EuropeNews',
          snippet:
           'Ich bin als Exilant gekommen und werde als Exilant gehen", schreibt Imad Karim, dem in der ARD Verbreitung von Fake News vorgeworfen wurde. Jetzt .',
          visible_link: 'https://de.europenews.dk/',
          rank: 4 },
      
        .... snip ....

        { link: 'https://www.neweurope.eu/',
          title: 'New Europe – Independent European News',
          snippet:
           'News · epa07062910 European Commission Vice President Valdis Dombrovskis (L) and Spanish Economy Minister Nadia Calvino Spain to shrink deficit, keep ...',
          visible_link: 'https://www.neweurope.eu/',
          rank: 10 } ] 
    } 
}
            

What if there are no results for a keyword?

There is a boolean flag in the results: no_results is set to true when the search keyword either has legitimately no results or if the search engine used a different search term instead and is displaying results for the corrected search term.

When the search term was automatically corrected, the results have a additional variable: effective_query. This variable is the search keyword, that was effectively used in the search. When effective_query is set, no_results is also set to true.