Become a Fan

Other Pages

About

My Photo

Recent Posts

Archives

More...

Disclosure

  • Disclosure
    Nobody pays me to write any of the copy on my blog, and should I ever have the good fortune that they do, I will declare it. My main employment is as the owner and principal of The Future Place consultancy. The Future Place provides two key services 1) training and services to industry and academic bodies and 2) consultancy services to companies. The details of the companies I work with are a private matter, but if I blog about any company who has paid The Future Place more than expenses recently (approx. two years) I will mention that they are a client. I hold equity in Virtual Surveys and provide consulting services to them from time to time. I am paid to run courses for a number of trade bodies and over the last few years clients have included ESOMAR, AMSRS, MRS, and MRIA.

My daughters eShop

« The book continues to make progress | Main | What is your favourite blog search engine? »

Easy to use Web Scraper

This is a really good video showing how to use OutWit a simple to use Web Scraper that sits as an extension to Firefox.

Remember, there are ethical considerations about using Web Scrapers. In particular

  1. Some services have Terms and Conditions that prohibit automation, for example Google does.
  2. When deciding whether something is truly in the public domain it is important to keep in mind the author's intent. If somebody posts a comment in on online discussion they might have the expectation that it would only be read by human, not scraped and searched.

However, if the web scraper is simply being used to change 20 minutes of cut and paste into 1 minute of scrape and post, then there is probably not a major difference.

December 30, 2009 in Technology |

Tags: market research, ray poynter, web scraping

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.

Subscribe to this blog's feed

The Festival of NewMR?

The Main Stage of the NewMR Festival will be on December 5th, the premier online event of the year.
Visit the NewMR website to find out more.

Twitter Updates

    follow me on Twitter

    Categories

    Technorati