Google News Scraper


Scrapes Google News.
 

Instructions

The scraper batch queries news.google.com, outputting a table of returns including URL, title, source, date, teaser text, and thumbnail image, .

Enter sources to be queried (optional, one per line, e.g. www.nyt.com), a set of key words (one per line) and a project name for the results file. Select desired number of results per query (default is 100, as greater numbers require multiple scrapes). In the advanced options you can also set a date range for your query, amongst other things.

Optionally, the scraper retrieves thumbnail images attached to each search return.

Sample project

Discover uses of "security fence," "apartheid wall" and "security barrier" in news media, outputting a table. Select all, and paste special in MS-Excel. Results may be visualized in a number of ways, including the following network diagram showing associations between terms and media sources.

See: R. Rogers and A. Ben-David, "Coming to Terms: A conflict analysis of the usage, in official and unofficial sources, of 'security fence,' 'apartheid wall,' and other terms for the structure between Israel and the Palestinian Territories," Media, Conflict & War, 2, 3, 2010, 202-229.[summary]

Course units using this tool

The Spheres
The Spheres Spheres as way of thinking about the Web Thinking of the Web in terms of spheres refers initially to the name of one of the most well known, the bl...

Other projects using this tool

Issue Image Analysis
Issue Animals Research With climate change, animals become endangered. Global warming as well as global cooling threatens the habitat of species, as animals migr...

I Attachment ActionSorted descending Size Date Who Comment
Apartheid_Wall_and_Security_Barrier_visualization.pngpng Apartheid_Wall_and_Security_Barrier_visualization.png manage 602 K 12 Dec 2008 - 12:13 MichaelStevenson R. Rogers and A. Ben-David "Coming To Terms" Figure One
googlenews_scraper.pngpng googlenews_scraper.png manage 8 K 05 Dec 2008 - 14:54 AnneHelmond  
Topic revision: r12 - 24 Feb 2015, ErikBorra
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback