Automation in .Net Part 2

HttpRequest and HttpResponse Classes

Let’s try to scrape from data from my own blog i.e. . We will try to extract all blog titles on home page.

There are various facets to scraping:

  1. We need to first have a URL that we need to scrape i.e.
  2. What all data needs to be scraped? In our case it’s Blog titles.
  3. Which verb is used by the URL to show data? In our case it’s a GET request
  4. The HTTP Headers required by website to serve the content properly.

Continue reading