.Net provides low level classes to natively work with HTTP. Two of such classes worth mentioning are:

1. HttpWebRequest
2. HttpWebResponse

To send a request to target URL we will create an instance of HttpWebRequest as follows: Continue reading →

HttpRequest and HttpResponse Classes

Let’s try to scrape from data from my own blog i.e. www.shaktitanwar.com . We will try to extract all blog titles on home page.

There are various facets to scraping:

1. We need to first have a URL that we need to scrape i.e. shaktitanwar.com
2. What all data needs to be scraped? In our case it’s Blog titles.
3. Which verb is used by the URL to show data? In our case it’s a GET request
4. The HTTP Headers required by website to serve the content properly.

Continue reading →

Automating applications is a serious business. Most business requires data coming from various sources. Usually there are 2 techniques that are used to fetch data from sources:

1.) Calling Provider API :

This is a White hat approach of getting data from data sources. Continue reading →