Description
- A) To implement a program (Web scrapper) in Java that take as input any URL say and do following things.
- It sends the request (http request) for the URL supplied.
- Get the page and extract all the text and html tags present in the page.
- Extracted text along with tags should be sent to an Excel file like <p> || welcome to pec (both in separate columns).
- Paste the extracted anchor tags i.e. </a> and the URLs in a separate file along-with the text so this excel with have two columns <text in anchor> <corresponding URL>.
- You can use String/ StringBuffer/ String Builder class function to perform the above operations
- Try to change the crawler developed above to Focused Web crawler. By crawling only those webpages that have information regarding any faculty of that institute.
- Devise a method to handle downloadable data present in the website like PDF, word links etc.
