Objectives:
- To learn how to crawl web pages using a library
- To learn automation using cronjobs
Using PHP, you can build a crawler that can crawl through Yahoo, Bing, or if you're feeling lucky, Google, to find the top 10 websites that appear as you search for the keyword (ex. "village 88").
You can watch a video we created to see how this could be done in PHP:
This was created for some of our students and may not look as good as other videos in the platform. Sample codes can be downloaded here.
Your task is to build a spiderbot that can crawl through a website. Display the top 5 results (title + link) that appear as you search for the keyword "software engineer" in: https://www.bing.com/
If you wanna level up more, what if you want to also set this up so that it would process/do this every day? There is a thing called cronjobs that allows you to run a script at a certain time interval, like every hour, every midnight, every Thursday, every third day of t...