I agree with rich59123 . You

PreciousReed — Fri, 01 Jul 2016 18:17:31 +0000

I agree with rich59123 . You can use rss feeds. Install and set up a Wordpress site. Install an RSS feeds plugin. Get the RSS feed url of the sites you are interested in, then place their url in your plugin configuration. Set the time of the post to be posted. That's it!

I agree with rich59123 . You

PreciousReed — Fri, 01 Jul 2016 18:17:30 +0000

How about rss feeds? I want

rich59123 — Sun, 26 Jun 2016 20:17:14 +0000

How about rss feeds? I want to display rss feeds possibly merge sort and colate the items on them from several list, choose some to be saved which will then be added to my database in a way that I can search for them by keywords, list and sort them? Thanks. Ive done it with mediawiki but found that it was resource heavy and was hoping there was a way with a smaller footprint or perhaps just more resource friendly

Greg, Thanks for the reply.

gonzales4monster — Thu, 19 Feb 2015 18:17:30 +0000

Greg,
Thanks for the reply. Imdb.com was just an example however I'd disagree with you on this (even if this is just and example ).
I will double check on the legality but I doubt posting the link to theirs site will be an issue. Please mind that I was referring to only posting the link with the title etc not coping description of the movies and other site content.
For the data volume, since I'm planning to pull only titles, this should be no issue. Even if they have 100k movie or more that is not a problem for these days DB engines and / or server hardware. .

Regards
G

Well, there are two thoughts

Greg K — Thu, 19 Feb 2015 09:09:52 +0000

Well, there are two thoughts that come to mind reading this...

1. The legality of scraping sites like IMDB... Probably against their terms of use for the site

2. Do you realize how huge that place is, updated daily, and how long it would take to crawl the entire site? Probably not really going to be feasible. (hint, go to google and put this as the search term site:imdb.com It comes back saying they have over 46 million pages indexed...

I really think you may want to rethink what you are wanting to do and narrow it down to something more doable than a broad scraping of sites.