I really need a web crawler...
Hi,
I provide a great service to my members, and the only thing I ask in return is that they link back to my site. Most of my members honour their agreement, but unfortunately, some do not.
My members are building up, so I don't have time to check through everybody's sites to see if the link is there. Is there a crawler (windows software or script based) that I can get to check their source code for me?
Thanks.
Oh, and by the way, the link doesn't have to be visible to humans, the point is SEO. As long as there's the href code, that's fine... and their sites are in a database.
James
andy206uk posted this at 15:38 — 4th February 2005.
He has: 1,758 posts
Joined: Jul 2002
You'd need to pay someone to write it for you. All it has to do is loop through a database of url's, read in the content of the page (html) and look for the code that you specify. Then set a flag in the database depending on whether or not it finds the code.
Easy aint it?
If you have perl or php enabled on your server and you have funds available I know a few people that could write this kind of script for you for a small fee.
Andy
james.thornton posted this at 16:50 — 4th February 2005.
They have: 42 posts
Joined: Jan 2005
Okay then. I was sure it was possible. I've got PHP and everything, so please point me to somebody who could help me out.
Thanks.
andy206uk posted this at 16:57 — 4th February 2005.
He has: 1,758 posts
Joined: Jul 2002
What kind of budget did you have in mind? I can reccomend different people depending on how much you have to spend.
Also, we're not allowed to discuss pricing of design and development jobs on this board due to price fixing laws so I'd appreciate it if you could drop me an email or private message so we can discuss this in private.
Andy
Want to join the discussion? Create an account or log in if you already have one. Joining is fast, free and painless! We’ll even whisk you back here when you’ve finished.