How much should I charge for this? - WickedFire - Affiliate Marketing Forum

alexb · 08-18-2008, 06:30 PM

Let's say that someone wants me to process a list of 125,000 domains and return the list with each domain's page rank and alexa rank. Let's say that I'm writing a script that can do this. How much should I charge for this service? I'm thinking price per thousand domains.

erect · 08-18-2008, 07:00 PM

Big ?: How long is the turnaround time?

You need to stay under the radar. It will take your server 3 days to cycle through 125k pages if doing 1 scrape every 2 seconds. I'm thinking you'll get your IP blocked every few hours if that's the case, but I can't tell you what the limit is because I honestly don't know.

Once your IP gets blocked by either service, there will be some additional hard costs involved on your part (new IP|domain|server).

If you have the PR & alexa scraping scripts already I'd probably go $3 per 1000 turned around in 7 days. I certainly would not do this for $3 if it's just 1000 sites but since you have a big number to work with...

Hooking this up wouldn't take me an hour or so worth of work, but if you don't have the functions in place you are looking at a much more labor intensive project.

micker · 08-18-2008, 07:14 PM

rather than deal with new IP's build in the AWS query fees in to the price.

xmcp123 · 08-18-2008, 07:18 PM

Quote:

Originally Posted by erect

Big ?: How long is the turnaround time?

You need to stay under the radar. It will take your server 3 days to cycle through 125k pages if doing 1 scrape every 2 seconds. I'm thinking you'll get your IP blocked every few hours if that's the case, but I can't tell you what the limit is because I honestly don't know.

Once your IP gets blocked by either service, there will be some additional hard costs involved on your part (new IP|domain|server).

If you have the PR & alexa scraping scripts already I'd probably go $3 per 1000 turned around in 7 days. I certainly would not do this for $3 if it's just 1000 sites but since you have a big number to work with...

Hooking this up wouldn't take me an hour or so worth of work, but if you don't have the functions in place you are looking at a much more labor intensive project.

sign up at 20 freehosts that allow PHP, and add a script to check pagerank onto each one.

erect · 08-18-2008, 07:25 PM

Quote:

Originally Posted by xmcp123

sign up at 20 freehosts that allow PHP, and add a script to check pagerank onto each one.

I've got about 10 "satellite" domains on cheap ass servers because I don't like to waste time with free hosts (some don't allow curl, some are already blocked, some are slow as piss, etc) $5/year hosts are not much better but not killed in the same fashion that free ones are.

I put a bunch of scraping scripts on each of them and just call randomly. It works pretty well for a $100 / year setup. This allows me to scrape whatever I want to and never expose my money servers.

Quick question, when scraping google (SERPs or PR) what's your experience as to how long in between queries you need to delay in order to not get banned? I've never had a problem, but I've also never hit 125k in a few hours.

xmcp123 · 08-18-2008, 08:12 PM

Quote:

Originally Posted by erect

I've got about 10 "satellite" domains on cheap ass servers because I don't like to waste time with free hosts (some don't allow curl, some are already blocked, some are slow as piss, etc) $5/year hosts are not much better but not killed in the same fashion that free ones are.

I put a bunch of scraping scripts on each of them and just call randomly. It works pretty well for a $100 / year setup. This allows me to scrape whatever I want to and never expose my money servers.

Quick question, when scraping google (SERPs or PR) what's your experience as to how long in between queries you need to delay in order to not get banned? I've never had a problem, but I've also never hit 125k in a few hours.

It depends. a 2 second seperation I've rarely(or never I think) gotten banned. 1 second seperation I get banned, but after a decent number of queries.
Anything less than that and it's quite fast.

08-18-2008, 07:00 PM	#2 (permalink)
erect Join Date: Jun 2007 Location: orbiting condo Posts: 3,188 iTrader: 15 / 94%	Big ?: How long is the turnaround time? You need to stay under the radar. It will take your server 3 days to cycle through 125k pages if doing 1 scrape every 2 seconds. I'm thinking you'll get your IP blocked every few hours if that's the case, but I can't tell you what the limit is because I honestly don't know. Once your IP gets blocked by either service, there will be some additional hard costs involved on your part (new IP\|domain\|server). If you have the PR & alexa scraping scripts already I'd probably go $3 per 1000 turned around in 7 days. I certainly would not do this for $3 if it's just 1000 sites but since you have a big number to work with... Hooking this up wouldn't take me an hour or so worth of work, but if you don't have the functions in place you are looking at a much more labor intensive project. Share Share this post on Digg Del.icio.us Technorati Twitter

08-18-2008, 07:14 PM	#3 (permalink)
micker Pretender to the throne Join Date: Oct 2006 Location: Detroit Posts: 318 iTrader: 0 / 0%	rather than deal with new IP's build in the AWS query fees in to the price. Share Share this post on Digg Del.icio.us Technorati Twitter

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
What to charge for this?	Enigmabomb	Shooting The Shit	6	07-26-2008 10:49 PM
How much should I charge for advertising on my podcast?	alexb	Shooting The Shit	6	05-06-2008 09:25 PM
I need CPA’s that charge a text message for Greece	glowleaf	Affiliate Marketing	0	04-14-2008 05:15 AM
uk credit cards that don't charge to purchase in foreign currency	s1mplygo1d	Shooting The Shit	4	03-19-2008 08:21 AM
How much should I charge for this?	alexb	Newbie Questions	3	03-07-2008 12:03 AM