Suggestions for Home Rolled Tracking System

Status
Not open for further replies.
xmcp123

xmcp123

Well-known member
Sep 20, 2007
4,145
95
48
Not Louisiana
www.slightlyshadyseo.com
Alright. So I've decided I'm officially annoyed with 3rd party tracking in general, and my current home rolled system is admittedly not up to the standards I would like.
I've been working on a list of things that I want to have in it, and was wondering if any WFers can think of other things I should add, or things that would become a problem down the road.
Or better yet if anyone knows something already like this other than tracking202, and not owned by Google.
  • Tracking of Keyword/Adgroup/Referrer/Campaign/Ad (obviously)
  • Tracking pixels or unique IDs similar to tracking202's import style.
  • Detection of certain keywords that are LEAST likely to lead to a conversion.
  • Recording bounce rates for landing pages, broken up by adgroup and traffic source
  • Built in cloaking(not for hiding keywords, but for optimizing landing pages behind the scenes)
  • Bot detection so they don't count in the stats.
  • Important: Ads and adgroups and such do not need to be MADE(like tracking202). They are visited, and exist forever after. Only campaigns/LPs are formally created.
  • After the url gets loaded with all the ugly variables, it is immediately recorded and refreshed to clear out the nastiness. (to an identical page, minus the ugliness)
So that's what I'm thinking so far. Any oversights/suggestions?
 


Tracking202 has a great system, just it is hosted somewhere else (where downtimes and prying eyes can get in the way). I would go for a script with similar features.

Also if/when such a project gets finished, give (or sell) us a self hosted version!
 
I made my own very basic tracking script a few weeks ago for yahoo/google. I was thinking about selling it as a self-hosted script but I couldn't come up with a way to keep it from just getting passed around the forum once one person bought it.
 
this is something i've been working on for a little while. it's no where near finished, but i'm using it while i continue to work on it.

at this point, i've got my tracking centralized at one domain and add a tracking pixel & an 'include' or module to my landing page/campaign site - which is kinda useless right now cause i haven't finished the reporting. this is also where i create my campaigns then export them to adwords etc. it's not really for managing the campaign, just launching them and tracking keywords > adgroups > campaigns. i only pass an internal keywords id as a subid to the networks. will also ad more keyword tools etc.
i'm just logging everything i can, and will let cron jobs reconcile all that data into reports, pushing the heaving lifting away from the landing page.

i've written it so far so i can manage all split/multivariate testing from the same place as well, but haven't fully implemented it.
bounce rate is something i hadn't thought of.
 
I just wrote something for myself over the weekend.....some of the stuff you mentioned is interesting. One thing I like is a page I made with all the raw search queries scraped out from the referrer information and listed along with the matched keywords from my campaign, meaning the keywords in adwords or ysm that triggered my search ad.

Also, bot detection is such a pain in the ass! My biggest issue so far for sure. It doesn't seem that bad when your looking at the user agent string and you see stuff like AdsBot-Google or whatever it is, but then you also notice that a lot of their requests are also from regular looking Mozilla user agent strings! Probably to make sure your not cloaking or something. Definitely can screw up your stats for sure.

I don't see the ugly variable stuff being an issue...but you could just point your links to the redirect url, that collects all this info, sets your cookies or whatever, and then redirects to a nice clean url without any variables in it. I never thought the url stuff would make a difference tho....maybe I'm wrong.

This might be a little unnecessary, but someone suggested to me that I add something to log the time that the user hits the site and then the time that the user converted, so you can see how long it took before the conversion. Right now I'm just logging the time the user hits the site to get an idea of which times bring the most conversions.

Good thread!
 
cardine said:
Tracking202 has a great system, just it is hosted somewhere else (where downtimes and prying eyes can get in the way). I would go for a script with similar features.

Also if/when such a project gets finished, give (or sell) us a self hosted version!
Click to expand...
I like tracking202, it's just obnoxious having to copy my ads in that way. If I want to try a lot of ads at once, it's much slower without dynamic creation. And yeah, once again with the remote hosting. I've heard nothing to make me especially suspicious, but the whole premise...I'd actually feel more comfortable if they had an evident revenue model on it other than "We'll work on it later".

And yup, I'm going to consider doing that. Although honestly, making pages "pretty" is not my thing. Nice graphs and such? Yeahhh. If people are fine with a simple, functional stat tracker with tables, then I may sell it or pass it around. Perhaps I'll leave the databases of clicks on your servers, and just have mine do the statistics processing, or maintain the IDs of the ads and such. That way I can control distribution without actually passing significant data.
phil9922 said:
I made my own very basic tracking script a few weeks ago for yahoo/google. I was thinking about selling it as a self-hosted script but I couldn't come up with a way to keep it from just getting passed around the forum once one person bought it.
Click to expand...
ZEND encoding or something may work. Just one of the optimizers. Or keep essential components on your own server, without too much data passing.
weneedsound said:
A stats update script, run by a cron job and text'd messaged to your phone?
Click to expand...
Ah! I've thought of that in the past, but totally blanked on it! Sounds excellent. I'm also thinking of including a super basic output of the results so someone can access it easily on an iphone and such. For people like me with a 12+ character password including numbers and symbols(go ahead, try and brute force it fishers), typing that in for all the affiliate networks is a tremendous hassle.

Keep the ideas coming!
but I'd also be interested to hear what people would be ok with and not ok with having on my server rather than yours.
 
natekapi said:
I just wrote something for myself over the weekend.....some of the stuff you mentioned is interesting. One thing I like is a page I made with all the raw search queries scraped out from the referrer information and listed along with the matched keywords from my campaign, meaning the keywords in adwords or ysm that triggered my search ad.
Click to expand...
Easily doable. I actually have a script done to do that more or less coded up. Although I'd have to adapt it for the various "search network" places.
Also, bot detection is such a pain in the ass! My biggest issue so far for sure. It doesn't seem that bad when your looking at the user agent string and you see stuff like AdsBot-Google or whatever it is, but then you also notice that a lot of their requests are also from regular looking Mozilla user agent strings! Probably to make sure your not cloaking or something. Definitely can screw up your stats for sure.
Click to expand...
Note the blackhat in the avatar. I'm pretty good at detecting bots. Actually, I'm going to have to be careful to ensure that people don't try and use however I detect them for cloaking. Call it a trade secret ;)
I don't see the ugly variable stuff being an issue...but you could just point your links to the redirect url, that collects all this info, sets your cookies or whatever, and then redirects to a nice clean url without any variables in it. I never thought the url stuff would make a difference tho....maybe I'm wrong.
Click to expand...
Probably doesn't make too much of a difference, but enough.

Once again, keep the ideas and such coming.
Also whatever you're ok with me having on my server, and what you're not.
This might be a little unnecessary, but someone suggested to me that I add something to log the time that the user hits the site and then the time that the user converted, so you can see how long it took before the conversion. Right now I'm just logging the time the user hits the site to get an idea of which times bring the most conversions.

Good thread![/quote]
 
Haha. Well help me out with my bot filtering cause it's pissing me off :)

One thing I'm doing is looking for direct hits to my site with all the variables in the url and no referrer, since no one would ever type my adwords destination url in directly, and filtering based on that too.
 
natekapi said:
Haha. Well help me out with my bot filtering cause it's pissing me off :)

One thing I'm doing is looking for direct hits to my site with all the variables in the url and no referrer, since no one would ever type my adwords destination url in directly, and filtering based on that too.
Click to expand...
Hint because I'm feeling zesty today: Use that as a way to determine how much resources you want to use on that IP. If there's no ref, reversedns, insert into database if it checks out. Although more and more google crawlers have no reverse DNS.
 
beejeebers said:
I just finished mine, the bot thing is a pain in the ass, but mine does almost everything on your wish list...it took more then a weekend to write too... :)
Click to expand...
Ah, nice!
I'm not expecting it to take a weekend or anything. I could probably do it in a week if I neglected other stuff, but this is going to be a pretty decent length project.
Especially if I release it to everyone and have to idiot proof it(not insinuating yall are idiots...but you know. There's a few in every bunch)
 
The hardest part is the beta. So many different server configurations to think about. I've gone through about 10 revisions in the past 2 days just because host a is totally different from host b. But I think the key is being able to host it on your own server.

No matter who you are or how good a guy you are, that just kills it. There is way too much money at risk to have your info "out there". I think that's where that tracking 202 script failed. The licensing is not hard to write either. Ah well...
 
feature necessary: cURL your way into copeac, cpae, etc and get those subids automagically instead of requiring copy/paste

[side note] I can't believe more affiliate companies don't ping one of your urls when a lead comes in. I saw one that did but then forgot which company it was. What a fucking feature that is easy to code (like only a few hours of work) that they all just don't include it, frustrating ...

Why not just search for bots by ip addresses as user_agents are too easy to forge or leave blank all together. If the ip doesn't check into the list, record the range and if that range hits over and over, just add it to your ip list of non-counted clicks

I totally agree with xmcp that t202 is WAY too cumbersome. I'd strip out 4 or 5 features and make the registration process of new sites just 1 page, just too much extra information. I mean shit, the links could almost be totally dynamic, just drop them in place and the system automatically adds the info it needs.

Take a look here for your reporting charts ... it's a free service (with commercial options) that allows you to just dump CSVs into their flash charts and it does a pretty good job rendering some pimp reports. Great for overviews and stuff. I don't do flash at all so I've been happy to use them in the past. It's easy to hack around and the clients tend to digg it.
 
I've been playing around with Piwik. They call themselves and open source alternative to Google analytics though they're very early in development. They do have events and a built in API though.
 
Ok, I haven't read the whole thread, but I'm going to release my article on basic link tracking here in a day or three. So when I'm sober tomorrow I may address some issues you guys have.

All you have to do is let me know what you want to accomplish and I'll let you know.

PM me ahead of time and hopefully I'll have an answer for you.

Cuz I'm cool like that.

If yall dicks, I ain't writing shit though. I would like input though on what people what to accomplish because that can only my what I write better.

oh yeah

ZEND encoding or something may work. Just one of the optimizers. Or keep essential components on your own server, without too much data passing.
Click to expand...
There is no such thing as protecting software. Period. It will get cracked and redistributed. Get over that ZEND shit.
 
Free keyword tracking

I have my free self made keyword tracking up for download on my site. ptaffiliate dot com. It does everything you guys have been describing, it free and already built. you host it yourself on one of your own domains.

all open source - if you make it better: pass it on.

cheers,
ptaffiliate.
 
Status
Not open for further replies.
Top Bottom