Beating Google ....

cheshire

cheshire

Señor Member
May 25, 2009
2,249
57
0
Tejas
So I'm killing time waiting on a new campaign to fire up and I started thinking about beating Google. People get pissed at them all the time but the fact of the matter is Google consistently provides the best results. The name Google is now part of the English language (Google it!) and Bing just sounds retarded, and Yahoo is dying the slow death. Google has a strangle hold on things with no sign of letting go.

So how does anyone take on a monster like Google? Open source.

It worked for the many Linux companies (RedHat, Ubuntu). So why not an Opensource search engine? Something people can download and install on their own server to run their own search engine?

I think this would have two big benefits:

Decentralizing search (making it easier for us to pay for ads on different search engines)

An algorithm that isn't a black box and can prove that it is truly un-biased ranking algorithm.

Each site could customize ratios to provide different results on each search engine.

I can see one con:

Access to code allows spammers/SEO's to exploit the shit out of it (but allowing each installation to tweak ratios would make it impossible to "optimize" for all search engines)

I'm positive this wouldn't instantly dethrone Google but it might allow some start ups to chip away at that market share and give us more search advertising options.

To be honest I can't believe this hasn't been done before. Maybe it's because all the geeks are elated with Google.. I've seen a few open source search engines like Sphinx but they're pretty much a joke. The opensource community puts so much effort into operating systems and hardware but nearly none into the access of information.

Discuss....
 


I actually like the idea of people being able to run niche search engines. ...maybe sort of like niche wikis or something. Or, as you said, people could tweak them. I think it's interesting.

I wonder how they'd rank on Google.

cheshire said:
So I'm killing time waiting on a new campaign to fire up and I started thinking about beating Google. People get pissed at them all the time but the fact of the matter is Google consistently provides the best results. The name Google is now part of the English language (Google it!) and Bing just sounds retarded, and Yahoo is dying the slow death. Google has a strangle hold on things with no sign of letting go.

So how does anyone take on a monster like Google? Open source.

It worked for the many Linux companies (RedHat, Ubuntu). So why not an Opensource search engine? Something people can download and install on their own server to run their own search engine?

I think this would have two big benefits:

Decentralizing search (making it easier for us to pay for ads on different search engines)

An algorithm that isn't a black box and can prove that it is truly un-biased ranking algorithm.

Each site could customize ratios to provide different results on each search engine.

I can see one con:

Access to code allows spammers/SEO's to exploit the shit out of it (but allowing each installation to tweak ratios would make it impossible to "optimize" for all search engines)

I'm positive this wouldn't instantly dethrone Google but it might allow some start ups to chip away at that market share and give us more search advertising options.

To be honest I can't believe this hasn't been done before. Maybe it's because all the geeks are elated with Google.. I've seen a few open source search engines like Sphinx but they're pretty much a joke. The opensource community puts so much effort into operating systems and hardware but nearly none into the access of information.

Discuss....
Click to expand...
 
2088451576_9d01774310.jpg
 
cheshire said:
S

An algorithm that isn't a black box and can prove that it is truly un-biased ranking algorithm.
Click to expand...

The whole point of google's algo (and google itself) is to provide biased search results. The vast majority of the time, their bias is very accurate. If the algo was free to analyze, it would be getting destroyed by spammers. Their bias is what makes Google the best.

99.9999% of google users probably can't ever tell the difference in results when google makes updates. Just because SEO webmasters are in an uproar doesn't mean everyone else cares, we ourselves are naturally biased to care about the minutiae that the populace at large doesn't care about.

The amount of engineering alone that went into making the algo what it is today would not emerge from an open source effort. Google has had thousands of PhD level engineers working on it for more than 10 years now. Not to mention the fact that the incentive for them to build a good algorithm as the backbone of their search engine is the fact that they profit out the ass from the ads they slap onto their SERPs.

I just don't think there's enough energy and enthusiasm out there to fuel such a project at the scale that Google is currently. Literally no other company in the world can crawl the internet as fast and comprehensively as they can. Plus, they have one of the largest server backbones in the world supporting their index, something that isn't feasible via decentralized machines a la SETI@Home.

Financial incentive is what drives search engine development. It always has.
 
dchuk said:
The whole point of google's algo (and google itself) is to provide biased search results. The vast majority of the time, their bias is very accurate. If the algo was free to analyze, it would be getting destroyed by spammers. Their bias is what makes Google the best.
Click to expand...

Perhaps, all search engines are going to be biased to some degree.

99.9999% of google users probably can't ever tell the difference in results when google makes updates. Just because SEO webmasters are in an uproar doesn't mean everyone else cares, we ourselves are naturally biased to care about the minutiae that the populace at large doesn't care about.
Click to expand...

My mom does, my dad does... I know because they've asked me things like "Google isn't giving me good links anymore, is there something better you geeks are using" or "I can't find anything I want on Google anymore" or "How do I turn off these giant pictures of the websites from popping up"

The amount of engineering alone that went into making the algo what it is today would not emerge from an open source effort. Google has had thousands of PhD level engineers working on it for more than 10 years now. Not to mention the fact that the incentive for them to build a good algorithm as the backbone of their search engine is the fact that they profit out the ass from the ads they slap onto their SERPs.
Click to expand...

That can be said for operating systems too. I mean Microsoft has been working on Windows Server for far longer and yet Linux came along and stole the show. I'm not saying something that kills Google would appear overnight... but slow and steady wins the race.

Their incentive is profit, plain and simple and if that means they can tweak the algorithm, or massage results here and there to make them more profit then they will do it. When a single company is providing the paid results and the "natural results" it's always going to tip in favor of a paid ad. I'm sure they have an entire team of PhD's just for that purpose.

I just don't think there's enough energy and enthusiasm out there to fuel such a project at the scale that Google is currently. Literally no other company in the world can crawl the internet as fast and comprehensively as they can. Plus, they have one of the largest server backbones in the world supporting their index, something that isn't feasible via decentralized machines a la SETI@Home.
Click to expand...

Perhaps, there isn't enough enthusiasm out there. Social is all the rage these days but I see that drying up in the distant future and/or decentralizing away from Facebook.

Financial incentive is what drives search engine development. It always has.
Click to expand...

Pretty much drives all markets yet some how Open source companies like Redhat and others have managed to profit from free software.
 
cheshire said:
Something people can download and install on their own server to run their own search engine?
Click to expand...

something like this maybe?

There is one that Leo Laporte mentioned a week back on a cast, but cant really remember which one. It's really on the tip of my tongue, as I've heard it before.

I found only this one unfortunately so far.

PS: Also, amazon is huge in search, it's just not implemented to do web crawls.
 
nickycakes said:
Your open-source search engine has about as much chance to dethrone google as linux has at dethroning windows/osx as a desktop os.
Click to expand...

Unfortunately, I have to agree with nickyCakes on this one. Yes, Linux runs online servers, but there only biggest achievement came after more than a decade of a no one cares war. I'm on the linux side, mind you. The main reason I switch to OSX was that it was finally fully UNIX, making it a direct relative to linux, so yeah, I can have my linux too, and have a nice interface, without all the hacking this and that, just to get a fucking internet connection... I recall PHLAK... that shit was the good old days, but then I decided to grow up and make monies online....
 
What's the point? I mean, Google owns search, move on to the next big thing... A mobile "something" has more potential to becoming the next big thing. It's like me saying... hey I don't like Windows anymore, and OSX is too sexy, let start another operating system.... At this late in the game. Google won Search, like Microsoft won Operatinging Systems, like IBM won at MainFrame computers, and Dell won at Personal Crappy Computers... Facebook is currently the social media king, and unlikely to be dethroned... Let's come up with a new idea that more innovative, and moves forward, and not backward.

P.S. http://www.sphider.eu/ [You can create your own search engine, complete open source, and allow it to search beyond your network of websites, and it's very accurate]
 
I don't think you realize how many hardware resources go into a search engine. Hope you have deep (VERY deep) pockets. There's a bit more to it than just installing some open-source PHP script on your dedicated box.

Just a tiny example of what Google does:

[ame="http://www.youtube.com/watch?v=VChOEvKicQQ"]Google's Hamina Data Center - YouTube[/ame]

Have fun competing!
 
The original two founders also didn't have much money when starting out. They approached a number of investors for funding. And finally went IPO and raised a lot of cash to afford buying servers like in the video.

Kiopa_Matt said:
I don't think you realize how many hardware resources go into a search engine. Hope you have deep (VERY deep) pockets. There's a bit more to it than just installing some open-source PHP script on your dedicated box.

Just a tiny example of what Google does:

Google's Hamina Data Center - YouTube

Have fun competing!
Click to expand...
 
Further, Google has been beaten by:

1) Baidu in China
2) Yandex in Russia
3) Naver in South Korea

There's a Chinese proverbs "Those that say it cannot be done, should not disturb the person doing it." My take is those who say it is impossible should not discourage those that are doing it.
 
  • Like
Reactions: Lalit_Burma
Except he isn't doing it. He isn't doing anything at all except suggesting that someone else do it.
 
You must log in or register to reply here.
Top Bottom