Why hasn't Microsoft bought Ska yet?

[minestrone]

I don't want or need to prove to you anything - if you choose to run your test in your own I might be able to dedicate some of my very valueable time checking the results and trying to explain why we have not picked certain links - there can be many valid reasons why it does not happen and in many of those cases why it doesn't matter.

Back in 2007/2008 when I was building first indexes I used comparison of data shown by Yahoo/Google to see how many we matched, this was done for 20 URLs belonging to domains of all sizes. We had very high match ratio especially identifying links from root domains rather than specific URLs, this does not mean it will happen for every URL - a lot depends on where links positioned and how valueable they are in grand scheme of things.

You do not have the processing power to map the internet. You can try and tell your customers that but you do not.

You are not google and you are not yahoo, you are just an immigrant who got state income.

[AtW]

You do not have the processing power to map the internet.

Well, yesterday we've crawled 1.2 bln URLs with total content size of 30 TB.

There are not many crawlers out there that do more than that regularly, probably total number is less than 10 for the whole world.

[adubya]

How do you know who's clicked on google ad links to your customers competitors if you're not google (or the competitor) ? Don't you only know that if people choose to use your search engine over google ?

(sorry if I misunderstand your business model)

[minestrone]

Well, yesterday we've crawled 1.2 bln URLs with total content size of 30 TB.

There are not many crawlers out there that do more than that regularly, probably total number is less than 10 for the whole world.

OK, take my bet then.

10 URLs.

You want to boast, I will take you boast.

You want to go for it?

[adubya]

Isn't his point that your 10 URLs will be so low volume (as in zero or thereabouts) that they just won't figure - and why should they ?

[AtW]

How do you know who's clicked on google ad links to your customers competitors if you're not google (or the competitor) ?

We are not doing that - when we find backlinks we add them to database so they can be analysed. This ain't google ad links and we have no idea if someone clicked on them, however in analysis it is possible to look at backlinks by their rank which is also based on who links to that page, after that manual review can quickly show if it's a worthwhile URL or not.

[AtW]

OK, take my bet then.

I've already replied to your proposal.

[AtW]

Isn't his point that your 10 URLs will be so low volume (as in zero or thereabouts) that they just won't figure - and why should they ?

ministrone has been going on for ages saying we never pick up links to his sites (he'll never name them so I could not check anything), now that he offered to buy 1000 links just like this for this "bet" is a sure give away that the people he buys them from charge very little (good single link can cost $100+ per month) and that means most certainly links will be in worthless places.

A lot of our customers and free users take advantage of our data to decide whether it's worth getting a link from particular domain/URL - it is easy to check if proposed URLs for those backlinks are in fact well linked in the first place, if they are then our crawlers should be recrawling them but if they got no backlinks then those are likely to be worthless pages. We'd still pick up lots of those but it takes time because we'll give priority to recrawling well linked, highly desireable URLs.

[adubya]

We are not doing that - when we find backlinks we add them to database so they can be analysed. This ain't google ad links and we have no idea if someone clicked on them, however in analysis it is possible to look at backlinks by their rank which is also based on who links to that page, after that manual review can quickly show if it's a worthwhile URL or not.

So you're tracking static links between sites not dynamic "clicks" ?

Interesting stuff.

[AtW]

So you're tracking static links between sites not dynamic "clicks" ?

Yes we track HTML links.

Well linked URLs however attract traffic to themselves and also possibly to pages they link out to and we use Alexa's traffic ranks on root domain level.