Thread: Best Navigation Architecture For Domain Database?

I've got a database here with the hosting history (new/deleted/transferred/dns added removed etc) for domains going back to 2000. In individual terms, it would be approximately 190 Million pages of content. Any ideas on how to build a navigation architecture (other than a search box)?

Regards...jmcc

Its down to what you want to be able to do?

Theres some nifty data mining architectures, data visualisation tools and navigation methods out there but you really need to start off with some idea of how you want to slice and dice the data.

What is you want ppl to do with the data? Find specifics?Carry out trend analysis? Visualise the data?

Originally Posted by Briask

Its down to what you want to be able to do?

Provide easily accessible domain hosting histories. The simplest way is to provide a search box. But I want to provide alternative searching methods and perhaps a directory structure. The directory structure is the hardest thing.

Regards...jmcc

Which TLDs are you covering?

It *might* make some sense to organise:

• by tld
• alphabetically
• number of characters in domain
• by nameserver
• by country
• by age (not sure about that one.. )

Originally Posted by blacknight

Which TLDs are you covering?

com/net/org/biz/info/eu/co.uk/mobi/asia for the moment.

by tld

This is the one I've implemented already.

alphabetically

Each list would be millions of domains long.[/quote]

number of characters in domain

That's a good one.

by nameserver

Could be tricky - there are loads of nameservers with one domain hosted but there are also nameservers with millions of domains hosted. The history of Godaddy's or Enom's nameservers is amazing.

by country

Can be done. But it might not be 100% accurate.

by age (not sure about that one.. )

This one is a very interesting option. It would be possible to generate complete lists of domains that were dropped and not reregistered. It would also be possible to identify tasted domains and domain tasters. It could be done on a monthly or yearly basis.

Regards...jmcc

alphabetically ... could be broken down further into ..

a -> aa ... ab ... ac .... th -> the...

listing them alphabetically with just the initial letter wouldn't make too much sense to me ... (unless it was something like dropped irish domains)

Also .. if you start getting spidered 190 million page hits could be ouch ... not to mention bots potentially collecting all the data from your system (though you've probably thought of that already)

Originally Posted by Forbairt

alphabetically ... could be broken down further into ..

a -> aa ... ab ... ac .... th -> the...

listing them alphabetically with just the initial letter wouldn't make too much sense to me ... (unless it was something like dropped irish domains)

Yep. This would narrow it down a bit.

Also .. if you start getting spidered 190 million page hits could be ouch ... not to mention bots potentially collecting all the data from your system (though you've probably thought of that already)

It gives me nightmares. Google is ok. But the morons in Microsoft and Yahoo with their multitude of braindead spiders would be continually hammering the sites. As for the scrapers, I have no problem with banning entire countries or Class A if they irritate me. The trick is maintaining a healthy balance between giving genuine users the info they want and blocking maggots.

Regards...jmcc

Originally Posted by jmcc

The trick is maintaining a healthy balance between giving genuine users the info they want and blocking maggots.

Given the nature of this ... would requesting people sign up be that big an issue ?

Considering a lot of sites are charging for this type of information ... a simple sign up wouldn't be out of the question in my mind

(remember you posting that you weren't doing it as a commercial venture ... as such on boards)

Originally Posted by Forbairt

Given the nature of this ... would requesting people sign up be that big an issue ?

It would be an extra layer of complexity. What I want to do is to make it as simple as possible to implement and run.

Considering a lot of sites are charging for this type of information ... a simple sign up wouldn't be out of the question in my mind

Possibly but only for the more intricate information. Sessions, users, authentication in php scares me.

(remember you posting that you weren't doing it as a commercial venture ... as such on boards)

I'm trying to remember writing that.

Regards...jmcc