| |
06-02-2008, 01:56 AM
| |||||
| |||||
 Are you using the vendors' feeds?
__________________ Hosting & Domains|Plesk Vps Hosting|Blog Tips|Films.ie|Gadgets|Monetisation Tips|Movie Chat Energise your forum! Click here for info  |
|
06-02-2008, 02:52 PM
| ||||
| ||||
| Nope .... the pricefindr bot crawls the websites and when it is refreshing the pricing information it notes changes in price from the last time it crawled the site. |
|
06-02-2008, 02:56 PM
| |||||
| |||||
| Most of the vendors you list provide feeds. I'd strongly recommend you look at using them, as scraping won't scale
__________________ Hosting & Domains|Plesk Vps Hosting|Blog Tips|Films.ie|Gadgets|Monetisation Tips|Movie Chat Energise your forum! Click here for info |
|
06-02-2008, 05:10 PM
| ||||
| ||||
| Thanks ...yeah agree for the major sites listed feeds would definitley be an easier option. Been trying out a more generic approach that I can adopt to other websites that don't supply feeds. Been using an open source project called nutch for the crawling and so far it's been scaling pretty well. Have set up a small two node cluster it uses the map/reduce algorithm to carry out the work. Haven't really pushed it might try adding some more sites and see how it gets on. Been thinking of adding DVDs in.. |
|
19-06-2008, 09:18 PM
| ||||
| ||||
| pricefindr.ie updated. Hi Guys, I have updated pricefindr.ie having added in some new verticals for DVD's and Games. The websites UI has been updated slightly but the main improvements has been back-end here I have moved the processing and number crunching into the cloud using amazon's EC2 to allow for future expansion. The index as it currently stands has 75,467 products in it. I'm currently looking to add additional sites to the pricefindr index so any you would recommend or suggest let me know. Any feedback on the site its look and feel etc would be greatly welcome as UI design isn't my strength Thanks |
|
21-06-2008, 04:08 AM
| ||||
| ||||
| Would you not pay someone to do some UI work for you? Even a student or a rentacoder or something. There's no nice way of saying the site looks pretty bad. I mean if you can't fire $30 for a template at the site, then, to quote a favourite phrase of mine, "you're at nothin'"! The idea could defeinitely work (though you'll need to think about how it pays for itself!), but jeepers get something done with the UI!!
__________________ URL.ie - Free URL truncation/shortening service. |
Linear Mode
