Why would you ever throw

Shaggy — Mon, 07 Dec 2009 15:14:40 +0000

Why would you ever throw data away? (Says the data packrat)
Normalisation will help with your storage size. A well placed index or three will keep you collecting data for a long, long time before more drastic measures need to be taken. Hundreds of millions of rows aren't out of reach on basic hardware.

If / when data volume become a problem, partition it off. Move it off to a different tablespace, or even a different database. There will come a day you'll wish you'd kept it.

Cheers,
Shaggy.

Hello teammatt3. Yes you are

benf — Sun, 06 Dec 2009 13:41:41 +0000

Hello teammatt3.
Yes you are right. I was not aware of denormalization looks interesting. But This is also what I thought to only keep the data in the database for no more than 2 years. It is better to keep the statistics from the year before so you can see trends in banner delivery. But only keeping the past 2 years keeps the volume down.

Thanks for the advice and if you can give me any more advice would be appreciated.

Is your ad server going to

teammatt3 — Sun, 06 Dec 2009 05:52:59 +0000

Is your ad server going to be serving 10,000 banners at a time? Be realistic. Everyone thinks their application is going to take off, and few ever do.

873,600 rows isn't a lot. Most of your data is numeric, and won't take up a lot of space. It can also be sorted and retrieved faster than textual data. 8 million records isn't bad at all either.

The data probably doesn't need to be stored for more than a year. Does a person really need to to see the IP address of someone who clicked a banner ad 13 months ago? Probably not.

I would highly recommend starting off with a normalized database, and if scalability becomes a problem, buy better hardware. And if the hardware doesn't fix the problem, then start caching, purging, and denormalizing your database.