Choice of approach

[fikkaud]

Hi,

I'd like some input on how to approach the following.

An application logs to a daily.csv file. It grows to about 20Mb, 5000lines, 10 columns a day.

 

I'd like to keep a MySql database of the items referenced in the csv. Mostly the data needing updating from the csv is a timestamp 'last recorded activity'

The other columns from csv rarely change and needs only 'on update', ie stuff moved to another physical location.

 

I've made one attempt in php running the whole csv line by line and updating every night. It's slow and does take 5-10mins(MyISAM). It involves a lot of queries ie. is CSVtimestamp > DBtimestamp for each line.

Would I be better off creating an array in php of the whole csv and organize it prior to the update statement?

 

Or should I go for a total different approach updating mysql on shorter intervals, even realtime?

(probably the solution lies in stuff I never knew about  )

 

Any suggestions is appreciated

regards

fikkaud