What's the best practice of loading logs into hdfs while using hive to do log analytic?
Subject What's the best practice of loading logs into hdfs while using hive to do log analytic?
Sorry if it is not appropriate to send one thread into two maillist.
I'm tring to use hadoop and hive to do some log analytic jobs.
And we want to use hadoop and hive to replace our old log analysic system.
collect logs from clients and then generate log files.
test which seems good for us.
hdfs or hive warehouse dir ?
these compressed log files into hdfs.
TABLE $tablename PARTITION (dt='$h') "
analysis job to start later.
mailist, and from what I have learned, this is not possible.
Or what's the common methods to handle problem I have describe above?
Can anyone give me some advices?