Loading Profile...
Recent activity
Subscribe to this feed
A comment on the question "Fuse on Cloudera" in Cloudera:
I am guessing this is the tutorial's new address:
http://developer.yahoo.com/hadoop/tut... – Andrew, on November 02, 2009 13:23-
Andrew replied on October 26, 2009 08:19 to the question "DBOutputFormat only inserts, does not update DB rows - alternatives?" in Cloudera:
Ok, mail sent... :) -
Andrew replied on October 26, 2009 06:21 to the question "DBOutputFormat only inserts, does not update DB rows - alternatives?" in Cloudera:
Hi Aaron!
I'll send you an e-mail then... I'll have to add this feature anyway because I need it, so it makes sense to do it in such a way that it makes it into core Hadoop / Sqoop.
Enjoy!
Andrew -
Andrew asked a question in Cloudera on October 25, 2009 17:38:
DBOutputFormat only inserts, does not update DB rows - alternatives?Hi!
I am trying to export the results of Hadoop processing to relational DB (MySQL in this case). I have tried using DBOutputFormat, but it seems a bit limited to me.
We are using hadoop to gather statistics and to post them to MySQL. The data flow is small enough so it wouldn't have much impact on our SQL server. However, the statistics are per-day and would also have to be updated, not just inserted (as the day goes on).
For example: we have the data that there were 2000 visits on some page today (by now), so we insert a row with this data in DB. But an hour later we need to update this row (not just insert a new one) with the new value.
Now, DBOutputFormat only allows inserts. The best solution I could find by now is to copy DBOutputFormat.java and DBConfiguration.java to my own project and change them (I can't just subclass them because of private properties and functions). But this just seems so... awkward. I am sure there must be a better way?
Can Sqoop help?
Thanks!
Andrew
