Todd's dashboard

Todd replied on November 24, 2009 04:45 to the problem "Unable to install cloudera-desktop." in Cloudera:

Hi everyone,

We've just pushed a new release of Cloudera Desktop (v0.3.0) which supports CentOS/RHEL 5 without any special python versions.

http://archive.cloudera.com/desktop/

Thanks
-Todd

A comment on the problem "Unable to install cloudera-desktop." in Cloudera:

Yep, RHEL 5 and CentOS 5 are compatible (we actually build on CentOS) – Todd, on November 23, 2009 20:05

Todd replied on November 23, 2009 16:44 to the problem "Unable to install cloudera-desktop." in Cloudera:

Hi Michael,

We will have an RPM available later today that should work on RedHat 5 (with Python 2.4) with no extra monkeying around. Be sure to sign up for the announcement list I linked above. If you shoot me an email offline in the meantime (todd at the domain you expect .com) I can probably get you a copy early if you'd like to give it a try.

Thanks
-Todd

Todd replied on November 23, 2009 16:05 to the question "can't import table" in Cloudera:

Hi Valentina,

Thanks for opening the JIRA. I noticed you're already watching it, so you should get updates as we work on it and eventually produce a patch.

The primary author of Sqoop is out of town at the moment, so it may be a week or two. Is this blocking you from getting work done? If so I can try to take a look, as I imagine it should be a pretty quick fix. Or, if you're a Java programmer yourself, feel free to take a crack at submitting a patch - we're always happy to have more Hadoop contributors!

-Todd

Todd replied on November 20, 2009 23:47 to the question "can't import table" in Cloudera:

Hi Kroshilina,

Regarding the error importing a table without primary keys, the issue is that Sqoop relies on the primary key in order to split the table into chunks to do the parallel import. I believe you can pick any non-primary key (preferably one that's indexed) and use --order-by to specify that, as the error suggests. That should allow Sqoop to generate the table. Assumedly this would be the case for your m2m mapping table - you'll have indexes on at least one of the two columns.

Regarding the 0 timestamp, I believe this is a bug. Would you mind submitting this bug report to the Apache "MAPREDUCE" JIRA at http://issues.apache.org/

Thanks
-Todd

Todd replied on November 17, 2009 07:05 to the problem "Stability issues on CDH2 with AWS." in Cloudera:

Hmm, that's certainly very strange. Can we sync up on IRC tomorrow, #cloudera, and I'll see what I can figure out?

Todd replied on November 17, 2009 06:55 to the problem "Stability issues on CDH2 with AWS." in Cloudera:

Hey Bradford,

Are you running these commands from your local box or from one of the nodes in the cluster?

If you're running from the local box, is it possible your IP address is changing around on your machine? If you have ElasticFox handy, you can verify that you've got your current IP address granted access in the your-cluster-master security group.

-Todd

Todd replied on November 17, 2009 00:18 to the question "Need help with installing Hadoop cluster on EC2" in Cloudera:

Hi,

1) Yes, you should most likely install the scripts and their dependencies on your local machine.

2) No, the idea is to follow the example and create a section for your cluster. You give it whatever name you like at this point, and then when you want to launch it, simply specify the same name in the command line at that time.

Hope that helps
-Todd

Todd replied on November 16, 2009 23:01 to the problem "thrift plugins not activated." in Cloudera:

Hi Desai,

Can you verify that you are running Cloudera's Distribution of Hadoop, and which version? The vanilla Apache Hadoop does not include certain plugin hooks which we rely on for the Cloudera Desktop integration.

-Todd

Todd replied on November 16, 2009 22:47 to the problem "thrift plugins not activated." in Cloudera:

Hi Desai,

Are you certain that the conf directories in which you've made the edits are the same conf directories being loaded by Hadoop? One simple way to ensure this is to introduce some malformed XML into the file, and try to restart the daemon. If it restarts correctly, you know you're editing a file in the wrong config directory. For example, just write "asdfsdfa" at the top of mapred-site.xml.

Thanks
-Todd

Todd replied on November 16, 2009 21:44 to the problem "Cloudera-desktop FS error" in Cloudera:

Hi Sagie,

This is difficult to diagnose over this forum. It seems to me like the version of hadoop running on your Cloudera Desktop server does not match the version of Hadoop running on your namenode. Since we've already verified the hadoop_bin setting, I'm not sure what's going on.

Would you mind popping by #cloudera on irc.freenode.net ? We may have better luck trying to diagnose your problem in a realtime conversation.

Thanks
-Todd

Todd replied on November 16, 2009 21:40 to the question "Reduce is very slow with "Too many fetch-failures"" in Cloudera:

Hi Son,

This problem is usually caused by the inability of the tasks on one nodes to access the tasktrackers running on other nodes.

Please check to make sure you have no firewalls running on your cluster.

Also be sure that your DNS setup is "sane". By this, I mean that if you take the IP of a machine and do reverse DNS to get a hostname, then forward-lookup that hostname back to an IP, it matches what you started with. This should work for all machines from every machine.

Thanks
-Todd

Todd replied on November 12, 2009 19:26 to the question "Running "grep" example in Cloudera Desktop using the JobDesigner" in Cloudera:

Hi Jiksoo,

How have you entered the parameters for your job?

My guess is that you've entered the pattern parameter including the single-quote characters above. When you run the command on from your shell, the single quotes tell the shell not to interpolate values inside the parameter. When using Desktop, however, we don't run your command line through a shell, so it's passing the literal quote characters as part of the pattern to the grep program.

Does this make sense?

-Todd

Todd replied on November 11, 2009 01:12 to the question "Running "grep" example in Cloudera Desktop using the JobDesigner" in Cloudera:

Hi Jiksoo,

It's possible we have some kind of bug with escaping or something.

We'll try to reproduce this bug here and let you know what we find.

Thanks
-Todd

Todd replied on November 11, 2009 00:48 to the problem "Unable to install cloudera-desktop." in Cloudera:

Hi Linh,

Unfortunately, I'm not entirely sure what's going on on your system.

I'm happy to report, though, that we will have a release out in the coming weeks that will support CentOS 5 natively on python 2.4 without any additional installation or mucking about.

Please be sure to subscribe to our announcement mailing list by emailing desktop-announce-subscribe@cloudera.com so that you hear when this release hits the ground.

Thanks
-Todd

Todd replied on November 10, 2009 01:11 to the problem "Unable to install cloudera-desktop." in Cloudera:

Linh: sounds like the RPM install actually failed, if that file is not present. When you installed the RPM, did it print any errors?

Todd replied on November 09, 2009 18:25 to the question "Monitoring items of Cluster Health Dashboard puzzled me" in Cloudera:

Hi,

Is it possible that you momentarily ran a tasktracker and datanode on that machine? If there was a point in time at which you were running these daemons, they will continue to exist in the health database; unfortunately we don't have a good way of removing these "zombie hosts" from the UI. The workaround is to do the following command to complete reset Desktop (NOTE: this will remove any job designs you've saved!):

$ sudo su -s /bin/bash cdesktop -c "rm -f /usr/share/cloudera-desktop/desktop.db ; /usr/share/cloudera-desktop/bin/desktop syncdb --noinput"

We have tested Desktop on clusters with a setup like yours and not seen this before.

-Todd

Todd replied on November 06, 2009 22:08 to the question "How to include third-party libraries in a MapReduce job?" in Cloudera:

Hi Ryan,

It sounds like you're passing an invalid argument to -libjars (pointing to a file that doesn't exist).

Can you paste the full command you're running, along with an "ls" in the directory where you're running it?

Also, I'm sort of confused why you are trying to add jars when running a streaming job with a python mapper. Are you using a custom inputformat?

-Todd

Todd replied on November 06, 2009 15:17 to the problem "Unable to install cloudera-desktop." in Cloudera:

Hi Jiksoo,

Yes, this is a good workaround for installing from the tar. I was confused and thought you were using the RPM, where we've never seen this issue before.

Glad you're up and running, now

-Todd

Todd replied on November 06, 2009 01:03 to the problem "Unable to install cloudera-desktop." in Cloudera:

Hi Jiksoo,

Thanks for the bug report,

Which distribution are you running? We've seen some errors like this during development, and want to recreate your environment in order to reliably track down the issue.

Linh: you can download RPMs from http://archive.cloudera.com/redhat/cd...

-Todd