Loading Profile...
Recent activity
Subscribe to this feed
Phil Whelan asked a question in Cloudera on August 12, 2009 20:14:
Hadoop - Configuring a slave's hostnameHi,
In Hadoop, I need to configure the hostname that a slave announces itself as to the master. I'm using EC2 and currently our machines have "hostname" configured as the external hostname. We have other services on our machines that currently depend on this setup.
Since Hadoop picks the hostname up from "hostname" (which resolves as an external IP) and EC2 machines cannot inter-communicate on their external IPs, it's causing some problems.
Is there a way to configure this on each slave? ie. "My hostname is XXX.XXX" or "My hostname is X.X.X.X"?
Thanks,
Phil-
Phil Whelan replied on July 29, 2009 22:18 to the question ""hadoop-ec2-ebs-beta" - purpose of master's EBS volumes?" in Cloudera:
Hi Alex,
That's a great answer to my question! Thank you very much.
Phil -
Phil Whelan replied on July 29, 2009 21:54 to the question ""hadoop-ec2-ebs-beta" - purpose of master's EBS volumes?" in Cloudera:
Hi Alex,
Sorry, maybe wrong terminology.
I have 1 master (with 2 EBS volumes) and 2 slaves (with 2 EBS volumes each). I assumed the example intended to use 2 EBS on the master as part of the hadoop storage space, and that the master would also be a node (3 nodes in total).
I'm seeing "Live Datanodes : 2" which only lists the slaves. And the total "Capacity" is roughly total to the 4 slave EBS volumes (rather than 6).
My question has two parts...
1) Can the master also be a node and add to the cluster capacity (even if it's a single core small ec2 instance)?
2) If not, why does the example allocate these EBS volumes for the master, since they do not seem to be being used.
Thanks,
Phil -
philwhln asked a question in Cloudera on July 29, 2009 20:35:
"hadoop-ec2-ebs-beta" - purpose of master's EBS volumes?Hi,
I'm following the example here with great success, but the master EBS volumes do not seem to be part of my hadoop cluster. Am I wasting these EBS volumes? Only the slaves are counted as "nodes".
http://www.cloudera.com/hadoop-ec2-eb...
Thanks,
Phil -
philwhln started following the idea "Add More Canadian Content" in boxee.
