Configuring the port numbers on OS X not working
I've finally gotten 0.18 running on my MacBook but am seeing some discrepancies.
My hadoop-site.xml contains:
...
<property>
<name>fs.default.name</name>
<value>hdfs://Ryans-MacBook.local:9000</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri's scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri's authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>
<property>
<name>mapred.job.tracker</name>
<value>Ryans-MacBook.local:9001</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>
...
However, I can only access the NameNode and JobTracker at ports 50070 and 50030. Why is that? Also, it seems that when I use localhost rather than Ryans-MacBook.local, I get a bunch of connection errors when I do a bin/start-all.sh. By the way the window title of the NameNode browser is set to "Hadoop Namenode 10.0.1.8:9000".
These are a few of the issues I've had which I can only guess are due to slight differences in OS X vs Linux.
Thanks,
Ryan
My hadoop-site.xml contains:
...
<property>
<name>fs.default.name</name>
<value>hdfs://Ryans-MacBook.local:9000</value>
<description>The name of the default file system. A URI whose
scheme and authority determine the FileSystem implementation. The
uri's scheme determines the config property (fs.SCHEME.impl) naming
the FileSystem implementation class. The uri's authority is used to
determine the host, port, etc. for a filesystem.</description>
</property>
<property>
<name>mapred.job.tracker</name>
<value>Ryans-MacBook.local:9001</value>
<description>The host and port that the MapReduce job tracker runs
at. If "local", then jobs are run in-process as a single map
and reduce task.
</description>
</property>
...
However, I can only access the NameNode and JobTracker at ports 50070 and 50030. Why is that? Also, it seems that when I use localhost rather than Ryans-MacBook.local, I get a bunch of connection errors when I do a bin/start-all.sh. By the way the window title of the NameNode browser is set to "Hadoop Namenode 10.0.1.8:9000".
These are a few of the issues I've had which I can only guess are due to slight differences in OS X vs Linux.
Thanks,
Ryan
1
person has this question
I have this question, too!
Tell me when someone answers.
The more people who ask this question, the more it gets noticed.
The more people who ask this question, the more it gets noticed.
-
Inappropriate?Hi Ryan,
HDFS and Hadoop MR actually run servers on multiple ports. 50070 and 50030 are the ports for the web interface. Other ports (commonly 8020 and 8021) are used for the communication amongst Hadoop servers and clients themselves. See http://www.cloudera.com/blog/2009/08/... for the list of relevant ports, including which configuration options you need to modify to adjust them.
-- Philip -
Inappropriate?Hey Philip,
Thanks for the reply. I had no idea that was the case. It makes sense now :)
-Ryan
Loading Profile...



EMPLOYEE