can't load nn_browsedfscontent.jsp

Error trying to browser the filesystem
After I set up my ec2 cluster, start the proxy, go to the home page,
then try to browse the fs by going to this link:
http://10.244.202.241:50070/nn_browse...
I get the following error:
the webpage "ip-10-244-186-32.ec2.internal:50075" cannot be found
DNS error occurred. Server cannot be found. The link may be broken.

Note I am able to load the homepage ok. And I am able to load the dfshealth.jsp
page ok too.

I am using cloudera hadoop-ec2 python scripts. The image is ami-6159bf08

Inappropriate?

Reply to this question

1 person has this question

I have this question, too! Tell me when someone answers.
The more people who ask this question, the more it gets noticed.

EMPLOYEE

Remove Inappropriate?
Delete or Edit

Todd, Employee, replied on July 26, 2009 22:41

Hi David,

Have you set up a SOCKS proxy to the EC2 cluster? You'll need that in order to access the web pages hosted by the datanode/tasktrackers. The "Browse DFS Content" JSP is hosted by datanodes, so it will need the proxy set up.

See the discussion about halfway down the instructions:

http://cloudera.com/hadoop-ec2

This page may also be of help: http://www.cloudera.com/blog/2009/05/...

Hope that helps
-Todd

this answers the question

1 Comment Add a comment This answers the question Mark as a company answer
Remove
David commented on July 27, 2009 03:34

Thanks for your reply Todd.
I do have the SOCKS proxy running. That is why I am able to load the NameNode link (http://10.244.186.226:50070/), which takes me to this page:
http://10.244.186.226:50070/dfshealth...

On that page there is a link, Browse the filesystem (http://10.244.186.226:50070/nn_browse...), which is the link I can't load.
I configure Firefox to use the socks proxy by
using the option: Automatic proxy configuration URL.
Remove Inappropriate?
Delete or Edit

Philip (flip) Kromer replied on July 27, 2009 13:21

I'm getting the same thing.

I think my SOCKS are on right: I can browse to
http://10.246.26.160:50030/jobtracker...
http://10.246.26.160:50070/dfshealth.jsp
and see the right thing. The dfsbrowse link, when I hover over it, is
http://10.246.26.160:50070/nn_browsed...
but I get a redirect after clicking to
http://domu-12-31-38-00-74-83.compute...
Which domain name my local computer can't resolve.

In the jobtracker pages, I can look at a job's details:
http://10.246.26.160:50030/jobdetails...
and I can see the list of failed tasks
http://10.246.26.160:50030/jobfailure...
but the links for each job are in the internal dns:
http://domu-12-31-38-00-85-53.compute...

One thing I noticed is that my cluster isn't in the same 255.255.0.0 net block:

Master: domU-12-31-38-02-15-52.compute-1.internal = 10.246.26.160
Slave: domu-12-31-38-00-74-83.compute-1.internal = 10.252.123.113
Slave: domu-12-31-38-01-7d-c4.compute-1.internal = 10.253.130.50

The SOCKS proxy .pac says
// Assume all nodes are in the same Class B
if (isInNet(host, "10.246.26.160", "255.255.0.0")) {
Even if I change this to
if (isInNet(host, "10.246.26.160", "255.0.0.0")) {
the behavior persists.

I also tried temporarily opening ports 50000-59999 to browse through the external interface. I could do so on that host but again, neighboring hosts became known by their internal domain names, not IP addresses.

Is there someplace in the /etc/hosts that I should set canonical IP <=> hostnames? I seem to recall there was some weirdness if the hostname and /etc/hosts mis-matched, but I don't remember anything more than that.

I’m unresolved internally

this answers the question

Add a comment This answers the question
EMPLOYEE

Remove Inappropriate?
Delete or Edit

Todd, Employee, replied on July 27, 2009 17:35

Hey guys,

If you're using FoxyProxy, you can set it up to route DNS requests through SOCKS. That will allow you to resolve the EC2 *.internal hostnames. To do so, go to Tools -> FoxyProxy -> Options, and then under "Miscellaneous" in the bottom left, choose "Use SOCKS proxy for DNS lookups".

Let me know if this doesn't fix the issue.

Thanks
-Todd

this answers the question

1 Comment Add a comment This answers the question Mark as a company answer
Remove
David commented on July 27, 2009 21:27

Hi Todd,

Thanks again for answering. I followed your instructions to use FoxyProxy and use Socks for DNS lookups.
I am able to get one step further along. Now
when I click on the "Browse the filesystem" link, it works and takes me
to a page titled: HDFS:/
But now none of the links on that page works.

An example of a link on that page that doesn't work is
Go back to DFS home (http://ip-10-244-183-146.ec2.internal...)

Any ideas?
EMPLOYEE

Remove Inappropriate?
Delete or Edit

Tom, Employee, replied on July 31, 2009 14:26

I think the PAC file needs an extra rule to resolve these addresses. I added

if (dnsDomainIs(host, ".ec2.internal")) {
return "SOCKS localhost:6666";
}

So the whole file became (your IP address will undoubtedly be different):

function FindProxyForURL(url, host) {
// Assume all nodes are in the same Class B
if (isInNet(host, "10.250.26.143", "255.255.0.0")) {
return "SOCKS localhost:6666";
}
if (dnsDomainIs(host, ".ec2.internal")) {
return "SOCKS localhost:6666";
}
return "DIRECT";
}

Hope this helps.

Tom

this answers the question

1 Comment Add a comment This answers the question Mark as a company answer
Remove
David commented on August 04, 2009 22:54

Thanks Tom. I followed your suggestion and it worked!

Reply to David's question

(Some HTML allowed)

Company participation

The mood in here

Products & Services

This topic is tagged

Related topics in Cloudera

can't load nn_browsedfscontent.jsp