zaki.rahaman's Profile

Recent activity

Subscribe to this feed

zaki.rahaman marked one of Carl Steinbach's replies in Cloudera as useful. Carl Steinbach replied to the problem "Waiting for Jobtracker to Stop Causes Script to Hang...".
zaki.rahaman replied on November 18, 2009 17:43 to the problem "Waiting for Jobtracker to Stop Causes Script to Hang..." in Cloudera:

resolved finally i think.... still getting some strange behavior at times

http://getsatisfaction.com/cloudera/t...

basically just added client_cidr option and so the urllib2 request is finally able to connect to the jobtracker once its running...

still having some weird issue on my own laptop as it is not reading in the config file.... (ive set it to have all permissions its in the right $HOME/.hadoop-ec2 dir and doesnt work even if i run hadoop-ec2 from this dir)

Do you think this solves the problem? Mark it as solved
zaki.rahaman replied on November 11, 2009 20:28 to the problem "Waiting for Jobtracker to Stop Causes Script to Hang..." in Cloudera:

Yea, the jobtracker seems to be starting fine and I can access it by ssh-ing into the master and going to localhost:50030. however when I try to access it form http://publicDNS:50030 on my local machine, the server doesn't respond and I suspect this is where the launch-cluster command is hanging... (i see the waiting for jobtracker to start ...... keep going forever) ... i checked the security group settings and 50030 is accessible

Do you think this solves the problem? Mark it as solved
zaki.rahaman replied on November 11, 2009 16:40 to the problem "Waiting for Jobtracker to Stop Causes Script to Hang..." in Cloudera:

I fixed the problem with the launch-cluster not reading the config (stupidly left on the brackets in the cluster-name) ... now its back to just hanging when it gets to the waiting for jobtracker to start.... from what i can tell the jobtracker is running on the master so i guess theres something else going on here...?

Do you think this solves the problem? Mark it as solved
zaki.rahaman replied on November 09, 2009 17:33 to the problem "Waiting for Jobtracker to Stop Causes Script to Hang..." in Cloudera:

Hey Carl/everyone else...

Any ideas? Still stuck...

Do you think this solves the problem? Mark it as solved
zaki.rahaman replied on November 03, 2009 17:08 to the problem "Waiting for Jobtracker to Stop Causes Script to Hang..." in Cloudera:

Yes, I'm using the correct version of the hadoop-ec2 scripts... and I do see the line before saying it's reading the config files... however the options remains empty...

heres the ec2-clusters.cfg file (i have a different one that i was using before with my own custom settings, this is just the one from the cloudera site)

[my-hadoop-cluster]
ami=ami-6159bf08
instance_type=c1.medium
key_name=kikin
availability_zone=us-east-1c
private_key=/Users/wenjiexu/Downloads/kikin.pem
ssh_options=-i %(private_key)s -o StrictHostKeyChecking=no

I'm launching it with the following command:
% hadoop-ec2 launch-cluster [my-hadoop-cluster] 3

Ive pasted the debug output here....
http://pastebin.com/m36a205b0

Not sure what else to do other than try and debug/refactor the config options reader or just hardcode my settings (which seems really inefficient)

Do you think this solves the problem? Mark it as solved
zaki.rahaman replied on November 02, 2009 22:34 to the problem "Waiting for Jobtracker to Stop Causes Script to Hang..." in Cloudera:

Yes, ec2-clusters.cfg is in the right location (.hadoop-ec2 in the home dir) and I just copied the basic config from the cloudera site (with the path to my EC2 key)

The entire debug output is obviously wayyy too long to cover here, but I think I've identified the main problem. For whatever reason, the options specified in the ec2-clusters.cfg file are not getting read by the hadoop-ec2 script (in the debug output theres an options={} line). I confirmed this as the script always fails to run unless an ami flag is specified on the command line (--ami = ami-id).

I thought this might be a Py 2.5/2.6 issue, but I got the same output using Py2.5. I was able to duplicate this behavior on a remote unix box as well as my local machine running os x 10.6 and on another mac as well. Not sure if it has to do with the hadoop-ec2 impl of the config parsing in Python (I'm not familiar with the Options/Config Parser modules) but maybe I'm doing something wrong as it doesn't look like anyone else has reported this problem.

Do you think this solves the problem? Mark it as solved
zaki.rahaman replied on October 30, 2009 23:13 to the problem "Waiting for Jobtracker to Stop Causes Script to Hang..." in Cloudera:

Hey Carl,

Thanks for the reply. So I found some other problems (tried this on 3 different machines py2.5 and 2.6). After turning the debugging on, I saw that the options from ec2-clusters.cfg are not being passed for some reason (I also know this bc it always fails to launch a cluster unless an ami is specified on the command line). Clearly, something is wrong with ConfigParser or somewhere else along the line. Your help here would reallly be appreciated.

Do you think this solves the problem? Mark it as solved
zaki.rahaman reported a problem in Cloudera on October 30, 2009 20:23:

Waiting for Jobtracker to Stop Causes Script to Hang...
I'm having trouble using the EC2 scripts... the only modification I made was to tell the main hadoop-ec2 script to use python 2.6 instead of 2.5... the script successfully launches the instances (master+slaves) but hangs when it gets to "Waiting for jobtracker to stop" .... any ideas?
zaki.rahaman marked one of Todd's replies in Cloudera as useful. Todd replied to the problem "Apt-get Install Fail UnsupportedClassVersionError".
zaki.rahaman replied on October 23, 2009 17:29 to the problem "Apt-get Install Fail UnsupportedClassVersionError" in Cloudera:

yea rookie mistake...

Do you think this solves the problem? Mark it as solved
zaki.rahaman reported a problem in Cloudera on October 23, 2009 15:26:

Apt-get Install Fail UnsupportedClassVersionError
I'm getting the following stack trace after running the hadoop-0.20 install on ubuntu...

root@hadoop:~# hadoop-0.20 version
Exception in thread "main" java.lang.UnsupportedClassVersionError: Bad version number in .class file
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:620)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:124)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:260)
at java.net.URLClassLoader.access$100(URLClassLoader.java:56)
at java.net.URLClassLoader$1.run(URLClassLoader.java:195)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.

not sure what's going on here as everything that should be there is as far as i know

zaki.rahaman's dashboard

About me

Recent activity