Loading Profile...
Recent activity
Subscribe to this feed
Juan Besa replied on March 15, 2009 19:50 to the question "Making the Binary File" in LUCI:
Mines around 674 MB... I used shorts and eliminated terms with more than 50000 and it took around 2-3 hours... I am starting to think it's a bit small!-
Juan Besa replied on March 15, 2009 05:20 to the question "Making the Binary File" in LUCI:
I used the randomAccessFile and it took about 2 hours. And then used scanner for the lines. -
Juan Besa replied on March 14, 2009 23:34 to the question "Java out of memory error" in LUCI:
You might not want to read the whole line. If the term has more than 50000 pages you just want to skip it. So instead of saving those gigantic lines in memory you can skip them.
So first read till you find the document frequency and if that is lower than 50000 save the line in a String.
Also if you find one with more than 50000 pages use
buffered_reader_file.skip(20*df) (The 20 is something like the minimum number of bytes per docId=termfrecuency pair)
so you skip a lot of bytes and then read the rest of the line. Saves tons of time on those long lines! -
Juan Besa replied on March 02, 2009 22:42 to the question "not reducing?" in LUCI:
My job stopped reducing at 44% just now it was on 100%map and 44% reduce... then it went to 99% map and is still on 44%reduce... apparently I lost the connection to palintir...
INFO ipc.Client: Retrying connect to server: palantir.ics.uci.edu/128.195.58.105:1751. Already tried 0 time(s).
09/03/02 14:02:39 INFO ipc.Client: Retrying connect to server: palantir.ics.uci.edu/128.195.58.105:1751. Already tried 1 time(s).
does anybody know if it will continue running or is it stuck? it hasn't moved for something like 30 minutes now... -
Juan Besa replied on March 02, 2009 22:42 to the question "not reducing?" in LUCI:
My job stopped reducing at 44% just now it was on 100%map and 44% reduce... then it went to 99% map and is still on 44%reduce... apparently I lost the connection to palintir...
INFO ipc.Client: Retrying connect to server: palantir.ics.uci.edu/128.195.58.105:1751. Already tried 0 time(s).
09/03/02 14:02:39 INFO ipc.Client: Retrying connect to server: palantir.ics.uci.edu/128.195.58.105:1751. Already tried 1 time(s).
does anybody know if it will continue running or is it stuck? it hasn't moved for something like 30 minutes now... -
Juan Besa replied on March 01, 2009 05:48 to the question "Wierd error" in LUCI:
Oh no... there it goes again. -
Juan Besa replied on March 01, 2009 05:47 to the question "Wierd error" in LUCI:
I actually killed that process. I think the problem may be I was missing something from the wiki and started a new process.
It hasn't disappeared and the one I started hasn't appeared. Any idea why? Or does it just take a while? -
Juan Besa asked a question in LUCI on March 01, 2009 04:38:
Wierd errorI keep getting this error message and it takes forever with 10 urls.
Error launching task
09/02/28 20:32:52 WARN mapred.JobClient: Error reading task outputhttp://hutz.ics.uci.edu:58352/tasklog...
09/02/28 20:32:52 WARN mapred.JobClient: Error reading task outputhttp://hutz.ics.uci.edu:58352/tasklog...
Does anybody know what it means? -
Juan Besa replied on March 01, 2009 00:03 to the question "Can't connect by ssh" in LUCI:
I finally got in! Maybe it's fixed? -
Juan Besa replied on February 28, 2009 23:45 to the question "Can't connect by ssh" in LUCI:
I'm using my mac so from the terminal I put.
ssh jbesavia@family-guy.ics.uci.edu
This has worked great during all previous assignments.
And I get the normal message (after putting my password)
Last login: Sat Feb 28 15:38:57 2009 from vp123144.reshsg.uci.edu
ICS CentOS managed "family-guy.ics.uci.edu" cluster.
This computer is still in beta, please report problems.
Files in /tmp and /var/tmp deleted on regular basis.
Files in /scratch deleted only when full.
E-mail us if you could make use of VirtualBox virtualization software.
--helpdesk@ics.uci.edu
But no prompt appears.
I have also tried connecting directly to a machine and that doesn't work either. -
Juan Besa asked a question in LUCI on February 28, 2009 23:37:
Can't connect by sshHi,
I haven't been able to connect to any computer all day... I am prompted for my password and that works. I get the welcome message and then no prompt appears...
Also just recently the class webpage failed to load.
My internets been working perfectly with all other pages...
Does anybody have the same problem? Or any idea what I can do?
Oh, I can connect to openlabs using winscp. -
Juan Besa replied on February 16, 2009 18:58 to the question "Can't connect to the DFS" in LUCI:
I changed machines and it worked. I also tried again today and it worked on the same computer it didn't work last time :)
I'm guessing port problem right? -
Juan Besa replied on February 16, 2009 05:46 to the question "Can't connect to the DFS" in LUCI:
I changed machines and it worked! still is it just a port thing? -
Juan Besa asked a question in LUCI on February 16, 2009 05:30:
Can't connect to the DFSI'm having trouble with the HDFS. I followed all the instructions in the wiki but apparently can connect to the hdfs. I get the following error when I try to copy the conf from my local machine to the dfs.
$ bin/hadoop dfs -copyFromLocal conf input
09/02/15 21:19:53 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 0 time(s).
09/02/15 21:19:54 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 1 time(s).
09/02/15 21:19:55 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 2 time(s).
09/02/15 21:19:56 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 3 time(s).
09/02/15 21:19:57 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 4 time(s).
09/02/15 21:19:58 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 5 time(s).
09/02/15 21:19:59 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 6 time(s).
09/02/15 21:20:00 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 7 time(s).
09/02/15 21:20:01 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 8 time(s).
09/02/15 21:20:02 INFO ipc.Client: Retrying connect to server: localhost/127.0.0.1:58273. Already tried 9 time(s).
Bad connection to FS. command aborted.
I also tried bin/hadoop dfs -help
and got the same error.
The only part of the Wiki where I had trouble was when I had allow login without password. The authorized_key file did not exist and I had to make it.
Does anybody know what the error means? or how to fix it?
Thanks!
Juan
