infection0's dashboard

infection0 replied on March 03, 2009 20:23 to the question "Jobs restarted?" in LUCI:

The server just restarted, so maybe you can have it running now.

infection0 replied on March 03, 2009 19:04 to the question "Jobs restarted?" in LUCI:

I can't even access the palantir site for now.

infection0 replied on March 03, 2009 00:11 to the question "Extension possible?" in LUCI:

"Due to the school wide system shutdown this weekend, the deadline for Assignment 6 will be extended 24 hours."

Check your e-mails.

:)

Thank god, everyone is still scrambling on palantir for resources.

infection0 marked one of Don Patterson's replies in LUCI as useful. Don Patterson replied to the question "Why does it get to 100% MAP and 100% REDUCE and then fail?".

infection0 replied on March 02, 2009 23:51 to the question "Question for URLs" in LUCI:

Yeah.

infection0 replied on March 02, 2009 22:12 to the question "job tracker down?" in LUCI:

Sweet. It's awesome that you could recover everything.

infection0 marked one of Art's replies in LUCI as useful. Art replied to the question "Why does it get to 100% MAP and 100% REDUCE and then fail?".

infection0 marked one of Art's replies in LUCI as useful. Art replied to the question "Why does it get to 100% MAP and 100% REDUCE and then fail?".

infection0 replied on March 02, 2009 21:45 to the question "Why does it get to 100% MAP and 100% REDUCE and then fail?" in LUCI:

Hmm. That is a good idea. Thanks. I will try that out. Do I need to increment the counter each loop or is setStatus sufficient?

infection0 replied on March 02, 2009 21:43 to the idea "Check for page being null" in LUCI:

Yeah, we realized this early on (thank god for connection errors on the input-10 file!)

try
{
Page page = myDownloader.download(PREPENDURL + URL, docID);
text = page.getText();
}
catch (NullPointerException e)
{
reporter.setStatus("Failed to fetch: " + URL);
return;
}

infection0 replied on March 02, 2009 21:42 to the question "job tracker down?" in LUCI:

It's down for me too...

infection0 replied on March 02, 2009 21:27 to the question "Why does it get to 100% MAP and 100% REDUCE and then fail?" in LUCI:

I've got all the updating in the reduce method:

reporter.incrCounter(Counters.REDUCER,1);
reporter.setStatus("REDUCE: " + term.toString());

All of the jobs I have run have gotten to 100% and then showed up as "Failed".

infection0 replied on March 02, 2009 12:33 to the question "Why does it get to 100% MAP and 100% REDUCE and then fail?" in LUCI:

You may notice that it is 4:30AM and yes I am working on my posting list...

infection0 asked a question in LUCI on March 02, 2009 12:33:

Why does it get to 100% MAP and 100% REDUCE and then fail?

My attempts to map/reduce get to 100% map and 100% reduce, but then it shows up as "FAILED" and only some of the parts are present. Is there any specific problem that I should address or is it just the quirky bugs of Hadoop/DFS? Should I just brute-force this until I get a complete copy of the my posting list?

infection0 marked one of Don Patterson's replies in LUCI as useful. Don Patterson replied to the question "Keeping count...".

infection0 replied on March 02, 2009 08:19 to the question "Question for URLs" in LUCI:

Oh yeah, that's what I meant. :) I typed "input" instead of "output" for some reason.

infection0 marked one of Yasser's replies in LUCI as useful. Yasser replied to the question "Question for URLs".

infection0 marked one of Yasser's replies in LUCI as useful. Yasser replied to the question "Few problems".

infection0 replied on March 02, 2009 08:08 to the question "Question for URLs" in LUCI:

Oh so basically we just find the spot in our input file that has our name as a term? OH. Wow I did a lot of unnecessary stuff.

infection0 replied on March 02, 2009 07:23 to the question "Few problems" in LUCI:

Thank you!