Philip (flip) Kromer's dashboard

Philip (flip) Kromer started following the problem "Duplicated group tweets" in Thing Labs.

Philip (flip) Kromer started following the problem "Tweets repeating themselves over and over, NOT RT's!" in Thing Labs.

Philip (flip) Kromer replied on July 28, 2009 11:09 to the question "Answering Cloudera configurator questions with NFS directories" in Cloudera:

Besides dfs.name.dir, does it make sense to direct fs.checkpoint.dir to the NFS?

I run a secondary namenode on a separate machine. Both the 2NN and NN have two local partitions as well as the NFS. My NFS partition isn't particularly fast, and I'm not completely clear how write-intensive the checkpoint operation is. On the other hand, the NFS server is fault-tolerant and is regularly backed up, and I like the idea of having the fs metadata be recoverable.

Right now, I direct dfs.name.dir to both a local drive and the NFS partition, and fs.checkpoint.dir to the NFS only. Do you have any advice on whether this is likely to be choking performance or unnecessarily redundant?

Philip (flip) Kromer replied on July 27, 2009 13:21 to the question "can't load nn_browsedfscontent.jsp" in Cloudera:

I'm getting the same thing.

I think my SOCKS are on right: I can browse to
http://10.246.26.160:50030/jobtracker...
http://10.246.26.160:50070/dfshealth.jsp
and see the right thing. The dfsbrowse link, when I hover over it, is
http://10.246.26.160:50070/nn_browsed...
but I get a redirect after clicking to
http://domu-12-31-38-00-74-83.compute...
Which domain name my local computer can't resolve.

In the jobtracker pages, I can look at a job's details:
http://10.246.26.160:50030/jobdetails...
and I can see the list of failed tasks
http://10.246.26.160:50030/jobfailure...
but the links for each job are in the internal dns:
http://domu-12-31-38-00-85-53.compute...

One thing I noticed is that my cluster isn't in the same 255.255.0.0 net block:

Master: domU-12-31-38-02-15-52.compute-1.internal = 10.246.26.160
Slave: domu-12-31-38-00-74-83.compute-1.internal = 10.252.123.113
Slave: domu-12-31-38-01-7d-c4.compute-1.internal = 10.253.130.50

The SOCKS proxy .pac says
// Assume all nodes are in the same Class B
if (isInNet(host, "10.246.26.160", "255.255.0.0")) {
Even if I change this to
if (isInNet(host, "10.246.26.160", "255.0.0.0")) {
the behavior persists.

I also tried temporarily opening ports 50000-59999 to browse through the external interface. I could do so on that host but again, neighboring hosts became known by their internal domain names, not IP addresses.

Is there someplace in the /etc/hosts that I should set canonical IP <=> hostnames? I seem to recall there was some weirdness if the hostname and /etc/hosts mis-matched, but I don't remember anything more than that.

Philip (flip) Kromer replied on March 28, 2009 05:10 to the question "Business Founders, Not Technical" in Capital Factory:

I also recommend Refresh Austin, Tech Happy Hour, Austin on Rails, etc. - great events, you'll learn something and you'll meet a lot of wicked-smart techie types. I don't believe any have meetings before the deadline, but Refresh and AoR both have "jobs" mailing lists that may yield a receptive partner.

Philip (flip) Kromer asked a question in Capital Factory on March 27, 2009 14:01:

Non-Compete/Non-Disclosure

Will the legal documents be posted soon? Specifically, are Capital Factory and those who review our application bound by non-disclosure/non-compete terms?

Philip (flip) Kromer replied on March 20, 2009 08:27 to the question "PIG nightly RPM builds" in Cloudera:

Pig-types is much faster, useful and mature than the 1.1 release.

Pig itself is so new that in many ways the SVN is a more robust choice. I've been using a nightly from right after they landed the 'types' branch (Jan 13th), as I know they did a lot of QA in order to land that. You'll want to choose a later nightly to ensure 1.9 compatibility.

(This would not stop me from using the rest of the Cloudera RPMs -- I'd simply manage my own Pig).

Philip (flip) Kromer started following the idea "Support for HBase" in Cloudera.

Philip (flip) Kromer started following the idea "Ubuntu support?" in Cloudera.

Philip (flip) Kromer started following the idea "Ubuntu support?" in Cloudera.

Philip (flip) Kromer started following the idea "Ubuntu support?" in Cloudera.

Philip (flip) Kromer replied on January 20, 2009 18:58 to the idea "Free up more screen for viewing." in OtherInbox:

+1 on better use of screen real estate. This was also my first reaction to using Otherinbox...

By the time I have my browser menu + toolbars, browser status bar, the otherinbox toolbar, the message index bar, the message bar, header text and the bottom newmsg/box bar only a VT100esque six lines of text remained.

A nitpick, I'm enjoying the site so far.

Philip (flip) Kromer replied on April 10, 2008 08:41 to the discussion "Ideas for improving infochimps.org" in Infochimps.org:

Thanks Nicolas! Fixed!

Philip (flip) Kromer marked one of Dethe Elza's replies in Infochimps.org as useful. Dethe Elza replied to the discussion "Ideas for improving infochimps.org".

Philip (flip) Kromer replied on April 05, 2008 18:01 to the discussion "Ideas for improving infochimps.org" in Infochimps.org:

I bet you are correct... the problem is that we have experts in data mining who are hack amateurs at website design driving this bus.

Realizing this, our approach is to keep things visually simple yet retain as much flexibility as possible in how we store the (rapidly evolving) metadata that characterize each dataset/field/contributor/collection. (So, for example, the long list of paragraph boxes that follow 'usage notes' are simply autogenerated from what's in the dataset description. When you see a whole paragraph box devoted to one footnote it's cause the page-drawing orangutang doesn't know a footnote from anything else).

So what I'm saying is: yes, it is probably jumbled, and some of that murkiness reflects incomplete clarity of concept. But improving apart from that is honestly beyond our expertise.

If you (or anyone else reading this) has a friend who does information & website design and would like to try their hand at a more forceful presentation of this information, we'd love their assistance. (Same goes for anyone really good at rails programming, database/scalable web server design, data visualization, nurturing online communities, IP law, programming frameworks design; and as always people to contribute data, convert data to open formats, and curate metadata.) This project will only succeed if we can get wikipedia/linux/etc style community involvement among people with diverse talents.

Philip (flip) Kromer started a conversation in Infochimps.org on March 09, 2008 04:26:

Ideas for improving infochimps.org

Infochimps.org is brand-new as of March 2008, and we're still furiously assembling its structure like a parent assembling a bike on Christmas eve.

If you have ideas about how to improve the site or to correct the many ways in which the site falls short, please share them! The TODO page at help.infochimps.org lists the immediate tasks we've set, so first take a gander over there if you can.

Philip (flip) Kromer asked a question in Infochimps.org on March 07, 2008 12:29:

What's broken on Infochimps.org?

Please let us know what's totally broken -- as in, it completely fails to work, not "designed stupidly" or "missing functionality".