Bryant's dashboard

Bryant replied on March 16, 2008 20:38 to the question "Linking firefox extension to the webserver?" in LUCI:

What approach is everyone using to download the current page from the Java application? I thought about using the websphinx stuff, but it's pretty process intensive it seems...

Bryant replied on March 16, 2008 19:55 to the question "Linking firefox extension to the webserver?" in LUCI:

Whoa, Vinh, that works perfectly! Thanks! :)

Bryant asked a question in LUCI on March 09, 2008 19:51:

Computing Query Weight?

I'm sort of confused about what values to use when calculating the weight of the query for the cosine score. I understand I'm supposed to treat the query like it's own document, but how does that change the values I need to use in the TF-IDF function?

If, for example, I have a one-word query, does that make both my term frequency and my document frequency for the query equal to 1? Would my corpus size also just be the size of my query? The issue with those numbers is that log(1) = 0, so my TF-IDF weight will be 0. That doesn't seem right...

I hope that makes sense :) Thanks!

Bryant started following the question "Multi or single word queries for Assignment 7?" in LUCI.

Bryant replied on March 04, 2008 04:50 to the question "Memory Issues When Constructing Posting List" in LUCI:

Thanks for the suggestions Kyle! I've been splitting my output into multiple files across multiple directories, which is working great, but it is really slow. At least the posting list construction is a one-time thing :)

Bryant asked a question in LUCI on March 03, 2008 20:45:

Memory Issues When Constructing Posting List

I'm having memory issues when building the posting list for the terms. Everything completely breaks down after I've handled roughly 35,000 pages. I've tried shortening the document URL, and that certainly helped. But I have no idea what I can do to take care of the next 35,000 in my set. What are you doing to avoid running out of memory?

Thanks :)

Bryant started following the discussion "How do I go about doing Assignment 6?" in LUCI.

Bryant replied on February 23, 2008 02:56 to the question "Making the XPI File" in LUCI:

I'm using 7-Zip as my archiving utility, and the directions from the site Patterson linked to worked fine for me (See 'deployment' at the bottom of http://www.ics.uci.edu/~djp3/classes/...).

First I zipped up the contents of my Chrome folder and named it 'sidebar.jar.' Then I edited my chrome.manifest file using the data provided on the website (renaming 'emptysidebar' to 'sidebar'). For the last step, I zipped up the entirety of my application, not just the files the directions specified. So inside my sidebar.xpi archive, I have install.rdf, chrome.manifest, and the entirety of my chrome folder (including the jar file).

I hope that works for you :-\

Bryant started following the question "Making the XPI File" in LUCI.

Bryant replied on February 22, 2008 02:14 to the idea "Useful resources for assignment 5" in LUCI:

Thanks for the sweet info :)

Do you have any hints on how to select and change the value of the element that displays the current page? I tried this, but it didn't do a thing:

onPageLoad: function(aEvent)
{
document.getElementById("currentPage").value = "Loaded"
}

Does it have anything to do with the fact that my Javascript is in the firefoxOverlay.xul file and the element I'm trying to change is in my sidebar.xul file?

Bryant started following the idea "Useful resources for assignment 5" in LUCI.

Bryant shared an idea in LUCI on February 14, 2008 22:13:

Remember to Set Max Depth

I was having issues with Assignment 4 -- the crawler would only crawl roughly half of the "easter egg" pages before stopping. It turns out the crawler defaults to only crawling pages that are 5 steps away from your seed. So if you're having the same problem, use the setMaxDepth() method and set it to something higher, like 10 or 50 or something :)

Bryant replied on February 11, 2008 01:00 to the question "Anyone know how to store the webGraph to a file? Anyone finish crawling all of wiki?" in LUCI:

Thanks for all your tips, Kyle! That did the trick for me :)

Bryant started following the question "Strange problem with Irvine page" in LUCI.

Bryant replied on February 10, 2008 02:37 to the question "Strange problem with Irvine page" in LUCI:

How did you manage to set other download parameters besides the page size? I always seem to overwrite previous changes whenever I change another download parameter (like the thread count).

Bryant replied on February 09, 2008 23:15 to the question "OutOfMemoryError" in LUCI:

Gwah! Now I'm not even getting an OutOfMemoryError. I'm getting this beast:

Exception in thread "Keep-Alive-SocketCleaner" java.lang.NoClassDefFoundError: Could not initialize class java.util.concurrent.TimeUnit
at sun.net.www.http.KeepAliveStreamCleaner.run(KeepAliveStreamCleaner.java:79)
at java.lang.Thread.run(Thread.java:619)

I marked my file output method as being "synchronized." Is that a bad thing?

Bryant asked a question in LUCI on February 09, 2008 23:13:

OutOfMemoryError

I'm getting an OutOfMemoryError after I visit 150-250 pages in Wikipedia. I'm not sure where it would be coming from; I don't think I'm storing any data as I go along (I'm writing stuff out to a file currently). The only thing I can think of is the set of Visited links. I'm using the default methods provided by the crawler (visited(link) and markVisited(link)). What approaches are you taking to avoid the OutOfMemoryError?

Bryant replied on February 02, 2008 19:45 to the question "Questions about lipograms" in LUCI:

Can lipograms occur mid-sentence? Also, wouldn't the "snake" token in your example terminate the lipogram?

Thanks!