Loading Profile...
Crawler runs successfully, but no analytics data is showing
Next problem. The crawler runs, but I still have no analytics data. The last lines in the crawler.out are:
2008-12-10 17:38:28,221 INFO [main] com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl - Seding crawling results of repository 8eecc26e-0122-31c1-8e86-59272a02c511
2008-12-10 17:38:52,002 INFO [main] com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl - Crawling results were sent for repository 8eecc26e-0122-31c1-8e86-59272a02c511
2008-12-10 17:38:52,002 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished repository https://svn.... 1/1
2008-12-10 17:38:52,002 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished crawling of repositories
2008-12-10 17:38:28,221 INFO [main] com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl - Seding crawling results of repository 8eecc26e-0122-31c1-8e86-59272a02c511
2008-12-10 17:38:52,002 INFO [main] com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl - Crawling results were sent for repository 8eecc26e-0122-31c1-8e86-59272a02c511
2008-12-10 17:38:52,002 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished repository https://svn.... 1/1
2008-12-10 17:38:52,002 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished crawling of repositories
I’m hopeful
1
person has this question
I have this question, too!
Tell me when someone answers.
The more people who ask this question, the more it gets noticed.
The more people who ask this question, the more it gets noticed.
Create a customer community for your own organization
Plans starting at $19/month
-
Inappropriate?Hi Keith!
Could you please send me (either here or to twitter) contents of your *.repo.conf file (with substituted repository path and username/password).
In logs on our side i see that crawling result come without the license key specified.
Thanks!
I’m happy to help programeter adopters!
-
Inappropriate?licencekey=0abc4d49d27c09827e9579538311b6a593731e38
path=https://svn....
username=
password=
type=SVN
trunkTitle=
technology=ALL
crawlingPeriod=YEAR -
Inappropriate?Just noticing something, is license mis-spelled in the .conf file, or is that the way the crawler expects the .conf file to read?
-
Inappropriate?yes, crawler expects it to be mis-spelled and also it is case-sensitive. So the line with license key should be:
licenceKey=0abc4d49d27c09827e9579538311b6a593731e38
Could you fix that please and try running crawler once again?
I’m sure that this was the problem
-
Inappropriate?Ok, I'm seeing analytics data, but I'm not seeing all of the team members - like me for instance.
I’m confused
-
Inappropriate?Hey! Your repository has been crawled till 2399 revision (which is somewhere in February 2006). So, you would need to run crawler few times more in order to get to December 2008.
While your repository is being crawled we will start preparing analytics for you!
I’m cheering Keith up!
1 Comment
Add a comment
This answers the question
-
We are updating GUI so that you would able to follow crawling state as well -
Inappropriate?Ok, is there a good way to tell when the whole repository has been crawled?
I’m waiting to see the results
-
Inappropriate?not at the moment :( but, we are planning on a release tomorrow and that will include such feature.
So far, when crawler hits the end, it will issue a log statement like: "Repository [XXXXX] has been measured till the end: ..." and if you ran the crawler after that then third to last statement on the log will be ""Repository XXXXX was already crawled lately!"
At the moment i see that you have almost reached October 2006. So i would prognose that you need to ran crawler for 3-4 times more.
I’m making release promises
-
Inappropriate?And once again there's a strange error in the .out
2008-12-11 15:47:01,734 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log entry from revision [Revision(Mon Nov 13 16:57:51 EST 2006[5314])] 1/3 for repo: https://svn....
2008-12-11 15:47:02,234 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log entry from revision [Revision(Mon Nov 13 16:57:51 EST 2006[5314])] 2/3 for repo: https://svn....
https://svn...
2008-12-11 15:47:02,640 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished repository https://svn.... 1/1
2008-12-11 15:47:02,640 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished crawling of repositories
Exception in thread "main" java.lang.NullPointerException
at com.goingsoftware.detective.measure.metrics.TouchedLinesMeasurer.measureExisting(TouchedLinesMeasurer.java:60)
at com.goingsoftware.detective.measure.metrics.AbstractMeasurer.measure(AbstractMeasurer.java:45)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measure(RepositoryCrawlerServiceImpl.java:391)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measureLog(RepositoryCrawlerServiceImpl.java:338)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlSvnRepository(RepositoryCrawlerServiceImpl.java:117)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlRepository(RepositoryCrawlerServiceImpl.java:60)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:81)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:64)
at com.goingsoftware.detective.crawler.RepositoryCrawler.main(RepositoryCrawler.java:46
I’m still irked
-
Inappropriate?I am checking on it. Very bizarre, cause at the moment it looks like some revisions in your repository don't have an author.
I’m fixing bugs
-
Inappropriate?That is odd, although I'm not sure that would be unexpected for a repository that is close to three years old and has probably been moved at least once. I'm fairly sure that several of these projects were imported from CVS.
Is there a way to tell the crawler to just crawl the last discrete time period - one year, two months, three weeks?
I’m confident there will be a work-around
-
Inappropriate?Yes, this is possible, but that will affect the accuracy of analytics (the longer period we take - the more accurate analytics are).
Anyway, i tried to fix this issue by making sure that author of modification is always defined. The fix wasn't tested thoroughly because it is quite difficult to reproduce issue without knowing exact circumstances. If that wouldn't work then we will get back to your suggestion to crawl only last few months.
Please download the latest version (should be 3.8.4) from your account at http://my.programeter.com and try to continue crawling.
Btw, now you can see how far repository was crawled on the project analytics page (former team analytics page).
I’m hoping for the best!
-
Inappropriate?I'm still getting an NPE with 3.8.4
2008-12-12 13:04:08,089 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log entry from revision [Revision(Mon Nov 13 16:57:51 EST 2006[5314])] 1/3 for repo: https://svn....
2008-12-12 13:04:08,698 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log entry from revision [Revision(Mon Nov 13 16:57:51 EST 2006[5314])] 2/3 for repo: https://svn....
2008-12-12 13:04:09,136 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished repository https://svn....
2008-12-12 13:04:09,136 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished crawling of repositories
Exception in thread "main" java.lang.NullPointerException
at com.goingsoftware.detective.measure.metrics.TouchedLinesMeasurer.measureExisting(TouchedLinesMeasurer.java:60)
at com.goingsoftware.detective.measure.metrics.AbstractMeasurer.measure(AbstractMeasurer.java:45)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measure(RepositoryCrawlerServiceImpl.java:391)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measureLog(RepositoryCrawlerServiceImpl.java:338)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlSvnRepository(RepositoryCrawlerServiceImpl.java:117)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlRepository(RepositoryCrawlerServiceImpl.java:60)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:81)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:64)
at com.goingsoftware.detective.crawler.RepositoryCrawler.main(RepositoryCrawler.java:46)
Did I uncover another strange problem with our SVN repository?
I’m unconcerned
-
Inappropriate?From the logs i am seeing that you are still using a 3.8.2 version (the previous one):
2008/12/12 12:58:53 Client 0abc4d49d27c09827e9579538311b6a593731e38 uses a crawler with version 3.8.2
(timezone is Eastern Standard Time)
Could you please re-check that? I suggest uninstalling the previous version of crawler (don't forget to back up repo.conf file) and then making a clean install of the latest version.
I’m determined to solve this mystery
-
Inappropriate?Downloaded a fresh version of 3.8.4, installed, double checked the .conf file and the proxy.properties file and I get the following:
The .out:
2008-12-12 15:42:46,961 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Starting crawling of repositories
2008-12-12 15:42:46,976 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Crawling repository https://svn./ 1/1
2008-12-12 15:42:46,976 INFO [main] com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl - Requesting repository state 8eecc26e-0122-31c1-8e86-59272a02c511
2008-12-12 15:42:48,273 INFO [main] com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl - Crawl state request was sent for reposutository 8eecc26e-0122-31c1-8e86-59272a02c511
2008-12-12 15:42:48,273 INFO [main] com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl - Blocking for 300000 ms until receiving a response ...
2008-12-12 15:42:48,977 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Repository state: CrawlState(Revision(Fri Sep 29 17:09:29 EDT 2006[4799]); lastcrawledDate -> 2006-09-29 17:09:29.0)
2008-12-12 15:42:48,977 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Crawling repository[https://svn./]
log4j: rolling over count=10485921
log4j: maxBackupIndex=3
log4j: Renaming file crawler-debug.log to crawler-debug.log.1
log4j: setFile called: crawler-debug.log, false
log4j: setFile ended
2008-12-12 15:43:22,573 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log revision [Revision(Fri Sep 29 17:24:46 EDT 2006[4800])] 1/2400 for repo: https://svn./
2008-12-12 15:43:22,573 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log entry from revision [Revision(Fri Sep 29 17:24:46 EDT 2006[4800])] 1/1 for repo: https://svn./
2008-12-12 15:43:23,026 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished repository https://svn./ 1/1
2008-12-12 15:43:23,026 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished crawling of repositories
Exception in thread "main" java.lang.NullPointerException
at com.goingsoftware.detective.measure.metrics.TouchedLinesMeasurer.measureExisting(TouchedLinesMeasurer.java:60)
at com.goingsoftware.detective.measure.metrics.AbstractMeasurer.measure(AbstractMeasurer.java:45)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measure(RepositoryCrawlerServiceImpl.java:391)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measureLog(RepositoryCrawlerServiceImpl.java:338)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlSvnRepository(RepositoryCrawlerServiceImpl.java:117)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlRepository(RepositoryCrawlerServiceImpl.java:60)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:81)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:64)
at com.goingsoftware.detective.crawler.RepositoryCrawler.main(RepositoryCrawler.java:46)
Any ideas?
I’m apparently the owner of the worst SVN repository ever
-
Inappropriate?Yep, now logs show that you use version 3.8.4
My fix didn't work - I'll have to dig deeper.
I’m on a mission
-
Inappropriate?Yay! Last exception was result of my stupid mistake in the code. I have rushed so much to get the working version out, that forgot to properly run unit tests.
I will make a new release soon and notify you when it's ready!
I’m full of hopes. sky is blue and sun is shining!
-
Inappropriate?Keith, please download the latest version of crawler and try using it. The new version is 3.8.5. This NPE should be gone and hopefully we will get your repository measured :)
I’m starting new week full of optimism
-
Inappropriate?That's great, but the my.programeter.com site isn't responding.
I’m sad I can't try out the lastest fix
-
Inappropriate?yeap, our hosting service provider had a major outage. now all servers seems to be back to normal.
I’m concerned with reliability
-
Inappropriate?Now I get a JMS exception:
2008-12-15 12:39:41,069 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished repository https://svn... 1/1
2008-12-15 12:39:41,069 INFO [main] com.goingsoftware.detective.crawler.RepositoryCrawler - Finished crawling of repositories
Exception in thread "main" org.springframework.jms.UncategorizedJmsException: Uncategorized exception occured during JMS processing; nested exception is javax.jms.JMSException: Could not post command: ActiveMQObjectMessage {commandId = 5, responseRequired = true, messageId = ID:SADERK745XP-2774-1229358014462-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = ID:SADERK745XP-2774-1229358014462-1:1:1:1, destination = queue://detective/crawl/results, transactionId = null, expiration = 0, timestamp = 1229362749381, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = 8eecc26e-0122-31c1-8e86-59272a02c511, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1631715, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {repository=8eecc26e-0122-31c1-8e86-59272a02c511, licenceKey=0abc4d49d27c09827e9579538311b6a593731e38, version=3.8.5}, readOnlyProperties = true, readOnlyBody = true, droppable = false} due to: java.io.IOException: Failed to post command: ActiveMQObjectMessage {commandId = 5, responseRequired = true, messageId = ID:SADERK745XP-2774-1229358014462-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = ID:SADERK745XP-2774-1229358014462-1:1:1:1, destination = queue://detective/crawl/results, transactionId = null, expiration = 0, timestamp = 1229362749381, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = 8eecc26e-0122-31c1-8e86-59272a02c511, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1631715, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {repository=8eecc26e-0122-31c1-8e86-59272a02c511, licenceKey=0abc4d49d27c09827e9579538311b6a593731e38, version=3.8.5}, readOnlyProperties = true, readOnlyBody = true, droppable = false} as response was: 500
at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:308)
at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:168)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:474)
at org.springframework.jms.core.JmsTemplate.send(JmsTemplate.java:548)
at com.goingsoftware.detective.crawler.web.AnalyticsPlatformClientImpl.sendCrawlResult(AnalyticsPlatformClientImpl.java:130)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:82)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:64)
at com.goingsoftware.detective.crawler.RepositoryCrawler.main(RepositoryCrawler.java:46)
Caused by: javax.jms.JMSException: Could not post command: ActiveMQObjectMessage {commandId = 5, responseRequired = true, messageId = ID:SADERK745XP-2774-1229358014462-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = ID:SADERK745XP-2774-1229358014462-1:1:1:1, destination = queue://detective/crawl/results, transactionId = null, expiration = 0, timestamp = 1229362749381, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = 8eecc26e-0122-31c1-8e86-59272a02c511, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1631715, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {repository=8eecc26e-0122-31c1-8e86-59272a02c511, licenceKey=0abc4d49d27c09827e9579538311b6a593731e38, version=3.8.5}, readOnlyProperties = true, readOnlyBody = true, droppable = false} due to: java.io.IOException: Failed to post command: ActiveMQObjectMessage {commandId = 5, responseRequired = true, messageId = ID:SADERK745XP-2774-1229358014462-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = ID:SADERK745XP-2774-1229358014462-1:1:1:1, destination = queue://detective/crawl/results, transactionId = null, expiration = 0, timestamp = 1229362749381, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = 8eecc26e-0122-31c1-8e86-59272a02c511, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1631715, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {repository=8eecc26e-0122-31c1-8e86-59272a02c511, licenceKey=0abc4d49d27c09827e9579538311b6a593731e38, version=3.8.5}, readOnlyProperties = true, readOnlyBody = true, droppable = false} as response was: 500
at org.apache.activemq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:62)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1255)
at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1659)
at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227)
at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:597)
at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:574)
at org.springframework.jms.core.JmsTemplate$4.doInJms(JmsTemplate.java:551)
at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:471)
... 5 more
Caused by: java.io.IOException: Could not post command: ActiveMQObjectMessage {commandId = 5, responseRequired = true, messageId = ID:SADERK745XP-2774-1229358014462-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = ID:SADERK745XP-2774-1229358014462-1:1:1:1, destination = queue://detective/crawl/results, transactionId = null, expiration = 0, timestamp = 1229362749381, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = 8eecc26e-0122-31c1-8e86-59272a02c511, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1631715, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {repository=8eecc26e-0122-31c1-8e86-59272a02c511, licenceKey=0abc4d49d27c09827e9579538311b6a593731e38, version=3.8.5}, readOnlyProperties = true, readOnlyBody = true, droppable = false} due to: java.io.IOException: Failed to post command: ActiveMQObjectMessage {commandId = 5, responseRequired = true, messageId = ID:SADERK745XP-2774-1229358014462-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = ID:SADERK745XP-2774-1229358014462-1:1:1:1, destination = queue://detective/crawl/results, transactionId = null, expiration = 0, timestamp = 1229362749381, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = 8eecc26e-0122-31c1-8e86-59272a02c511, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1631715, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {repository=8eecc26e-0122-31c1-8e86-59272a02c511, licenceKey=0abc4d49d27c09827e9579538311b6a593731e38, version=3.8.5}, readOnlyProperties = true, readOnlyBody = true, droppable = false} as response was: 500
at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:33)
at org.apache.activemq.transport.http.HttpClientTransport.oneway(HttpClientTransport.java:91)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:40)
at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:74)
at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:79)
at org.apache.activemq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1233)
... 12 more
Caused by: java.io.IOException: Failed to post command: ActiveMQObjectMessage {commandId = 5, responseRequired = true, messageId = ID:SADERK745XP-2774-1229358014462-1:1:1:1:1, originalDestination = null, originalTransactionId = null, producerId = ID:SADERK745XP-2774-1229358014462-1:1:1:1, destination = queue://detective/crawl/results, transactionId = null, expiration = 0, timestamp = 1229362749381, arrival = 0, brokerInTime = 0, brokerOutTime = 0, correlationId = 8eecc26e-0122-31c1-8e86-59272a02c511, replyTo = null, persistent = true, type = null, priority = 4, groupID = null, groupSequence = 0, targetConsumerId = null, compressed = false, userID = null, content = org.apache.activemq.util.ByteSequence@1631715, marshalledProperties = null, dataStructure = null, redeliveryCounter = 0, size = 0, properties = {repository=8eecc26e-0122-31c1-8e86-59272a02c511, licenceKey=0abc4d49d27c09827e9579538311b6a593731e38, version=3.8.5}, readOnlyProperties = true, readOnlyBody = true, droppable = false} as response was: 500
at org.apache.activemq.transport.http.HttpClientTransport.oneway(HttpClientTransport.java:86)
... 16 more
I’m a bit worn out by all of this
-
Inappropriate?This is problem on our side... I've checked the logs - our server crashed due to memory troubles. Could you please set "crawlingPeriod" to QUARTER in the repo.conf file.
Please ran the crawler once more after that (you don't need to reinstall anything)
I am really sorry for all this mess. As a "candy": can I offer programeter services for free for three months for your development team?
I’m trying to be positive
-
-
Inappropriate?I seem to be running smoothly, can I update the crawler to run YEAR instead of QUARTER, or should I continue to use the QUARTER setting?
I’m rockin'!
-
Inappropriate?Good point. I am a bit scared that YEAR setting might provoke memory problems again (as your repository seems quite big).
But let's try YEAR setting if we will have troubles with that, then i would ask you to go back to QUARTER setting. Is that OK with you?
I’m eager to finally being able to provide analytics
-
Inappropriate?I'm ok with continuing to analyze my repository in small chunks.
I’m excited to get this done.
-
Inappropriate?Hi guys, I'm now having strange exceptions.
2008-12-23 13:58:34,924 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log revis
ion [Revision(Thu Jul 03 14:11:38 EDT 2008[26695])] 29/600 for repo: https://svn.garmin.com/inet/
2008-12-23 13:58:34,924 INFO [main] com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl - Measuring log entry
from revision [Revision(Thu Jul 03 14:11:38 EDT 2008[26695])] 1/1 for repo: https://svn.garmin.com/inet/
2008-12-23 14:04:16,268 INFO [main] com.goingsoftware.detective.measure.metrics.checkstyle.CheckstyleAuditListener - Exception occu
red: File: C:\DOCUME~1\saderk\LOCALS~1\Temp\tmp-QueryOptionTest6856544738253589006.java. Message: Got an exception - java.lang.NullP
ointerException Line: 0 Column: 0
com.goingsoftware.detective.measure.metrics.SyntaxException: Got an exception - java.lang.NullPointerException
at com.goingsoftware.detective.measure.metrics.checkstyle.CheckstyleAuditListener.addError(CheckstyleAuditListener.java:59)
at com.puppycrawl.tools.checkstyle.Checker.fireErrors(Checker.java:485)
at com.puppycrawl.tools.checkstyle.api.AbstractFileSetCheck.fireErrors(AbstractFileSetCheck.java:205)
at com.puppycrawl.tools.checkstyle.TreeWalker.process(TreeWalker.java:337)
at com.puppycrawl.tools.checkstyle.TreeWalker.process(TreeWalker.java:590)
at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:239)
at com.goingsoftware.detective.measure.metrics.checkstyle.CheckstyleMeasurer.measureExisting(CheckstyleMeasurer.java:41)
at com.goingsoftware.detective.measure.metrics.AbstractMeasurer.measure(AbstractMeasurer.java:45)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measure(RepositoryCrawlerServiceImpl.java:391)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measureLog(RepositoryCrawlerServiceImpl.java:33
8)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlSvnRepository(RepositoryCrawlerServiceImpl
.java:117)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlRepository(RepositoryCrawlerServiceImpl.ja
va:60)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:81)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:64)
at com.goingsoftware.detective.crawler.RepositoryCrawler.main(RepositoryCrawler.java:46)
2008-12-23 14:04:16,268 ERROR [main] com.goingsoftware.detective.measure.metrics.AbstractMeasurer - Meaurement failed C:\DOCUME~1\sa
derk\LOCALS~1\Temp\tmp-QueryOptionTest6856544738253589006.java
com.goingsoftware.detective.measure.metrics.SyntaxException: Got an exception - java.lang.NullPointerException
at com.goingsoftware.detective.measure.metrics.checkstyle.CheckstyleAuditListener.addError(CheckstyleAuditListener.java:59)
at com.puppycrawl.tools.checkstyle.Checker.fireErrors(Checker.java:485)
at com.puppycrawl.tools.checkstyle.api.AbstractFileSetCheck.fireErrors(AbstractFileSetCheck.java:205)
at com.puppycrawl.tools.checkstyle.TreeWalker.process(TreeWalker.java:337)
at com.puppycrawl.tools.checkstyle.TreeWalker.process(TreeWalker.java:590)
at com.puppycrawl.tools.checkstyle.Checker.process(Checker.java:239)
at com.goingsoftware.detective.measure.metrics.checkstyle.CheckstyleMeasurer.measureExisting(CheckstyleMeasurer.java:41)
at com.goingsoftware.detective.measure.metrics.AbstractMeasurer.measure(AbstractMeasurer.java:45)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measure(RepositoryCrawlerServiceImpl.java:391)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.measureLog(RepositoryCrawlerServiceImpl.java:33
8)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlSvnRepository(RepositoryCrawlerServiceImpl
.java:117)
at com.goingsoftware.detective.measure.services.RepositoryCrawlerServiceImpl.crawlRepository(RepositoryCrawlerServiceImpl.ja
va:60)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:81)
at com.goingsoftware.detective.crawler.RepositoryCrawler.crawl(RepositoryCrawler.java:64)
at com.goingsoftware.detective.crawler.RepositoryCrawler.main(RepositoryCrawler.java:46)
log4j: rolling over count=10485851
Any clues?
thanks, -
Inappropriate?Hi Keith!
Does the crawler crash after that?
At the moment to me it seems that it should continue working after such exceptions. Usually they mean that parser we use is unable to parse some of the files.
I’m wishing merry christmas
-
Inappropriate?It seems to run well enough after that, but I have seen recently some out of memory errors when it finishes a repository scan. I'm not sure the two issues are related.
I'll update this thread today if I encounter any more issues.
thanks,
EMPLOYEE
