Search:

Note that I have also included Possible_Leaks_Objects_Retained_By_Inner_Class_Back_References report in the same archive, which might be useful as well.

Hi Costin,

Tried to do more analysis with MAT, but it didn't show much of detail, however I got a trial license of YourKit analyzer and here are some reports:...

Hi Costin,

Please find reports from the most recent run.

https://dl.dropbox.com/u/95015919/08_02_heapdump-1343944821881_Leak_Suspects.zip...

Hi Costin,

I have deployed the app in another environment and will run several tests against it, will post the results as soon as I am done.


Sincerely,
David

Do you think I should go ahead and do more testing, or the issue is in the CACHE_CLASSES hashmap, which is also used in the hadoop-core-0.20.2-cdh3u3?

Can we also go back and elaborate why did we...

Yeah, it was easier to do local testing and I was hoping it would give some insight, but anyway I will perform another batch of tests.

We are using Cloudera distribution of hadoop, specifically...

Hi Costin,

Yes, I am using the latest snapshot and all of the submitted jobs are Tool implementations.
I will perform more testing in another environment and let you know the results.
...

Hi Costin,

I run several tests and I saw the following message:

URLs: [file:/project-2.9-SNAPSHOT/WEB-INF/classes/job-jars/test-job.jar]
Parent CL: ContextLoader@conductor
System CL:...

Hi Costin,

I browsed around and didn't find a new artifact for the patched version, are these correct places to look?
...

Could you please share your email address or shoot an email to davidgev at gmail dot com, since I am not able to attach full Leak report (it exceeds size limit of 100KB).


Sincerely,
David

Hi Costin,

Executed Leak Suspects report from Eclipse Memory Analyzer, attached please find the report in HTML format.
5070


Will try the new build today and let you know whether the issue is...

Hi Costin,

I tried to attach larger image, but seems there is a size limitation on the attachments, sorry for that.

I did several heap dumps, I am using jvisualvm for that. I will send you Heap...

Hi Costin,

Here is a new thread dump, please note that the application is working already for 2 days, so you can compare this with the previous one.

5068

Sincerely,
David

Thanks Costin,

The first thing I did today in the morning, is tuned some JVM GC properties, namely added:

-XX:+UseConcMarkSweepGC
-XX:+UseParNewGC
-XX:+CMSPermGenSweepingEnabled...

Hi Costin,

Thanks for taking look at it, I will try and let you know how it works for me.

One quick question: I just faced an issue in a long running Spring Batch process that submits jobs...

I was also wondering, is it possible to add more logging when running hadoop tool tasklets, such as to show Jar file, Tool Class, Hadoop Arguments, Hadoop Job Configuration Properties (the ones that...

When I am running the job from console, the counter values are printed after the job is done.
From the SHDP perspective, you are right, they are actually incremented, but I can't see them in console.

Suppose that each job calculates some stats during execution, for example:


TOTAL_USERS
ACTIVE_USERS
SUBSCRIBER_USERS
...


These are incremented in the reducer, as it finds more Users...

Ok, so usually when I am running the job from console after the job ends, all these statistics are printed out, so I can crawl the logs and extract important information for reporting purposes, such...

We are using org.apache.hadoop.mapred.Reporter for doing some reporting when running hadoop jobs.
We have enumeration with the counters and we do reporter.incrCounter(Enum, 1), and after a specific...

Hi Costin,

Is there a way to output Job Counters of the executing Hadoop Tool Tasklet?


Sincerely,
David

Hi guys,

Did anyone encounter a situation like this?

Hi,

I think it means that something else already registered that cache, probably some other dependency in your project.
Make sure to view the dependency tree and exclude duplications. You can use...

Hi everyone,

I am using Spring Batch and exposing some of the MBeans using the following code:



<!-- registry and serverConnector are used for connecting from a remote machine -->
...

Ahh, I see. We are using Maven Enforcer, since having duplicate classes in the classpath can potentially cause unknown issues and that has been the case several times when we had transitive...