Donnerstag, 25. Februar 2010

Heap Dump Analysis with Memory Analyzer, Part 2: Shallow Size

In the second part of the blog series dedicated to heap dump analysis with Memory Analyzer (see previous post here) I will have a detailed look at the shallow size of objects in the heap dump. In Memory Analyzer the term shallow size means the size of an object itself, without counting and accumulating the size of other objects referenced from it. For example the shallow size of an instance of java.lang.String will not include the memory needed for the underlying char[].

At first look this seems like a clear definition and a relatively boring topic to read about. So why did I decide to write about it? Because despite of the understandable definition, it is not always straightforward (for the tool developers) to calculate the shallow size, or (for a user) to understand how the size was calculated. The reasons? – different JVM vendors, different pointer sizes (32 / 64 bit), different dump formats, insufficient data in some heap dumps, etc … These factors could lead to small differences in the shallow sizes for objects of the same type shallow sizes being displayed for objects of the same type, and thus to questions.

Is it really important to know the precise size? Not necessarily. If you got a heap dump from an OutOfMemoryError in your production system, and MAT helps you to easily find the leak suspect there – let’s say it is some 500Mb big object - then the shallow size of every individual object accumulated in the suspect's size doesn’t really matter. The suspect is clear and you can go on and try to fix the problem.

On the other hand, if you are trying to understand the impact of adding some fields to your “base” classes, then the size of the individual instance can be of interest.

In the rest of the post I would have a look at the information available (or missing) in the different snapshot formats, explain what MAT displays as shallow size in the different cases, and try to answer some of the questions related to the shallow size which we usually get. If you are interested, read further.

As I mentioned already, the various snapshot formats contain different pieces of information about the objects. I will look at each of them separately, and additionally differentiate between object instances and classes. For more information on the different heap dump formats see part one of the blog series.

Instance Size in HPROF Heap Dumps

The heap dumps in HPROF binary format do not provide the correct size of each instance. What they provide is the number of bytes used to store the necessary data in the heap dump, but not the number of bytes the VM really needs to store the instance in the heap. Therefore, in MAT we have to (and attempt to) model how the VM would store the instance and how much memory it would need.

The approach we have to calculate the sizes is the following:

Instance Shallow Size = [object header] + space for fields of super class N + [some bytes because of alignment] + … + space for own fields + [some bytes because of alignment]

The sizes originally provided in the hprof file do not contain the object header and the additional space the VM uses to have the object addresses aligned in a certain way. These are namely the parameters we guess on our own. Does this always work? No. Unfortunately not. In the Bugzilla entry 231296 you can find some discussions on the topic, and also what the current state is. Here is just a short summary:

  • The formula we use gives correct results for dumps from 64 bit Sun VMs (1.4, 1.5, 1.6)

  • With the formula we use to calculated the sizes for dumps from 32 bit Sun VMs we observed: correct results for 1.6 dumps; small deviations for a handful of objects in 1.5 dumps

  • For the special case of a x64 Sun VM with compressed OOPs we have no solution at the moment. We haven't found a way to guess from the HPROF file that the pointers were compressed

  • Instance Size in IBM System Dumps (read with DTFJ)

    The DTFJ provides already the correct instance size for the objects, and in MAT we don’t have to do any guessing – the instance sizes are correct. What needs to mentioned is that it may happen that two instances of the same class and in the same heap dump have different shallow sizes. The Memory Analyzer was not prepared until recently to handle such a case. More information on when exactly such a difference could appear and some discussions on the necessary changes can be found in Bugzilla entry 301228.

    Instance Size in PHD Dumps

    The sizes provided in the PHD (Portable Heap Dumps from IBM JVMs) dumps are also correct and MAT just displays them without any further computations.

    Class Size in HPROF

    The HPROF format does not provide information about the memory needed for a class - for bytecode, for jitted code, etc... For every class the Memory Analyzer will show as shallow size the sum of the shallow sizes of all static fields of the class.

    Class Size in IBM System Dumps

    DTFJ provides more information about the classes sizes. The shallow size for classes reported by MAT includes the size of all methods (bytecode and jitted code sections) and also the on heap size of the java.lang.Class object.

    Class Size in PHD Dumps

    The PHD dumps do not contain information about the method sizes. The shallow size for classes in PHD dumps is just the size of the java.lang.Class object.

    Shallow Size of a Set of Objects

    The "Shallow Size" column appears in many views where objects have been aggregated in groups based on different criteria. The shallow size of a set of objects is just the sum of the shallow sizes of the individual objects in the set. There are two things to mention here, which have raised questions in the past:

  • in a class histogram (i.e. when objects are aggregated based on their class) the table may contain more than one entry with the same class name. This happens if the same class is loaded with more than one class loaders. If one is interested in the total shallow size of the instances of all classes, one can filter the histogram and sum up the sizes

  • the classes are added to the record for java.lang.Class. This means that the shallow size in the histogram entry for java.lang.Class is the sum of the shallow sizes of all classes (calculated as described above in the "Class Size ..." paragraphs.

  • Summary

    My personal view is that if one is using the Memory Analyzer to find the root cause of an OutOfMemoryError, then the shallow sizes of the individual objects are not that important.

    If for a given purpose one needs to understand in detail the sizes of objects, then it is important to remember that they depend on the concrete JVM and heap dump type, and that in some cases the displayed sizes are not given by the VM but are calculated in MAT. I hope that the short overview given in this blog could be helpful for better understanding these details.

    What Comes Next?

    In the next post I plan to write again about size, but a different one - the retained (or keep alive) size of objects and object sets.

    18 Kommentare:

    1. i would like to ream more, regarding performance monitoring im quite novice but i have issue on WAS 7 that i HAVE to fix it.

    2. Where is part 3? I'm looking forward to read it :)

    3. oh my god! you update your post so slowly

    4. Here is  my testimony on how I was cured of HIV by Dr Akhigbe,with his natural herbal medicine.  on a regular basis in efforts to help others when I could. As you may know, each donation is tested.  Well, on July 6th I had a meeting with a Red Cross representative and was told that I had HIV. “What went through your mind when you heard that "Rose" Good question reader! To be honest, I thought my life was over, that I would ever find love, get married, have children or anything normal. Lucky for me I have an amazing support system.  My family supported me then I never thought that I was invincible to STD s or pregnancy or anything else parents warn their kids about. I just didn’t think about it. I was in a monogamous relationship and thought that I asked the right questions.  We even talked about marriage Scary.  During that time I was in college and donated blood on a re as well. who helped me in search of cure through the media.there we saw a good testimony of sister 'Kate' about the good work of Dr Akhigbe natural herbal medicine cure.then I copied his email address and contacted him. A good herbalist doctor with a good heart, he is kind, loving and caring. He replied back to my message and told me what to do. After a week the doctor sent me my herbal medicine and instructed me how to take it.Yes it worked very well, after drinking it I went to the hospital for another test and everything turned negative. What a wonderful testimony I can never forget in my life. Dr Akhigbe is a man who gave me a life to live happily forever so all I want you all to believe and know that cure of HIV is real and herbs is a powerful medicine and it works and heals.  Dr Akhigbe also used his herbal medicine to cure diseases like:   HERPES, DIABETES, SCABIES, HEPATITIS A/B, STROKE, CANCER, ALS, TUBERCULOSIS, ASTHMA, PENIS ENLARGEMENT, MALARIA, LUPUS, LIVER CIRRHOSIS, DEPRESSION, HIV/AIDS, EPILEPSY, BACTERIAL, DIARRHEA, HEART DISEASES, HIGH BLOOD PRESSURE, PARKINSON'S, ALZHEIMER, COLD URTICARIA, HUMAN PAPILLOMAVIRUS,INSOMNIA,  BACTERIAL VAGINOSIS, SCHIZOPHRENIA, JOINT PAIN, STOMACH PAIN, CHROME DISEASES, CHLAMYDIA, INSOMNIA HEARTBURN, ,  THYROID, MAR BURG DISEASES, MENINGITIS, ARTHRITIS, BODY WEAK, SMALLPOX, DENGUE, FEVER, CURBS, CHANCRE, AND OTHERS VARIOUS DISEASES/ VIRUS.   You are there and you find yourself in any of these situations, kindly contact Dr Akhigbe now to help you get rid of it. Here is his email address:      or you can write to him on whats app with his phone number:   +2349010754824.
        My appreciation is to share his testimony for the world to know the good work Dr Akhigbe has done for me and he will do the same for you.

    5. Nice Post...
      Where can I register myself to access Microsoft applications?
      The only secured and well trusted place online to get an authorised Microsoft account is If you are a new user, you must know that to access any Microsoft application, users must have a product key. Just like any other paid software it ensure that you are using authentic software with unlimited services and no one has access to your account.

    6. There are multiple NBN New Development Applications accessible for the people willing to employ broadband services. However, there are manifold factors that need to be considered while deciding which one is suitable for the place.

      Here, in this brief article, we spoke about all those points that need to be mentioned beforehand having the NBN Services.

      value services

    7. Contract Law Assignment Help online. We are the most popular and affordable service provider in the USA, with a team of experts who make sure to curate the content within a short period of time by gathering relevant information from reliable sources.

      Read More :

    8. Sharp as can be clear, man! It's genuinely what I need!! Much respected! Appreciative to you. Individuals like Hitler and Mussolini changed into a befuddling distortion on culture and human stunning new disturbing new astonishing surprising unforeseen new development. In this case you are not able to Find WPS Pin on HP Printer then visit our website. We help you to all printer issues.

    9. You cause them to cover zones on your blog. I will return once more.
      Hi I providing hosting in 80% off. For more information visit my website click here Hostinger Coupon 2021

    10. How do I recover my cash app account without email?
      Hello, do you want to recuperate your money application account without an email then you have picked the correct way to realize How do I recover my cash app account without email? On this way, you will handily get the method to recuperate a money application account without email. To get that methodology you simply need to tap on this given connection here and from that point onward, it involves only a couple of moments.


    12. – – The great performance and dependability of HP wireless printers are well known. HP has a lot of expertise producing printers because they've been in the printing market for a long time.

    13. atuoyo post utme past questionsSo much happy to read this blog post, you have many information put together on this article and i really appreciate your effort in sharing it

    14. Positive site, where did u come up with the information on this posting? I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work. Do you know What Is A Good Clicking Speed? If not, then visit check out my blog and find out about it here.