Donnerstag, 25. Februar 2010

Heap Dump Analysis with Memory Analyzer, Part 2: Shallow Size

In the second part of the blog series dedicated to heap dump analysis with Memory Analyzer (see previous post here) I will have a detailed look at the shallow size of objects in the heap dump. In Memory Analyzer the term shallow size means the size of an object itself, without counting and accumulating the size of other objects referenced from it. For example the shallow size of an instance of java.lang.String will not include the memory needed for the underlying char[].

At first look this seems like a clear definition and a relatively boring topic to read about. So why did I decide to write about it? Because despite of the understandable definition, it is not always straightforward (for the tool developers) to calculate the shallow size, or (for a user) to understand how the size was calculated. The reasons? – different JVM vendors, different pointer sizes (32 / 64 bit), different dump formats, insufficient data in some heap dumps, etc … These factors could lead to small differences in the shallow sizes for objects of the same type shallow sizes being displayed for objects of the same type, and thus to questions.

Is it really important to know the precise size? Not necessarily. If you got a heap dump from an OutOfMemoryError in your production system, and MAT helps you to easily find the leak suspect there – let’s say it is some 500Mb big object - then the shallow size of every individual object accumulated in the suspect's size doesn’t really matter. The suspect is clear and you can go on and try to fix the problem.

On the other hand, if you are trying to understand the impact of adding some fields to your “base” classes, then the size of the individual instance can be of interest.

In the rest of the post I would have a look at the information available (or missing) in the different snapshot formats, explain what MAT displays as shallow size in the different cases, and try to answer some of the questions related to the shallow size which we usually get. If you are interested, read further.

As I mentioned already, the various snapshot formats contain different pieces of information about the objects. I will look at each of them separately, and additionally differentiate between object instances and classes. For more information on the different heap dump formats see part one of the blog series.

Instance Size in HPROF Heap Dumps

The heap dumps in HPROF binary format do not provide the correct size of each instance. What they provide is the number of bytes used to store the necessary data in the heap dump, but not the number of bytes the VM really needs to store the instance in the heap. Therefore, in MAT we have to (and attempt to) model how the VM would store the instance and how much memory it would need.

The approach we have to calculate the sizes is the following:

Instance Shallow Size = [object header] + space for fields of super class N + [some bytes because of alignment] + … + space for own fields + [some bytes because of alignment]

The sizes originally provided in the hprof file do not contain the object header and the additional space the VM uses to have the object addresses aligned in a certain way. These are namely the parameters we guess on our own. Does this always work? No. Unfortunately not. In the Bugzilla entry 231296 you can find some discussions on the topic, and also what the current state is. Here is just a short summary:

  • The formula we use gives correct results for dumps from 64 bit Sun VMs (1.4, 1.5, 1.6)

  • With the formula we use to calculated the sizes for dumps from 32 bit Sun VMs we observed: correct results for 1.6 dumps; small deviations for a handful of objects in 1.5 dumps

  • For the special case of a x64 Sun VM with compressed OOPs we have no solution at the moment. We haven't found a way to guess from the HPROF file that the pointers were compressed

  • Instance Size in IBM System Dumps (read with DTFJ)

    The DTFJ provides already the correct instance size for the objects, and in MAT we don’t have to do any guessing – the instance sizes are correct. What needs to mentioned is that it may happen that two instances of the same class and in the same heap dump have different shallow sizes. The Memory Analyzer was not prepared until recently to handle such a case. More information on when exactly such a difference could appear and some discussions on the necessary changes can be found in Bugzilla entry 301228.

    Instance Size in PHD Dumps

    The sizes provided in the PHD (Portable Heap Dumps from IBM JVMs) dumps are also correct and MAT just displays them without any further computations.

    Class Size in HPROF

    The HPROF format does not provide information about the memory needed for a class - for bytecode, for jitted code, etc... For every class the Memory Analyzer will show as shallow size the sum of the shallow sizes of all static fields of the class.

    Class Size in IBM System Dumps

    DTFJ provides more information about the classes sizes. The shallow size for classes reported by MAT includes the size of all methods (bytecode and jitted code sections) and also the on heap size of the java.lang.Class object.

    Class Size in PHD Dumps

    The PHD dumps do not contain information about the method sizes. The shallow size for classes in PHD dumps is just the size of the java.lang.Class object.

    Shallow Size of a Set of Objects

    The "Shallow Size" column appears in many views where objects have been aggregated in groups based on different criteria. The shallow size of a set of objects is just the sum of the shallow sizes of the individual objects in the set. There are two things to mention here, which have raised questions in the past:

  • in a class histogram (i.e. when objects are aggregated based on their class) the table may contain more than one entry with the same class name. This happens if the same class is loaded with more than one class loaders. If one is interested in the total shallow size of the instances of all classes, one can filter the histogram and sum up the sizes

  • the classes are added to the record for java.lang.Class. This means that the shallow size in the histogram entry for java.lang.Class is the sum of the shallow sizes of all classes (calculated as described above in the "Class Size ..." paragraphs.

  • Summary

    My personal view is that if one is using the Memory Analyzer to find the root cause of an OutOfMemoryError, then the shallow sizes of the individual objects are not that important.

    If for a given purpose one needs to understand in detail the sizes of objects, then it is important to remember that they depend on the concrete JVM and heap dump type, and that in some cases the displayed sizes are not given by the VM but are calculated in MAT. I hope that the short overview given in this blog could be helpful for better understanding these details.

    What Comes Next?

    In the next post I plan to write again about size, but a different one - the retained (or keep alive) size of objects and object sets.

    23 Kommentare:

    1. i would like to ream more, regarding performance monitoring im quite novice but i have issue on WAS 7 that i HAVE to fix it.

    2. Where is part 3? I'm looking forward to read it :)

    3. oh my god! you update your post so slowly

    4. Here is  my testimony on how I was cured of HIV by Dr Akhigbe,with his natural herbal medicine.  on a regular basis in efforts to help others when I could. As you may know, each donation is tested.  Well, on July 6th I had a meeting with a Red Cross representative and was told that I had HIV. “What went through your mind when you heard that "Rose" Good question reader! To be honest, I thought my life was over, that I would ever find love, get married, have children or anything normal. Lucky for me I have an amazing support system.  My family supported me then I never thought that I was invincible to STD s or pregnancy or anything else parents warn their kids about. I just didn’t think about it. I was in a monogamous relationship and thought that I asked the right questions.  We even talked about marriage Scary.  During that time I was in college and donated blood on a re as well. who helped me in search of cure through the media.there we saw a good testimony of sister 'Kate' about the good work of Dr Akhigbe natural herbal medicine cure.then I copied his email address and contacted him. A good herbalist doctor with a good heart, he is kind, loving and caring. He replied back to my message and told me what to do. After a week the doctor sent me my herbal medicine and instructed me how to take it.Yes it worked very well, after drinking it I went to the hospital for another test and everything turned negative. What a wonderful testimony I can never forget in my life. Dr Akhigbe is a man who gave me a life to live happily forever so all I want you all to believe and know that cure of HIV is real and herbs is a powerful medicine and it works and heals.  Dr Akhigbe also used his herbal medicine to cure diseases like:   HERPES, DIABETES, SCABIES, HEPATITIS A/B, STROKE, CANCER, ALS, TUBERCULOSIS, ASTHMA, PENIS ENLARGEMENT, MALARIA, LUPUS, LIVER CIRRHOSIS, DEPRESSION, HIV/AIDS, EPILEPSY, BACTERIAL, DIARRHEA, HEART DISEASES, HIGH BLOOD PRESSURE, PARKINSON'S, ALZHEIMER, COLD URTICARIA, HUMAN PAPILLOMAVIRUS,INSOMNIA,  BACTERIAL VAGINOSIS, SCHIZOPHRENIA, JOINT PAIN, STOMACH PAIN, CHROME DISEASES, CHLAMYDIA, INSOMNIA HEARTBURN, ,  THYROID, MAR BURG DISEASES, MENINGITIS, ARTHRITIS, BODY WEAK, SMALLPOX, DENGUE, FEVER, CURBS, CHANCRE, AND OTHERS VARIOUS DISEASES/ VIRUS.   You are there and you find yourself in any of these situations, kindly contact Dr Akhigbe now to help you get rid of it. Here is his email address:      or you can write to him on whats app with his phone number:   +2349010754824.
        My appreciation is to share his testimony for the world to know the good work Dr Akhigbe has done for me and he will do the same for you.

    5. Nice Post...
      Why my HP printer displays an offline error when it's not?
      Offline error in the HP printer occurs when due to network connectivity issues, or you have not set your HP printer rightly on the computer. When your HP printer says offline, then the first step you need to take is to check the router and printer connection; if you found it okay, you will have to take some advanced measures. Sometimes, an outdated printer driver also causes an offline error; updating them will effectively resolve the issue. You will have to visit the HP Support website, follow a few instructions on the screen, click printer under let's identify your printer to get started. Now choose download drivers under the Driver section, and within a few seconds, the file gets downloaded to your computer. After downloading, double-click over the installer file and keep following instructions to install it on your computer. These steps will resolve the offline problem.

    6. Nice Post...
      Where can I register myself to access Microsoft applications?
      The only secured and well trusted place online to get an authorised Microsoft account is If you are a new user, you must know that to access any Microsoft application, users must have a product key. Just like any other paid software it ensure that you are using authentic software with unlimited services and no one has access to your account.

    7. There are multiple NBN New Development Applications accessible for the people willing to employ broadband services. However, there are manifold factors that need to be considered while deciding which one is suitable for the place.

      Here, in this brief article, we spoke about all those points that need to be mentioned beforehand having the NBN Services.

      value services

    8. Contract Law Assignment Help online. We are the most popular and affordable service provider in the USA, with a team of experts who make sure to curate the content within a short period of time by gathering relevant information from reliable sources.

      Read More :

    9. Great worked and good for share great Blog. It is just amazing and so convincing for those who are finding the Best Car Transport In Hyderabad. LogisticGuru.PVT is really a great Car Transport In Hyderabad service provider. LogisticGuru.PVT is dealing with affordable Car transportation services. The company tends to cover a wide range of service. Visit their website and get more information about it.

    10. Sharp as can be clear, man! It's genuinely what I need!! Much respected! Appreciative to you. Individuals like Hitler and Mussolini changed into a befuddling distortion on culture and human stunning new disturbing new astonishing surprising unforeseen new development. In this case you are not able to Find WPS Pin on HP Printer then visit our website. We help you to all printer issues.

    11. You cause them to cover zones on your blog. I will return once more.
      Hi I providing hosting in 80% off. For more information visit my website click here Hostinger Coupon 2021

    12. Our Assignment Help can be your lifesaver if you have to submit a lot of assignments that need to be completed within a short period of time. We have experts who can curate the best content for you by gathering relevant information from reliable sources and then formatting it in a customized format that has been mentioned by the university or college.

    13. HP Wireless Printer Setup

      Connecting a printer over a USB cable is pretty easy. But connecting a wireless printer can become very challenging quite often. Follow the HP wireless printer setup process to install it in no time.

      1. Take the printer out of the box and remove all tapes from it before starting.
      2. Plug in the power cord, turn on the printer and install the print cartridges. Allow your printer to complete its startup routine, which includes printing of an alignment page (if applicable).
      3. Go to the manufacturer's website and download drivers for your printer model.
      4. Install the drivers and follow the instructions during hp wireless printer setup process.
      5. In the meantime, go to printer's control panel and use wireless printer setup wizard.
      6. Identify your wifi network and connect the printer to it using its correct credential.
      7. Complete the wireless printer setup process by connecting to your printer model as it would appear during installation.

      hp wireless printer setup

    14. Canon printer support phone number

      Canon wireless printer setup is very easy if compared with other available printers in the market. In case of any problem or confusion, canon printer support phone number is easy to get from the supplied manual. Follow canon printer setup process to install canon wireless printer.

      1. Take the printer out of the box and remove all tapes from it before starting.
      2. Plug in the power cord, turn on the printer and install the print cartridges. Allow your printer to complete its startup routine, which includes printing of an alignment page (if applicable).
      3. Go to the manufacturer's website and download drivers for your printer model.
      4. Install the drivers and follow the instructions during canon wireless printer setup process.
      5. In the meantime, go to priter's control panel and use wireless printer setup wizard.
      6. Identify your wifi network and connect the printer to it using its correct credential.
      7. Complete the wireless printer setup process by connecting to your printer model as it would appear during installation.

      canon printer support number

    15. Brother Printer Setup

      We are one of the most reliable, and independent 3rd party printer technical support and service providers, providing 24/7 online or on-call technical support services to the users of brother printer at a very nominal charge. Our printer experts have extensive knowledge to set up brother wireless printer in the simple ways. For brother printer setup, you can call our Brother printer support experts 24/7.

      Brother Printer Support

    16. Pogo games not loading

      Check for correct User Name or Screen Name. Most of the time we rely on the browser’s auto-complete feature and select wrongly entered screen name in the past.
      Passwords are case sensitive. Therefore, always check the letter’s cases of the entered password. If possible, type the password as you use in a notepad and then copy and paste the letters in the password field. Below is a list of things you should always check before entering into troubleshooting.
      1. If any of your family member shares the same account, ask them if they might have changed the screen name or password.
      2. Look into your email inbox to get the actual screen name when you had registered with Pogo
      3. If you find any point relevant to your case, you can go to the article explaining how to reset password of your Pogo account and follow all the steps carefully to reset your pogo account password. in problems

    17. Gmail not working

      When your email stops working or gets hacked, don’t panic. Call Email Support Phone Number and let the highly skilled Specialists get you up running.
      Troubleshooting steps you can try to fix basic email problems:
      1: Make sure you’re connected to the web.
      2: Close the app, then reopen it.
      3: Sign out, then sign back in.
      4: Clearing the cookies and history of browser
      5: Check your browser extensions or applications and remove unwanted extensions

      Gmail not receiving email

    18. Ethylene, also known as ethene or green ethylene, is an example of a hydrocarbon.

      ethylene production

    19. Nepal Tour Packages Wonderful post! I am getting the wonderful ideas to explore top popular tourist attractions of Nepal. Travelers can learn important tips to get the best deals for Nepal tourism packages. This blog is very interesting for me. I am very happy for this amazing blog.