Linux misreports process size with heap multi-mapping

There is this issue in Linux when a process maps the same physical memory to multiple addresses at once ("heap multi-mapping").

For example, Java's ZGC garbage collector does this.

When heap multi-mapping occurs, Linux (at least the versions I use) miscalculates (overreports) the resident process size. This affects top and all programs which query the OS for process sizes.

In effect, I have no more way of knowing how big a process actually is.

This screenshot is what top shows for a ZGC-enabled Java program with <<200 MB of Java objects. So in reality, the process is around 500 MB, not several GB.

So, to get to the question: Does anyone know a Linux version (or patch or tool) that can correctly report process sizes in this case?

Edit: I now made my own little tool specifically for the ZGC use case.

Answer

You could ask the kernel about how the process uses memory (/proc/PID/smaps) and then do the math based on the reported Shared and Private memory. The shared memory must only be counted once.

Below is some code that parses smaps and sums up the various types of memory usage for a list of pids. The unique set size (uss) memory is the private memory of the process, and might give a better idea about how much memory the process actually uses. Ultimately it becomes a question of how you would like to factor the shared memory into the equation.

rss = '.+?Rss:\s+(\d+)'
pss = '.+?Pss:\s+(\d+)'
shared_clean = '.+?Shared_Clean:\s+(\d+)'
shared_dirty = '.+?Shared_Dirty:\s+(\d+)'
priv_clean = '.+?Private_Clean:\s+(\d+)'

priv_dirty = '.+?Private_Dirty:\s+(\d+)'
MEM_REGEXP = /#{rss}#{pss}#{shared_clean}#{shared_dirty}#{priv_clean}#{priv_dirty}/m

def self.get_memory_map( pids)

    memory_map = {}
    #memory_map[ :pids_searched] = pids
    memory_map[ :pids_found] = {}
    memory_map[ :rss] = 0
    memory_map[ :pss] = 0

    memory_map[ :shared_clean] = 0
    memory_map[ :shared_dirty] = 0
    memory_map[ :rss_shared] = 0
    memory_map[ :priv_clean] = 0
    memory_map[ :priv_dirty] = 0
    memory_map[ :uss] = 0

    pids.each do |pid|
        begin
            # We don't want to see any "/proc/9723: No such file or directory" messages in stderr

            lines = nil
            Util.silently { lines = File.read( "/proc/#{pid}/smaps") }
        rescue
            lines = nil
        end
        if lines
            lines.scan(MEM_REGEXP) do |rss,pss,shared_clean,shared_dirty,priv_clean,priv_dirty|
                memory_map[ :pids_found][pid] = true
                memory_map[ :rss] += rss.to_i
                memory_map[ :pss] += pss.to_i

                memory_map[ :shared_clean] += shared_clean.to_i
                memory_map[ :shared_dirty] += shared_dirty.to_i
                memory_map[ :rss_shared] += shared_clean.to_i + shared_dirty.to_i
                memory_map[ :priv_clean] += priv_clean.to_i
                memory_map[ :priv_dirty] += priv_dirty.to_i
                memory_map[ :uss] += priv_clean.to_i + priv_dirty.to_i
            end
        end
    end
    memory_map[ :pids_found] = memory_map[ :pids_found].keys

    return memory_map
end