=head1 Ideas and Plans for Memory Profiling with NYTProf

XXX I<This mostly a dumping ground for thoughts, and links at the moment>.
Somewhere to gather the info so it'll be easier to find when someone
wants to work on it.

It's also focussed on runtime profiling of total memory usage over time
(rather than a detailed analysis of who 'owns' what memory as some particular
point in time, like the end of the program).

NYTProf currently only measures time and supports a limited range of "clocks"
(realtime vs cputime). For profiling memory we need to add a new kind of
"clock" that measures memory usage. Since we're generalizing the concept of
what gets measured (and how we get the info from the system) a better name
than "clock" would be "probe".

Conjectural terminology:

    "Probe" means some measuring mechanism like get_clock(), times(), getrusage(),
    that may yield multiple pieces of information with a single call.

    "Measure" is one specific item generated by a probe.

    Probe "time"   uses times(), measures: "time.user", "time.user+sys" etc
    Probe "clock"  uses clock_gettime(), measures: "clock.realtime", "clock.monotonic" etc
    Probe "rusage" uses getrusage(), measures: "rusage.majflt", "rusage.nvcsw" etc
    Probe "memory" measures: "memory.bytes", "mem.allocs" etc
    Probe "arena"  measures: "arena.svs", "arena.bytes" etc

Generalize the concepts of probes. Have a structure defining a 'probe' with
pointers to functions to get the values, subtract values to get relative ticks,
return the tick units etc. Give them names and attributes (cpu, realtime etc).
User could then pick a probe by name. By default we'd pick the best available
realtime probe.
Use the subtraction logic where we currently handle times in the statement and
subroutine profilers.

=head1 Email threads

"Memory profiling in Devel::NYTProf?: - June 2009
http://groups.google.com/group/develnytprof-dev/browse_frm/thread/1df4cba3001cd4e4/136812b44e9f7631
Talking about the problems of measuring memory usage of the whole process re:
http://blog.robin.smidsrod.no/index.php/2009/05/26/memory-footprint-of-popular-cpan-modules

"Memory profiling possibilities in NYTProf" - September 2009
http://groups.google.com/group/develnytprof-dev/browse_frm/thread/c711c132216a3cea/035012e3dc2971ec
This includes a detailed overview of the issues.

"profiling memory" - Dec 2009
http://groups.google.com/group/develnytprof-dev/browse_frm/thread/5ffd24200866b0c1/201b58c18d826aaa
Nicholas Clark offers an experimental patch that intercepts malloc and free
and makes NYTProf measure memory usage.

=head1 Possibly Relevant Perl Modules

Per-process memory information:

    http://search.cpan.org/perldoc?Devel::Mallinfo

Arena, stash, and pad based memory reporters:

    http://search.cpan.org/perldoc?Devel::Gladiator
    http://search.cpan.org/dist/Internals-DumpArenas/
    http://search.cpan.org/dist/Internals-GraphArenas/
    http://search.cpan.org/perldoc?Devel::DumpSizes
    http://search.cpan.org/perldoc?Devel::Arena

Per-object memory size reporters:

    http://search.cpan.org/perldoc?Devel::Size
    http://search.cpan.org/perldoc?Devel::Size::Report

Others:

    http://search.cpan.org/dist/Devel-Memalyzer/
    http://search.cpan.org/perldoc?Devel::Memalyzer::Plugin::ProcSmaps

=head1 Other Items of Interest

"Memory Efficient Perl" slides by jjore

    http://docs.google.com/present/view?id=dg7kgpct_24cjs3c9fv
    http://diotalevi.isa-geek.net/~josh/090402/frontend.png

Other profile/memory visualization tools

    http://netjam.org/spoon/viz/
    http://java.dzone.com/announcements/visualvm-12-great-java

http://blogs.perl.org/users/alex_balhatchet/2012/01/debugging-memory-use-in-perl---help.html

http://stackoverflow.com/questions/8715611/can-i-use-dtrace-on-os-x-10-5-to-determine-which-of-my-perl-subs-is-causing-the

=cut
