umm_malloc manual merge with upstream

[mhightower83]

Noteworthy Changes:

• ESP.getHeapFragmentation() is now calculated with blocks. Previously the calculation was done with bytes.
• If frequent calls are made to ESP.getHeapFragmentation(), -DUMM_INLINE_METRICS would reduce long periods of interrupts disabled caused by frequent calls to umm_info(). Instead, the computations get distributed across each malloc, realloc, and free. This appears to require an additional 116 bytes of IRAM vs using UMM_STATS with UMM_INFO.
• When both UMM_STATS and UMM_INLINE_METRICS are defined, macros and structures have been optimized to reduce duplication.
• ESP.getFreeHeap(), umm_free_heap_size(), xPortGetFreeHeapSize(), system_get_free_heap_size(), ... previously reported values that were 8 bytes too large. In contrast ESP.getMaxFreeBlockSize() was not affected.
• Fixes the divide by zero exception that was caused by calling ESP.getHeapStats(NULL, NULL, &hfrag); with a fully allocated heap.

New functions from upstream:

• int umm_usage_metric( void ), returns the ration of 100 * (used/free blocks). Returns -1 when free blocks is zero.
• int umm_fragmentation_metric( void ), returns the same information as ESP.getHeapFragmentation(). When UMM_INLINE_METRICS is defined ESP.getHeapFragmentation() is a thin wrapper to umm_fragmentation_metric().

* Edited to reflect current PR content.

[mhightower83]

• ESP.getMaxFreeBlockSize() is now adjusted to allow for umm_heap overhead. It is now possible for a malloc(ESP.getMaxFreeBlockSize());

@devyte While working through memory boundary calculations on HeapMeteric.ino, I find myself questioning whether I should have done this. At the moment I am on the fence.

[devyte]

I think not. The current behavior is that it's not possible to do malloc(maxBlock), which can be unexpected. However, from your explanations on gitter, even if you manage to make it work for maxBlock, it could potentially fail e. g. for maxBlock-overhead, because then the remaining space can't be divided. And that is also unexpected, which means the original cause for fixing it isn't resolved.
Like you said, too many corner cases, and hence too fragile.
I suggest reverting that. A comment has already been added to the docs stating that the maxBlock means free mem chunk, and that size can't be malloced due to overhead.

[earlephilhower]

After re-reading this a couple times it LGTM. The only non-debug path change I caught was the realloc that exactly fits (uncommon, so hard to test but on reading the code looks good). The debug/freeblock stuff I'm less worried about even though the changes are larger since it's not anywhere near as critical as the main malloc code.

@devyte, @d-a-v, can we please get your eyes on this?

[devyte]

Nothing jumps out at me

[mhightower83]

In umm_realloc:

• The #if defined(UMM_REALLOC_MINIMIZE_COPY) section tracks the content of the upstream with some local macros added. Back when I made my 1st update to umm_malloc PR, I found the upstream had been refactored and removed the defragmenting properties that were originally present. It took some looking to see the logic, it didn't have any comments to make it stand out.
• I added the #elif defined(UMM_REALLOC_DEFRAG) to recreate and preserve the defragmenting functionality that was lost. This is the default build option we have set in umm_malloc_cfg.h. I have not done any structured testing to confirm; however, I think this to be the best option when considering the amount of reallocates that can occur with the Strings library.

The upstream umm_malloc is a .c program and often does pointer math with void * pointers. When the .c is changed to .cpp the C++ compiler does not accept these. I previously changed most of them to char * in my previous sync up with the upstream. This time round unless a char * value was useful later, I leaned toward using uintptr_t casting.

[d-a-v]

I like UMM_INLINE_METRICS even it is still not enabled by default, maybe some more perf tests need to be performed.
Thanks for this fine work, thank you for keeping it sync with upstream.