Malloc breaks the heap if (freeblocksize - 16) < newblocksize < (freeblocksize - 7)

# 08 Oct 2014

I would expect that when I use malloc to allocate a block on the heap, one of the following things would happen: Either the block would be allocated correctly and I would get a pointer to it; or there would be no room for the block and I'd get a null pointer. However, I've found that if the size of the block I'm allocating is greater than (freeblocksize - 16) and less than (freeblocksize - 7) (where freeblocksize is the size of the last free block on the heap), the malloc statement returns a pointer, but the heap is corrupted.

The following code demonstrates this:

#include "mbed.h"

int main() {
    // The following lines print out a list of the blocks on the heap
    // The free block size is 0xfd4 
    fprintf(stderr, "\nHeap at start\n");
    __heapvalid((__heapprt)fprintf,stderr,1);
    // We now allocate a block close to the size of the free block
    // If the block size falls between the (0xfd4 - 15) and (0xfd4 - 8),
    // malloc returns a valid pointer but the heap is corrupted.
    // If the block is larger or smaller, the malloc statement operates
    // correctly.
//    char* ptr = (char*)malloc (0xfd4 - 16);   // Works correctly
    char* ptr = (char*)malloc (0xfd4 - 15);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 14);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 13);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 12);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 11);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 10);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 9);    // Fails
//    char* ptr = (char*)malloc (0xfd4 - 8);    // Fails
//    char* ptr = (char*)malloc (0xfd4 - 7);    // Works correctly
    fprintf(stderr, "\nHeap after allocation, ptr = %p\n", ptr);
    __heapvalid((__heapprt)fprintf,stderr,1);
    char* ptr2 = (char*)malloc (20);
    fprintf(stderr, "\nHeap after another allocation, ptr2 = %p\n", ptr2);
    __heapvalid((__heapprt)fprintf,stderr,1);
}

Is this something I'm doing wrong or is it a bug? Can anyone think of a workaround?

I would expect that when I use malloc to allocate a block on the heap, one of the following things would happen: Either the block would be allocated correctly and I would get a pointer to it; or there would be no room for the block and I'd get a null pointer. However, I've found that if the size of the block I'm allocating is greater than (freeblocksize - 16) and less than (freeblocksize - 7) (where freeblocksize is the size of the last free block on the heap), the malloc statement returns a pointer, but the heap is corrupted. The following code demonstrates this: <<code>> #include "mbed.h" int main() { // The following lines print out a list of the blocks on the heap // The free block size is 0xfd4 fprintf(stderr, "\nHeap at start\n"); __heapvalid((__heapprt)fprintf,stderr,1); // We now allocate a block close to the size of the free block // If the block size falls between the (0xfd4 - 15) and (0xfd4 - 8), // malloc returns a valid pointer but the heap is corrupted. // If the block is larger or smaller, the malloc statement operates // correctly. // char* ptr = (char*)malloc (0xfd4 - 16); // Works correctly char* ptr = (char*)malloc (0xfd4 - 15); // Fails // char* ptr = (char*)malloc (0xfd4 - 14); // Fails // char* ptr = (char*)malloc (0xfd4 - 13); // Fails // char* ptr = (char*)malloc (0xfd4 - 12); // Fails // char* ptr = (char*)malloc (0xfd4 - 11); // Fails // char* ptr = (char*)malloc (0xfd4 - 10); // Fails // char* ptr = (char*)malloc (0xfd4 - 9); // Fails // char* ptr = (char*)malloc (0xfd4 - 8); // Fails // char* ptr = (char*)malloc (0xfd4 - 7); // Works correctly fprintf(stderr, "\nHeap after allocation, ptr = %p\n", ptr); __heapvalid((__heapprt)fprintf,stderr,1); char* ptr2 = (char*)malloc (20); fprintf(stderr, "\nHeap after another allocation, ptr2 = %p\n", ptr2); __heapvalid((__heapprt)fprintf,stderr,1); } <</code>> Is this something I'm doing wrong or is it a bug? Can anyone think of a workaround?

Erik -

# 08 Oct 2014

How do you notice the heap being corrupted, and which target are you using (Your LPC812 and LPC11u24 use uARM, while your LPC1768 uses regular ARM, and they have different malloc algorithmes).

Tim Barry

# 08 Oct 2014

I'm using the LPC1768. I noticed the heap corrupting when checking some allocation / deallocation functions in a larger piece of software using "heapvalid". I just happened to have a block of data that fell within the size limits I've mentioned above. In my code example after allocating the large block, the call to

__heapvalid((_heapptr)fprintf,stderr,1);

just returns:

------- heap validation complete

with no block information. When I allocate to the heap again it returns:

alloc block 1000034c size  18
alloc block 10000364 size fd4
alloc block 10001338 size 10007b00
------- heap validation complete

i.e. there are no free blocks listed

Tim Barry

# 13 Oct 2014

Can anyone from mbed look into this? I need to use the heap in my product and I currently have no workaround.

Martin Kojtal

# 13 Oct 2014

Hello Tim,

how do you check the size using heapvalid? What's the output? You shared in your first post the program, but I would like to see also the output from the console (entire output it prints, will be more clear to users) , plus also heapstats() , as it provides some other details.

Tim Barry

# 13 Oct 2014

Hi Martin,

I've modified the code to include heapstats:

#include "mbed.h"

int main() {
    // The following lines print out a list of the blocks on the heap
    // The free block size is 0xfd4 
    fprintf(stderr, "\nHeap at start\n");
    __heapvalid((__heapprt)fprintf,stderr,1);
    __heapstats((__heapprt)fprintf,stderr);
    // We now allocate a block close to the size of the free block
    // If the block size falls between the (0xfd4 - 15) and (0xfd4 - 8),
    // malloc returns a valid pointer but the heap is corrupted.
    // If the block is larger or smaller, the malloc statement operates
    // correctly.
    char* ptr = (char*)malloc (0xfd4 - 16);   // Works correctly
//    char* ptr = (char*)malloc (0xfd4 - 15);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 14);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 13);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 12);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 11);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 10);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 9);    // Fails
//    char* ptr = (char*)malloc (0xfd4 - 8);    // Fails
//    char* ptr = (char*)malloc (0xfd4 - 7);    // Works correctly
    fprintf(stderr, "\nHeap after allocation, ptr = %p\n", ptr);
    __heapvalid((__heapprt)fprintf,stderr,1);
    __heapstats((__heapprt)fprintf,stderr);
    char* ptr2 = (char*)malloc (20);
    fprintf(stderr, "\nHeap after another allocation, ptr2 = %p\n", ptr2);
    __heapvalid((__heapprt)fprintf,stderr,1);
    __heapstats((__heapprt)fprintf,stderr);
}

This produces the following output:

Heap at start
alloc block 1000034c size  18
free block  10000364 size fd4 next=00000000
------- heap validation complete
4052 bytes in 1 free blocks (avge size 4052)
1 blocks 2^10+1 to 2^11

Heap after allocation, ptr = 10000368
alloc block 1000034c size  18
alloc block 10000364 size fc8
free block  1000132c size   c next=00000000
------- heap validation complete
12 bytes in 1 free blocks (avge size 12)
1 blocks 2^2+1 to 2^3

Heap after another allocation, ptr2 = 10001330
alloc block 1000034c size  18
alloc block 10000364 size fc8
alloc block 1000132c size  18
free block  10001344 size 100c next=00000000
------- heap validation complete
4108 bytes in 1 free blocks (avge size 4108)
1 blocks 2^11+1 to 2^12

Each call to heapvalid produces a list of allocated blocks, ending with a free block. The size of the free block matches the size returned by heapstats. All is working as I would expect.

Now if I change the size of the allocated block to fall within the limits I've mentioned in my original post:

//    char* ptr = (char*)malloc (0xfd4 - 16);   // Works correctly
    char* ptr = (char*)malloc (0xfd4 - 15);   // Fails
//    char* ptr = (char*)malloc (0xfd4 - 14);   // Fails

the following output is produced:

Heap at start
alloc block 1000034c size  18
free block  10000364 size fd4 next=00000000
------- heap validation complete
4052 bytes in 1 free blocks (avge size 4052)
1 blocks 2^10+1 to 2^11

Heap after allocation, ptr = 10000368
------- heap validation complete
0 bytes in 0 free blocks (avge size 0)

Heap after another allocation, ptr2 = 10001340
alloc block 1000034c size  18
alloc block 10000364 size fd4
alloc block 10001338 size 1018
------- heap validation complete
4092 bytes in 1 free blocks (avge size 4092)
1 blocks 2^10+1 to 2^11

The call to heapvalid after the block is allocated (line 25 of code) doesn't produce a list of allocated blocks or free blocks and the call to heapstats (line 26 of code) returns 0 free bytes. After an allocation of 20 bytes (line 27) the call to heapvalid produces a list of allocated blocks, which doesn't include the 20 bytes just allocated and doesn't include any free blocks. The call to heapstats suggests that there are 4092 free bytes which is actually what I'd expect, but it doesn't match the output from heapvalid.

Maybe it's heapvalid that is returning bad info??

Thanks in advance for your help.

Hi Martin, I've modified the code to include heapstats: <<code>> #include "mbed.h" int main() { // The following lines print out a list of the blocks on the heap // The free block size is 0xfd4 fprintf(stderr, "\nHeap at start\n"); __heapvalid((__heapprt)fprintf,stderr,1); __heapstats((__heapprt)fprintf,stderr); // We now allocate a block close to the size of the free block // If the block size falls between the (0xfd4 - 15) and (0xfd4 - 8), // malloc returns a valid pointer but the heap is corrupted. // If the block is larger or smaller, the malloc statement operates // correctly. char* ptr = (char*)malloc (0xfd4 - 16); // Works correctly // char* ptr = (char*)malloc (0xfd4 - 15); // Fails // char* ptr = (char*)malloc (0xfd4 - 14); // Fails // char* ptr = (char*)malloc (0xfd4 - 13); // Fails // char* ptr = (char*)malloc (0xfd4 - 12); // Fails // char* ptr = (char*)malloc (0xfd4 - 11); // Fails // char* ptr = (char*)malloc (0xfd4 - 10); // Fails // char* ptr = (char*)malloc (0xfd4 - 9); // Fails // char* ptr = (char*)malloc (0xfd4 - 8); // Fails // char* ptr = (char*)malloc (0xfd4 - 7); // Works correctly fprintf(stderr, "\nHeap after allocation, ptr = %p\n", ptr); __heapvalid((__heapprt)fprintf,stderr,1); __heapstats((__heapprt)fprintf,stderr); char* ptr2 = (char*)malloc (20); fprintf(stderr, "\nHeap after another allocation, ptr2 = %p\n", ptr2); __heapvalid((__heapprt)fprintf,stderr,1); __heapstats((__heapprt)fprintf,stderr); } <</code>> This produces the following output: <<code>> Heap at start alloc block 1000034c size 18 free block 10000364 size fd4 next=00000000 ------- heap validation complete 4052 bytes in 1 free blocks (avge size 4052) 1 blocks 2^10+1 to 2^11 Heap after allocation, ptr = 10000368 alloc block 1000034c size 18 alloc block 10000364 size fc8 free block 1000132c size c next=00000000 ------- heap validation complete 12 bytes in 1 free blocks (avge size 12) 1 blocks 2^2+1 to 2^3 Heap after another allocation, ptr2 = 10001330 alloc block 1000034c size 18 alloc block 10000364 size fc8 alloc block 1000132c size 18 free block 10001344 size 100c next=00000000 ------- heap validation complete 4108 bytes in 1 free blocks (avge size 4108) 1 blocks 2^11+1 to 2^12 <</code>> Each call to heapvalid produces a list of allocated blocks, ending with a free block. The size of the free block matches the size returned by heapstats. All is working as I would expect. Now if I change the size of the allocated block to fall within the limits I've mentioned in my original post: <<code>> // char* ptr = (char*)malloc (0xfd4 - 16); // Works correctly char* ptr = (char*)malloc (0xfd4 - 15); // Fails // char* ptr = (char*)malloc (0xfd4 - 14); // Fails <</code>> the following output is produced: <<code>> Heap at start alloc block 1000034c size 18 free block 10000364 size fd4 next=00000000 ------- heap validation complete 4052 bytes in 1 free blocks (avge size 4052) 1 blocks 2^10+1 to 2^11 Heap after allocation, ptr = 10000368 ------- heap validation complete 0 bytes in 0 free blocks (avge size 0) Heap after another allocation, ptr2 = 10001340 alloc block 1000034c size 18 alloc block 10000364 size fd4 alloc block 10001338 size 1018 ------- heap validation complete 4092 bytes in 1 free blocks (avge size 4092) 1 blocks 2^10+1 to 2^11 <</code>> The call to heapvalid after the block is allocated (line 25 of code) doesn't produce a list of allocated blocks or free blocks and the call to heapstats (line 26 of code) returns 0 free bytes. After an allocation of 20 bytes (line 27) the call to heapvalid produces a list of allocated blocks, which doesn't include the 20 bytes just allocated and doesn't include any free blocks. The call to heapstats suggests that there are 4092 free bytes which is actually what I'd expect, but it doesn't match the output from heapvalid. Maybe it's heapvalid that is returning bad info?? Thanks in advance for your help.

Tim Barry

# 23 Oct 2014

Hi Martin,

Any thoughts on what might be going wrong here?

Thanks Tim

Geremia G

# 26 Oct 2014

What i would think, is about heapvalid and heapstats being dodgy. Looking at here http://www.keil.com/support/man/docs/armlib/armlib_chr1359122851244.htm in the note it tells "If you are using the default one-region memory model, heap memory is allocated only as it is required. This means that the amount of free heap changes as you allocate and deallocate memory." The example shows that heapstats does not returns the whole free heap space and, in my ignorance, i would think the end addr is something like predefined at begin, or if you malloced and freed a big block, that's the new end addr for calculating free space. I would think when you malloc in that weird size range, you probably fall on the exact boundary of such "block", so a "next" one is not created. Unfortunally i don't know how malloc really works, so this is just my thought.

From what i see from your output, when you tell it fails, in real you don't get a null pointer from malloc as you stated in first post. Are you sure your buffers are really corrupted? i mean, without using hepstats and heapvalid as checks, just check if malloc rets null ptr, fill buffers and check back if really corrupted.

Important changes to forums and questions