Is my understanding correct that when a string or a dynamic array is extended it might result in its existing content being released to the heap?

If so, is it possible to ensure that this is zeroed or randomised first, without having to do it manually?

Currently not, although such behaviour could easile be introduced as an option.

Current HeartBleed frenzy getting you (or your bosses) scared ? :)

Michael.

Mark Morgan Lloyd wrote

Is my understanding correct that when a string or a dynamic array is extended it might result in its existing content being released to the heap? If so, is it possible to ensure that this is zeroed or randomised first, without having to do it manually?

You can install a memory manager that does this for all (de)allocations and then calls through to the original memory manager. There is no way to only do this for strings and dynamic arrays, and I don't think it would be a good idea to do so. Not all passwords are strings, so that would probably mostly give a false sense of security.

Jonas

Using a memory manager would reliably wipe strings etc. when reallocated (i.e rather than when an assignment didn't trigger reallocation). On the other hand it would have the overhead of also overwriting blocks that the user knew were being freed and could audit first, as well as stuff that was being freed as part of the RTL operation.

It's that latter case- where the RTL is copying something without the user being aware- that I think is significant.

:-) No, but I don't think enough people are focussing on the real problem which is that the OpenSSL developers were letting sensitive data leak to the freelist.

If, when they wrote the code some years ago, they'd been rigorous in their handling of passwords and private keys then the current bug- introduced in 2012- would have been far less serious.

Such stuff from RTL operations may include getmem'd buffers used to read private key data from disk.

It's that latter case- where the RTL is copying something without the user being aware- that I think is significant.

Either the memory manager guarantees trashing, or it does not. Having half-solutions are worse than nothing at all, because it gives a false sense of security. Even the best audits are guaranteed to miss things (as in case of the person who wrote and the person who reviewed/audited the patch that introduced the heartbleed bug), and as you mentioned there is no control over the internals of the RTL (which has not been written at all from the point of view that buffers that might sometimes contain sensitive data should always be cleansed).

Jonas

Correct.

Having looked at the openssl library, I can say it's a miracle it seen such a mess of macros and whatnot, a good showcase of why I think C is not a good language choice. So, not surprised that it contains leaks :(

OTOH, I think people are hugely exaggerating the problem, considering it was introduced relatively recently and that I got my security update before it hit the newspapers.

That is of course not to say that it shouldn't be fixed and people shouldn't bother. But the way it is presented is more about scaring people than anything else. Hysterics...

Michael.

Michael Van Canneyt wrote

OTOH, I think people are hugely exaggerating the problem, considering it was introduced relatively recently and that I got my security update before it hit the newspapers.

The exploit code was also on github before news about the bug hit the newspapers. There is even some evidence it may have been exploited for at least 3 months already, maybe longer (because unless you used some special intrusion detection system rules, it left no traces at all in the log files, so there's only very little data to go on).

Also, the fact that you updated your server so quickly, doesn't mean that everyone did. Our university's mail servers were only patched yesterday morning (more than 24 hours after the story broke), because they needed time to prepare the patching (don't ask, I don't know the details). I bet tons of credentials and private data has been accessed over the past days all over the world.

That is of course not to say that it shouldn't be fixed and people shouldn't bother. But the way it is presented is more about scaring people than anything else. Hysterics...

I very strongly disagree. All certificates and login data used with vulnerable services over the past year or so should be considered compromised. It will probably take months before all affected certificates are replaced (if that ever happens for most of them), and many of the replaced and hence potentially compromised certificates will probably never be revoked. The result is a huge increase in chances for man-in-the-middle attacks, not to mention all the compromised login data and private information (emails, bank statements, ...).

Jonas

Like I said, this is not to say that no action should be taken. I expect that all sensitive sites (banks, google, etc) have taken immediate action.

That the login of my local tennis/pool/golf club was compromised is not really so scary, sorry.

Anyway, getting off topic.

The main point is that in FPC you can install a memory manager that wipes out any memory when getting or releasing it, if you want to make your software more secure that way.

Michael.

Michael Van Canneyt wrote

The main point is that in FPC you can install a memory manager that wipes out any memory when getting or releasing it, if you want to make your software more secure that way.

how would one go about doing this? i learned in my TP3/6 days to use fillchar on everything to ensure that it was filled with 0x00... especially my data files... when looking at them with a hex editor, my OCD would hit strongly because the data file was "not clean and holding only my data"...

i don't know how one would go about cleaning released memory as someone else asked about (eg: extending an array or string or etc)... once the memory is released, it is no longer accessible, right?

agreed...

The ideal is to overwrite sensitive data with random bytes, since even the length of a zero block can be useful to an attacker.

i don't know how one would go about cleaning released memory as someone else asked about (eg: extending an array or string or etc)... once the memory is released, it is no longer accessible, right?

But since the deallocated memory is going to a local heap, sooner or later you're likely to get that back as a new block. That, as I understand it, is what happened in OpenSSL.

The worst case would be if a cautious programmer zeroed everything that he was freeing explicitly, without realising that any strings he extended were going back into the heap intact so now stood out like a sore thumb. Anybody who was able to inspect the heap would see only strings that had subsequently been expanded:

     password := getFromUser();          // Probably about 7 chars
     password += #$00 + systemName();    // Leaves password on heap
     saveToDB(Tiger2(password));
     zeroString(password)                // Length doesn't change
   end;                                  // Zeroed block freed to heap

"waldo kitty" wrote

On 4/11/2014 5:03 AM, Michael Van Canneyt wrote: > > The main point is that in FPC you can install a memory manager that

wipes out

> any memory when getting or releasing it, if you want to make your

software more

> secure that way.

[snip]

i don't know how one would go about cleaning released memory as someone

else asked about (eg: extending an array or string or etc)... once the memory is released, it is no longer accessible, right?

That's where the heap manager comes in. In FPC every allocation that is not done manually through OS functions (namely string/array allocation, class allocation, getmem() and new()) go through the heap manager of which you can register a custom one. This heap manager hooks the allocation and deallocation calls and thus you could provide a manager that clears deallocated memory (or fills with random data) before it's passed on to the default heap manager.

Regards,
Sven

Attached is an implementation that allows you to specify:

What to do when allocating memory (zero out, randomize, nothing) What to do when freeing memory (zero out, randomize, nothing).

Careful, it does not play well with the heaptrc unit.

If there is an interest in such thing, we can add it to the RTL.

Michael.

Michael Van Canneyt wrote

Attached is an implementation that allows you to specify: What to do when allocating memory (zero out, randomize, nothing) What to do when freeing memory (zero out, randomize, nothing).

very nice! thank you sir! i'm still learning FPC and all its fancy (to me compared to TP/BP) features :)

Careful, it does not play well with the heaptrc unit.

why is that? is there a way to fix it? i do use the heaptrc unit to try to ensure that i'm not leaving stuff behind...

If there is an interest in such thing, we can add it to the RTL.

as miniscule as it is compared with others', here's my vote in the positive ;)

It works together, no worries there, but heaptrc does it's own 'trashing' a bit: it writes some housekeeping data, so released zeroed-out memory is not 100% zeroed-out.

The regular heap manager itself does this too, but heaptrc just a bit more...

Duly noted :)

Michael.

Michael Van Canneyt wrote

Attached is an implementation that allows you to specify:

A few comments:
- allocmem already zeros the memory. No need to do it a second time
- Getmem and Allocmem can return nil (dependent on mem manager, sometimes on ReturnNilIfGrowHeapfails). A test in ZeroMem and RandomMem on nil would be "Safe".
- SafeReAllocMem doesn't clear/scramble the memory in case a realloc moves the memory block to a different place.

Cheers

Indeed, copy&paste from getmem. Removed the zeroing.

- Getmem and Allocmem can return nil (dependent on mem manager, sometimes on ReturnNilIfGrowHeapfails). A test in ZeroMem and RandomMem on nil would be "Safe".

Indeed :) Good point, I have added this check :)

- SafeReAllocMem doesn't clear/scramble the memory in case a realloc moves the memory block to a different place.

You are right. A logic error. ReallocMem is the more tricky one. Revised.

That's why I post such quick code; for peer review.

Thanks for pointing these out, revised version attached. Again, comments/reviews welcome.

Michael.

In case of a block move, the memory (OP) is already freed when you clear/scramble it. I'm afraid you can't simply re-use the underlying ReAllocMem (think cmem for example).

You're welcome :)

Ludo

Can I add my positive vote vote to that please, for the specific case where a string etc. is expanded. As others have said, if a programmer /explicitly/ releases unwiped sensitive data that's his problem.

Hmh.
You are right of course. Well, that makes it actually simpler. I will look at this tonight.

Thanks once more,

Michael.