I am new to databases and trying to understand the basic concepts. I have learned how to delete data in a database.  But one of my friends told me that you should never delete data in a database.  Rather, when its no longer needed, it's better to simply mark it or flag it as 'not in use'.

Is that true? If so, how would a big company like IBM handle their data for a hundred or more years?

As with all these things the answer is "it depends".

If the user is ever likely to want the data back then your friends are right - you don't really delete just mark the record as "deleted". This way when the user changes their mind you can recover the data.

However, if the deleted data is more than a certain time period old (a year for example) you might decide to really delete it from the live tables but keep it in either an archive table or even just on back up should the user ever want it back. In this way you can keep the amount of data (live and recently deleted) to a minimum.

However, if the data is ephemeral or easily recreated you may well decide to actually delete the data.

There is one class of data that you have to delete - and that's personal data that the user doesn't want you to hold any more. There may be local laws (e.g. in the EU) that makes this a mandatory requirement (thanks Gavin)

Equally there may be rules that require you not to delete data, so before deciding anything check with any regulatory authorities on what you need to do to comply with the law.
