Recently I took a peak at a default mysql installation and found one interesting, to not say scary, thing. SQL_MODE comes set to empty string and if you dig into the documentation you find that... you find nothing! Yes you heard me, whatever a blank SQL_MODE means is not known yet is what most of the people run with because this little fact is advertised nowhere, especially not in the post installation guidelines.

Playing around what can be empirically observed is that mysql behaves very loosely, for example by truncating strings or rounding numbers. So unless your application is very strict about this, you may very well have corrupted data in several places.
What mysql will do in those cases is to throw a warning, so to some extent you have a chance to figure out what happened, but it's hard to find applications doing proper error checking, let alone watching out for warnings (although some people do).

What SQL_MODE='' breaks

Here's a brief list of what breaks when defaulting to an empty sql_mode:

• Invalid dates will be allowed and converted to 0000-00-00
• Division by zero errors on insert/update will insert a NULL and return a warning
• Zero dates (0000-00-00) will be allowed
• Integers will be rounded to upper boundary and a warning will be raised
• Strings will be truncated and a warning will be raised

This is really bad for your data! You can read more about SQL_MODE settings and implications here.

Conclusions

Default SQL_MODE harms your data, you're advised to change it. Personally I suggest TRADITIONAL, which implies STRICT_ALL_TABLES and STRICT_TRANS_TABLES, and those are enough to keep your data sane (as a result some applications might break, so be careful).
This is quite bad of Mysql, both because it leads to corrupted data, but most importantly because it's not advertised as it should in the post installation document, which leaves people exposed. And I don't actually see any reason for such default setting if not backward compatibility with poorly coded applications, which could be read as an attempt from Mysql to minimize the chance of complaints from its users even tho they are effectively helping bad applications to prosper.