The fact that custom object associations are only many-to-many is a big detriment. Two big reasons.
Cross object associations can't be display on views. For example, because company-to-contact relationship is a one-to-many. This means a contact can only have one company and thus there is no conflict when adding company information on a contact view - which is permissible by the system.
The same things applies to workflows that may be used to copy data from a property on one object to another. For instance, if you have data on a company and you want the contacts to inherit that data via a workflow, that's possible. But if you want to contact to inherit data from an associated custom objects, it's not possible.