Well Knowage does everything you are looking for (kind of). Knowage combines several open source tools into one product.
From what I can tell, you only need the following:
1) BIRT and/or Jasper reporting engines
2) QBE models and worksheets.
3) an OLAP engine like Mondrian
4) The cockpit engine (for dashboards)
Knowage had all the admin features to manage roles, users etc.
HOWEVER: It comes really down to what you are willing to pay.There is a free community edition and a commercial edition. Knowage has recently has gone the same route as Pentaho and Tibco Jaspersoft. They have commercialized their open source based offering. Knowage used to be unique (when they were SpagoBI) in that they were 100% open sourced and gave us enterprise features with their product.
Now they have a community edition which has a subset of features. So if you want to pay big bucks, then Knowage has everything you want. But then again so do the commercial versions of Jaspersoft and Pentaho. Not sure if Knowage's pricing is more competitive.
You seem to require the multi-tenant feature to handle multiple companies. That is no longer part of the community edition. Also a key feature if you are a consulting company, is the ability to import and export documents/reports. This feature is also now part of the enterprise edition.
Bottom Line: If you want a community edition, I think it has more features that Pentaho and Jaspersoft, but you really are limited. If you are technically capable, you can always work with the prior version of Knowage called SpagoBI. Version 5.1 is the most stable (but no longer supported and takes a bit of work to find). It has everything you want, but you will have to manage changes/upgrades yourself from source code. It has older versions of Jasper reports and Birt, but you can upgrade yourself fairly easily.