25 Unpopular and Contradictory Opinions on Data and Engineering

1. If your Data Product does not generate revenue, it’s an internal tool

2. Many companies will get more value out of a half-decent Business Intelligence team than they will trying to build Data Products

3. Data Engineering is a form of Software Engineering

4. Data engineering projects often do not require the complexity seen in software engineering projects

5. Complex data engineering projects are not more complex than software engineering projects; they just used a specialized set of tools

6. Centralized data platforms are good

7. Centralized data teams often are not

8. Your data team structure should compliment the rest of the organization

9. More data analysts probably won’t make your data team more effective but more analysts might help your organization use data more effectively

10. More data engineers might make your data platform better

11. “Data” as a standalone function has created more confusion than any other function. UX and User Research might be a close second

12. Data PMs are only warranted when your product is VERY technically complex

13. Data PMs are often a byproduct of misaligned business goals

14. The data infrastructure design and tooling at Meta, Netflix and AirBnb are not going to solve your startup’s problems

15. Most of your tooling decisions will not matter

16. A handful of your tooling choices will cause acute frustration

17. Don’t use multiple vendor tools when one would suffice; the marginal additional value for a dedicated solution is seldom worth the additional complexity

18. Even the most pragmatic engineers get caught up in the vendor hype cycle

19. Many companies using Snowflake, BigQuery and Redshift could just be using rollup tables in Postgres

20. Save a few exception, there is now little differentiation in the Data Warehouses; most novel feature sets are now table stakes

21. That data catalog you’ve been scoping out is probably a waste of time

22. A metadata catalog is very much worth it

23. You don’t need a new tool to have a metadata catalog

24. Your organization’s Data Strategy has less to do with its data than you think

25. If you think your Data Strategy is complete, you should revisit it

---