Skip to content

Recently Updated

06-03: 202204291059 - Managing separation of concerns in the data domain will be solved on an organizational level.

06-03: 202204271244 - Data Engineers being disconnected from data lake-centric consumers is an organizational not an architectural issue

06-03: 202204281117 - Success is the intersection of discipline and habit

06-03: 202204201153 - Our brain uses separated channels to do two things in parallel, but we cannot focus on two things simultaniously.

06-03: 202202171517 - People mistake note taking with thought capturing

06-03: 202204101040 - Web3 sidelines businesses in hope for revenue

06-03: 202204101040 - Living intense experiences is an important complement to a good life

06-03: 202204281623 - For Media platforms people driven engagement is significantly more critical as opposed to brands/outlet produced content.

06-03: 202202071255 - Its necessary to identify goals to achieve, but even more important is to formulate a path towards it.

06-03: 202202161010 - Organizing ourselves to obtain goals depends on our executive function

Tags

consumer-analytics (1)

data-analytics (2)

202204271244 - Data Engineers being disconnected from data lake-centric consumers is an organizational not an architectural issue

202204291059 - Managing separation of concerns in the data domain will be solved on an organizational level.

platform-business (1)

productivity (2)

zettelkasten (1)

202204271244 - Data Engineers being disconnected from data lake-centric consumers is an organizational not an architectural issue

Barr Moses describes this as a major drawback of modern data architectures, where EL jobs are planned and executed in separation of the transformation layer.¹
From my personal experience this issue is rooted in analytics roles (Data Scientist, Data Analysts) are organizationally disconnected from the data engineering team.
In growing organisations in which analysts work embedded in particular functions/teams of the business, there will always be the need for a central data team in my opinion. This team makes sure that data engineers and data analysts / analytics engineers derive usable staged versions of the raw data that arrive in a given data lake. Preventing embedded business oriented roles to rely on "brittle 600 lines SQL query to get their answer".

Barr Moses - Is the modern data warehouse broken?↩

Tags

#data-analytics

Last updated on 6/3/2022