Comparison of data architectures
To Nha Notes | May 27, 2023, 3:45 p.m.
There is a transactional storage software layer that runs on top of an existing data lake and makes it work more like a relational database. The competing open-sourced options for this layer include Delta Lake, Apache Iceberg, and Apache Hudi.
| Characteristic | Relational data warehouse | Data Lake | Modern Data Warehouse | Data Fabric | Data Lakehouse | Data Mesh |
| Year introduced | 1984 | 2010 | 2011 | 2016 | 2020 | 2019 |
| Centralized vs Decentralized | Centralized | Centralized | Centralized | Centralized | Centralized | Decentralized |
| Storage Type | Relational | Object | Relational and Object | Relational and Object | Object | Domain-specific |
| Schema Type | Schema-on-Write | Schema-on-Read | Schema-on-Read and Schema-on-Write | Schema-on-Read and Schema-on-Write | Schema-on-Read | Domain-specific |
| Data Security | High | Low to Medium | Medium to High | High | Medium | Domain-specific |
| Data Latency | Low | High | Low to High | Low to High | Medium to High | Domain-specific |
| Time to Value | Medium | Low | Low | Low | Low | High |
| Total cost of solution | High | Low | Medium | Medium to High | Low to Medium | High |
| Supported use cases | Low | Low to Medium | Medium | Medium to High | High | High |
| Difficulty of development | Low | Medium | Medium | Medium | Medium to High | High |
| Maturity of technology | High | Medium | Medium to High | Medium to High | Medium to High | Low |
| Company skillset needed | Low | Low to Medium | Medium | Medium to High | Medium to High | High |
References
The ebook Deciphering Data Architectures of James Serra