It’s going to have a very big hill to climb if it’s playing in a space where duckdb already has a hold. Duck has probably been my favourite technology find in the last few years. Awesome tech.
Cool! But as a data engineer I don't know when I would ever use this. Getting data into a centralized place so it can be joined and queried easily is like prio 1 for any data team.
I'm sure SREs will really love me doing expensive adhoc queries against production postgres /s
I've yet to work in enterprises big enough to have multi cloud data warehouses though, maybe it's more useful in that setting?
Sorry, just trying to understand. Why would I use this over duckdb. duckdb already has plugins for a lot of databases. Is the syntax the advantage?
ClickHouse also supports a lot of data sources and has a local mode where you just use a single binary with local-only access.
Coincidentally, I wrote an article today on how I use it for similar scenarios. It can fetch from S3, multiple databases at once, and so on.
And you get all the benefits of a database when you need to join or postprocess data from multiple sources.
https://rushter.com/blog/clickhouse-data-processing/
It’s going to have a very big hill to climb if it’s playing in a space where duckdb already has a hold. Duck has probably been my favourite technology find in the last few years. Awesome tech.
I’ll still check this out though.
I agree
Cool! But as a data engineer I don't know when I would ever use this. Getting data into a centralized place so it can be joined and queried easily is like prio 1 for any data team.
I'm sure SREs will really love me doing expensive adhoc queries against production postgres /s
I've yet to work in enterprises big enough to have multi cloud data warehouses though, maybe it's more useful in that setting?