Can someone summarize Snowflake's unique technical value? I'm quite familiar with both Redshift (I would summarize it as Postgres adapted to sharded, columnar OLAP functioning) and BigQuery (there is a famous paper explaining the architecture). Also with more traditional databases such as MySQL, PostgreSQL, SQL Server, and columnar OLAP databases like Vertica. I explored the website a little bit, but couldn't construe a clear statement of the technical architectural value. Some of the comments here are valuable, but I'm missing a clearer "big picture" overview. Thanks!
I've worked with a large Postgres cluster before (~1PB of data) and have been experimenting with Snowflake recently. I would say there's two clear technical advantages of Snowflake over Redshift. First is there's no maintenance when using Snowflake. You just signup for a Snowflake account, upload a CSV, and you can start querying the data. This is in contrast to Redshift where you have to manually provision a cluster, resize it as you add more data, etc.
The second is their pricing. Storing data in Snowflake costs the same as it would cost to store in S3. The tradeoff is you also have to pay based on how long your queries take. Depending on your workload this can result in a massive cost savings. If you access only small amounts of your data infrequently, it's like you're storing the data in S3 and you only have to pay a bit more when accessing the data. This is in contrast to Redshift where you have to pay for the full cost of the cluster regardless of whether you are actually querying the data or not.
Snowflake also has a ton of quality of life improvements compared to Redshift. One really nice thing is you can change the amount of compute used for any individual query. For example, if you have one specific slow query, you can allocate 4x the compute for that one query, pay 4x as much while the query is running, and get the query to run 4x faster (ultimately costing you the same amount as if you used 1x the compute).
One neat thing is there's ultimately only one "Snowflake instance" in each region. Everyone's tables are in the same instance, but you can only access the tables you have permission to access. This allows you to easily share data between different Snowflake accounts. You can store the data in one account and query it from another.
So the core value proposition is really strong and it also has a bunch of extra features that are all pretty useful at the end of the day.
This post focused on Snowflake solely from a business point of view. I'm considering writing another one that focuses on it from a technical point of view.
Snowflake owes much of its performance benefits to "micro-partitions" [1]. BigQuery is a worthwhile comparison. MySQL, PostgreSQL, SQL Server, and Vertica are not close equivalents.
FWIW, if you're looking at PostgreSQL-at-humungo scale, there are a few options around. TimescaleDB, Citus (which Azure now offers as a service called Hyperscale DB) and there are others I always feel terrible for overlooking.
I work for VMware and get along well with folks who work on Greenplum. It's still doing massive workloads with massive amounts of data for lots of customers, has the ability to operate over blob stores with predicate pushdowns and recently merged up to parity with the PostgreSQL 12 upstream[0]. It's the fruit of a six year effort to return to the upstream from a heavily modified fork of 8.3. A truly monumental effort.
Snowflake’s value is that they provide the same technical products as amazon/google/etc, but are not amazon/google/etc. Some shops like buying into the google ecosystem, some are afraid of vendor lock in.
Probably other things, but many companies exist just to be alternatives to faang. If you’re good enough, you surpass that intention.
I saw them a year or two ago positioning themselves in contrast to Redshift and BigQuery. I though "these guys are building something for Microsoft to acquire" (my thought was, something with a more modern OLAP architecture than SQL Server, which they can offer via Azure). Naive me, they were so much more business savvy than that...
