count(distinct xx)优化 本页介绍天翼云TeleDB数据库的count(distinct xx)优化案例。 count(distinct xxx)发生在CN节点,对于TP类业务,需要操作的数据量少的情况下,性能开销是没有问题的,而且往往比下推执行的性能开销还要小。但如果一次要操作的数据量比较大的AP类业务,则网络传输就会成功瓶颈。 针对distinct,可以改写为group by写法,将数据去重操作下推到DN上执行,提高效率。 下面是针对count(distinct xx)写法的SQL优化案例: 1. 测试数据准备: teledb CREATE TABLE t1(f1 serial not null unique,f2 text,f3 text,f4 text,f5 text,f6 text,f7 text,f8 text,f9 text,f10 text,f11 text,f12 text) distribute by shard(f1); NOTICE: Replica identity is needed for shard table, please add to this table through "alter table" command. CREATE TABLE Time: 89.938 ms teledb insert into t1 select t,md5(t::text),md5(t::text),md5(t::text),md5(t::text),md5(t::text),md5(t::text),md5(t::text),md5(t::text),md5(t::text),md5(t::text),md5(t::text) from generateseries(1,1000000) as t; INSERT 0 1000000 Time: 14849.045 ms (00:14.849) teledb analyze t1; ANALYZE Time: 1340.387 ms (00:01.340) 2. 改写前的SQL执行计划和耗时: teledb explain (verbose) select count(distinct f2) from t1; QUERY PLAN Aggregate (cost103320.00..103320.01 rows1 width8) Output: count(DISTINCT f2) > Remote Subquery Scan on all (dn01,dn02,dn03,dn04,dn05,dn06,dn07,dn08,dn09,dn10) (cost100.00..100820.00 rows1000000 width33) Output: f2 > Seq Scan on public.t1 (cost0.00..62720.00 rows1000000 width33) Output: f2 (6 rows) Time: 0.748 ms postgres