Archive

January 2016

Hive on Tez vs Hive on Spark on CDH, and What Is Legacy Now

Apache Tez was introduced as a faster, DAG-based execution engine for Hive and other Hadoop workloads, often noticeably faster than classic MapReduce in ETL pipelines. This article explains what Tez brought to Hive, how it fit into CDH-era deployments, and why the old practice of hand-compiling Tez against CDH 5.x is now a legacy pattern rather than a recommended approach. It also covers the Spark side: how Spark reads Hive tables today through SparkSession with Hive support, why that is not the same as Hive running on an execution engine, and the manual HiveContext workaround CDH 5.3 needed. Apache Tez was designed as a low-latency, DAG-based execution engine for Hadoop. It replaced many of the heavyweight MapReduce patterns used by early Hive deployments with more efficient execution graphs, reusing containers and avoiding unnecessary materialization steps. In practical terms, switching Hive from MapReduce to Tez often made ETL and reporting jobs noticeably faster on the sam...