org.apache.spark.sql.hive.test

Class TestHiveContext

Object

org.apache.spark.sql.SQLContext

org.apache.spark.sql.hive.HiveContext

org.apache.spark.sql.hive.LocalHiveContext

org.apache.spark.sql.hive.test.TestHiveContext

All Implemented Interfaces:

com.typesafe.scalalogging.slf4j.Logging, java.io.Serializable

Direct Known Subclasses:

TestHive

public class TestHiveContext
extends LocalHiveContext

A locally running test instance of Spark's Hive execution engine.

Data from testTables will be automatically loaded whenever a query is run over those tables. Calling reset will delete all tables and other state in the database, leaving the database in a "clean" state.

TestHive is singleton object version of this class because instantiating multiple copies of the hive metastore seems to lead to weird non-deterministic failures. Therefore, the execution of test cases that rely on TestHive must be serialized.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	TestHiveContext.QueryExecution Override QueryExecution with special debug workflow.
class	TestHiveContext.TestTable

Constructor Summary

Constructors

Constructor and Description
TestHiveContext(SparkContext sc)

Method Summary

Methods

Modifier and Type	Method and Description
boolean	cacheTables()
scala.util.matching.Regex	describedTable()
TestHiveContext.QueryExecution	executePlan(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)
java.io.File	getHiveFile(String path)
scala.Option<java.io.File>	hiveDevHome() The location of the hive source code.
java.io.File	hiveFilesTemp()
scala.Option<java.io.File>	hiveHome() The location of the compiled hive distribution
scala.collection.Seq<TestHiveContext.TestTable>	hiveQTestUtilTables()
java.io.File	inRepoTests()
void	loadTestTable(String name)
String	metastorePath()
scala.collection.mutable.HashMap<String,TestHiveContext.TestTable>	registerTestTable(TestHiveContext.TestTable testTable)
void	reset() Resets the test instance by deleting any tables that have been created.
scala.collection.Seq<String>	runSqlHive(String sql) Runs the specified SQL query using Hive.
scala.collection.mutable.HashMap<String,TestHiveContext.TestTable>	testTables() A list of test tables and the DDL required to initialize them.
String	warehousePath()

Methods inherited from class org.apache.spark.sql.hive.HiveContext

createTable, hivePlanner, hiveql, hql

Methods inherited from class org.apache.spark.sql.SQLContext

cacheTable, createParquetFile, createSchemaRDD, logicalPlanToSparkQuery, parquetFile, registerRDDAsTable, sparkContext, sql, table, uncacheTable

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.typesafe.scalalogging.slf4j.Logging

logger

Constructor Detail

TestHiveContext

public TestHiveContext(SparkContext sc)

Method Detail

warehousePath

public String warehousePath()

Overrides:

warehousePath in class LocalHiveContext

metastorePath

public String metastorePath()

Overrides:

metastorePath in class LocalHiveContext

hiveHome

public scala.Option<java.io.File> hiveHome()

The location of the compiled hive distribution

hiveDevHome

public scala.Option<java.io.File> hiveDevHome()

The location of the hive source code.

runSqlHive

public scala.collection.Seq<String> runSqlHive(String sql)

Description copied from class: HiveContext

Runs the specified SQL query using Hive.

executePlan

public TestHiveContext.QueryExecution executePlan(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)

hiveFilesTemp

public java.io.File hiveFilesTemp()

inRepoTests

public java.io.File inRepoTests()

getHiveFile

public java.io.File getHiveFile(String path)

describedTable

public scala.util.matching.Regex describedTable()

testTables

public scala.collection.mutable.HashMap<String,TestHiveContext.TestTable> testTables()

A list of test tables and the DDL required to initialize them. A test table is loaded on demand when a query are run against it.

registerTestTable

public scala.collection.mutable.HashMap<String,TestHiveContext.TestTable> registerTestTable(TestHiveContext.TestTable testTable)

hiveQTestUtilTables

public scala.collection.Seq<TestHiveContext.TestTable> hiveQTestUtilTables()

cacheTables

public boolean cacheTables()

loadTestTable

public void loadTestTable(String name)

reset

public void reset()

Resets the test instance by deleting any tables that have been created. TODO: also clear out UDFs, views, etc.