Tajo provides a feature to use Hive functions directly without re-compilation or additional code.
Only thing to do is registering path to a directory for jar files containing your hive functions.
You can do this by set
tajo-site.xml like the following.
<property> <name>tajo.function.hive.jar-dir</name> <value>/path/to/hive/function/jar</value> </property>
The path should be one in local filesystem. HDFS directory is not supported because of JAVA URI compatability problem.
The path must point to a directory, not a file. And multiple directory entries are not allowed. However, it is possible to load multiple jar files.
Using in detail¶
Tajo reads hive functions override
org.apache.hadoop.hive.ql.exec.UDF class. Function name is used as specified in
@Description annotation. If it doesn’t exist, Tajo uses full qualified class name as function name. For example,
it can be like this :
select com_example_hive_udf_myupper('abcd'), so it is recommended to use Description annotation.
Additionally if some function signature duplicate occurs, it may throw
Parameter type / Return type¶
Hive uses Writable type of Hadoop in functions, but Tajo uses its internal Datum type. So only some Writable types are supported currently by internal converting. They are listed below.
(*) They are in org.apache.hadoop.hive.serde2.io package, others are in org.apache.hadoop.io package.
1. Currently, Hive UDAF is not supported. Old UDAF interface is deprecated in Hive, and new GenericUDAF interface cannot be applied because of function design difference between Tajo and Hive. For same reason, new GenericUDF functions are not supported in Tajo.
2. Because HDFS directory is not supported, Hive UDF jar files should be copied to each worker directory and each path should be specified in tajo-site.xml.